Researchers at the University of Cornell (United States) have developed Spelling, a ring driven by artificial intelligence (AI) designed to identify and translate sign language, through microsonar technology. Although there are some solutions designed to recognize the fingerprint, they have usually not been adopted by the community of deaf people and with auditory difficulties due to their little practicality and voluminous size.
In this sense, in the face of developing a device capable of capturing all subtle and complex movements of the fingers when executing the sign language, the researchers have created the Spelling ring. Thus, driven by AI and equipped with microsonar technology, this ring can continuously and in real time the finger spelling, specifically, in American sign language (ASL) to later pass it to text.
As the researchers have explained in a statement on their website, Spelling currently allows uses such as the introduction of text in computers or ‘smartphones’ through the dactylological spelling, which is used to spell words without corresponding signs, such as the proper names or technical terms. However, according to researchers, with greater development, the Speness ring could “revolutionize the translation of ASL through the continuous monitoring of words and complete sentences with signs.”
Regarding its operation, it is enough to place Spelling on the thumb and, through its integrated microphone and speaker, emit and receive inaudible sound waves that track the movements of the hand and fingers of the user. These movements are complemented with those registered by means of a minigiroscope also integrated into the device, which consists of a 3D printed housing of a size similar to that of a 20 cents coin.
Following this line, an algorithm of deep learning of its own development processes the data registered by the microsonar and predicts the letters described with the hand in ASL in real time and “with a precision similar to that of many existing systems that require more ‘hardware'”, as Hyunchul Lim, leader of the research. As explained, Speness has been evaluated with 20 American sign language users, both experienced and beginners.
These users naturally spell more than 20,000 words of different length and, as a result, the precision of Spelling stood between 82 and 92 percent, depending on the difficulty of the word in question. This is because the AI system is trained to recognize 26 hand forms associated with each letter of the alphabet although, as they have nuanced, it is a challenge given that the variation when representing the letters with gestures “can be significant”, as explained by the Professor of Information Sciences of the University of Cornell, co -author of the article, Cheng Zhang.
With all this, the purpose of Speness is to pave the way “towards a more diverse and inclusive access to computational resources” to benefit the community of users who use ASL language. However, researchers have nuanced that “there is still much to do” to develop devices capable of carrying out a complete recognition of this language.