Amrita Vishwa Vidyapeetham researchers develop India’s first device to turn eye signs into speech

Mumbai, June 14, 2023: Researchers at Humanitarian Technology (HuT) Labs of Amrita Vishwa Vidyapeetham, ranked the 7th Best University in India in the 2023 National Institution Ranking Framework, have developed Indias first device that enables people with speech impairment to communicate with the world by using simple eye signs.
 

The device, called Netravaad, consists of a camera, display, speaker, controller and a rechargeable battery that lasts for six hours on a single charge. The camera detects the eye sign of the user, which is converted into alphabet, word or a sentence by a customized AI algorithm called Sharani. The detected word and sentences are displayed on the screen. In addition, the voice through speakers helps the external world understand the patients feelings, and the patient also gets a feel of speaking to others. Netravaad will advance in its development and become available in regional languages such as Malayalam and Hindi. Additionally, the user interface will be highly interactive, ensuring a comfortable experience for users.

 
Said Prof. Rajesh Kannan Megalingam, Director, Humanitarian Technology (HuT) Labs, Associate Professor, ECE Dept. Head, Amrita Center for Advanced Robotics, “For communication, we created a new language using eye signs, called Netravaani. It allows users to utilize all 26 English alphabets to create any word and sentence. It also supports the numbers 0 to 9, with which the users can create any number. Thought the device is currently set in English, Netravaad is also undergoing regional language trials in Malayalam and Hindi. More languages are planned to be added in the future.We intend to make a highly interactive user interface to increase the comfort level of patient as its future phase.”
 

Netravaad is portable and lightweight device that can be adjusted to any position for the patients’ comfort. It can be used while sitting in a chair, lying on the bed or even while travelling. The Netravaad Unit houses a display monitor, camera, speaker, a smash box and a rechargeable 24V battery. The complete unit is housed in a box and positioned on a swivel platform that can be adjusted to face in any direction as needed. The camera detects the users eye movements and converts them into words or sentences, which are displayed on the screen and verbalized through the speaker.

Prof. Kannan further added, “Netravaad supports two different modes, English 1 and English 2. In the first mode, patients can select the ten most used words for day-to-day activities, allowing for quick communication with caretakers or family members for any immediate need. The English 2 mode has word, sentence, and number sub-modes. In the word or sentence sub-mode, users can form any word or sentence, respectively, with eye signs. An advanced prediction algorithm also allows them to select pre-defined words or sentences. In the number sub-mode, the users can communicate numbers. The patient can choose any of the mode or sub-mode using the eye sign. The device is user-friendly. Patients are giving training on the device according to their health condition, after which they can easily communicate using Netravaad.”

The patients are trained before they can use Netravaad. The duration of the training varies depending on the patients health condition. In addition to training, all the instructions for using Netravaad are displayed on the screen and voice instructions are given using the speaker. After completion of the training, the patient can easily follow the instructions given by the device and communicate with the external world through the voice of Netravaad.
 

Eye sign language is a process to estimate and track the line of sight of a person, or simply, where a person is looking. It has mainly five categories of eye signs including left, right, top, close and center.