Glasses That Turn Speech Into Sign

The device shows sign language from spoken words right away, helping deaf people understand speech in places like school and work.

Two students from Strawberry Fields High School in Chandigarh — 14-year-old Advit Agrawal and 17-year-old Thakur Garg — have received a design patent from the Government of India for a wearable device that helps the deaf and hard of hearing communicate.

Their device, GestureX, is a pair of glasses that convert spoken words into sign language in real time. The patent, registered under Class 16-06, protects the design of the tool, which focuses on a gap in assistive communication technology.

– Advertisement –

The idea started when Advit, during his Indian Sign Language (ISL) classes, began thinking about ways to support people who use sign language. He had worked on robotic arms that turned text into signs. Thakur had worked on converting sign language into speech. They decided to work together on converting speech into visual signs and created GestureX.

GestureX uses a Raspberry Pi microprocessor, a microphone, and Python-based speech recognition software to convert spoken words into text. This text is matched with a sign language video from a digital library and shown in real time on an OLED screen inside the glasses. The current version supports American Sign Language (ASL), British Sign Language (BSL), and Indian Sign Language (ISL).

The design includes features like directional audio detection, text-to-sign mapping, and a frame that rests on the ear. Though still a prototype, the students are testing and adjusting its performance.

Advit and Thakur see GestureX as a tool that can support communication in places like schools, workplaces, and hospitals, where access to spoken language is needed. They plan to increase production by improving the design for cost and aligning it with the Make in India initiative. A separate technology patent is also in process.

They currently work independently but are open to mentorship and partnerships with groups working in assistive technology or inclusion. For them, the project is not just a technical task but an effort to build something that supports access through design and technology.