AI Deaf Translator: A Web Chat Bridge for Thai Sign Language

Communication between deaf and hearing individuals in Thailand can be difficult, especially when real-time interpretation services are unavailable. Thai Sign Language has its own grammar, structure, and variation, making automated translation a complex technical and social challenge.

AI Deaf Translator was developed by AiCE sophomore students Sippapas Pronanunt, Saritwatt Khanthakamolmart, Jirapat Kulruchakorn, and Jirat Kositchaiwat, under the guidance of Dr. Sarun Gulyanon. The project builds a real-time web chat platform that supports communication between Thai text and Thai Sign Language.

The team designed the system around two translation directions. In text-to-sign mode, a hearing user types Thai text, and the system converts the sentence into Thai Sign Language word order before displaying a sequence of sign video clips. In sign-to-text mode, a deaf user performs signs through a webcam, and the system recognizes the gesture and converts it into Thai text.

The technical pipeline uses MediaPipe Holistic to extract body and hand landmarks from video input. These landmark sequences are then processed by machine learning models, including a Temporal Convolutional Network and an LSTM-based metric learning model.
For text-to-sign conversion, the team used large language model prompting to restructure Thai sentences into Thai Sign Language grammar before mapping words to sign video clips.

The web application includes real-time chat functionality through Socket.IO, message persistence through a database, and a separated inference architecture using FastAPI for machine learning workloads. This separation improves system stability and keeps heavy model processing apart from the main chat service.

The project achieved strong validation results on a focused daily-word vocabulary, while also identifying important limitations. Real-world webcam performance is more difficult than validation testing because users sign at different speeds, angles, lighting conditions, and distances from the camera. Thai Sign Language datasets are also limited, making generalization a continuing challenge.

AI Deaf Translator is a powerful example of assistive technology built in a Thai context. It shows how CMKL students can work with low-resource language challenges, multimodal AI, real-time systems, and inclusive design. The project also highlights the importance of involving deaf communities in future vocabulary expansion, usability testing, and system refinement.

Project Members: Sippapas Pronanunt, Saritwatt Khanthakamolmart, Jirapat Kulruchakorn, Jirat Kositchaiwat

Advisor: Dr. Sarun Gulyanon and Dr. Sovaritthon Chansaengsee
Industry Advisor: Ratchasuda College, Mahidol University

Domain: Assistive Technology, Thai Sign Language, Computer Vision, Real-Time Web Applications, Multimodal AI