Text-to-sign translation using animated avatars

We are developing tools to translate text into sign language using animated avatars. Such tools have several potential applications. They could support communication between deaf and hearing people in situations where human sign language interpreters cannot be employed, they could give sign language users easier access to written and spoken information, and they could help people in learning sign language.

Cautionary note

We should emphasise that, to support face-to-face interaction between deaf and hearing people, a qualified human sign language interpreter should, whenever available, always be preferred over a machine translation system. Still, it is worth investigating the extent to which a machine translation system making use of avatar technology can be of help in situations in which a human interpreter cannot be employed or for purposes other than supporting face-to-face interaction, such as providing access to information or providing support in learning sign language.

A case study: text-to-sign translation for COVID-19 healthcare

It is challenging for healthcare professionals to communicate with Deaf patients (Felllinger, Holzinger and Pollard, 2012), even more so in times of COVID-19 (Mckee, Moran and Zazove, 2020). Sign language interpreters cannot always enter hospitals and clinics, facemasks make lipreading impossible (Grote and Izagaren,2020) and interpreting via videorelay is not always viable. Building on the JASigning avatar engine, we are developing a tool which translates sentences that are frequently used in the diagnosis and treatment of COVID-19 from Dutch into Sign Language of the Netherlans (NGT). Click on the button below to see a demo of this tool.

Additional resources for healthcare professionals

Useful advice for health care professionals about communicating with deaf patients can be found on the Wat Telt! website. This website also contains a video teaching you fifteen signs that could help you in communicating with deaf patients. Would you like to learn more? Sign up for a two-day course on providing assistance to deaf and hard of hearing patients at RINO. This course is taught by Dr. Anika Smeijers (Amsterdam Medical Centre), Prof. Onno Crasborn (Radboud University Nijmegen) and Roos Wattel (Wat Telt!).

A screenshot of the ATLAS interview segment on SignLab Amsterdam.

©2020 SignLab Amsterdam
: signlab.amsterdam@gmail.com