Text-to-sign translation using animated avatars
We are developing tools to translate text into sign language using animated avatars. Such tools have several potential applications. They could support communication between deaf and hearing people in situations where human sign language interpreters cannot be employed, they could give sign language users easier access to written and spoken information, and they could help people in learning sign language.
We should emphasise that, to support face-to-face interaction between deaf and hearing people, a qualified human sign language interpreter should, whenever available, always be preferred over a machine translation system. Still, it is worth investigating the extent to which a machine translation system making use of avatar technology can be of help in situations in which a human interpreter cannot be employed or for purposes other than supporting face-to-face interaction, such as providing access to information or providing support in learning sign language.
A case study: a sign language avatar for railway announcements
Public transport organisations, such as Dutch railway operator Nederlandse Spoorwegen (NS), generally communicate important messages through spoken announcements in the train or station. As a result, Deaf people miss crucial information. Because of varying levels of reading proficiency, textual information is not always a solution. Moreover, video footage is not scalable and will not yield satisfactory results when individual video parts need an update. Virtual sign language avatars can provide consistent, anonymous and scalable translations. For the development of a comprehensible and fluent sign language avatar, collaboration between experts in language technology, avatar technology and sign language has been fundamental. Click on the button below or on the image of the avatar to see a demo of this tool.
A case study: text-to-sign translation for COVID-19 healthcare
It is challenging for healthcare professionals to communicate with Deaf patients (Felllinger, Holzinger and Pollard, 2012), even more so in times of COVID-19 (Mckee, Moran and Zazove, 2020). Sign language interpreters cannot always enter hospitals and clinics, facemasks make lipreading impossible (Grote and Izagaren,2020) and interpreting via videorelay is not always viable. Building on the JASigning avatar engine, we have developed a tool which translates sentences that are frequently used in the diagnosis and treatment of COVID-19 from Dutch into Sign Language of the Netherlands (NGT). Click on the button below or on the image of the avatar to see a demo of this tool.
