SignLab Amsterdam was founded in 2020, with the ambition to leverage recent advances in artificial intelligence, computer vision and computer graphics to add new computational and applied dimensions to the long tradition of theoretical sign language research at the University of Amsterdam. Our team includes both deaf and hearing researchers, with diverse backgrounds and types of expertise. Our research agenda is driven by fundamental scientific and technological challenges but also by urgent societal issues. We collaborate with several non-academic organisations to maximise the societal impact of our work.

---

Societal challenges

Globally, around 70 million people are deaf. Evidently, deaf people have no sensory access to spoken language. What is perhaps less evident is that written texts (e.g., captions, educational materials, healthcare information) are difficult to process for many deaf people as well. To understand this as a hearing person, imagine learning to read in a language with an unfamiliar alphabet (for instance, Thai, if you are used to the Roman alphabet) without being told how the characters in the alphabet are pronounced. Limited access to important public services, including education and healthcare, leads to unequal opportunities and social exclusion, as witnessed for instance by high rates of unemployment and depression among deaf adults.

Deaf children are especially vulnerable. Most deaf children (95%) have hearing parents, and only a small percentage of these parents (in the Netherlands, around 10%) manage to learn sign language. As a consequence, during the first years of their lives, many deaf children receive little language input that is accessible to them, which in turn often negatively affects their linguistic and cognitive development, as well as their mental well-being throughout the lifespan.

---

Scientific challenges

The language sciences have focused predominantly on spoken languages. We still know relatively little about the linguistic structure of sign languages. A complicating factor is that there is often a large amount of variation, both in terms of lexicon and in terms of grammar, even within a single sign language community. An important theoretical challenge is to characterise sign languages in a way that does justice to this variation while also capturing what is not subject to variation.

Another fundamental question concerns the relationship between conventionalised elements of sign languages and non-conventionalised, depictive elements which are commonplace in visual communication (especially alongside sign in purely visual communication but also alongside speech in multimodal communication), how these elements are integrated structurally and semantically.

---

Technological challenges

Sign language technologies are also highly underdeveloped. For instance, while tools like ChatGPT and Google Translate already work for more than 100 spoken languages, they don't work for any sign language yet. A major stumbling block, both for scientific and for technological progress, is that sign language data is lacking both in quantity and in quality. Sign languages do not have a written form. Most available data is therefore in the form of annotated videos, where annotations usually consist in a label for each sign occurring in the video and a translation of each utterance. Putting together such annotated video datasets is highly time-consuming. As a result, existing datasets are relatively small. For instance, the Corpus NGT (Nederlandse Gebarentaal, Sign Language of the Netherlands) currently contains almost 180.000 annotated signs, while the Corpus Spoken Dutch contains 9.000.000 words with extensive annotations, i.e., 50 times as many, and the latter is just one out of several large Dutch corpora. Moreover, the quality of these sign language datasets for scientific and technological purposes is low. Videos are 2D representations of a 3D reality. Furthermore, they often involve occlusion and motion blur. Annotations are often not consistent across datasets. Consequently, much information is lost in the process of measuring and representing the movement of signers. The field needs a much stronger foundation when it comes to data collection methodologies and data representation formats.

---