Visual communication. Theoretical, empirical, and applied perspectives (ViCom)

DFG Priority Programme (start 2022)

Prof. Dr. Cornelia Ebert
Johann Wolfgang von Goethe-Universität Frankfurt am Main (coordinator)

Prof. Dr. Markus Steinbach
Georg-August-Universität Göttingen


Description

ViCom investigates the special features and linguistic significance of visual communication. This comprises sign languages as fully developed natural languages which exclusively rely on the visual channel for communication, but also visual means that enhance spoken language such as gestures. It aims at disclosing the specific characteristics of the visual modality as a communication channel and its interaction with other channels (especially the auditory channel) to develop a comprehensive theoretical linguistic model of human communication and its cognitive foundations.

ViCom focuses on the investigation of sign languages and gestures as prime examples of visual human communication. Additionally, the research agenda will comprise the investigation of gestures in didactic and therapeutic contexts, gestural aspects of vocal communication and in the written modality, pictorial narratives as well as visual communication strategies of non-linguistic species and in multimodal human-computer interaction systems. In theoretical linguistic research, both sign language and gestures, which have been recognised as fruitful research topic only very recently, are currently analysed by application of the linguistic vocabulary established for spoken languages. Only little attention has been paid to the modelling of the specific properties of the visual transmission channels. And it is now becoming evident that the formal linguistic repertoire needs to be extended to meet the specific requirements of visual communicative means.

However, although gesture research is a new and emerging topic in theoretical linguistics, it has been a central topic for a long time in other disciplines, such as cognitive linguistics, semiotics, psychology, neurosciences, animal communication, cultural studies, or conversation analysis. Likewise, cognitive oriented analyses of sign languages have highlighted the modality-specific (visual) properties of sign languages for a long time. From an applied perspective, the need for natural multimodal human-computer interfaces has been recognised for some time already. And finally, gestures are important for didactic and therapeutic uses due to their well-established role in language acquisition and learning.

So, while there already exists a substantial body of empirical data and a considerable amount of insights on the form and functions of visual communication, this knowledge is distributed across disciplines and has not been collected and unified in a comprehensive way yet. ViCom as a highly interdisciplinary priority programme brings together the above-mentioned different disciplines and research traditions under a theoretical, empirical and applied perspective and will use these new insights to develop a new comprehensive theoretical model of language and communication that is able to account for all modalities. It will also help to advance linguistic methodology and technological, therapeutic, and didactic environments.