The relation between selfhood and intersubjectivity is currently debated in philosophical psychology. It has been stressed, that the mental self representation is significantly affected by others. EMOTIC aims to develop an experimental framework to investigate the plasticity of the self representation using individualized face avatars and real-time face tracking.





The mental representation of one’s own face, i.e. self-face representation, is important in the constitution of personal identity. We recognize our face faster and pay more attention to it than to other familiar faces. Instead of a static self-face representation, it’s plasticity is widely accepted. This means, we continuously match and adjust the self-face representation according to our perception, for instance when we recognize ourself in a mirror. However, in case of congenital or acquired neurological or psychiatric disorders (e.g. dementia, shizophrenia), misrecognition occurs due to severely distorted self-representations.

Psychological and neuroscientific research actively investigates the plasticity of the self-face. The enfacement illusion is a widely-established methodological paradigm. It describes experimental conditions under which participants temporarily integrate other’s facial features, e.g. that of an avatar, into their own self-face representation. Conventionally, visuo-tactile stimuli are used in an artificial experimental setting.

Within EMOTIC, we are extending the conventional enfacement illusion for investigation of the plasticity of self-representation in inter-facial communication. We are developing an experimental framework for real-time manipulation of face avatars. In comparison to the conventional approach, our experimental framework increases ecological validity and opens-up new directions for behavioral research.

Creation of Realistic Avatars

We are establishing realistic individualized avatars in our 3D portrait studio “Camera Facialis”. Highly-resolved face scans are captured with minimal manual effort using our fully-automated reconstruction pipeline. We apply statistical shape analysis to a large database of face scans for the creation of morphable face models that capture inter- and intra-individual facial variability. Our Facial Expression Morphable Model (FexMM) allows the convenient creation of individualized avatars, e.g., for participants in enfacement experiments. Further, researchers can specifically manipulate a wide range of static and dynamic facial features of the created avatars through our FexMM.

In contrast to other avatars which were designed by artistic experts, our avatars improve the perceived photographic and behavioral realism. This is also important when dynamic instead of static facial stimuli are used to foster the ecological validity of psychological experiments.

Experimental Framework for Real-Time Avatar Manipulation

We are developing a framework for behavioral experiments based on real-time tracking of facial pose and expressions using video cameras or webcams. Our Open Virtual Mirror Framework (OMVF) can run on standard computer hardware with a latency fairly below 200ms, which is seen as a critical threshold to perceive actions as self-initiated. With our framework, we are able to increase the sense of agency over comparable virtual mirror frameworks in which avatars can only imitate the participant’s head pose. The OMVF offers a lightweight programming interface that allows convenient usage from existing software for psychological experiments such as PsychoPy.  The integration of the FexMM facilitates creation of various experimental conditions, individualization of avatars for a participants, and manipulation of a large variety of facial features on-the-fly.






An Extended 3D Morphable Face Model with Applications in Experimental Psychology Doctoral thesis, Technische Universität Berlin, Stefan Zachow (Advisor), 2023 Martin Grewe BibTeX
Enfacement Manipulation in Transmitted Inter-Facial Communication
The Open Virtual Mirror Framework for Enfacement Illusions - Enhancing the Sense of Agency With Avatars That Imitate Facial Expressions Behavior Research Methods, 2022 Carl Martin Grewe, Tuo Liu, Andrea Hildebrandt, Stefan Zachow BibTeX
Enfacement Manipulation in Transmitted Inter-Facial Communication
Facial Landmark Detection on 3D Surface Scans using Geometric Deep Learning Master's thesis, Freie Universität Berlin, Martin Grewe, Tamaz Amiranashvili, Stefan Zachow (Advisors), 2021 Sofi Tiwari BibTeX
Enfacement Manipulation in Transmitted Inter-Facial Communication
Orientation-invariant Dense Correspondence using Graph Convolutional Neural Networks Master's thesis, Universität Potsdam, Martin Grewe, Tamaz Amiranashvili, Zachow Stefan (Advisors), 2021 Dev Punjabi BibTeX
Enfacement Manipulation in Transmitted Inter-Facial Communication
Statistical Learning of Facial Expressions Improves Realism of Animated Avatar Faces Frontiers in Virtual Reality, Vol.2, pp. 1-13, 2021 Carl Martin Grewe, Tuo Liu, Christoph Kahl, Hildebrandt Andrea, Stefan Zachow BibTeX
Enfacement Manipulation in Transmitted Inter-Facial Communication
Symmetric or not? A holistic approach to the measurement of fluctuating asymmetry from facial photographs Personality and Individual Differences, Vol.166, pp. 1-12, 2020 Selina Weiss, Carl Martin Grewe, Sally Olderbak, Benjamin Goecke, Laura Kaltwasser, Andrea Hildebrandt BibTeX
Enfacement Manipulation in Transmitted Inter-Facial Communication