The relation between selfhood and intersubjectivity is currently debated in philosophical psychology. It has been stressed, that the mental self representation is significantly affected by others. EMOTIC aims to develop an experimental framework to investigate the plasticity of the self representation using individualized face avatars and real-time face tracking.
Background
The mental representation of one’s own face, i.e. self-face representation, is important in the constitution of personal identity. We recognize our face faster and pay more attention to it than to other familiar faces. Instead of a static self-face representation, it’s plasticity is widely accepted. This means, we continuously match and adjust the self-face representation according to our perception, for instance when we recognize ourself in a mirror. However, in case of congenital or acquired neurological or psychiatric disorders (e.g. dementia, shizophrenia), misrecognition occurs due to severely distorted self-representations.
Psychological and neuroscientific research actively investigates the plasticity of the self-face. The enfacement illusion is a widely-established methodological paradigm. It describes experimental conditions under which participants temporarily integrate other’s facial features, e.g. that of an avatar, into their own self-face representation. Conventionally, visuo-tactile stimuli are used in an artificial experimental setting.
Within EMOTIC, we are extending the conventional enfacement illusion for investigation of the plasticity of self-representation in inter-facial communication. We are developing an experimental framework for real-time manipulation of face avatars. In comparison to the conventional approach, our experimental framework increases ecological validity and opens-up new directions for behavioral research.
Creation of Realistic Avatars
We are establishing realistic individualized avatars in our 3D portrait studio “Camera Facialis”. Highly-resolved face scans are captured with minimal manual effort using our fully-automated reconstruction pipeline. We apply statistical shape analysis to a large database of face scans for the creation of morphable face models that capture inter- and intra-individual facial variability. Our Facial Expression Morphable Model (FexMM) allows the convenient creation of individualized avatars, e.g., for participants in enfacement experiments. Further, researchers can specifically manipulate a wide range of static and dynamic facial features of the created avatars through our FexMM.
In contrast to other avatars which were designed by artistic experts, our avatars improve the perceived photographic and behavioral realism. This is also important when dynamic instead of static facial stimuli are used to foster the ecological validity of psychological experiments.
Experimental Framework for Real-Time Avatar Manipulation
We are developing a framework for behavioral experiments based on real-time tracking of facial pose and expressions using video cameras or webcams. Our Open Virtual Mirror Framework (OMVF) can run on standard computer hardware with a latency fairly below 200ms, which is seen as a critical threshold to perceive actions as self-initiated. With our framework, we are able to increase the sense of agency over comparable virtual mirror frameworks in which avatars can only imitate the participant’s head pose. The OMVF offers a lightweight programming interface that allows convenient usage from existing software for psychological experiments such as PsychoPy. The integration of the FexMM facilitates creation of various experimental conditions, individualization of avatars for a participants, and manipulation of a large variety of facial features on-the-fly.
