The relation between selfhood and intersubjectivity is currently debated in philosophical psychology. It has been stressed, that the mental self representation is significantly affected by others. EMOTIC aims to develop an experimental framework to investigate the plasticity of the self representation using individualized face avatars and real-time face tracking.

 

 

 

Background

The mental representation of one’s own face, i.e. self-face representation, is important in the constitution of personal identity. We recognize our face faster and pay more attention to it than to other familiar faces. Instead of a static self-face representation, it’s plasticity is widely accepted. This means, we continuously match and adjust the self-face representation according to our perception, for instance when we recognize ourself in a mirror. However, in case of congenital or acquired neurological or psychiatric disorders (e.g. dementia, shizophrenia), misrecognition occurs due to severely distorted self-representations.

Neuroscientific research actively investigates the plasticity of the self-face. The enfacement illusion is a widely-established methodological paradigm. It describes experimental conditions under which participants temporarily integrate other’s facial features, e.g. that of an avatar, into their own self-face representation. Conventionally, visuo-tactile stimuli are used in an artificial experimental setting.

Within EMOTIC, we are extending the conventional enfacement illusion for investigation of the plasticity of self-representation in inter-facial communication. We are developing an experimental framework for real-time manipulation of face avatars. In comparison to the conventional approach, our experimental framework increases ecological validity and opens-up new directions for behavioral research.

 

Real-Time Manipulation of Face Avatars

For our experiments, we are establishing realistic individualized avatars in our 3D portrait studio “Camera Facialis”. We are generating highly-resolved avatars with minimal manual effort using our fully-automatic reconstruction and face matching pipeline. Each avatar is integrated in our statistical face model, which contains morphological patterns related to various inter- and intra-individual factors including sex, ethnicity, and expression dynamics.

3D Face Avatar

 

We are developing a framework for behavioral experiments based on real-time tracking of facial pose and motion from standard video cameras. The tracked dynamics are transferred to the avatar which is shown to the participants. During transfer, not only the conventional conditions of the enfacement illusion can be altered (synchronity vs. asynchronity). The semantic morphological patterns of the avatar and our statistical face model allow to specifically manipulate various factors related to the perception of faces.

Face Tracking