Accurate measurements of 3D geometry and texture of the human face play an important role in medical applications, affective computing or behavioral sciences. Besides resolution and accuracy, important requirements for large-scale assessment are reliability and ease of use of scanning devices and analysis software. This project aims to develop a flexible and extensible photogrammetric setup, called “Camera Facialis”, combined with an advanced processing pipeline for the reconstruction of highly detailed facial performances in dense correspondence. In fully-automatic fashion, the pipeline produces semantically annotated 3D face models in dense correspondence to a reference face ready for advanced morphological statistics as well as generation of statistical shape models or blendshape rigs for intra- and interindividual facial appearance.

Single-Shot 3D Face Capture

The current setup used for high-resolution scanning of 3D faces in Camera Facialis

The basis of Camera Facialis is a flexible and versatile passive multi digital single-lens reflex (DSLR) setup. Currently, eight DSLRs and two studio flashes are triggered synchronously to capture images of the facial surface at 36MP resolution per device. The software downloads the images to a processing unit where they are fed into the reconstruction pipeline. This allows for large-scale data acquisition necessary for intra- and individual facial morphology analysis. Using our setup, we have successfully captured more than 2000 faces.

3D surface and texture reconstruction of a face as obtained from our processing pipeline without the need for manual editing.

Fully-Automatic Processing Pipeline

Automated 3D surface and texture reconstruction from calibrated stereo images.

To enable the large-scale reconstruction of 3D facial surfaces, we have developed a fully-automatic processing pipeline. Domain specific knowledge about appearance and geometry of the human face is integrated into our algorithms to render the process as reliable as possible. Using standardized stereo-photogrammetric procedures in combination with robust geometric priors, we are able to reconstruct 3D surfaces in dense correspondence to a reference face, from conventional calibrated stereo-images.

Transfer of dense correspondense to an individual surface. Our pipeline can transfer any number of landmarks from the reference.

Measurement of Facial Motion

The analysis and simulation of facial motion is of particular interest in perception research and facial surgery. Linear interpolation is commonly used to morph the facial surface between two static expressions but this neglects the complex and non-linear interaction of the facial tissue. We are enabling continuous time-dependent measurements by development of a markerless tracking system. Based on low-cost commodity hardware that is integrated into Camera Facialis, the goal is to estimate vector fields that capture facial motion densely estimated over the entire facial surface.

Samples from a motion sequence of an evolving expression captured with our setup.