Accurate measurements of 3D geometry and texture of the human face play an important role in medical applications, affective computing or behavioral sciences. Besides resolution and accuracy, important requirements for large-scale assessment are reliability and ease of use of scanning devices and analysis software. This project aims to develop a flexible and extensible photogrammetric setup, called “Camera Facialis”, combined with an advanced processing pipeline for the reconstruction of highly detailed facial performances in dense correspondence. In fully-automatic fashion, the pipeline produces semantically annotated 3D face models ready for advanced morphological statistics as well as generation of statistical shape models.
Single-Shot 3D Face Capture
The current setup used for high-resolution scanning of 3D faces in Camera Facialis
The basis of Camera Facialis is a flexible and versatile passive multi digital single-lens reflex (DSLR) camera setup. Currently, eight DSLRs and two studio flashes are triggered synchronously to capture images of the facial surface at 36MP resolution per device. The software downloads the images to a processing unit where they are fed into the reconstruction pipeline. This allows for large-scale data acquisition necessary for intra- and individual facial morphology analysis. Using our setup, we have successfully captured more than 2000 faces.
3D surface and texture reconstruction of a face as obtained from our processing pipeline without the need for manual editing.
Fully-Automatic Processing Pipeline
To enable the large-scale reconstruction of 3D facial surfaces, we have developed a fully-automatic processing pipeline. Domain specific knowledge about appearance and geometry of the human face is integrated into our algorithms to guarantee reliable data processing. Using standardized stereo-photogrammetric procedures in combination with robust geometric priors, we are able to reconstruct 3D surfaces in dense correspondence from conventional calibrated images.
Automated 3D surface and texture reconstruction from calibrated stereo images.
Measurement of Facial Motion Trajectories
The analysis and simulation of facial motion is of particular interest in perception research and facial surgery. Linear interpolation is commonly used to morph the facial surface between two static expressions but this neglects the complex and non-linear interaction of the facial tissue. We are enabling continuous measurements by development of a markerless tracking system for facial motion capture. Based on synchronized video devices integrated into Camera Facialis, the goal is to estimate trajectories that describe facial motion in combination with the highly detailed 3D surface scans.
Reconstructed 3D motion trajetories from a stereo image sequence captured in sync with the high-resolution DSLR camera images.
