Examples from ancient studies and psychology
For many years, the humanities and mathematics were considered as two sciences that have little to do with one another. During the last decade, however, researchers on both sides realized the potential lying in the respective other field. In this article, we present four examples illustrating this potential.
© mapsland.com
Modeling the Past:
Innovation Spreading in the Prehistoric World
How can we understand processes that happened in prehistoric times when there were no written records of events?
One possibility is to use available archaeological findings and build a good mathematical model that we can study. However, most of the existing archaeological data is sparse and uncertain; lots of information is unknown and there is no procedure to repeat the history and obtain new data. To deal with these problems, we work closely with researchers from the humanities, whose expertise enhances our studies and provides new links in our modeling approach.
Let us consider a more specific problem: How did innovations spread in the prehistoric world?
One of the impactful innovations that spread from the Near East to Europe between 6200 and 3000 BC, was the wool-bearing sheep. The change from herding hairy sheep to woolly sheep was an essential driving force for the later textile production. But the exact spreading path is unknown. Since the knowledge of an innovation and its usage was passed on from one person to another, we can expect the spreading path to be strongly connected to the human mobility dynamics. In order to study this, we built a mathematical model that allows us to simulate both human migration and innovation spreading. These simulations can be used to analyze dynamical properties of the two processes, explore how they are coupled, and discover the important factors that could have affected this innovation spreading.
Geographical area of our interest for wooly sheep spreading. a Suitability landscape constructed from environmental data; the assumed origin of the woolly sheep is around Tell Sabi Abyad (star). b Agents move in the suitability landscape and adopters (red) of the innovation can pass on the innovation to neighboring non-adopters (yellow) with a certain probability.
Through a fruitful collaboration with experts from the Cluster of Excellence TOPOI, we gained access to the archaeological and environmental data. We used this data to build a suitability landscape that indicates how suitable a particular region was for herding woolly sheep in the respective period (see figure 1a). Then, we constructed an agent-based model (ABM) in order to simulate possible spreading scenarios. An ABM consists of a set of rules describing the behavior of agents and their interaction patterns with each other and the environment. In our model, an agent represents a group of people in the ancient world. Agents move in the suitability landscape by following the so-called dynamical equation, according to which they move freely but with a bias toward regions that are attractive for herding woolly sheep. At the same time, agents can interact socially with other agents in their vicinity and pass on the innovation with a certain probability. By applying this abstract model to the example of the woolly sheep (figure 2), we can generate time series for the spreading process (see figure 3)(FN:N. Djurdjevac Conrad, D. Fürstenau, A. Grabundžija, L. Helfmann, M. Park, W. Schier, B. Schütt, C. Schütte, M. Weber, N. Wulkow, J. Zonker: Mathematical Modeling of the Spreading of Innovations in the Ancient World, Accepted to eTopoi, Journal for Ancient Studies, 7, 2018.). This opens up new research questions like: What are the best parameters of our model? How did the innovation spread between different geographical regions? With our research, we try to answer such questions and, thus, help understand processes in prehistoric times.
Snapshots of one realization of the wool-bearing sheep innovation spreading in the prehistoric world.
INFO Box
This research is done in collaboration with Brigitta Schütt (Freie Universität Berlin, FB Geowissenschaften and TOPOI, Berlin, Germany), Wolfram Schier (Freie Universität Berlin, FB Geschichts- und Kulturwissenschaften and TOPOI, Berlin, Germany), Daniel Fürstenau (Freie Universität Berlin, FB Wirtschaftswissenschaft and Einstein Center Digital Future, Berlin, Germany), and their coworkers. It has been partially funded by the Cluster of Excellence TOPOI – The Formation and Transformation of Space and Knowledge in Ancient Civilizations and ECMath (Einstein Center for Mathematics Berlin).
Accessing Hidden Texts in Egyptian Papyri
One of the best sources of information about our cultural origin is provided by written texts.
Top: Photograph of sealed legal texts. Bottom: Physically unfolded package. (ÄMP Berlin, SMB)
For example, excavations in Elephantine, a small island in the Nile in Egypt, brought to light large quantities of papyri, telling us 4,000 years of cultural history of various religious, ethnic, and linguistic groups that lived there (FN:V. M. Lepper [Hrsg.]: Forschung in der Papyrussammlung – Eine Festgabe für das Neue Museum; Ägyptische und Orientalische Papyri und Handschriften des Ägyptischen Museums und Papyrussammlung Berlin, Bd 1, Akademie-Verlag, 2012.) (FN:D. Baum, N. Lindow, H.-C. Hege, V. Lepper, T. Siopi, F. Kutz, K. Mahlow, H.-E. Mahnke: Revealing Hidden Text in Rolled and Folded Papyri. Appl. Phys. A, 123, pp. 171–177, March 2017.). The papyri are typically rolled or folded (see figure 4). However, for papyri that are too fragile to be physically unfolded or unrolled, the writings are inaccessible.
Our aim is to make such papyri readable. For this, we acquire a 3-D tomographic image of it, reconstruct the papyrus geometrically on the computer, virtually unfold and unroll it, and finally visualize the writing on it (FN:D. Baum, N. Lindow, H.-C. Hege, V. Lepper, T. Siopi, F. Kutz, K. Mahlow, H.-E. Mahnke: Revealing Hidden Text in Rolled and Folded Papyri. Applied Physics A, 123, pp. 171–177, March 2017.). A prerequisite is that the tomographic method depicts the writing with a sufficiently high contrast. For documents written with ferrous ink, this is the case when using X-ray-based tomography.
Volume rendering of CT scan of rolled mock-up papyrus. Virtually unfolded mock-up from CT scan seen on the left.
For virtual unrolling, we depict cross sections in high detail, allowing the user to define manually closed polylines that approximate the contours (figure 6, left); using interpolation and extrapolation, similar contours are computed in other planes (figure 6, right). Ensuring that the number of points in each polyline is equal, a quadrangular 2-D mesh is implicitly defined, representing a surface; this surface is flattened such that differences in the distan ces between neighbored contour points are minimized (figure 5).
Virtual unrolling of a papyrus roll. Right: Red contours are interactively set; yellow and blue contours are computed by inter- and extrapolation. Left: Depiction of a cross section.
For virtual unfolding, each step of the physical folding process is virtually undone.
After a series of steps, the unfolded papyrus becomes topologically equivalent to a role, which is then unrolled, as described before. In each step, we provide the user with an overview of the current state using volume rendering (see figure 7, left); furthermore, we display cross sections in high detail in which the user can interactively draw polylines. For the actual unfolding, Moving Least Squares deformation is applied based on these polylines to unfold the package as rigidly as possible. After all folds have virtually been undone, we unroll the roll as described previously. In a final step, an equalization is performed that largely eliminates all distortions that accumulated during the whole procedure. Then the writing is visualized using contrast-enhancing techniques (figure 7, right).
Left: Computer-tomographic (CT) scan of folded mock-up papyrus. Right: Virtually unfolded papyrus from CT scan seen on the left.
INFO Box
This project was carried out in collaboration with the archaeologist Verena M. Lepper (Ägyptisches Museum und Papyrussammlung (ÄMP) – Staatliche Museen zu Berlin (SMB) – Stiftung Preußischer Kulturbesitz), the physicists Heinz-Eberhard Mahnke (Freie Universität Berlin, FB Physik and TOPOI, Berlin, Germany) and Ingo Manke (Helmholtz-Zentrum Berlin), and their coworkers. It was financially supported by the Starting Grant ELEPHANTINE of the European Research Council (ERC), the Beauftragte der Bundesregierung für Kultur und Medien (BKM), the Cluster of Excellence TOPOI (The Formation and Transformation of Space and Knowledge in Ancient Civilizations) of the Deutsche Forschungsgemeinschaft (DFG).
3-D Mesh Morphing for Psychological Experiments
Understanding how object recognition and categorization is performed by the human brain is a topic that has been studied in psychology for many years. As a result, it is widely believed that for this task our brain builds up a mental representation of objects. Studies have shown that such mental representations are not fixed but remain flexible and are subject to continuous short- and long-term adaptation. The majority of such studies have so far concentrated on human faces, the recognition of which plays a very essential role in our daily life. Hence, it is not clear how these findings generalize to the recognition and categorization of other objects.
Overlay of images representing a global morphing from a lobster to a crab.
Therefore, in an interdisciplinary project, perceptual psychologists and biologists in the Cluster of Excellence “Image Knowledge Gestaltung” have been investigating object categorization on the biological categories of crabs and lobsters. In particular, they are investigating how a previously seen input stimulus influences the categorization decision of a test person. This effect is known as adaptation effect and has been shown to exist for the recognition of human faces (FN:C.M. Grewe, G. Le Roux, S.-K. Pilz, S. Zachow: The Various Facets of Facial Expressions. Proceedings of the IEEE Conference on Automatic Face and Gesture [accepted for publication], 2018.). For their study, they produced images in which categorization-relevant features of crabs and lobsters were systematically morphed using standard 2-D image-processing software. This resulted in intermediate images between a crab and a lobster that were then used to study the adaptation effect (FN:A. Reindl, T. Strobach, C. Becker, G. Scholtz, T. Schubert: Crab or Lobster? Mental Principles underlying the Categorization of crustaceans by biology experts and non-experts. Zoologischer Anzeiger – A Journal of Comparative Zoology, 256, pp.28–35, 2015.). Unfortunately, these images showed a rather poor resemblance with real crabs and lobsters, which may bias the experimental results.
(8)Global morphing from a lobster (far left) to a crab (far right).
(9) Even though only the carapace is modified, the images show a smooth transition of the whole object without any noticeable artifacts at the joints between the carapace and other parts of the body. Local morphing of the carapace from a lobster (far left) to the carapace of a crab (far right) while keeping all other parts fixed as in the lobster.
To create more realistic-looking images, we utilize 3-D shape morphing using state-of-the-art methods from differential geometry. This allows us to easily create natural-looking 3-D shapes that are in between a crab and a lobster (see figure 8). Controlled psychological experiments, however, require local mesh morphing that transforms one part (e.g. the carapace) of the animal at a time. To achieve this goal, we developed an approach that combines two previously proposed methods for mesh morphing to obtain a natural-looking transformation of selected parts of the mesh, while keeping the rest of the mesh as stable as possible (see figure 9). From the interpolated representations, standardized images were created that much better resemble real crabs and lobsters even though artificial. Currently, new psychological experiments are being carried out using these new images, thus greatly improving the experimental conditions.
Facial Morphology and its Applications
Our face has an immense significance for interpersonal communication. Sympathy and antipathy for others often depend on our first impression.
Already within milliseconds, stereotypes may have been triggered by an opponent’s face. The perception of facial expressions determines our interpretation of statements (see figure 10). Similarly, we always use facial expressions – consciously or unconsciously – to emphasize what we say. Therefore, the face and its perception have been the subject of art and science for centuries and are still intensively researched.
We have probably all encountered the experience of drawing conclusions from interpreting other people’s facial expressions. Disputes up to serious conflicts may have arisen from a (mis)interpretation of facial expressions. Hence, facial expressions and their deliberate use are intensively trained, not only in acting but also for successful negotiations (e.g. in business and politics). Image source: https://twitter.com/realdonaldtrump
People with congenital or acquired facial malformations, or palsy due to nerve damage, are often stigmatized. An aim of facial plastic surgery is to correct such dysplasia in view of a normal or natural facial anatomy. To plan and perform surgical modifications of faces in such a way that both desired aesthetic visual appearance and expressions are considered is still a challenge.
A small percentage of the population suffers from an autism-spectrum disorder. Those people often have difficulties in interpreting facial expressions. It is difficult for them to imagine what other people think or feel, which causes impairments of social interaction. The training of expression interpretation and production therefore supports their social skills. Another small group of the population has difficulties in recognizing familiar faces.
Expressions are formed by the actions
of facial muscles. In our research,
we are studying the resulting effects
on the facial surface.
They suffer from so-called face blindness (prosopagnosia), which is a neurological disorder of the brain that needs to be identified and thoroughly tested for any treatment.
Together with our collaborators from anthropology, we have studied regular facial development in male and female children. Nonlinear growth trajectories were determined for the use in the early diagnosis of syndromes. Similarly, typical aging effects like facial sagging have been investigated morphometric-ally. For the analysis and synthesis of facial expressions (computational facial morphology), we have established a 3-D morphable model that is able to synthesize various individual nuances beyond typical expression categories [5]. Their significance for expression perception is being investigated in a broad user study as part of the Mimik-Explorer shown at the German Hygiene Museum in Dresden.
In collaboration with psychologists, we are developing novel experiments for the investigation of self-representation based on virtual-reality techniques. Our goal is to gain understanding of the role of facial identity in (virtual) communication. For the planning of facial surgery, our work allows the assessment of expressions before and after treatments (FN:S. Zachow: Computational Planning in Facial Surgery. Journal of Facial Plastic Surgery, 31[5], pp. 446-462, 2015.). For example, in facial palsy, a detailed planning of facial nerve reanimation along with an objective follow-up evaluation method can be established.
INFO Box
With applications in anthropology, medicine, psychology, and affective computing in mind, we target questions like: (a) How do facial proportions vary in a normal or healthy range, (b) what kinds of growth and aging trajectories can be observed during childhood or in elderly people, (c) are there characteristic patterns for facial expressions giving rise to an objective classification scheme, and (d) is it possible to employ such a scheme to artificially synthesize plausible expressions? These are all driving questions in research that motivate an in-depth analysis of the almost endless diversity of real faces.
Digital Facial Morphology
Expression scans from the ZIB 3-D Face Database with photographic texture containing high-resolution skin details.
Computational Analysis of Facial Morphology and Expressions
Computational facial morphology offers the opportunity to apply mathematical methods for the analysis of facial shape and proportions as well as skin appearance and expression. Based on image and geometry processing, statistics, and machine learning, the relation of facial morphology to attributes like sex, age, culture, and so forth, can be studied. Similarly, characteristic patterns describing the change in shape and appearance due to expressions can be precisely extracted and quantified from a representative set of facial performances. Our research comprises (1) the development of new techniques for the acquisition of high-resolution 3-D faces, (2) their integration into a large-scale database, and, using this, (3) establishing a foundation of morphological analysis as well as applications in psychology and medicine.
Camera Facialis – 3-D Facial Performance Capture
Human face perception is particularly sensitive to the subtlest details. For the acquisition of high-resolution facial details including small wrinkles resulting from micro expressions, the 3-D portrait studio Camera Facialis (see figure 12) has been established at ZIB (FN:C. M. Grewe, L. Schreiber, S. Zachow: Fast and Accurate Digital Morphometry of Facial Expressions. Journal of Facial Plastic Surgery, 31[05], pp. 431–438, 2015). In contrast to expensive and laborious commercial setups common in entertainment productions and media art, Camera Facialis has been built on top of affordable optical devices and commodity hardware to enable ultrafast, high-fidelity measurements for a very detailed capture of facial performances in a large-scale manner (see figure 14).
The 3-D portrait studio Camera Facialis
uses multiview photogrammetry to
capture high-resolution 3-D facial
performances.
Nuances of facial expressions synthesized using expression patterns determined by statistical shape analysis.
ZIB 3-D Face Database
Photometric reconstruction of 3-D faces is usually a tedious manual process even for expert users. To establish a comprehensive large-scale 3-D face database, we have developed a fully automated reconstruction pipeline for high-throughput data acquisition. This has been achieved by consequent integration of statistical shape templates. Detailed 3-D facial scans result from specifically optimized algorithms (FN:C. M. Grewe, S. Zachow: Fully Automated and Highly Accurate Dense Correspondence for Facial Surfaces. Computer Vision – ECCV Workshops, LNCS Vol. 9914, Springer, pp. 552–568, 2016.). That way, several thousand faces have already been integrated into the 3-D face database, which is continuously extended for digital facial morphology.
Digital 3-D Facial Morphology
The statistical analysis of shape and appearance requires a well-defined mathematical framework. Descriptors are needed to transform a face into a typically high-dimensional space, where similarities or distances, correlations, or clusters can be analyzed. At ZIB, we are developing correspondence-based descriptors that take the Riemannian structure of shape spaces into account (FN:C. von Tycowicz, F. Ambellan, A. Mukhopadhyay, S. Zachow: An Efficient Riemannian Statistical Shape Model using Differential Coordinates. Journal of Medical Image Analysis, 43, pp. 1–9, 2018.). We enable the analysis of facial features up to the level of wrinkles and skin pores by establishing accurate dense correspondence with our techniques (see figure 15) [5].
In the Camera Facialis, 3-D face scans in dense correspondence to an animatable surface mesh are acquired. On top of a neutral face scan (left), arbitrary expression patterns extracted from the ZIB face database can be transferred, for instance, the smile expression shown on the right.
