This research explores convex optimization for deformable registration of 3D templates to 2D image data from multiple cameras, with the generalized projection model.
Introduction
3D imaging of human body is an important component of modern medical practice. The most common modalities of 3D medical imaging are:
- Computed Tomography (CT). While producing very accurate 3D images, they subject patients to high dosage of ionizing radiation which increases likelihood of cancer
- Magnetic Resonance Imaging (MRI). The imaged 3D structures are accurate but the apparatus is expensive, lacks portability and inapplicable on patients with metallic implants and prosthesis.
Biplanar radiography. Offers an alluring alternative to CT and MRI. The ability to register a CT generated 3D template to biplanar radiographs would offer invaluable clinical insights. However, such registrations are challenging owing to the significant differences between a generic category-level template and the actual shape to be reconstructed. Interestingly, the biplanar radiography has many parallels with the setup of motion capture, which has received many advanced methods for fitting a generic template to some captured marker positions.
Generalization to multiple cameras. We generalize the biplanar radiography setup to that of a deforming object being imaged by multiple cameras (not necessarily two). The generalized camera model does not put any constraint on the sightlines (c.f. perspective, orthographic, weak-perspective, etc.), as shown below:
Method
We solve the problem of non-rigid registration as well as camera pose estimation with a convex relaxation approach. The overview is shown below:
