2D X-ray images play a crucial role for the diagnosis and the therapy planning in orthopaedics. This imaging technique is not only widely available but is also, in contrast to more advanced 3D imaging methods like CT or MRI, considered a fast and inexpensive procedure. However, musculoskeletal anatomy is inherently 3D, and accurate assessment of the patients anatomy is therefore largely dependent upon the individual surgeon’s experience and ability to assess the 3D anatomy by analysis of 2D images only. To overcome this limitation, and in order to enable reliable, quantitative planning of surgical interventions that is less dependent upon the experience of the surgeon, the goal of this project is to provide a method aimed at 3D reconstruction of musculoskeletal anatomy from standard 2D projection X-ray images.

Introduction

Despite the increasing availability of 3D image acquisition methods like computed tomography (CT), conventional  X-ray images are still the method of choice for diagnosis and treatment planning in orthopedics. X-rays however only depict a 2D projection of the anatomy of interest, which renders assessing its 3D parameters such as the anatomical shape a challenging task for a human observer (eg. the surgeon). The aim of this project is to develop computer-aided techniques that reconstruct 3D models from a single or a few X-rays, describing both the patient-specific shape and bone density distribution of an anatomy of interest. As an ultimate goal, such patient-specific models will allow for precise joint replacement planning, follow-up and biomechanical analysis, providing essential 3D information solely based on 2D X-rays.

3D-Reconstruction Methods

In previous work (Lamecker et al. 2006, Dworzak et al. 2008, Dworzak et al. 2010, see also THORAX project), we search for a 3D model representation of the anatomy of interest that best depicts the patient-specific 2D shape in the reference X-ray(s). Many variations of the 3D model's shape are projected according to the known X-ray setup, i.e. given a relative position of the X-ray (point) source to the detector. The shape that maximizes the similarity between the model's projection and the X-ray image(s) is assumed to be the best approximation of the true 3D anatomy. Previously, we evaluated the similarity by comparing the projected contours of the model and silhouettes from the reference X-ray. Although contours or silhouettes of 3D anatomical models can be computed very efficiently, they first have to be extracted from the individual patient's X-ray images in a manual or automatic manner. Even more problematic is to use contour information only, as this ignores important information on the interior structures (Fig. 1).

The methods developed in this project aim at overcoming the limitations of previous approaches by regarding as much information contained in the reference X-ray images as possible. Instead of comparing silhouettes or contours, we project virtual X-ray images from the 3D models that simulate a X-ray screening according to the clinical setup. The virtual X-ray images are compared directly to the reference X-rays by means of their intensity distribution, taking the bone-interior structures into account. This way, we increase the robustness of the reconstruction process while determining not only the patient-specific anatomical shape but also its bone-interior density information.

Fig. 1. A 3D model of the proximal femur (upper thigh bone) that incorporates volumetric density information (a). Compared to a projection of contour lines (b), a virtual X-ray (c) depicts the internal density structure of the bone and therefore better resembles the femur in the clinical X-ray (d).

Fast Generation of Virtual X-ray Images

The reconstruction framework has to evaluate many anatomical shape candidates in order to improve the reconstruction quality by identifying the best among several locally optimal solutions. It consequently has to produce large quantities of virtual X-rays (e.g. 10 000 or more images). An essential goal therefore is to quickly compute both variations of anatomical models and their corresponding projections. In addition, the virtual X-rays should depict the density information of the anatomy of interest as accurately as possible to allow for intensity-based comparison with the clinical X-rays.

We developed a hardware-accelerated method that achieves these goals (Ehlke et al. 2013). Compared to previous work, its main contributions are:

  • A GPU-algorithm for fast rendering of tetrahedral meshes with higher-order polynomial density functions that represent the shape and density distribution of anatomical structures. Our approach simplifies the tessellation of tetrahedra and can be implemented on the GPU without any explicit branching or looping.
  • A method that varies (deforms) both anatomical shape and density information on the GPU and concurrently generates virtual Xray images from the deformed anatomy model.

We evaluated the GPU-algorithm and compared to previous hybrid methods, where the model is deformed on the CPU and projected using the GPU. When regarding more than 40 deformation parameters and higher polynomial degrees of the density function (d = 2 and d = 3), our GPU approach is 6 to 7 times faster than a multi-threaded deformation on the CPU that does not account for the projection time. A comparison of the GPU approach to a hybrid CPU/GPU implementation reveals a performance gain of even 8 to 9 times in the same scenario. We further performed a quality evaluation indicates that our method generates virtual X-rays similar to ground truth images from CT (see Fig. 2).

Fig. 2. Comparison of virtual X-ray images to ground truth data. The projections (a) and (b) show close-ups of the right ilium (pelvis) and were generated by applying our method to a tetrahedral mesh of 252k tetrahedra and density function degrees d = 0 and d = 3 respectively. A ground truth projection from CT (c) and a clinical X-ray image (d) of the same pelvis are given for reference. Images (e) and (f) depict the differences of the virtual X-rays (a) and (b) to the ground truth (c) with red indicating positive error, blue negative error.

Anatomical Modelling for 3D-Reconstruction

The reconstruction is performed by searching among a large set of candidates (variations) for the shape that best represents the patient-specific anatomy. It is crucial that anatomically “meaningful” shape candidates are selected, since an anatomically impossible shape might have a similar 2D appearance than the real, patient-specific anatomy and the similarity evaluation in 2D space might thus lead to wrong conclusions. Finding the “right” shape from one or few X-ray images is an ill-posed problem. We therefore introduce a-priori statistical information about the anatomy of interest that acts as a prior during reconstruction, restricting the search space to anatomically meaningful representations.

Previously, statistical shape models (SSMs) proved to be an adequate prior in medical registration problems such as 3D segmentation of CT- and MRI data (Seim et al., 2008, see also Atlas-based 3D Image Segmentation). SSMs utilize the Principal Component Analysis (PCA) to express statistically significant anatomical variations within a training population. We enrich SSMs with volumetric bone-interior information learned from CT and utilize these Statistical Shape and Intensity Models (SSIMs) as a prior for the reconstruction process. The problem of finding the right shape is thus translated into deriving a statistical instance of the SSIM, such that the patient-specific anatomical shape and bone-interior density depicted in the reference images is approximated best.

A treatment plan in orthopaedics does often not only involve single bony anatomies, but rather whole joint structures (e.g. in total hip replacement). The individual components of the joint are oriented in various poses, depending on the patient's physical condition and pose in front of the X-ray scanner. To cope with different joint postures in the reference X-ray images, we further extend the SSIMs to articulated statistical shape and intensity models (ASSIMs). The ASSIM of the hip developed at ZIB represents the hip cup as a ball joint, in which each proximal femur (upper thigh bone) rotates with three degrees of freedom (see Fig. 3). The rotational center of the hip joint is part of the statistical analysis of the model and learned from the training population.

Fig. 3: Articulated Statistical Shape and Intensity Model of the hip

Fig. 3. Articulated Statistical Shape and Intensity Model (ASSIM) of the hip, expressing two different joint postures. Depicted here is the articulated surface of the model (beige) and the bone-interior density distribution (black and white).

Resources

 

Publications

2021
Unsupervised Detection of Disturbances in 2D Radiographs 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pp. 367-370, 2021 Laura Estacio, Moritz Ehlke, Alexander Tack, Eveling Castro-Gutierrez, Hans Lamecker, Rensso Mora, Stefan Zachow BibTeX
DOI
3D Reconstruction from 2D X-ray Images
2020
3D Reconstruction of Anatomical Structures from 2D X-ray Images Doctoral thesis, Technische Universität Berlin, Stefan Zachow, Hans Lamecker (Advisors), 2020 Moritz Ehlke BibTeX
DOI
3D Reconstruction from 2D X-ray Images
2016
Influence of pelvic tilt on functional acetabular orientation Technology and Health Care, 25(3), pp. 557-565, 2016 Robert Zahn, Sarah Grotjohann, Heiko Ramm, Stefan Zachow, Matthias Pumberger, Michael Putzier, Carsten Perka, Stephan Tohtz BibTeX
DOI
3D Reconstruction from 2D X-ray Images
2015
3D Assessment of Osteosynthesis based on 2D Radiographs Proceedings of the Jahrestagung der Deutschen Gesellschaft für Computer- und Roboterassistierte Chirurgie (CURAC), pp. 317-321, 2015 (preprint available as ZIB-Report 15-47) Moritz Ehlke, Mark Heyland, Sven Märdian, Georg Duda, Stefan Zachow PDF (ZIB-Report)
PDF (ZIB-Report)
PDF (ZIB-Report)
BibTeX
3D Reconstruction from 2D X-ray Images
Assessing the Relative Positioning of an Osteosynthesis Plate to the Patient-Specific Femoral Shape from Plain 2D Radiographs ZIB-Report 15-21 (Appeared in: Proceedings of the 15th Annual Meeting of CAOS-International (CAOS)) Moritz Ehlke, Mark Heyland, Sven Märdian, Georg Duda, Stefan Zachow PDF
PDF
BibTeX
URN
3D Reconstruction from 2D X-ray Images
Towards Robust Measurement Of Pelvic Parameters From AP Radiographs Using Articulated 3D Models Computer Assisted Radiology and Surgery (CARS), 2015 (preprint available as ZIB-Report 15-11) Moritz Ehlke, Thomas Frenzel, Heiko Ramm, Mohsen Akbari Shandiz, Carolyn Anglin, Stefan Zachow PDF (ZIB-Report)
BibTeX
3D Reconstruction from 2D X-ray Images
2014
Robust Measurement of Natural Acetabular Orientation from AP Radiographs using Articulated 3D Shape and Intensity Models ZIB-Report 14-12 (Presented at the 14th Annual Meeting of CAOS-International (CAOS)) Moritz Ehlke, Thomas Frenzel, Heiko Ramm, Hans Lamecker, Mohsen Akbari Shandiz, Carolyn Anglin, Stefan Zachow PDF
BibTeX
URN
3D Reconstruction from 2D X-ray Images
2013
Articulated Statistical Shape Models Master's thesis, Humboldt-Universität zu Berlin, Beate Meffert (Advisor), 2013 Matthias Bindernagel BibTeX
3D Reconstruction from 2D X-ray Images
Fast Generation of Virtual X-ray Images for Reconstruction of 3D Anatomy IEEE Transactions on Visualization and Computer Graphics, 19(12), pp. 2673-2682, 2013 (preprint available as ZIB-Report 13-38) Moritz Ehlke, Heiko Ramm, Hans Lamecker, Hans-Christian Hege, Stefan Zachow PDF
PDF (ZIB-Report)
BibTeX
DOI
URN
3D Reconstruction from 2D X-ray Images
2012
Efficient projection and deformation of volumetric intensity models for accurate simulation of X-ray images ZIB-Report 12-40 (Presented at the 12th Annual Meeting of CAOS-International (CAOS)) Moritz Ehlke, Heiko Ramm, Hans Lamecker, Stefan Zachow PDF
BibTeX
URN
3D Reconstruction from 2D X-ray Images
Efficient projection and deformation of volumetric shape and intensity models for accurate simulation of X-ray images Eurographics Workshop on Visual Computing for Biomedicine (NVIDIA best poster award), 2012 Moritz Ehlke, Heiko Ramm, Hans Lamecker, Stefan Zachow BibTeX
3D Reconstruction from 2D X-ray Images
Simulating X-ray images from deformable shape and intensity models on the GPU Master's thesis, Technische Universität Berlin, Bernd Bickel, Bernd Mahr, Stefan Zachow (Advisors), 2012 Moritz Ehlke BibTeX
3D Reconstruction from 2D X-ray Images
2009
3D Reconstruction of the Human Rib Cage from 2D Projection Images using a Statistical Shape Model Int. J. Comput. Assist. Radiol. Surg., 5(2), pp. 111-124, 2009 Jalda Dworzak, Hans Lamecker, Jens von Berg, Tobias Klinder, Cristian Lorenz, Dagmar Kainmüller, Heiko Seim, Hans-Christian Hege, Stefan Zachow BibTeX
DOI
3D Reconstruction from 2D X-ray Images
2008
Towards model-based 3-D reconstruction of the human rib cage from radiographs Proc. 7. Jahrestagung der Deutschen Gesellschaft für Computer-Roboterassistierte Chirurgie (CURAC), pp. 193-196, 2008 Jalda Dworzak, Hans Lamecker, Jens von Berg, Tobias Klinder, Cristian Lorenz, Dagmar Kainmüller, Heiko Seim, Hans-Christian Hege, Stefan Zachow BibTeX
3D Reconstruction from 2D X-ray Images
2006
Atlas-based 3D-shape reconstruction from x-ray images Proc. Int. Conf. of Pattern Recognition (ICPR2006), pp. 371-374, Vol.Volume I, 2006 Hans Lamecker, Thomas Wenckebach, Hans-Christian Hege BibTeX
DOI
3D Reconstruction from 2D X-ray Images
Atlas-basierte 3D-Rekonstruktion des Beckens aus 2D-Projektionsbildern Bildverarbeitung für die Medizin 2006: Algorithmen - Systeme - Anwendungen, pp. 26-30, 2006 Hans Lamecker, Thomas Wenckebach, Hans-Christian Hege, Georg N. Duda, Markus Heller BibTeX
DOI
3D Reconstruction from 2D X-ray Images