The causes of low back pain are not yet fully understood. One essential part of a better understanding is the knowledge about the relationship between spinal morphology and lower back pain (LBP).
So far, spinal shapes are most commonly classified into four types according to the Roussouly scheme [6] and some of its population-based computational extensions [4], which only look at two-dimensional sagittal views. In other words, these classes do not take into account the three-dimensional (3D) shape of the entire human spine. Although one previous study developed a 3D classification scheme, it does not correlate those classes with LBP and kinematics [5].
Our project is part of the DFG Research Unit 5177 “The Dynamics of the Spine: Mechanics, Morphology, and Motion towards a Comprehensive Diagnosis of Low Back Pain”, where we are aiming to investigate the spinal 3D morphology and motion in association with LBP based on cohort data.
For this purpose, we will develop a statistical 3D shape model of the spine that will incorporate the variation in spinal morphology of both the complete spinal shape as well as the individual vertebrae.
We will then individualize these shape models to static and dynamic measurements of subjects, which will help us to find correlations between spinal morphology and functional impairments or LBP.
Our main goal is to establish an extended morphological score for spinal diagnostics and further research.
Data Collection and Segmentation
For creating statistical shape models, we first need a large pool of segmented medical spine imaging data. Therefore, we are collecting, and processing a wide range of subject data from the German National Cohort (GNC) [1], Berliner Rückenstudie, and the VerSe Challenge [2].
A special focus lies on the Berliner Rückenstudie, which is a study within our research collaboration, where our partners take extensive measurements related to the human lumbar spine of up to 3000 subjects, and further processes the data within the research unit. To create a database of spinal shapes we will segment the spine from tomographic CT and MR image data, while individually classifying all vertebrae.
Challenges
While manually segmenting spinal structures in MR images to generate ground truth data for the training of machine learning algorithms, we are facing several challenges (Figure 2), which we aim to address step-by-step.
The low contrast between bones and surrounding soft tissue in MRIs leads to incomplete anatomical structures in the segmentations (structural sparsity). Furthermore, the high slice distance of the images causes missing information in between the slices (inter-slice sparsity). The image noise and the inhomogeneity of image intensity add up to the difficulty of the manual segmentation process, especially of smaller vertebral structures.
Due to the aforementioned challenges, the manual segmentation work can be an expensive process that requires prior anatomical knowledge to interpolate invisible bony structures. Therefore, we want to automate this process. However, even creating a large enough high-quality dataset for training an algorithm poses a challenge (data scarcity).
We aim to address all of these challenges and develop a segmentation algorithm that can learn from scarce and sparse data to automatically segment given MRI databases (GNC and the Berliner Rückenstudie). A special focus lies on segmenting individual vertebrae including the vertebral processes, arches, facets, and foramina.
Pseudo-Labeling: Addressing the Data-Scarcity
The purpose of pseudo-labeling is to increase data size and quality to a certain extent by employing an iterative process. The idea is to train a neural network on an initial small dataset with structural sparsity, and then to (re-)predict the segmentations for seen and unseen images [3] (Figure 3).
Figure 4: Pseudo-labeling Architecture
This method seems to correct minor errors in the manual segmentations (Figure 4) by learning the correct labels from other training samples.
Figure 5: Improvement of segmentation by re-prediction; red: manual segmentation with errors, green: 1st re-prediction, blue: 2nd re-prediction, red-blue overlay: manual vs. 2nd re-prediction, magenta: regions of improvement
Unpaired Domain Adaptation: Completing Holes in Anatomical Structures
As mentioned before, it can be challenging to segment complete bony structures in MRIs. In comparison, CT images have great contrast between bones and the rest of the image. This makes it easier to segment CTs even for non-experts. Usually, a simple intensity thresholding can separate the bone from the rest of the image. If we had exactly one corresponding registered CT image for each MRI, we could just use the segmentations from CTs. However, this kind of data is not available to us.
Therefore, we are working on generating synthetic CTs from MRIs, or vice versa. The computer vision community has been working on image-to-image translation methods without having paired training data for a while now (e.g. CycleGAN [4]). Lately, there has also been some work in the medical imaging domain (Matsuo et al. [5]). The basic idea of these methods is to implicitly align the data distributions from one domain to another using neural networks.
The challenge here is to ensure exact correspondences between the bones of the two image modalities and quantify the mismatch automatically.
Our preliminary results show that we can translate real CTs to synthetic images with the appearance of an MRI (Figure 6). Evaluating image appearances can be done with the FID score. However, the evaluation of the image correspondence between the two domains remains a challenge.
Figure 6: CT-to-MRI Translation
3D Reconstruction: Interpolating Gaps in-between Slices
MRIs can have a slice distance higher than 3.3 mm, which can lead to gaps in the segmentations. This inter-slice sparsity needs to be interpolated these gaps with plausible information.
Figure 7: Reconstructing vertebrae from parallel (top) and orthogonal (bottom) slices
In the last couple of years, the computer vision community has been researching 3D reconstruction techniques using neural networks. Recent work in our group has also successfully reconstructed single vertebrae from sparse segmentation data (Amiranashvili et al. [6]).
So far, most of these methods seem to work only on smaller-sized objects. However, our goal is to reconstruct the complete spine.
References
[1] Bamberg et al.: Whole-Body MR Imaging in the German National Cohort: Rationale, Design, and Technical Background. Radiology 277(1): 206 – 20, 2015.
[2] Sekuboyina et al.: VerSe: A Vertebrae labeling and segmentation benchmark for multi-detector CT images. Medical Image Analysis, 2021.
[3] Ziyan et al.: Revisiting nnU-Net for Iterative Pseudo Labeling and Efficient Sliding Window Inference. MICCAI FLARE Challenge, 2022.
[4] Zhu et al.: Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. ICCV, 2017.
[5] Matsuo et al.: Unsupervised-learning-based method for chest MRI–CT transformation using structure constrained unsupervised generative attention networks. Scientific reports, 12(1):11090, 2022.
