Machine Learning Approaches for Enhanced, Shape Model Based 3D Image Segmentation
Fully automatic segmentation of arbitrary anatomical structures from 3D medical image data is a challenging, yet unsolved problem. Though fully automatic segmentation is essential for further clinical analysis, complexity of anatomical structures across population makes a generalized segmentation scheme extremely challenging. Moreover, specific challenges of different imaging modalities have so far hindered the possibility of a general purpose fully automatic 3D segmentation framework. Statistical 3D shape models have proven to be valuable shape priors that are to be deformed within their range of normal variation in shape to match the respective image information. Within the project, we are aiming to combine Machine Learning along with the statistical shape priors for getting a step closer to a general 3D image segmentation approach. In particular, Machine Learning techniques for image matching based on intensity will be developed in order to improve both the model building as well as the segmentation process.
Image-based Cost Functions
Principal Component Analysis (PCA) on local intensity profiles has not proven to beneficially act as a robust cost function. Random Forest Regression Voting (RFRV), though a powerful method for 2D image data, turned out to be impractical for 3D data, due to huge memory consumption and computational time. Dictionary Learning (DL) does not require any heuristics and is general enough to be applied across anatomies and modalities. DL operations are matrix operations, thus being efficiently evaluated.
Joint Dictionary Learning
Given 3D image data and accordingly segmented anatomical structures of interest, rotational invariant histograms of oriented gradients (HoG) are sampled at the structures’ boundaries. These feature samples are used as input for learning a dictionary.
A second dictionary is learnt for background image information. A combined dictionary of foreground and background features has been established, acting as a cost function for image segmentation.
Cost Function for a test patch: Sum of residuals from representations by the two dictionaries.
Clustered Appearance Models
Local appearance patterns of anatomical structures in medical image data play an important role in determining the location, yet the variation of these patterns across localities are challenging for traditional appearance models. Dictionary Learning, even though widely used as an appearance model, lacks a systematic approach for comparison between dictionaries for subsequent clustering. A dictionary comparison framework is proposed for generating clustered appearance models useful for fully automatic segmentation.
We consider available dictionary comparison metrics and propose two new metrics for comparison. Three novel meta comparison schemes are also developed for systematic comparison of the metrics in appearance dictionary context. Furthermore, a novel non-overlapping dictionary clustering approach is developed for clustered appearance modeling on anatomical structures.
Initial Seeds (upper row) and the resulting clusters (bottom row) by our dictionary clustering algorithm. Notice the similarity of clusters for different number of initial seeds: (a) 50, (b) 150, (c) 250, (d) 350 and (e) 450.
Publications
2021 |
|||
Jianning Li, Pedro Pimentel, Angelika Szengel, Moritz Ehlke, Hans Lamecker, Stefan Zachow, Laura Estacio, Christian Doenitz, Heiko Ramm, Haochen Shi, Xiaojun Chen, Franco Matzkin, Virginia Newcombe, Enzo Ferrante, Yuan Jin, David G. Ellis, Michele R. Aizenberg, Oldrich Kodym, Michal Spanel, Adam Herout, James G. Mainprize, Zachary Fishman, Michael R. Hardisty, Amirhossein Bayat, Suprosanna Shit, Bomin Wang, Zhi Liu, Matthias Eder, Antonio Pepe, Christina Gsaxner, Victor Alves, Ulrike Zefferer, Cord von Campe, Karin Pistracher, Ute Schäfer, Dieter Schmalstieg, Bjoern H. Menze, Ben Glocker, Jan Egger | AutoImplant 2020 - First MICCAI Challenge on Automatic Cranial Implant Design | IEEE Transactions on Medical Imaging, 40(9), pp. 2329-2342, 2021 |
BibTeX
DOI |
2017 |
|||
Anirban Mukhopadhyay, Suchendra Bhandarkar | Biharmonic Density Estimate - a scale space descriptor for 3D deformable surfaces | Pattern Analysis and Application, pp. 1-13, 2017 |
BibTeX
DOI |
Avan Suinesiaputra, Pierre Albin, Xenia Alba, Martino Alessandrini, Jack Allen, Wenjia Bai, Serkan Cimen, Peter Claes, Brett Cowan, Jan D'hooge, Nicolas Duchateau, Jan Ehrhardt, Alejandro Frangi, Ali Gooya, Vicente Grau, Karim Lekadir, Allen Lu, Anirban Mukhopadhyay, Ilkay Oksuz, Nripesh Parajuli, Xavier Pennec, Marco Pereanez, Catarina Pinto, Paolo Piras, Marc-Michael Rohe, Daniel Rueckert, Dennis Saring, Maxime Sermesant, Kaleem Siddiqi, Mahdi Tabassian, Lusiano Teresi, Sotirios Tsaftaris, Matthias Wilms, Alistair Young, Xingyu Zhang, Pau Medrano-Gracia | Statistical shape modeling of the left ventricle: myocardial infarct classification challenge | IEEE Journal of Biomedical and Health Informatics, 2017 |
BibTeX
DOI |
David Wilson, Carolyn Anglin, Felix Ambellan, Carl Martin Grewe, Alexander Tack, Hans Lamecker, Michael Dunbar, Stefan Zachow | Validation of three-dimensional models of the distal femur created from surgical navigation point cloud data for intraoperative and postoperative analysis of total knee arthroplasty | International Journal of Computer Assisted Radiology and Surgery, 12(12), pp. 2097-2105, 2017 |
PDF
BibTeX DOI |
2016 |
|||
Anirban Mukhopadhyay, Fatih Porikli, Suchendra Bhandarkar | Detection and Characterization of Intrinsic Symmetry of 3D Shapes | Proceedings of IEEE International Conference on Pattern Recognition, 2016 |
BibTeX
|
Anirban Mukhopadhyay, Arun Kumar, Suchendra Bhandarkar | Joint Geometric Graph Embedding for Partial Shape Matching in Images | IEEE Winter Conference on Applications of Computer Vision, pp. 1-9, IEEE, IEEE Winter Conference on Applications of Computer Vision (WACV), 2016 |
BibTeX
|
Anirban Mukhopadhyay, Oscar Morillo, Stefan Zachow, Hans Lamecker | Robust and Accurate Appearance Models Based on Joint Dictionary Learning Data from the Osteoarthritis Initiative | Lecture Notes in Computer Science, Patch-Based Techniques in Medical Imaging. Patch-MI 2016, pp. 25-33, Vol.9993, 2016 |
BibTeX
DOI |
Anirban Mukhopadhyay | Total Variation Random Forest: Fully automatic MRI segmentation in congenital heart disease | RAMBO 2016, HVSMR 2016: Reconstruction, Segmentation, and Analysis of Medical Images, pp. 165-171, Vol.LNCS 10129, 2016 |
BibTeX
DOI |
2015 |
|||
Anirban Mukhopadhyay, Ilkay Oksuz, Marco Bevilacqua, Rohan Dharmakumar, Sotirios Tsaftaris | Data-Driven Feature Learning for Myocardial Segmentation of CP-BOLD MRI | Functional Imaging and Modeling of the Heart, pp. 189-197, Vol.9126, Lecture Notes in Computer Science, 2015 |
BibTeX
DOI |
Ilkay Oksuz, Anirban Mukhopadhyay, Marco Bevilacqua, Rohan Dharmakumar, Sotirios Tsaftaris | Dictionary Learning Based Image Descriptor for Myocardial Registration of CP-BOLD MR | Medical Image Computing and Computer-Assisted Intervention -- MICCAI 2015, pp. 205-213, Vol.9350, Lecture Notes in Computer Science, 2015 |
BibTeX
DOI |
Anirban Mukhopadhyay, Ilkay Oksuz, Marco Bevilacqua, Rohan Dharmakumar, Sotirios Tsaftaris | Unsupervised myocardial segmentation for cardiac MRI | Medical Image Computing and Computer-Assisted Intervention -- MICCAI 2015, pp. 12-20, Vol.LNCS 9351, 2015 |
BibTeX
DOI |
2012 |
|||
Sara Orgiu | Automatic liver segmentation in contrast enhanced CT data using 3D free-form deformation based on optimal graph searching | Master's thesis, Politecnico di Milano, Pietro Cerveri, Hans Lamecker (Advisors), 2012 |
BibTeX
|
2011 |
|||
Maximilien Renard | Improvement of Image Segmentation Based on Statistical Shape and Intensity Models | Master's thesis, Université libre de Bruxelles, Nadine Warzée, Olivier Debier, Hans Lamecker, Stefan Zachow (Advisors), 2011 |
BibTeX
|