Machine Learning Approaches for Enhanced, Shape Model Based 3D Image Segmentation

Image-based Cost Functions

Principal Component Analysis (PCA) on local intensity profiles has not proven to beneficially act as a robust cost function. Random Forest Regression Voting (RFRV), though a powerful method for 2D image data, turned out to be impractical for 3D data, due to huge memory consumption and computational time. Dictionary Learning (DL) does not require any heuristics and is general enough to be applied across anatomies and modalities. DL operations are matrix operations, thus being efficiently evaluated.

Joint Dictionary Learning

Given 3D image data and accordingly segmented anatomical structures of interest, rotational invariant histograms of oriented gradients (HoG) are sampled at the structures’ boundaries. These feature samples are used as input for learning a dictionary.

A second dictionary is learnt for background image information. A combined dictionary of foreground and background features has been established, acting as a cost function for image segmentation.

Cost Function for a test patch: Sum of residuals from representations by the two dictionaries.

Clustered Appearance Models

Local appearance patterns of anatomical structures in medical image data play an important role in determining the location, yet the variation of these patterns across localities are challenging for traditional appearance models. Dictionary Learning, even though widely used as an appearance model, lacks a systematic approach for comparison between dictionaries for subsequent clustering. A dictionary comparison framework is proposed for generating clustered appearance models useful for fully automatic segmentation.

We consider available dictionary comparison metrics and propose two new metrics for comparison. Three novel meta comparison schemes are also developed for systematic comparison of the metrics in appearance dictionary context. Furthermore, a novel non-overlapping dictionary clustering approach is developed for clustered appearance modeling on anatomical structures.

Initial Seeds (upper row) and the resulting clusters (bottom row) by our dictionary clustering algorithm. Notice the similarity of clusters for different number of initial seeds: (a) 50, (b) 150, (c) 250, (d) 350 and (e) 450.