Nonlinear registration and template driven segmentation

Abstract Introduction Registration is the process of the alignment of medical image data. Keyphrases nonlinear registration template driven segmentation different people intrinsic shape difference nonuniform manner broad structure local nonlinear shape difference various disease state nonlinear deformation data set introduction registration normal anatomical variability shape difference harvard medical school different specie physical deformation process nonrigid organ medical image data.

Powered by:. Our inverse consistency constraint was achieved by forcing the residual fields defined on both images to approach zero and is thus fully symmetric. Evaluating the performance of nonlinear image registration algorithms is difficult due to lack of gold standard and the unavailability of a ground truth [ 23 — 24 ].

In the absence of the ground truth, we used three criteria for evaluating the registration performance. Then the properties of transformations were inspected to ascertain that the transformations are one- to-one mapping eg. OASIS consists of stripped stripped of extramenigeal tissues and intensity corrected MR images with an istropic resolution of 1mm 3 of subjects in the age range of 18 to 96 years.

We randomly selected 20 adult brains from OASIS dataset to construct an average brain atlas to demonstrate the quality of our image registration algorithm. And we have registered an 89 years old female brain from OASIS to our atlas to demonstrate the performance of our algorithm even when the deformation was relatively large.

Deformation based segmentation is widely used as an objective criterion for evaluating the performance of registration algorithm [ 23 — 24 ]. In the deformation based segmentation, all the images that undergo registration are manually parcellated into different structures or segmented into different tissues by experts.

In the absence of ground truth, the manually segmented maps by experts are considered to represent the truth. We compared the similarity between the manually labeled maps with the deformation based results. This dataset consists of T1-weighted image volumes from 18 brains along with manual tissue segmentation and anatomical structures. After registration, the manually segmented images and anatomical parcellation maps of these 17 image volumes were deformed to the first image volume.

The 17 deformed maps were compared with the manually produced maps available at the IBSR website of the first image volume using the Dice similarity index DSI [ 26 ]. DSI is given by:. In the above equation denotes the number of elements in the segmented image set. We compared the performance of our algorithm with the two most popular nonlinear registration algorithms: the demons algorithm [ 12 ] and the B-spline based free form deformation FFD model [ 27 ]. Multiresolution paradigm in which the images were successively down sampled by factors of 4, 2, and 1 full resolution.

The number of iterations for these levels of resolution were set to , , and 32, respectively. We also investigated the effect of number of iterations on the convergence of MI. The parameter setting for ITK demons was same as that was used for our method. Three levels of multiple-resolutions were also used in the B-spline based FFD method with the grid control points spacing of 40mm along each direction.

The number of iterations and the number of steps were 10 at each resolution. From coarse to fine, the length of steps were 20mm, 10mm, and 5mm, respectively. The top row shows axial source image A and the target image B to be registered. The registration results with and without ICC are shown in the second and third rows, respectively. For example, the deformed image shown in the second row appears to be very similar to the original image first row.

However, the deformed image produced without ICC third row , especially in the ventricular region and parts circled by red, does not appear to be well registered. More specifically, the ICE produced by the registration method without ICC was observed to be as large 11 voxels, with a mean value around 4 voxels. Registration results with and without ICC.

First row: axial MR images A source and B target image to be registered. The intensity range of ICE maps in the second row is from 0 to 0. The histograms of the logarithm of Jacobian determinants with and without ICC, were calculated for the ventricles and are shown in Fig. The mean values of logarithm of Jacobian shown in Figs. But, this is not the case for the histograms without ICC Figs.

Histogram of the log Jacobian determinant for the registration with ICC measured in the ventricular area for: A source image and B target image. We have compared the performance of the registration algorithms that includes 1 both steps 4 and 6, 2 step 6 alone with our fully symmetric residual method, and 3 demons bijective scheme in step, 6.

This makes it difficult to plot the variation of the total cost function with the number of iterations. We plotted RMS since it is closely related to E inv in the total cost function. These results also show that the similarity terms produced with steps 4 and 6 is only 0. Also, the MI produced by demons bijective scheme shows very significant fluctuations. Figure 4 shows the brain atlas produced from 20 brains after registering to one individual brain template.

The 20 brains were randomly selected from the OASIS dataset with the age varying from 30 to 60 years. The created atlas shown in Fig. The brain atlas is remarkably sharp and some of the fine features can be seen clearly in Fig.

The sharpness of the averaged brain itself does not guarantee the performance of the registration, but at least validates the optimization procedure [ 9 ]. Since our proposed method maintains the deformation field along forward and reverse directions, we calculated the Jacobian determinant of the deformation field on the subject and template image spaces separately for the 20 registrations used to create the averaged brain atlas Fig.

The positive Jacobian determinant for all voxels Fig. The ICE measured on both images is shown in Fig. As can be seen in Fig. This demonstrates that our proposed algorithm can produce inverse consistent registration with sub-voxel ICE. Axial MR images from two sections: A individual template image and B corresponding slices from the atlas based on twenty brains. As a demonstration that our method can deal with relatively large deformation, we registered an 89 year old brain from the OASIS dataset to the atlas that we created.

As can be observed from Figs. The shrinkage and dilation of ventricles in the Jacobian maps can be clearly seen in Figs. The values of the Jacobian determinant varied from 0. Results of registration of an elderly female brain and the created atlas: A elderly individual brain, B section from the atlas, C atlas deformed to the space of individual brain, D individual brain deformed to the atlas space.

The Jacobian determinant maps in the individual image and the atlas spaces are shown in E and F respectively. The 17 deformed maps were compared with the manually produced maps of the first image volume.

We have also compared the performance of our algorithm with the two most popularly used nonlinear registration algorithms: the demons algorithm [ 12 ] and the B-spline based FFD model [ 27 ]. As an example, Fig. For comparison, the corresponding IBSR01 image and the manually segmented tissue map are shown in Fig.

Visual inspection, particularly the areas circled by red and green, shows that the segmentation map produced by our method is superior to that of demons and FFD algorithms. We have also parcellated cerebellum, hippocampus, putamen, caudate, lateral ventricles and thalamus using the three methods and the results are summarized in Table 1.

Usually, DSI above 0. As can be seen from Table 1 , while all the methods produced high DSI, our method yielded superior results for all the six structures. Quantitative analysis based on two tailed paired t-test indicates that our method produced significantly higher similarity measure on five structures except for caudate.

Nonlinear Registration of T1 to Template. Introduction to Neurohacking In R. Enroll for Free. This Course Video Transcript. From the lesson Extended Image Processing In this section, we will discuss the different types of registration and how one would go through processing a multi-sequence MRI scan, as well as wrapper functions that make the process much easier.

Nonlinear Registration of T1 to Template To improve alignment, we introduced a non-linear registration step that incorporates a novel hybrid cost function based on surface and volume. Our algorithm furthermore included a new multi-level feature weighting for shape averaging. Training set consists of MR images and manual labels of controls and patients Figure 1A. Labels are converted into surface meshes using spherical harmonics and point distribution model SPHARM-PDM that ensure shape-inherent point-wise correspondences across subjects Styner et al.

Each surface is mapped onto its corresponding MRI. In the beginning of the segmentation step, the pair of each template image and its MTL surface are mapped on the test image. As the test image does not have its own surface, the surface features extracted on the test image are from the surface of each template.

By comparing the features extracted from each template and those from the test image, Surface- with volume-derived similarity measures for optimal template selection are then computed to select an optimal subset n a Figure 1B Next, a non-linear registration that is driven by volume-based intensities and features sampled on evolving template surfaces is performed to improve alignment between each template in the subset n a and the individual MRI Figure 1B The motivation of using this hybrid registration was to improve the boundary fitting by weighting the features extracted using deformable surfaces as well as to use a consistent similarity measurement in all the steps.

After choosing a smaller subset n b , templates are then averaged using adaptive weighting combined with local averaging, which creates the final segmentation Figure 1B -3,4. The test image's features are updated during the series of the steps including template selection, non-linear registration and weighted averaging as the image and the surface deform. In this manner, the similarity of the deformable surface and the target MTL border is expected to increase and the surface gets a similar shape to the true MTL boundary.

Figure 1. HybridMulti automatic hippocampal segmentation steps. Flowchart of the proposed algorithm in A , steps 2 and 4 are illustrated only for the HP. The segmentation procedure consists mainly of two: template library construction and automated segmentation of mesiotemporal structures. A Template library construction. B Automatic segmentation of MTL structures. At a given surface vertex v , we define three spherical neighborhoods of 3, 5, and 7 mm radius.

These spheres are subdivided into an inner region IR and outer region OR with respect to the surface boundary, where we compute the following texture features Kim et al. Previously we demonstrated that each feature almost equally contributed to the segmentation accuracy and observed the optimal result using all the features. Notably, we did not use the shape features proposed in our previous surface-based framework Kim et al.

The deformation in the current study is instead governed directly by a volume-based non-linear registration see section Boundary-Weighted Non-linear Registration of Template Subset to Test MRI. From the template library, we first select a subset of candidates that are most similar to the test image.

To that end, we compute the hybrid similarity O total that combined surface-based O surface and volume-based O volume similarity term between each template j and the test MRI i using:. The surface-based similarity O surface is defined as:. O surface is calculated across all surface vertices v. O volume can be any similarity function including the cross-correlation or the normalized mutual information NMI that quantifies statistical intensity distribution dependency of two images A and B Studholme et al.

The computation of cross-correlation is generally faster while the NMI is more robust in similarity of multi-modal images compared to each other. For computational efficiency, we compute O volume within a mask defined by dilating the current template label three times. The number of selected templates n a was empirically determined to maximize O total see section Parameter Selection.

Accordingly, the deformation field d is estimated as:. O smooth is a smoothness term to constrain the estimated deformation. We employed a type of freeform deformation models defined in Collins et al. To improve the registration accuracy, we increase the weight of voxels on and nearby the target boundary by incorporating a similarity measure derived from the template surface evolving during the registration with the original volume similarity.

To estimate the deformation field, we redefine the Equation 3 as:. O vol , reg is the correlation coefficient over a volume of interest here, a geometric union of all MTL template labels in the library, subsequently dilated 5 times for more extensive spatial coverage as in Collins and Pruessner A larger weight w surf , reg moves S MTL , S more rapidly to areas presenting with feature characteristics similar to those on the surface of the template image.

This method is applied to non-linear optimization problems for which derivatives may not be known and is robust against the local minima problem. This function has been used as the standard optimization method in the non-linear registration algorithm Collins et al.

The non-linear registration in the previous section Boundary-weighted Non-linear Registration of Template Subset to Test MRI is applied to decrease shape variability and to increase similarity between the template-subset and test image. We determine n b empirically, which will be evaluated in the section Parameter Optimization. Optimal global weights for these n b templates are calculated using the similarity function Equation 2 as in Kim et al.

Analogously, we define the weighted mean and SD of features at a given vertex v i by:. Similarity from Equation 2 can be formulated for the template-subset n b :. In the above formulas, weights are determined by maximizing the similarity between the n b template-subset and test image.

The cost function O subset is optimized using the multivariate derivative-free Nelder-Mead approach Lagarias et al. We determine weights at each sampling vertex, and interpolate these weights to vertices at the next finer level l 1. We define the locally weighted average surface as:. The algorithm stops when Equation 11 stops increasing or l reaches preset l max to prevent from an extensive computation. The proposed multi-level approach using different subdivisions is mainly for coarse-to-fine spatial fitting and the use of this strategy avoids the introduction of a constraint term preventing from local minima while the surface shape gets finer.

The Ethics Committee of the Montreal Neurological Institute and Hospital approved the study and written informed consent was obtained from all participants.

MR images were acquired on a 1. Images underwent intensity non-uniformity correction Sled et al. MTL structures were manually segmented by an expert using the protocol described in Bernasconi et al. We also acquired 3T T1-weighted images on Siemens Trio Tim scanner using a channel phased-array head coil.

This data was used to evaluate whether the algorithm consistently selected the same or similar parameter values for different dataset. Based on maximal Dice overlap index between automated and manual labels, the following parameters were chosen empirically: weight of surface-based similarity w surfac e to select the optimal subset as in Equation 1 ; weight of surface-based similarity w surf , reg used in non-linear registration; size of initial template-subset n a ; size of final template-subset n b ; and finest subdivision l max in local weighting.

The optimal parameters that resulted in most accurate segmentation were selected for each training-set. We segmented the test-set based on their corresponding training-set and the parameters.

We repeated this process three times while all the three sets were tested. Segmentation accuracy was evaluated at the following stages: i initial n a template-subset selection; ii non-linear registration; iii final n b template-subset selection; iv global and local weighted averaging.

We compared accuracy at each stage to that of the previous stage using paired t -tests. Keeping proportions of controls and patients constant, we randomly selected 40 subjects as a test-set.

We repeated this process 20 times to avoid a possible bias. We evaluated automated segmentation accuracy at these smaller template library sizes. Significances of all statistical tests were adjusted for multiple comparisons using Bonferroni-correction.

The parameters resulting in the best segmentation accuracy were selected at very similar values between the 3 test-sets when using a three-fold cross validation. Use of the cross-correlation or NMI as the similarity function did not make a difference in segmentation accuracy.