NA-MIC Project Weeks

Back to Projects List

3D Medical Registration and Segmentation with Elastix and MONAI Label

Key Investigators

• Konstantinos Ntatsis (Leiden University Medical Center, the Netherlands)
• Andres Diaz-Pinto (NVIDIA & King's College London, United Kingdom)

Presenter location: In-person

Project Description

This project aims to investigate the application of itk-elastix (a python wrapping of Elastix) for image registration in combination with MONAI Label for segmentation/classification. Depending on the time/people availability, we will work in one or more sub-projects.

Initial sub-project: We will starty by training a single modality MONAI Label model on Elastix-aligned brain images (T1, T2, FLAIR, etc) using SynthSeg as the source of annotations. SynthSeg is a tensorflow-based deep learning segmentation tool for brain MRIs. It consists of a generative network that produces the synthetic images and a 3D U-Net trained to do the segmentation. The only input (training data) is the training labels so no real images are used.

We will use SynthSeg to produce annotations as “ground truth” on a publicly available dataset like BRATS (multimodal + non-healthy brains) or OASIS (temporal/monomodal + healthy brains). Elastix will be used for the co-registration of the different modalities or temporal images and achieve segmentation via registration.

Other possible sub-projects:

• Extend the whole brain segmentation model available in the Model Zoo, Use Elastix to perform affine registration of the data in the MNI305 space.
• Compare registration performance between cross-modal registration (CT-MRI) versus intra-modal registration via synthesised MRI (MRI_syn - MRI). MONAI for the synthesis and elastix for the registration. What would a suitable dataset be?
• Train MONAI Label model for automatic landmark identification in e.g. lung images (dataset) . Landmarks can be used either to assist registration with elastix OR elastix can be used to validate the landmark accuracy. 3D Slicer can be used to visualize the landmarks and ease the qualitative evaluation.
• … any other idea that is interesting to people, feel free to propose it!

Objective

1. Working code, jupyter notebooks, any other artifacts etc that demonstrate the combination of itk-elastix and MONAI Label. They will be helpful for users that would like to solve similar problems.

Approach and Plan

1. Configure and run Elastix
2. Setup and run MONAI Label
3. Make sure they work together nicely (e.g. output of Elastix should be suitable for MONAI, or the reverse)
4. Improve the results (a bit)
5. Polish and store the code/documentation/results so that they are helpful for future generations

Progress and Next Steps

1. Preliminary registration of the BRATS dataset. Several details need to be sorted out still.
2. …

Illustrations

Example of the unregistered images for a subject in the BRATS dataset:

Background and References

• Elastix repo: https://github.com/SuperElastix/elastix
• itk-elastix (python wrapping): https://github.com/InsightSoftwareConsortium/ITKElastix
• MONAI Label: https://monai.io/label.html
• itk-torch-bridge: https://docs.monai.io/en/latest/data.html#module-monai.data.itk_torch_bridge