Learn how GE Healthcare used AWS to build a new AI model to interpret MRI.

MRI images are naturally complex and contain a lot of data.

This required developers training large language models (LLMs) for MRI analysis to segment captured images into 2D. However, this is only an approximation of the original image, limiting the model’s ability to analyze complex anatomy. This causes problems in complex cases involving brain tumors, skeletal disorders or cardiovascular diseases.

However, GE Healthcare appears to have overcome this major obstacle by debuting the industry’s first full-body 3D MRI research-based model (FM) at this year’s AWS re:Invent. For the first time, models can use full 3D images of their entire bodies.

GE Healthcare’s FM is built from the ground up on AWS. Few models have been designed specifically for medical imaging, such as MRI, and are based on more than 173,000 images from more than 19,000 studies. The developers say they were able to train the model with five times less computing power than previously needed.

GE Healthcare has not yet commercialized the basic model. It is still in the evolutionary research phase. Mass General Brigham, the initial evaluator, will soon begin testing.

“Our vision is to get these models into the hands of technology teams working across the health system, giving them powerful tools to develop research and clinical applications faster and more cost-effectively,” Parry Bhatia, chief AI officer at GE HealthCare, told VentureBeat. .”

Real-time analysis of complex 3D MRI data possible

Although this is a groundbreaking advancement, generative AI and LLM are not new areas for the company. Bhatia explained that the team has been working with cutting-edge technology for more than a decade.

One of its flagship products is AIR Recon DL, a deep learning-based reconstruction algorithm that allows radiologists to obtain clear images more quickly. This algorithm reduces scan times by up to 50% by removing noise from raw images and improving signal-to-noise ratio. Since 2020, 34 million patients have been scanned with AIR Recon DL.

GE Healthcare will begin work on MRI FM in early 2024. Because the model is multimodal, it can support image-to-text retrieval, link images and words, and segment and classify diseases. Bhatia said the goal is to provide medical professionals with more details in a single scan than ever before, allowing for faster, more accurate diagnosis and treatment.

“This model has significant potential to improve medical procedures such as biopsies, radiation therapy, and robotic surgery by enabling real-time analysis of 3D MRI data,” Dan Sheeran, GM of Healthcare and Life Sciences at AWS, told VentureBeat. .” he said.

It has already outperformed other publicly available research models on tasks including prostate cancer and Alzheimer’s disease classification. Image retrieval achieved up to 30% accuracy in matching text descriptions to MRI scans. That may not seem all that impressive, but it’s a big improvement over the 3% capability seen by similar models.

“We’ve reached a point where we’re delivering really solid results,” Bhatia said. “The implications are enormous.”

Do more with (much less) data

The MRI process requires several different types of data sets to support the different techniques for mapping the human body, Bhatia explained.

For example, a technique known as T1-weighted imaging emphasizes fatty tissue and reduces the water signal, while T2-weighted imaging enhances the water signal. The two methods are complementary and create a full picture of the brain that helps clinicians detect abnormalities such as tumors, trauma or cancer.

“MRI images come in all shapes and sizes, just like looking at a book in all different formats and sizes, right?” Bhatia said.

To overcome the challenges posed by diverse data sets, developers introduced a “scale and adapt” strategy to enable the model to handle and respond to different variations. Additionally, data may be missing in some areas. For example, the image may be incomplete, so we taught the model to ignore these instances.

“Instead of getting stuck, we taught the model to skip the gaps and focus on what was available,” Bhatia said. “Think of this as solving a puzzle with a missing piece.”

The developers also used semi-supervised student-teacher learning, which was especially useful when data was limited. Using this method, two different neural networks are trained on both labeled and unlabeled data, and the teacher generates labels that help students learn and predict future labels.

“We are now using a lot of these self-supervised techniques that don’t require huge amounts of data or labels to train large models,” Bhatia said. “We can learn more from these raw images than we could in the past, which reduces dependency.”

This helps ensure the model works well in hospitals with fewer resources, older machines and different types of datasets, Bhatia explained.

He also emphasized the importance of the model’s multimodality. “In the past, a lot of technologies were unimodal,” Bhatia said. “We will only look at images and text. But now it’s becoming multimodal, so you can go from image to text, text to image, so you can take a lot of the work that you used to do in the past as separate models and really unify your workflow.”

He emphasized that researchers only use data sets to which they have rights. GE Healthcare has partners who license de-identified data sets and carefully adheres to compliance standards and policies.

Solve computational, data problems using AWS SageMaker

Undoubtedly, there are many challenges in building these sophisticated models, such as limited computational power for gigabyte-sized 3D images.

“This is a huge amount of 3D data,” Bhatia said. “You have to get this into the model’s memory, which is a really complicated problem.”

To overcome this, GE Healthcare built on Amazon SageMaker, which provides high-speed networking and distributed training capabilities across multiple GPUs, and leveraged Nvidia A100 and Tensor Core GPUs for large-scale training.

“Due to the data size and model size, we cannot send it to a single GPU,” Bhatia explained. SageMaker allowed us to customize and scale jobs across multiple GPUs that could interact with each other.

Additionally, developers were able to read and write datasets faster using Amazon FSx on Amazon S3 object storage.

Bhatia pointed out that another challenge is cost optimization. Amazon’s Elastic Compute Cloud (EC2) allowed developers to move unused or infrequently used data to lower-cost storage tiers.

“Leveraging Sagemaker to train these large-scale models, primarily for efficient, distributed training across multiple high-performance GPU clusters, has been one of the critical components that has really helped us move faster,” said Bhatia.

He emphasized that all components are built from a data integrity and compliance perspective, taking into account HIPAA and other regulatory regulations and frameworks.

Ultimately, “These technologies can really help us streamline and innovate faster, as well as reduce administrative burden, improve overall operational efficiency and ultimately drive better patient care. Now we can provide more personalized care.” Because there is.”

Serves as a basis for other specialized fine-tuning models

Although the current model is limited to the MRI domain, researchers see great opportunity for expansion into other areas of medicine.

Sheeran noted that historically, AI in medical imaging has been constrained by the need to develop customized models for specific conditions in specific organs and thus the need for expert annotation for each image used for training.

However, this approach is “inherently limited” and poses generalization problems because the way the disease manifests itself varies in each individual.

“What we really need is thousands of such models and the ability to quickly generate new models as we encounter new information,” he said. High-quality labeled datasets for each model are also essential.

Now, generative AI allows developers to pre-train a single base model that can serve as the basis for other specialized fine-tuned models downstream, rather than training separate models for each disease/organ combination.

For example, GE Healthcare’s model could be extended to areas such as radiation therapy, where radiologists spend significant time manually marking organs that may be at risk. It could also help reduce scanning times during X-rays and other procedures that currently require patients to sit still in machines for long periods of time, Bhatia said.

“We’re not just expanding access to medical imaging data through cloud-based tools,” Sheeran said. “We are changing how we leverage that data to drive AI advancements in healthcare.”