Nvidia is enhancing its medical AI and digital twin capabilities

Nvidia is enhancing its medical AI and digital twin capabilities ...

Nvidia has been a leader in providing AI and digital twin infrastructure for the medical community. Its various offerings improve diagnostics, the development of new medical devices, medical research, and drug development. At the Fall GTC Conference, Nvidia announced several new medical tools, partnerships, and workflows.

Kimberly Powell, Nvidia's vice president of healthcare, said the GTC is a truly unique healthcare event. From things like surgery to pharmaceutical research, she explains how AI and accelerated computing are progressing the field.

Highlights include:

  • Release of MONAI 1.0, a new domain-specific AI framework that improves AI imaging workflows to medical diagnostics and robotics.
  • Migration of Clara Holoscan from MGX to IGX to simplify medical tool and robot development, deployment and management. 
  • BioNeMo extends Nvidia’s large language model (LLM) to support protein, DNA and chemical analysis workflows. 
  • Partnership with MIT’s and Harvard’s Broad Institute to accelerate human genomics research. 

These various announcements complement and extend each other. Let's go through them one at a time.

Medical imaging is simplified by MONAI.

In April 2020, Nvidia and King's College London introduced MONAI, a medical imaging software that allows for artificial intelligence to streamline workflows. This allows for raw imaging data to be transformed into interactive digital twins for analysis or diagnosis, or to guide surgical instruments. The platform has received over 600,000 downloads, half of them in the last six months.

Monai 1.0 is now available for download, with several essential features. Interactive labeling can cut by 75% the time required to label data for training AI models. Auto3D supports AutoML algorithms for automatically selecting machine learning models for 3D segmentation and interpretation. Model Zoo supports federated learning to increase the privacy of medical data.

The medical metaverse is industrialized by IGX.

Clara Holoscan MGX was introduced by Nvidia earlier this year as a reference design for a medical device platform. On top of Nvidia's new IGX platform for robotics, Clara Holoscan IGX greatly reduces the time it takes to integrate Holoscan into new products with integrated security and management capabilities.

Clara Holoscan MGX has provided equipment for over 70 leading companies, including Siemens Healthineers for MRI, Olympus for endoscopy, and Intuitive Ion for better lung biopsies. New Clara IGX products include Activ Surgical's hyperspectral blood flow imager, Moon Surgical's robotic-assisted surgeon, and Proximie's telepresence surgery system.

“We realized that what we're developing for these medical device use cases is actually applicable to a much wider market,” said Powell. “Industrial automation and smart factories all have a similar robotics pipeline that needs to be executed on the far edges of the network and incorporate things like functional safety so that humans and robots can be at the same time.”

Powell said the platform helps reduce the amount of latency needed for new applications, and that they've set the goal of keeping latency to 50 milliseconds. The latest version of Holoscan can do straight-up video processing in under 10 milliseconds and supports more than 30 simultaneously running AI algorithms at less than 50 milliseconds.

Powell said they are coordinating Clara with Nvidia's Isaac platform for robotics and Omniverse platform for industrial digital twins. "We're leveraging everything the company makes, and we're connecting these platforms together because robotics isn't unique in healthcare as it is in other fields," Powell said.

BioNeMo speaks proteins.

The new BioNeMo Framework from Nvidia aids medical researchers in preparing and modeling large biomolecular language models at supercomputing scales. It extends efforts like the Nvidia NeMo Megatron framework and research projects like AlphaFold that analyze proteins to support DNA, protein, and chemical research.

Each domain has its own distinct method of encoding data into strings, including DNA, proteins, and chemicals.

Powell said the world has over 10,000 illnesses but only 500 cures. "We need to enhance numerical and experimental methods with AI in order to explore the near infinite chemistry and protein space." The Nvidia BioNeMo LLM framework and cloud services will accelerate the development of artificial intelligence (AI) that understands chemistry and biology.

Four pretrained models are included in the new framework. ESM-1, developed by Meta AI Labs, processes amino acid sequences to model properties and functions. OpenFold helps visualize proteins. MegaMolBART can help anticipate chemical reactions, optimize mixtures, or generate new ones. ProtT5 extends the capabilities of protein large language models to sequence generation.

Powell said Nvidia is providing BioNeMo as both a framework and a service. The framework will help researchers develop new pre-trained language models at any scale for chemistry, protein, DNA, and RNA. It also supports data transformations necessary for biomolecules. In October, Nvidia plans to provide early access to the BioNeMo service.

The Nvidia-Broad collaboration accelerates innovation.

Nvidia has also announced a long-term collaboration with the Massachusetts Institute of Technology and Harvard University, a leading genetics research organization and tools supplier.

Clara Parabricks' computational genomics framework is being transferred to the Broad Institute's Terra cloud platform, used by 25,000 top medical researchers. Initially, they intend to support six new workflows. For example, a new whole genome sequencing workflow running on GPUs shortens the process from a day to an hour and cuts the cost in half compared to a CPU approach.

Nvidia is also adding a new deep learning model to the Broad Institute's genome analysis toolkit, which more than 100,000 researchers use.

Powell believes that combining the broad Institute's expertise with Nvidia's technology expertise might expedite the deployment of new AI medical innovations from years to months.

You may also like: