AI in Medical Imaging (MRI / DICOM / neuroimaging)

Verified multi-source research synthesis (5 rounds, Jun/2026) + empirical PoC on a real case (petroclival meningioma, repo rodrigo/mri-paula). Focus on segmentation, volumetry, and 3D visualization of brain MRI. Axes: open×paid, self-hosted×API.

Open + self-hosted (mature)

Brain structures + volumetry (FreeSurfer family — gold standard)

  • FastSurfer (Apache-2.0) — CNN; whole-brain 95 classes (DKT atlas) + CerebNet (cerebellum) + HypVINN (hypothalamus); volumetry in minutes, cortical thickness ~1h. https://github.com/Deep-MI/FastSurfer
  • SynthSeg (Apache-2.0) — contrast/resolution-agnostic (up to 10mm slice, no retraining); v2.0 = cortical parcellation + mm³ volumes + ICV + QC. arXiv:2107.09559
  • FreeSurfer — recon-all pipeline (subcortical: hippocampus, amygdala, thalamus + surface). Proprietary non-OSI license.

DICOM + 3D platform

  • 3D Slicer (BSD) — full DICOM (DICOMwebDIMSE), 2D3D4D segmentation, best open 3D-render hub. <https:/slicer.org>

Segmentation frameworks

  • nnU-Net (Apache-2.0) — self-configuring, SOTA; basis of TotalSegmentator. Nature Methods 2021.
  • TotalSegmentator (Apache-2.0) — >100 structures CT+MR; brain in MR is weak (brain_structures task is CT-only/licensed).

Medical foundation models (2024-2026, promptable)

Model License Highlight Brain limitation
BiomedParse (MS, Nature Methods 2024) open text-prompt, 82 objects, 9 modalities semantic recognition, not diagnostic
MedSAM2 (2025) Apache-2.0 3D+video (SAM2.1), public weights+data aggregate metrics, not brain-specific
SAM-Med3D open 3D by points, 247 categories mediocre brain performance vs. specialized
VISTA3D (NVIDIA/MONAI, CVPR 2025) uncertain 127 classes 3D auto+interactive CT-centric

Foundation models via API/cloud (all = dev tool, NO clearance)

  • Google MedGemma 1.5 — open-weights (HAI-DEF), 1st public LLM to interpret 3D CT/MRI volumes; HF + Vertex. MedLM (text) deprecated 2025-09-29.
  • Microsoft MedImageParse 3D (Azure AI Foundry) — 3D CT/MRI segmentation by text, 16 categories incl. "brain anatomies"; open-weights tier + premium serverless.
  • MedImageInsight — embeddings 9 modalities (incl. MRI), 2D-slice.
  • NVIDIA VISTA-3D NIM / MONAI Cloud APIs — CT-centric, research-only.
  • AWS HealthImaging — DICOM data-plane, not analysis.

None has FDA/CE registration — not for clinical use.

  • Cortechs.ai NeuroQuant — FDA 510(k) K170981K241098, CE; 3D T1 volumetryneurodegeneration. NeuroQuant Brain Tumor: dedicated FDA product for meningiomametastasisglioma + longitudinal volume tracking.
  • icometrix icobrain — FDA K181939 + CE-MDR; MS/white-matter lesions; hybrid edge-to-cloud access (on-prem icobridge connector pseudonymizes before egress).
  • Siemens AI-Rad Companion Brain MR, Brainreader Neuroreader (FDA K140828) — volumetry/morphometry.
  • Unconfirmed (rate-limited during verification): Combinostics, Pixyl.Neuro, Quantib/DeepHealth, Qynapse, Qure.ai qMRI.
  • CT, outside MRI: Viz.ai, RapidAI, Aidoc, Brainomix (stroke in CT/CTA).

Structural gaps

  • Vessels/Circle of Willis in MRA: usable — TopCoW (PMID 38235066) + nnU-Net (Dice ~0.90) + eICAB (14 segments). Aneurysm in MRA still immature.
  • Cranial nerves: immature research — only DTI/dMRI tractography (MRtrix3).
  • Bone/skull base in MRI: MR is poor for bone → synthetic-CT (radiotherapy: Philips MRCAT, Spectronic). Gap.

Empirical PoC (real case — petroclival meningioma ~5.2 cm)

Repo rodrigo/mri-paula (poc-segmentacao/). Central finding: every generic automatic/promptable method fails on a skull-base tumor contiguous with the venous sinus it invades:

Method Volume Largest axis (report: 5.2 cm)
Naive region-grow 28–56 cm³ 6.8–22.8 cm (leaks)
Classic vessel suppression (TOF+SWI) collapses/1127 cm³ fragile
MedSAM2 (3D box) 42–50 cm³ 8.2–8.5 cm (overflows)
MONAI brats SegResNet (4-seq) 59 cm³ 8.2 cm (best localized; inflates w/ peritumoral)
BiomedParse (text "meningioma") 4.1 cm³ 22.4 cm (sparse fragments = miss)
grow-from-seeds (random walker) 48.5 cm³ 4.9 cm (✅ ~6% of report, no leak)

Verdict: empirically confirmed — grow-from-seeds (tumor+background seeds, headless equivalent of 3D Slicer) was the only one that didn't leak AND matched the report measurement (4.9 vs. 5.2 cm). Every generic automatic/promptable method fails (over-segments or disperses). Clinical alternative: NeuroQuant Brain Tumor (FDA). A trained model (MONAI) localizes better than a generic promptable one; the ideal would be meningioma-specific (BraTS-MEN — no public weights). Detail + scripts + masks: poc-segmentacao/RESULTS-medsam2.md.

Engineering lessons (PoC, T4/py3.13)

  • py3.13 wheels force torch cu124 (cu130 breaks w/ driver 12.4); BiomedParse requires

    detectron2+transformers+safetensors+timm and cuDNN off on the T4 (CUDNN_STATUS_NOT_INITIALIZED).

  • Gated models on HF: classic Read token (fine-grained inference-only gives 403) + accept the license on the account that owns the token.