Enhancing Brain MRI Anomaly Detection and Reasoning with ROI Rethink and Synthetic Data

Shangkun Li1, Jie Xu1, Yi Guo1, Zeju Li1, Yuanyuan Wang1
1Fudan University
MICCAI 2026 (Early Accept)
Paper Code Model

Abstract

Medical vision-language models typically generate diagnoses through single-pass inference without indicating which image regions support their conclusions. This lack of spatial grounding limits clinical utility: outputs cannot be audited, and models may hallucinate findings on normal scans. We present BrReMark (Brain Rethink via ROI Marking), a framework that introduces explicit region marking into brain MRI diagnosis. The model first generates hypotheses about potential abnormalities and grounds them through explicit bounding box marking, then verifies conclusions by re-examining the marked evidence. Training combines supervised fine-tuning on structured reasoning trajectories with reinforcement learning using a composite reward over localization accuracy and diagnostic reasoning. Furthermore, we integrate a domain randomization-based pathology synthesis augmentation strategy to improve the model's generalizability to out-of-distribution (OOD) data. On internal benchmark, BrReMark improves mAP50 from 0.74% to 37.54% compared to the base model, while achieving 21.57 Clinical F1 and 45.26% diagnostic accuracy. On NOVA OOD benchmark, it also achieves competitive overall performance with a 45.7% reduction in false positives compared to the state-of-the-art, indicating reduced hallucination on rare pathologies.

Method

BrReMark Framework

Left: Data curation and training pipeline. he training data consists of real slices from seven public brain MRI datasets (including both normal and abnormal cases), and synthetic slices generated via SynthSeg with EDT-weighted pathology injection. Right: BrReMark inference pipeline. Given a brain MRI and query, the model first generates a hypothesis and invokes mark_bbox to localize suspicious regions (Turn 1), then re-examines the marked image to verify findings and produce a grounded diagnosis (Turn 2).

Two-Turn Mark-and-Rethink Paradigm

We formulate brain MRI diagnosis as a two-turn visual dialogue that mirrors clinical reasoning. In Turn 1, the model proposes a hypothesis and generates a bounding box to localize the suspicious region (or null for normal images). In Turn 2, the marked image is fed back — the model re-examines the highlighted region to verify or revise its hypothesis, producing a grounded final diagnosis with an auditable reasoning trace.

Two-Stage Training

Stage 1 (SFT): Teaches the model the two-turn format and structural syntax (<think>, <tool_call>, <rethink>, <answer>) using Gemini-generated reasoning chains.

Stage 2 (GRPO): Aligns reasoning with clinical requirements via a composite reward: format compliance (rfmt), IoU-based localization (rloc), and LLM-as-judge semantic evaluation (rllm). Clinical safety constraints include hallucination penalty, modality gating, and synthetic reward masking.

Synthetic Pathology Injection

To enhance OOD generalization, real lesion masks are injected into healthy brain label maps via EDT-weighted sampling, then rendered into domain-randomized MRI volumes. Synthetic data is used only in RL for spatial targeting, with rllm explicitly masked to prevent clinical hallucinations.

Main Results

We evaluate BrReMark across three clinical tasks: Anomaly Localization, Image Description, and Differential Diagnosis on both in-distribution and OOD (NOVA) benchmarks. Despite with a compact size of 7B parameters, BrReMark achieves state-of-the-art performance across all tasks.

Table 1: Anomaly Localization Results Table 2: Description and Diagnosis Results

Key Findings

  • Anomaly Localization: Achieves mAP30 of 64.93 on our benchmark, outperforming Qwen2.5-VL-72B by +9.77 points. On the OOD NOVA benchmark, reaches mAP30 of 33.05.
  • Hallucination Suppression: Reduces detection false positives from 1,782 to 812 (45.7% reduction). Clinical safety score improves from 61.64% to 66.27%.
  • Diagnostic Accuracy: Achieves 45.26% diagnostic accuracy, surpassing Lingshu-32B (45.07%), with a reasoning quality score of 60.30.
  • OOD Generalization: Synthetic pathology injection drives mAP30 from 5.24 (Lingshu-7B baseline) to 33.05 on NOVA.

Citation

@inproceedings{li2026brremark,
  title={Enhancing Brain MRI Anomaly Detection and Reasoning with ROI Rethink and Synthetic Data},
  author={Li, Shangkun and Xu, Jie and Guo, Yi and Li, Zeju and Wang, Yuanyuan},
  booktitle={International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI)},
  year={2026}
}