MedVision Logo MedVision: Benchmarking Quantitative Medical Image Analysis

Yongcheng Yao1 , Yongshuo Zong1 , Raman Dutt1 , Yongxin Yang3 ,
Sotirios A Tsaftaris2 , Timothy Hospedales1
1School of Informatics, University of Edinburgh
2School of Engineering, University of Edinburgh
3Queen Mary University of London
arXiv GitHub Docs Data Models Demo Project

MedVision is a dataset and benchmark for quantitative medical image analysis, including detection, tumor/lesion (T/L) size estimation, and angle/distance (A/D) measurement tasks.

🗂️ Images & Annotations

MedVision consolidates 22 public medical imaging datasets — BraTS24, MSD, OAIZIB-CM and others — into one uniformly structured resource: 29K 3D images and 11.2M annotated 2D slices, carrying 24.3M single-instance annotations across the three quantitative tasks — or 45.3M multi-instance annotations with the per-sample quality/size filters lifted. Neither figure counts instances: several boxes or clusters of the same target on one slice are one annotation in both. The imaging spans five modalities — X-ray (XR), CT, MRI, ultrasound (US) and PET — across many anatomical regions.

Source images are kept as 3D volumes reoriented to RAS+ (a canonical right-anterior-superior axis convention), which makes plane definitions consistent across datasets that were originally stored with different orientations. Because most vision-language models consume 2D images, MedVision does not ship pre-cut slices: the loader slices volumes to 2D on the fly along any of the three anatomical planes — axial, coronal or sagittal — at load time. This keeps the on-disk footprint tied to the volumes themselves (a full copy is around 1 TB) rather than to an exploded set of PNGs.

Segmentation masks. Every dataset except Ceph-Biometrics-400 ships with segmentation masks: dense manual ground truth drawn by expert annotators, and the source of the label names shown in each task’s label map below. To download the image and mask files, load any of a dataset’s detection configs — MedVision distributes only the annotations, and the loader fetches and preprocesses the raw imaging into the dataset folder you specify.

Read more in the documentation: what MedVision holds · the four annotation types · multi-instance vs single-instance annotations

📊 Dataset Statistics

Annotation counts per dataset for annotation v1.1.1, across the three quantitative tasks — detection (Box), tumor/lesion size (T/L), and biometrics (A/D) — with an enlarged panel for the smaller datasets. The two sets differ only by filtering: single-instance keeps a target only when it is a single, large-enough instance, while multi-instance keeps every annotated target whatever its instance count or size.

Single-instance annotation counts per dataset across the MedVision benchmark (annotation v1.1.1)
Multi-instance annotation counts per dataset across the MedVision benchmark (annotation v1.1.1)

🔎 Dataset Explorer

Narrow the MedVision benchmark to the subset you need, then copy the exact loading command. Pick a body part, choose one or more anatomy labels, and select an imaging modality — the explorer lists the datasets that fit, with a ready-to-run load_dataset(...) snippet for each matching test config. Covers the three quantitative tasks: detection (bounding box), tumor/lesion size, and biometrics (angle/distance).

Acknowledgements

This work was supported by the United Kingdom Research and Innovation (grant EP/S02431X/1), UKRI Centre for Doctoral Training in Biomedical AI at the University of Edinburgh, School of Informatics.

BibTeX

@misc{yao2026medvisionbenchmarkingquantitativemedical,
    title={MedVision: Benchmarking Quantitative Medical Image Analysis}, 
    author={Yongcheng Yao and Yongshuo Zong and Raman Dutt and Yongxin Yang and Sotirios A Tsaftaris and Timothy Hospedales},
    year={2026},
    eprint={2511.18676},
    archivePrefix={arXiv},
    primaryClass={cs.CV},
    url={https://arxiv.org/abs/2511.18676}, 
}