MedVision Logo MedVision: Benchmarking Quantitative Medical Image Analysis

Yongcheng Yao1 , Yongshuo Zong1 , Raman Dutt1 , Yongxin Yang3 ,
Sotirios A Tsaftaris2 , Timothy Hospedales1
1School of Informatics, University of Edinburgh
2School of Engineering, University of Edinburgh
3Queen Mary University of London
arXiv GitHub Docs Data Models Demo Project Data Explorer

MedVision is a dataset and benchmark for quantitative medical image analysis, including detection, tumor/lesion (T/L) size estimation, and angle/distance (A/D) measurement tasks.

All posts

Measuring What Matters: 50× More Tumor & Lesion Measurements

When a doctor follows a cancer, one of the most important questions is also the simplest: how big is it? Size helps decide the stage of a disease, the choice of treatment, and, scan after scan, whether that treatment is working.

MedVision tests whether AI models can answer that question from medical images. To grade an AI, you need an answer key: a large collection of images where the true size of each tumor or lesion is already known. Dataset v1.4.0 rebuilds how the tumor/lesion measurements are made for all 12 of our tumor and lesion collections, and the result is dramatic: the number of measurements grows from about 76 thousand to 3.8 million, fifty times more.

Where the measurements come from

Each scan in these collections comes with outlines drawn by experts marking diseased tissue. For every outlined spot, we fit an oval around it and record two numbers: its longest diameter and its width across, in millimeters, imitating how a radiologist sizes a tumor in the clinic. Those numbers are what an AI model is graded against.

The fitting is automatic, so the rules deciding what gets measured and what counts as a trustworthy measurement shape the whole dataset. v1.4.0 rethinks those rules.

Why we rebuilt the measurement scheme

Nothing in the earlier versions was wrong. Every measurement they published was sound, and it still is. But the earlier rules were cautious in ways that left many real lesions unmeasured, and how many were being left out had never been evaluated. The new scheme is built around three principles.

Decide what to measure in millimeters, not pixels. Medicine sizes disease in physical units, so the measurements should be selected in physical units too. The earlier scheme measured a spot only if it covered enough pixels, and a pixel is not a fixed physical size: it varies from scanner to scanner and even between viewing directions within a single 3D scan. The very same pixel rule could ask for about 5 mm from one direction of a scan but almost 9 mm from another, so real lesions in between went unmeasured. The new scheme measures any spot that is roughly 2 millimeters across, adjusted to what each scan can actually resolve. One standard, applied equally to every image.

Judge a measurement by its quality, not its orientation. The earlier scheme kept an extra caution test that was, in practice, easiest to pass for shapes lined up with the image grid. Real tumors usually sit at an angle, so many perfectly good measurements of tilted lesions were set aside. The new scheme drops that test, and most of the 50× growth comes from welcoming those tilted lesions back.

Check every number against physical common sense. Fitting millions of shapes automatically means rare glitches are inevitable. The new scheme adds four independent sanity checks, including that nothing can be thinner than a single pixel of the image and nothing can be longer than its own region allows.

Just as important, v1.4.0 asks a question no earlier version had answered: of everything the experts outlined, how much actually receives a measurement? The next two sections show the new measurements and that coverage.

What 3.8 million measurements look like

Box plot of longest diameter distributions for all 38 tumor and lesion types in MedVision v1.4.0

All 3.8 million tumor/lesion measurements at a glance. Each row is one of the 38 tumor or lesion types; the box shows where the middle half of its measurements fall and the whiskers span the central 90%, in millimeters.

Typical sizes range from about 8 mm for the smallest brain tumor cores to over 4 cm for kidney tumors, and every type spreads across a wide range; the question “how big is it?” has a different answer for every disease. Even the extremes checked out: the single largest measurement, 54 cm, turned out to be genuine disease spread along the length of the body in a scan covering the whole body, not a glitch.

How much did we measure?

We counted, for the first time. Across the 12 collections the expert outlines contain about 7 million separate spots, and 55% of them now carry a measurement. That may sound low until you look at what the unmeasured ones are: almost all are tiny spots a few pixels across. A quarter of everything outlined is just one or two pixels, fragments too small for any scan to size meaningfully, let alone a clinician.

Coverage of tumor and lesion measurements by type, size, and collection in MedVision v1.4.0

How much of the outlined disease carries a measurement (i.e., recall). Coverage climbs steeply with size: for tumors and lesions at least 20 millimeters across, it is above 97% in every one of the 12 collections, and 99% overall.

In other words, what goes unmeasured is stray specks in the outlines: once a lesion reaches a size that matters in the clinic, it is almost certainly measured.

What it means for you

If you use MedVision, the images are exactly the ones you already have; only the tumor/lesion measurements were rebuilt, and they are now far larger in number, fairer across scans, and free of impossible values. Every earlier version (<1.4.0) remains available unchanged, so past results stay reproducible. For the technically inclined, the full release note has every rule, number, and verification behind this post.

Acknowledgements

This work was supported by the United Kingdom Research and Innovation (grant EP/S02431X/1), UKRI Centre for Doctoral Training in Biomedical AI at the University of Edinburgh, School of Informatics; the Edinburgh International Data Facility (EIDF) and the Data-Driven Innovation Programme at the University of Edinburgh.

BibTeX

@misc{yao2026medvisionbenchmarkingquantitativemedical,
    title={MedVision: Benchmarking Quantitative Medical Image Analysis}, 
    author={Yongcheng Yao and Yongshuo Zong and Raman Dutt and Yongxin Yang and Sotirios A Tsaftaris and Timothy Hospedales},
    year={2026},
    eprint={2511.18676},
    archivePrefix={arXiv},
    primaryClass={cs.CV},
    url={https://arxiv.org/abs/2511.18676}, 
}