MathLLMs/MathVision
Dataset Β· https://huggingface.co/datasets/MathLLMs/MathVision
Measuring Multimodal Mathematical Reasoning with the MATH-Vision Dataset [π» Github] [π Homepage] [π Main Leaderboard ] [π Open Source Leaderboard ] [πΏ Wild Leaderboard ] [π Visualization] [π Paper] πΏ NEW: MATH-Vision-Wild MATH-Vision-Wild is a photographic, real-world variant of MATH-Vision. The same testmini problems are physically captured on printed paper, iPads, laptops, and projectors under varying lighting and angles β the conditions VLMs actuallyβ¦ See the full description on the dataset page: https://huggingface.co/datasets/MathLLMs/MathVision.
First seen by Oriel on .
AI summary
MATH-Vision is a benchmark of 3,040 visual mathematics problems showing a substantial gap between multimodal models and humans, with MATH-Vision-Wild exposing further photographic robustness issues.
- The dataset covers 16 mathematical disciplines and five difficulty levels, using high-quality problems sourced from real math competitions.
- MATH-Vision-Wild re-captures problems on printed paper, iPads, laptops, and projectors under varying lighting and angles.
- Among listed models, only o4-mini improved on photographed problems, reaching 57.2%; other models declined by 12.35% to 52.58%.
- The dataset provides Hugging Face loading examples, leaderboards, visual examples, and ongoing corrections for reported question errors.
- The benchmark evaluates multimodal mathematical reasoning across subjects and is intended for researchers comparing LMM capabilities and robustness.
#multimodal reasoning #ζ°ε¦ benchmarks #computer vision #large multimodal models
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days Β· Up to 120 recorded samples per source and metric, with reposts kept separate Β· First and latest retained
Likes
Total likes reported by the source.
- Hugging Face Datasets: 182, latest observation ; 0 over observed period since
Downloads (30d)
Rolling 30-day downloads. A change in this value is not new downloads during the observed period.
- Hugging Face Datasets: 10,064, latest observation ; +17 over observed period since
Aliases
| Source | External ID | URL |
|---|---|---|
| Hugging Face Datasets | MathLLMs/MathVision | https://huggingface.co/datasets/MathLLMs/MathVision Read discussion |