RESEARCHMonitorNEXT 12 MONTHS
What Does MMLU Actually Measure? A Psychometric Audit of Difficulty Structure in Aggregate Benchmark Scores
arXiv cs.CL — Computation and Language
Factual evidence
What the source reports
Item Response Theory analysis of 1,000 models reveals aggregate MMLU scores measure factual retrieval capacity rather than reasoning ability.