RESEARCHInvestigateNOW
Don't Claim Benchmark-Oriented Optimization Improves General Coding Capability -- Diverse Evaluation Is Required
arXiv cs.LG — Machine Learning
Factual evidence
What the source reports
New research shows benchmark optimization on SWE-bench and LiveCodeBench fails to reflect real-world enterprise coding capabilities.
Open source