Reference library
Measuring Massive Multitask Language Understanding
Paper summary
Introduces a broad multiple-choice benchmark spanning academic and professional subjects. Initial experiments expose uneven accuracy and poor awareness of errors even in large language models.
Relevance to the project
Relevant to optional LLM evaluation and the need for multiple observables. A single aggregate score can conceal weak domains or misleading confidence.
A fixed knowledge benchmark does not measure self-maintenance, emergence, or lifetime learning.
Source record
- Reference ID
- mmlu
- Archived version
- 2009.03300v3
- Summary provenance
- reference-library/papers/mmlu/metadata.json · summary
- Review status
- initial orientation summary; not a critical full-paper review
- Reading scope
- Selected abstract/opening and available closing discussion/conclusion passages in the preserved text extract; broader project notes provide context. No reproduction of experiments, exhaustive methods review, or equation/figure audit.