Reference library
SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model
Paper summary
SmolLM2 develops a small language model through extensive multi-stage training and specialized mathematical, code, and instruction data. Ablations and data-mixture refinement explain how corpus quality and allocation affect capability.
Relevance to the project
An optional small-model reference and example of data-versus-architecture controls. Small inference size can still hide a large external training investment.
The paper's models and datasets are not downloaded or adopted by this task.
Source record
- Reference ID
- smollm2
- Archived version
- 2502.02737v1
- Summary provenance
- reference-library/papers/smollm2/metadata.json · summary
- Review status
- initial orientation summary; not a critical full-paper review
- Reading scope
- Selected abstract/opening and available closing discussion/conclusion passages in the preserved text extract; broader project notes provide context. No reproduction of experiments, exhaustive methods review, or equation/figure audit.