Reference library
Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research
Paper summary
Introduces Dolma, a large openly documented text corpus, with source mixtures, filtering, and construction details. Intermediate-data analyses and training experiments make data choices more inspectable.
Relevance to the project
A reproducibility example for recording inputs and processing stages. Relevant mainly to optional language-model research rather than present simulation inputs.
The paper is preserved locally; the full dataset has not been downloaded.
Source record
- Reference ID
- dolma
- Archived version
- 2402.00159v2
- Summary provenance
- reference-library/papers/dolma/metadata.json · summary
- Review status
- initial orientation summary; not a critical full-paper review
- Reading scope
- Selected abstract/opening and available closing discussion/conclusion passages in the preserved text extract; broader project notes provide context. No reproduction of experiments, exhaustive methods review, or equation/figure audit.