Reference library
Distilling the Knowledge in a Neural Network
Paper summary
Transfers predictive information from an ensemble or large teacher to a smaller model using softened output distributions. Experiments show that a compact student can retain useful knowledge beyond training only on hard labels.
Relevance to the project
A future model-compression reference and a comparison for transmission of behavior rather than exact structure. Teacher, data, and loss remain explicit external machinery.
Distillation is not biological inheritance or spontaneous reproduction.
Source record
- Reference ID
- distillation
- Archived version
- 1503.02531v1
- Summary provenance
- reference-library/papers/distillation/metadata.json · summary
- Review status
- initial orientation summary; not a critical full-paper review
- Reading scope
- Selected abstract/opening and available closing discussion/conclusion passages in the preserved text extract; broader project notes provide context. No reproduction of experiments, exhaustive methods review, or equation/figure audit.