Reference library
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
Paper summary
Averages weights of models fine-tuned from a common pretrained starting point. Experiments find improved accuracy and robustness without ensemble inference costs when the solutions occupy compatible low-error regions.
Relevance to the project
A future comparison for combining learned mechanism candidates. Check parameter alignment and resulting dynamics before treating averaging as meaningful inheritance or recombination.
Averaging unrelated networks can fail; shared origin matters.
Source record
- Reference ID
- model-soups
- Archived version
- 2203.05482v3
- Summary provenance
- reference-library/papers/model-soups/metadata.json · summary
- Review status
- initial orientation summary; not a critical full-paper review
- Reading scope
- Selected abstract/opening and available closing discussion/conclusion passages in the preserved text extract; broader project notes provide context. No reproduction of experiments, exhaustive methods review, or equation/figure audit.