Reference library
Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer
Paper summary
Maximal Update Parametrization supports transferring tuned hyperparameters from smaller neural networks to larger ones. Theory and experiments examine settings in which useful learning-rate and related choices remain stable with scale.
Relevance to the project
Potentially reduces future external-training sweeps for mechanism networks. Use explicit scaling conventions and validate transfer instead of assuming ordinary parameters scale unchanged.
The result does not select the engine's evolutionary procedure or runtime update scale.
Source record
- Reference ID
- muparam
- Archived version
- 2203.03466v2
- Summary provenance
- reference-library/papers/muparam/metadata.json · summary
- Review status
- initial orientation summary; not a critical full-paper review
- Reading scope
- Selected abstract/opening and available closing discussion/conclusion passages in the preserved text extract; broader project notes provide context. No reproduction of experiments, exhaustive methods review, or equation/figure audit.