Reference library
Understanding the difficulty of training deep feedforward neural networks
Paper summary
Studies initialization and nonlinearities in deep supervised feedforward networks. The initial input-count uniform heuristic differs from the proposed fan-in/fan-out compromise. Variance reasoning assumes approximately linear activations and independence; logistic saturation and depth affect training. Classification experiments support the proposed scheme in those settings.
Relevance to the project
Assistant interpretation: provides a precise comparison for configurable input-count scaling and cautions about saturation. It does not establish autonomous dynamics or preservation after plastic updates.
Reading is focused, not exhaustive; no independent replication. Focused full-text reading of initialization experiments, activation/saturation discussion and sections 3–4 variance derivation, including equations 1 and 16; not an exhaustive experimental or figure audit.
Source record
- Reference ID
- pmlr-glorot10a
- Archived version
- Published AISTATS 2010, PMLR 9:249–256
- Summary provenance
- reference-library/papers/pmlr-glorot10a/metadata.json · summary
- Review status
- focused primary full-text methods reading; not exhaustive critical review
- Reading scope
- Focused full-text reading of initialization experiments, activation/saturation discussion and sections 3–4 variance derivation, including equations 1 and 16; not an exhaustive experimental or figure audit.