Reference library
On the difficulty of training recurrent neural networks
Paper summary
Analyzes vanishing and exploding gradients in recurrent networks through geometric and dynamical arguments. Gradient-norm clipping and a soft constraint are proposed and evaluated as training remedies.
Relevance to the project
A stability reference if mechanism networks are trained externally. It also motivates measuring recurrent sensitivity, although gradient difficulty differs from intrinsic activity failure.
Clipping training gradients does not by itself bound autonomous runtime state.
Source record
- Reference ID
- pmlr-pascanu13
- Archived version
- Publisher/institution PDF advertised by article metadata
- Summary provenance
- reference-library/papers/pmlr-pascanu13/metadata.json · summary
- Review status
- initial orientation summary; not a critical full-paper review
- Reading scope
- Selected abstract/opening and available closing discussion/conclusion passages in the preserved text extract; broader project notes provide context. No reproduction of experiments, exhaustive methods review, or equation/figure audit.