Reference library
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
Paper summary
Derives activation-dependent initialization for deep rectifier networks and evaluates PReLU image classifiers. The forward calculation uses input second moments because rectifier outputs are not centered. Independence and symmetric preactivations yield variance 2/fan-in for ReLU; changing the negative slope changes the factor.
Relevance to the project
Assistant interpretation: a gain cannot be selected independently of activation and signal distribution. Supervised CNN results do not determine a recurrent plastic network's gain.
Reading is focused, not exhaustive; no independent replication. Focused full-text reading of PReLU definition and section 2.2 forward/backward initialization derivation; selected convergence experiment discussion. No exhaustive ImageNet, figure or implementation audit.
Source record
- Reference ID
- cvf-he2015-rectifiers
- Archived version
- Published ICCV 2015 open-access proceedings PDF
- Summary provenance
- reference-library/papers/cvf-he2015-rectifiers/metadata.json · summary
- Review status
- focused primary full-text methods reading; not exhaustive critical review
- Reading scope
- Focused full-text reading of PReLU definition and section 2.2 forward/backward initialization derivation; selected convergence experiment discussion. No exhaustive ImageNet, figure or implementation audit.