Reference library
Self-Normalizing Neural Networks
Paper summary
The paper constructs feedforward self-normalizing networks using SELU activations and weight statistics so that a mean/variance map has an attracting fixed point within a stated domain. The mapping uses independent inputs sharing moments and an approximately Gaussian weighted sum motivated by the central limit theorem. It handles restricted deviations of weight sum and squared norm, and presents computer-assisted contraction reasoning whose detailed proof is in supplementary material. Its normalization target concerns activation moments across samples/layers.
Relevance to the project
Useful for stating the covariance and activation assumptions behind variance-based scaling rather than importing a feedforward rule directly into a recurrent plastic network. A fixed point of a moment map is not a guarantee of nonconstant individual trajectories, and SELU-specific results do not silently choose a new F/H activation or numerical gain.
Independence/Gaussian assumptions can fail with narrow correlated recurrent stages; the stated moment and weight domains constrain the theorem. Only published main-paper selected methods/proof sketch read, not supplementary proof or experiments. No claim of strict proof validation, stability with arbitrary online plastic weights, or absence of repeated trajectories.
Source record
- Reference ID
- self-normalizing-networks-2017
- Archived version
- Published NeurIPS 2017 proceedings PDF, 10 pages; supplementary proof payload not acquired
- Summary provenance
- reference-library/papers/self-normalizing-networks-2017/metadata.json · summary
- Review status
- Targeted full-text methods review; neither exhaustive full-paper reading nor independent reproduction
- Reading scope
- PDF pages 3–5 read in full, covering Definition 1, SELU construction, the mean/variance map, Theorem 1 and its proof sketch, and Theorem 2 statement. Front matter checked. Pages 1–2 orientation/figures, remainder of paper and supplementary detailed proofs not critically reviewed; no proof verification or reproduction.