Reference library
Gated Delta Networks: Improving Mamba2 with Delta Rule
Paper summary
Gated DeltaNet combines adaptive memory gating with delta-rule updates in a linear recurrent architecture. Gating enables forgetting while targeted updates manage associations; a parallel algorithm supports hardware-efficient training.
Relevance to the project
A concrete memory-update comparison for bounded histories and mutable state. Test retention, forgetting, dynamics, and compute separately from language benchmark accuracy.
Externally trained recurrent memory is not a validated evolutionary mechanism.
Source record
- Reference ID
- gated-deltanet
- Archived version
- 2412.06464v3
- Summary provenance
- reference-library/papers/gated-deltanet/metadata.json · summary
- Review status
- initial orientation summary; not a critical full-paper review
- Reading scope
- Selected abstract/opening and available closing discussion/conclusion passages in the preserved text extract; broader project notes provide context. No reproduction of experiments, exhaustive methods review, or equation/figure audit.