Reference library
Retentive Network: A Successor to Transformer for Large Language Models
Paper summary
RetNet combines parallel training, recurrent inference, and chunkwise recurrent processing through a retention mechanism. Language-model experiments evaluate this alternative to conventional Transformer attention.
Relevance to the project
A comparison for balancing stateful computation and GPU parallelism. Chunk boundaries and carried state could inform future performance prototypes.
Sequence-model throughput is not evidence of intrinsic sustained neural activity.
Source record
- Reference ID
- ret-net
- Archived version
- 2307.08621v4
- Summary provenance
- reference-library/papers/ret-net/metadata.json · summary
- Review status
- initial orientation summary; not a critical full-paper review
- Reading scope
- Selected abstract/opening and available closing discussion/conclusion passages in the preserved text extract; broader project notes provide context. No reproduction of experiments, exhaustive methods review, or equation/figure audit.