Reference library
RWKV: Reinventing RNNs for the Transformer Era
Paper summary
RWKV combines time-based recurrent mixing with a formulation supporting parallel training. It aims for constant per-step inference state and computation while retaining language-model capability at scale.
Relevance to the project
An architecture comparison for efficient stateful mechanisms. Distinguish parallelism over neurons from the source's parallel training over sequence positions.
Constant per-token inference cost assumes a fixed model and state size.
Source record
- Reference ID
- rwkv
- Archived version
- 2305.13048v2
- Summary provenance
- reference-library/papers/rwkv/metadata.json · summary
- Review status
- initial orientation summary; not a critical full-paper review
- Reading scope
- Selected abstract/opening and available closing discussion/conclusion passages in the preserved text extract; broader project notes provide context. No reproduction of experiments, exhaustive methods review, or equation/figure audit.