Reference library
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
Paper summary
Transformer-XL carries prior-segment representations through segment-level recurrence and uses a compatible relative positional scheme. Evaluations show better long-range language modeling and reduced repeated computation during inference.
Relevance to the project
A comparison for bounded directly accessible histories with information carried across time. Specify which snapshots are retained and how stale state is interpreted.
Language dependency length is not a guarantee of indefinite causal memory.
Source record
- Reference ID
- transformer-xl
- Archived version
- 1901.02860v3
- Summary provenance
- reference-library/papers/transformer-xl/metadata.json · summary
- Review status
- initial orientation summary; not a critical full-paper review
- Reading scope
- Selected abstract/opening and available closing discussion/conclusion passages in the preserved text extract; broader project notes provide context. No reproduction of experiments, exhaustive methods review, or equation/figure audit.