Reference library
Are Transformers Universal Approximators of Sequence-to-Sequence Functions?
Paper summary
Establishes universal approximation results for continuous sequence-to-sequence functions by Transformers. Attention builds contextual mappings, while positional encodings remove the restriction to permutation-equivariant functions under the theorem's conditions.
Relevance to the project
Helps compare representational assumptions when choosing how F accesses incoming slots and histories. Explicit order or identity changes what distinctions a mechanism can express.
The theorem does not guarantee efficient training or a practical network size.
Source record
- Reference ID
- arxiv-1912-10077
- Archived version
- arXiv preprint; retrieved version indicated in PDF
- Summary provenance
- reference-library/papers/arxiv-1912-10077/metadata.json · summary
- Review status
- initial orientation summary; not a critical full-paper review
- Reading scope
- Selected abstract/opening and available closing discussion/conclusion passages in the preserved text extract; broader project notes provide context. No reproduction of experiments, exhaustive methods review, or equation/figure audit.