Reference library
Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks
Paper summary
Attention-based set encoders model interactions between elements. SAB/ISAB are equivariant; pooling produces an invariant output. Inducing points give O(nm) attention rather than full O(n²). The max-regression table favors max pooling over the attention model, cautioning against blanket superiority.
Relevance to the project
Assistant interpretation: candidate ordering and contextual competition are separate questions for a neural connection selector. Compare shared independent scoring with contextual set processing before choosing a mechanism. No selector architecture or training objective is adopted.
No guarantee of useful rewiring, evolutionary discoverability, stability or GPU speed. Linear scaling assumes fixed inducing-point count. Universality proof not reviewed; approximation claims are not lossless fixed-width encoding guarantees.
Source record
- Reference ID
- pmlr-lee19d
- Archived version
- Published ICML 2019, PMLR 97:3744–3753
- Summary provenance
- reference-library/papers/pmlr-lee19d/metadata.json · summary
- Review status
- focused primary full-text methods reading; not exhaustive critical review
- Reading scope
- Focused published full-text reading: introduction, sections 2–3 (attention blocks, symmetry, complexity and stated universality), section 4 cautions and section 5 opening/Table 1. Local opening page identity cross-check. Supplementary proofs, remaining experiments and figures not critically reviewed; no independent replication.