Reference library
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Paper summary
BERT pretrains bidirectional Transformer representations using masked text and then adapts them to downstream language tasks. Joint access to left and right context improves the evaluated understanding tasks relative to earlier pretraining baselines.
Relevance to the project
Illustrates how access restrictions and training objectives shape representations. Its bidirectional context is an offline comparison, while causal temporal histories require a different availability contract.
Pretraining and fine-tuning are externally specified learning procedures.
Source record
- Reference ID
- bert
- Archived version
- 1810.04805v2
- Summary provenance
- reference-library/papers/bert/metadata.json · summary
- Review status
- initial orientation summary; not a critical full-paper review
- Reading scope
- Selected abstract/opening and available closing discussion/conclusion passages in the preserved text extract; broader project notes provide context. No reproduction of experiments, exhaustive methods review, or equation/figure audit.