Reference library
A Simple and Effective Pruning Approach for Large Language Models
Paper summary
Wanda prunes pretrained language-model weights using weight magnitude multiplied by corresponding input activation magnitude. It avoids retraining and expensive second-order reconstruction, and evaluates how much language-model performance remains after pruning.
Relevance to the project
Provides an activity-sensitive baseline for later connection-removal experiments or optional model compression. Compare performance and actual sparse execution costs before transferring the criterion to changing neural graphs.
Pruning a trained LLM is different from locally evolving topology.
Source record
- Reference ID
- wanda
- Archived version
- 2306.11695v3
- Summary provenance
- reference-library/papers/wanda/metadata.json · summary
- Review status
- initial orientation summary; not a critical full-paper review
- Reading scope
- Selected abstract/opening and available closing discussion/conclusion passages in the preserved text extract; broader project notes provide context. No reproduction of experiments, exhaustive methods review, or equation/figure audit.