Task Context Shapes Component Roles in Language Models
Large language models reuse the same attention heads across tasks, but identifying these shared heads does not explain what they do differently in each task.
ProofPaper ↗
Key points
- A natural source of inspiration is task-dependent interactions among shared brain regions, which motivate a network view of language-model computation.
- Here, we compare fixed model components across tasks using identical graph facts, statements specifying which directed connections are present.
- We measure how interventions affect attention heads and the activation state in later layers (influence), as well as the answer (importance).
- These findings show that task context changes both which graph facts the shared components respond to and how their outputs affect later computation.
Sources (1)
- [1]Task Context Shapes Component Roles in Language ModelsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 05:40 PM
Large language models reuse the same attention heads across tasks, but identifying these shared heads does not explain what they do differently in each task.
A natural source of inspiration is task-dependent interactions among shared brain regions, which motivate a network view of language-model computation.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 9, 2026Taking Tokenization off the Host for Agentic LLM Serving
- Oct 9, 2026Attention Sees, MLP Refuses: Tracing the Attention-to-MLP Safety Pathway in LLMs
- Oct 9, 2026Qwen/Qwen-Image-2.1-Turbo
- Oct 8, 2026BeliefScope: Diagnosing Evidence-Driven Revision and Pressure-Induced Shifts in Large Language Models
- Oct 8, 2026SparseDecoding: Decoding-Aware Pruning for Accurate and Efficient LLM Inference
- Oct 7, 2026Amortized Off-Policy Evaluation for LLMs