In-context learning

Timeline

1. Oct 8, 2026 · Research paper · 1 source
   Compile the Table: Query-Calibrated Operator Compression for Tabular In-Context Learning
   We propose QCOC (Query-Calibrated Operator Compression), which exploits the exchangeability and repeated use of in-context examples by compiling their full KV cache once into compact memory shared across subsequent queries.
2. Oct 8, 2026 · Research paper · 1 source
   Do LLMs Learn from Rewards in Context? : Rethinking the role of reward in In-Context Reinforcement Learning
   LLM agents increasingly improve at inference time by accumulating experience in context rather than by updating parameters.
3. Oct 8, 2026 · Research paper · 1 source
   SFT-as-Context Mitigates Forgetting in Supervised Fine-Tuning
   We introduce SFT-as-context, a training-free method in which the parent model uses the SFT model's response as context to answer the query.
4. Oct 7, 2026 · Research paper · 1 source
   What can linear attention learn from nonlinear teachers in-context?
   Linear attention is a tractable model for understanding the mechanisms governing in-context learning in transformers.
5. Oct 7, 2026 · Research paper · 1 source
   BEANS-Next and ROOTS: Broadening Audio-Language Capabilities for Bioacoustics
   In this work, we introduce BEANS-Next, a benchmark grounded in a taxonomy of bioacoustics tasks spanning acoustic perception, biological category recognition, scene understanding, and in-context learning.
6. Oct 7, 2026 · Research paper · 1 source
   When to Unpair: Regulating Pairing Dependence in Medical Visual In-Context Learning
   Visual in-context learning (ICL), well suited to label-scarce medical imaging, uses support image-label pairs to demonstrate input-output mappings, while the labels collectively indicate the requested task.
7. Oct 7, 2026 · Research paper · 1 source
   Efficient Provably Private Classification with a Tabular Foundation Model
   Here we introduce PrivTab, an easy to use tabular foundation model for differentially private classification that embeds a privacy mechanism within its architecture.
8. Oct 7, 2026 · Research paper · 1 source
   Leaner Transformers Can Easily Learn to Cluster
   Recent work shows that transformers can exactly perform Lloyd's algorithm for $k$-means clustering with $n$ points in $d$ dimensions with an embedding size $d{\textsf{emb}} = d+k$ (thus, requiring attention projection matrices of size $(d+k)^2$).
9. Oct 6, 2026 · Research paper · 1 source
   Spatial Induction Heads: In-Context Learning of Multidimensional Cellular Automata
   We introduce spatial induction heads, two-layer gather-and-match circuits in which the first layer reconstructs the relevant spatial neighborhood and the second matches the resulting configuration against earlier occurrences.
10. Oct 6, 2026 · Research paper · 1 source
    GeneICL: A Tabular Foundation Model for Bulk Transcriptomics
    Towards this end, we introduce GeneICL, a 4.2M-parameter tabular foundation model combining a semi-synthetic pretraining prior built from measured bulk expression profiles with a parameter-efficient recurrent architecture.
11. Oct 6, 2026 · Research paper · 1 source
    Towards In-Parameter Memory Augmentation for Large Language Models
    Recently Large Language Models (LLMs) and LLM-based agents increasingly need to incorporate knowledge acquired after pretraining, e.g., domain facts, user preferences, documents, and interaction experience.
12. Oct 6, 2026 · Research paper · 1 source
    The Standardization Trap: Certifying Joint Label Processing in Tabular Foundation Models
    Linear regression and kernel smoothing offer tractable explanations of in-context learning: in both, the features determine the weight assigned to each context label.
13. Oct 6, 2026 · Research paper · 1 source
    TICDA: Tabular In-Context Data Attribution
    We introduce TICDA, a method that measures the influence of every demonstration in the context directly from linear surrogates trained on TFM latent embeddings, in a single forward pass and at negligible cost.
14. Oct 6, 2026 · Research paper · 1 source
    Continuous Memory Machines
    To that end, we introduce the Continuous Memory Machine (CMM), a recurrent architecture with matrix-valued short- and long-term memory states serving distinct functional roles.
15. Oct 6, 2026 · Research paper · 1 source
    Adaptive Mean Estimation by In-Context Learning: A Gradient-Flow Analysis
    Prior Fitted Networks (PFNs) such as TabPFN now rival established statistical procedures across prediction and estimation tasks.
16. Oct 6, 2026 · Research paper · 1 source
    Adversarially Trained Linear Transformers Are Optimal Robust In-Context Learners for Gaussian Mixtures
    Specifically, we show that, for a family of Gaussian-mixture classification tasks, a sufficiently deep linear transformer adversarially trained across tasks can asymptotically attain the robust Bayes error on previously unseen tasks through in-context learning from clean demonstrations.
17. Sep 29, 2026 · Research paper · 1 source
    In-context Robot Learning Made Simple: A Democratized Recipe for Manipulation Tasks
    We study robotic in-context learning (ICL), an emerging paradigm that enables robots to infer and execute tasks from visual demonstrations.

Often appears with