SynAct: Synergistic Active Feedback Learning for LLM Agents
We present SynAct, an active learning framework that treats expert attention as the binding constraint and allocates feedback at the level of individual decisions.
ProofPaper ↗
Key points
- LLM agents executing multi-step workflows fail in two fundamentally different ways: recoverable slips, which further attempts can repair, and knowledge gaps, which retrying cannot resolve.
- Existing approaches either avoid expert feedback, allowing agents to reinforce their own errors, or allocate feedback at the task level without identifying consequential decisions.
- SynAct ranks decisions by the mutual information between the selected action and the terminal outcome across repeated rollouts, queries an expert at the highest-value decisions, and distills the resulting corrections into transferable rules that the agent can reuse without expert access at deployment.
- We theoretically characterize what this signal identifies under finite repeated sampling: with k rollouts and at least m observations per action, outcome-constant decisions have zero mutual information, while outcome-separating decisions have mutual information at least H2(m/k) bits, yielding exact separation for any threshold t in (0,H2(m/k)).
Sources (1)
- [1]SynAct: Synergistic Active Feedback Learning for LLM AgentsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 07:20 PM
We present SynAct, an active learning framework that treats expert attention as the binding constraint and allocates feedback at the level of individual decisions.
LLM agents executing multi-step workflows fail in two fundamentally different ways: recoverable slips, which further attempts can repair, and knowledge gaps, which retrying cannot resolve.
Extractive summary: sentences quoted from the sources.
