Ask Your Advisor: Learning When to Consult Stronger Models in Agentic Systems

Key points

• To balance task performance and computational cost, agentic workflows can use a lower-cost model for execution and selectively consult a stronger model.1
• In such executor-advisor systems, the executor decides when to seek advice, making escalation depend on self-assessment, although current language models struggle to detect where their own reasoning fails.1
• Our approach formulates consultation as a sequential resource-allocation problem.1
• At workflow contact points, ARIA maps execution evidence to a compact state abstraction and selects SILENT or CONSULT, with action values learned by Monte Carlo control from cost-penalized outcomes.1

Sources (1)

• [1]Ask Your Advisor: Learning When to Consult Stronger Models in Agentic Systems
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 05:19 PM

  We introduce ARIA, a meta-controller that learns when to consult by balancing task success against consultation cost.

  To balance task performance and computational cost, agentic workflows can use a lower-cost model for execution and selectively consult a stronger model.

Extractive summary: sentences quoted from the sources.