CueRator: Agentic Search for Symbolic Rules to Adapt Frozen Multimodal Encoders
We propose CueRator, an agentic framework for policy-aware decision-rule discovery, which adapts frozen contrastive multimodal encoders by searching for the decision rule that converts their cross-modal similarities into predictions.
ProofPaper ↗
Key points
- Large language model agents have been used to search over symbolic structures such as programs and equations.
- We validate it on open-vocabulary audio-visual event perception, where existing methods involve a trade-off between adaptivity and generalization to unseen categories: trained modules adapt at the cost of generalization, and fixed rules the reverse.
- The framework pairs a symbolic formulation for generalization with a lightweight policy that predicts its parameters per video for adaptivity.
- A report-guided multi-agent loop discovers the formulation offline, evaluating each candidate on its expressive ceiling and on whether a trained policy can realize it.
Sources (1)
- [1]CueRator: Agentic Search for Symbolic Rules to Adapt Frozen Multimodal EncodersarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 07:18 AM
We propose CueRator, an agentic framework for policy-aware decision-rule discovery, which adapts frozen contrastive multimodal encoders by searching for the decision rule that converts their cross-modal similarities into predictions.
Large language model agents have been used to search over symbolic structures such as programs and equations.
Extractive summary: sentences quoted from the sources.