JoyInside-VLM-Agent: Product-Conditioned Multimodal Policy Learning for Embodied Agents
We present JoyInside-VLM-Agent, a ReAct-style multimodal agent developed through product-conditioned trajectory construction and staged policy learning for robotic platforms and smart hardware.
ProofPaper ↗
Key points
- Embodied agents must translate multimodal observations and user requests into executable actions while adapting to product constraints, changing goals, and execution feedback.
- A closed-loop sandbox generates multimodal interaction trajectories covering goal revisions and failure recovery, which are checked against execution outcomes and semantic criteria.
- Supervised fine-tuning and multi-teacher on-policy distillation precede reinforcement learning with DPER (Dual-Source Privileged On-Policy Self-Distillation with Token-Level Evidence Routing).
- Training rewards combine generative process evaluation, multi-judge rankings, and task-queue consistency.
Sources (1)
- [1]JoyInside-VLM-Agent: Product-Conditioned Multimodal Policy Learning for Embodied AgentsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 03:54 AM
We present JoyInside-VLM-Agent, a ReAct-style multimodal agent developed through product-conditioned trajectory construction and staged policy learning for robotic platforms and smart hardware.
Embodied agents must translate multimodal observations and user requests into executable actions while adapting to product constraints, changing goals, and execution feedback.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 9, 2026OGPO: Offline Goal-conditioned Policy Optimization for Recoverable Vision-Language-Action Models
- Oct 9, 2026Continually Post-training Small Models via Synthetic Self-distillation
- Oct 9, 2026Rethinking Relative Policy Differences Transfer of On-Policy Distillation
- Oct 8, 2026Distilling Routed 3D Privilege for Spatial Reasoning in Vision-Language Models
- Oct 7, 2026Enabling Preference-driven Unlearning in Few-step Distilled Text-to-Image Diffusion Models
- Oct 7, 2026MIRROR: From Imitation to Internalization in LLM Personalization
