AgentGarten: Code Worlds for Evolving Agents
We introduce AgentGarten, a framework that couples simulators and game engines with a shared neural renderer to build real-time interactive environments.
Key points
- Interactive virtual worlds allow agents to learn through exploration and interaction.
- What agents can learn is bounded by the environments they practice in, which must be faithful, with consistent state, rules, and dynamics, and realistic, with observations that follow the real-world visual distributions.
- To build the neural renderer, we adapt a pretrained video model to geometry conditions, distill it with our proposed Adversarial Forcing, and optimize inference for real-time interaction.
- In AgentGarten, agents perceive the world through visual observations, interact with it in real time, and improve by distilling each round of experience into playbooks that subsequent agents inherit and refine.
Sources (2)
- [1]AgentGarten: Code Worlds for Evolving AgentsHugging Face Daily Papers · Oct 8, 12:00 AM
We introduce AgentGarten, a framework that couples simulators and game engines with a shared neural renderer to build real-time interactive environments.
Interactive virtual worlds allow agents to learn through exploration and interaction.
- [2]AgentGarten: Code Worlds for Evolving AgentsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 05:32 PM · same content
Extractive summary: sentences quoted from the sources.