Relational Abstractions for Spatial Reasoning with Diffusion Models

Key points

• Diffusion models excel at image synthesis, but they remain limited in their ability to reliably satisfy structured spatial reasoning constraints.1
• In conditional data distribution modeling tasks with implicit logical structure, such as puzzles defined by visible clues paired with consistent solutions, state-of-the-art generative models tend to approximate pixel-space distributions without learning the underlying logical rules required for inference.1
• We show that the relational knowledge derived from object-centric representations enriches diffusion models with structural primitives, allowing them to effectively guide the generative representation space during both training and inference, and enabling conditional image generation that satisfies reasoning constraints.1
• Additionally, we introduce a large-scale generative spatial reasoning benchmark with four datasets inspired by human-solvable puzzles.1

Sources (1)

• [1]Relational Abstractions for Spatial Reasoning with Diffusion Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 10:00 AM

  To address this limitation, we present a novel framework for spatial reasoning with diffusion models that leverages unsupervised object discovery and abstractions of object relations.

  Diffusion models excel at image synthesis, but they remain limited in their ability to reliably satisfy structured spatial reasoning constraints.

Extractive summary: sentences quoted from the sources.