S-Space: Exploring Spatial Workspace in Multimodal Language Models

Key points

• We uncover S-Space, a linear representation subspace in which object-token representations encode continuous coordinates along horizontal, vertical, and distance axes.1
• Although we identify these axes by differentiating spatial-answer scores with respect to object-token activations, S-Space is intrinsic rather than induced by specific data or prompts, remaining readable under non-spatial questions.1
• We show that S-Space functions as an internal spatial workspace of MLLMs. Perception encodes object locations into it, forming spatial representations that can be directly read out to support accurate spatial judgments.1
• Direction-word readouts reveal systematic coupling between cardinal and image-relative reference frames.1

Sources (2)

• [1]S-Space: Exploring Spatial Workspace in Multimodal Language Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 12:48 PM

  Multimodal language models (MLLMs) exhibit promising spatial reasoning capabilities, yet the internal mechanisms supporting them remain poorly understood.

  We uncover S-Space, a linear representation subspace in which object-token representations encode continuous coordinates along horizontal, vertical, and distance axes.

• [2]S-Space: Exploring Spatial Workspace in Multimodal Language Models
  Hugging Face Daily Papers · Oct 9, 12:48 PM · same content

Extractive summary: sentences quoted from the sources.