The Communication Bottleneck: A Round-Trip Study of Tree-Structured Expression Serialization in Language Models

Key points

• When language models reason in chain-of-thought or exchange free-text intermediates, they serialize structured information into natural language.1
• How much tree-structured compositional content survives this bottleneck?1
• We propose a round-trip protocol that answers this question empirically for tree-structured expressions.1
• Together these results identify tree-structured expression serialization as a primary limiting factor when models communicate hierarchical structure through natural language.1

Sources (1)

• [1]The Communication Bottleneck: A Round-Trip Study of Tree-Structured Expression Serialization in Language Models
  Apple Machine Learning Research · Sep 29, 12:00 AM

  The Communication Bottleneck: A Round-Trip Study of Tree-Structured Expression Serialization in Language Models

  When language models reason in chain-of-thought or exchange free-text intermediates, they serialize structured information into natural language.

Extractive summary: sentences quoted from the sources.