Structure Tax: How Structured Output affects LLMs Performance

Key points

• We re-examine this claim by evaluating a battery of models, datasets and schemas, measuring task accuracy, confidence calibration, and hidden-state geometry.1
• The tax turns out to depend on schema design rather than on structure per se: reasoning-first field ordering matches or exceeds free-form accuracy, while answer-first ordering causes steep drops, particularly in smaller models.1
• Format sensitivity scales inversely with a task's own structural constraints, and schemas that preserve reasoning order also improve calibration with CKA showing greater separability between correct and incorrect representations in middle transformer layers.1
• Our findings indicate that properly designed structured formats can match or exceed free-form performance, reframing the critical question from whether to structure' to how to structure' for optimal reasoning preservation.1

Sources (1)

• [1]Structure Tax: How Structured Output affects LLMs Performance
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 02:39 PM

  Deploying large language models in production often requires constraining outputs to structured formats such as JSON or XML, and prior work treats the resulting accuracy loss as an inherent `structure tax'.

  We re-examine this claim by evaluating a battery of models, datasets and schemas, measuring task accuracy, confidence calibration, and hidden-state geometry.

Extractive summary: sentences quoted from the sources.