STRUCTURALCOST: A controlled reading time dataset for modeling human sentence processing difficulty
We introduce STRUCTURALCOST, a self-paced reading dataset of 475 participants and 40,800 observations isolating the processing cost of long-distance subject-verb dependency resolution.
ProofPaper ↗
Key points
- We replicate a low-powered psycholinguistic finding at NLP scale, namely that human reading times at the main verb increase with dependency length, driven by syntactic embedding beyond linear distance.
- Different language models -- spanning n-gram models, SSMs, and transformers -- partially mirror this graded difficulty profile, yet underestimate the integration cost humans incur, with a gap that persists across architectures and model sizes.
- This suggests these models capture the predictive component of human processing but not the full integration cost that working memory imposes.
- STRUCTURALCOST provides data needed to drive progress toward evaluating the cognitive plausibility of language models.
Sources (1)
- [1]STRUCTURALCOST: A controlled reading time dataset for modeling human sentence processing difficultyarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 12:00 PM
We introduce STRUCTURALCOST, a self-paced reading dataset of 475 participants and 40,800 observations isolating the processing cost of long-distance subject-verb dependency resolution.
We replicate a low-powered psycholinguistic finding at NLP scale, namely that human reading times at the main verb increase with dependency length, driven by syntactic embedding beyond linear distance.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 5, 2026LiquidAI/d1-omni-600M
- Oct 5, 2026MC-Sparse: Deconstructing and Closing the Dense-Sparse Attention Gap in Diffusion Transformers
- Sep 30, 2026Cloudflare/clef-flash
- Sep 29, 2026microsoft/AesCode-32B
- Sep 29, 2026Language Models for Text Classification: From Bag-of-Words to Jev
- Aug 26, 2026vllm-project/vllm v0.28.0