Isotropic Yet Undecodable: The Sequential Content-Sufficiency Gap in Latent-Predictive Text Representations

Key points

• An information-theoretic decomposition separates input ambiguity, representation loss, and readout mismatch.1
• We construct recoverable views where perfect agreement and joint isotropic Gaussianity coexist with zero target information, and establish limits imposed by deterministic canonical anchors.1
• These results motivate CANOPE, a nonautoregressive framework with ordered latent canvases, canonical-token supervision, and geometric regularization.1
• On 3,930 LJSpeech validation utterances, frozen PL2 with a trained MatchaTTS readout yields 21.54% word error rate (WER) on corrupted text, versus 99.22% for frozen PL0, while end-to-end MatchaTTS reaches 10.93%.1

Sources (1)

• [1]Isotropic Yet Undecodable: The Sequential Content-Sufficiency Gap in Latent-Predictive Text Representations
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 07:52 AM

  We study sequential content sufficiency by investigating whether a representation retains the ordered target information available in its input.

  An information-theoretic decomposition separates input ambiguity, representation loss, and readout mismatch.

Extractive summary: sentences quoted from the sources.