Judging in Latent Space: Efficient Generative Reward Modeling via Semantics-Preserving Compression

Key points

• Reward modeling often requires jointly representing and reasoning over multiple evaluation criteria, yet verbalizing this process token by token can incur substantial inference cost.1
• By using the structure of rubric-guided evaluations to guide compression, LatentGRM learns compact continuous trajectories that support autonomous pairwise judgments without generating textual assessments.1
• Under matched training data and backbones, LatentGRM achieves competitive aggregate preference accuracy relative to explicit Supervised Fine-Tuning (SFT) judges at both 4B and 8B scales.1
• Across four benchmark domains, LatentGRM-8B compresses evaluation trajectories by 8.9--9.2x and reduces total judge inference time by 6.1--7.0x at vote@5.1

Sources (2)

• [1]Judging in Latent Space: Efficient Generative Reward Modeling via Semantics-Preserving Compression
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 10:05 AM

  We introduce LatentGRM, a latent evaluation framework built on semantic chunking, compression, and reconstruction.

  Reward modeling often requires jointly representing and reasoning over multiple evaluation criteria, yet verbalizing this process token by token can incur substantial inference cost.

• [2]Judging in Latent Space: Efficient Generative Reward Modeling via Semantics-Preserving Compression
  Hugging Face Daily Papers · Oct 8, 12:00 AM · same content

Extractive summary: sentences quoted from the sources.