Reading the Room: Foundations, Design, and Challenges of Normative Competence in LLMs

Key points

• Human communities are governed by normative systems: shared standards that produce norms dictating acceptable behavior, enforced through community sanctioning.1
• Thus, alignment requires normative competence: the ability to discern from interaction alone what norms a community enforces without relying on static pretrained knowledge.1
• We show that baseline LLM agents fail to learn norms even when doing so would improve their accuracy.1
• We then experiment with various normative modules -- architectural components for norm inference -- finding that norm-following is highly sensitive to both the style of norm and the model powering the normative module, suggesting a lack of generalizability.1

Sources (1)

• [1]Reading the Room: Foundations, Design, and Challenges of Normative Competence in LLMs
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 09:02 PM

  We introduce a multi-agent community debate setting, where access to debate is governed by synthetic norms, to study normative competence in isolation from pretraining exposure.

  Human communities are governed by normative systems: shared standards that produce norms dictating acceptable behavior, enforced through community sanctioning.

Extractive summary: sentences quoted from the sources.