Principled Under Pressure: Post-Training Decides Whether LLMs Act on Their Own Moral Judgment

Key points

• An agent that says an action is wrong and then takes it anyway is a different failure from one that does not know better, and evaluations of stated values cannot see it.1
• On OLMo-3-7B-Instruct, the model takes the action it judged wrong on about one in five pressuring scenarios, more often than on the same scenarios with the pressure removed.1
• Across four instruct models the gap depends on the post-training recipe: OLMo-3 and Meta's Llama-3.1-8B-Instruct carry it; Tulu 3 shows none on the whole panel (above about 0.01 in probability) or on its own most-pressuring scenarios; Qwen2.5-7B-Instruct shows none on the whole panel (above about 0.02) and is unresolved on its own (0.083, -0.028 to 0.195).1
• On both models that carry it, reasoning about the stakes before acting moves the choice back toward the model's own judgment, against a same-length non-moral task, with or without the pressure; on OLMo-3, naming the norm at stake does about a third of that.1

Sources (1)

• [1]Principled Under Pressure: Post-Training Decides Whether LLMs Act on Their Own Moral Judgment
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 04:52 PM

  Language models increasingly act as agents.

  An agent that says an action is wrong and then takes it anyway is a different failure from one that does not know better, and evaluations of stated values cannot see it.

Extractive summary: sentences quoted from the sources.