On the Effectiveness-Fluency Trade-Off in LLM Conditioning: A Systematic Study

Key points

• Controlling the output of Large Language Models (LLMs) is a central challenge for their reliable deployment, yet a clear understanding of the involved trade-offs remains elusive.1
• We find that efficient steering methods frequently achieve conditioning at a steep cost to fluency.1
• Furthermore, we identify a critical yet previously overlooked interaction with the training paradigm: activation steering methods are far less effective on instruction-tuned models than on their base counterparts.1
• We introduce Dynamically Scaled Activation Steering (DSAS), a method-agnostic steering framework that decouples when to steer from how to steer.1

Sources (1)

• [1]On the Effectiveness-Fluency Trade-Off in LLM Conditioning: A Systematic Study
  Apple Machine Learning Research · Sep 30, 12:00 AM

  On the Effectiveness-Fluency Trade-Off in LLM Conditioning: A Systematic Study

  Controlling the output of Large Language Models (LLMs) is a central challenge for their reliable deployment, yet a clear understanding of the involved trade-offs remains elusive.

Extractive summary: sentences quoted from the sources.