PEFT

Timeline

1. Oct 8, 2026 · Open-source release · 1 source
   huggingface/trl v1.15.0
   SFT, DPO, KTO, GRPO, RLOO and Distillation now score tokens with a fused LM head: a Triton kernel projects the hidden states through the LM head in tiles and reduces to per-token log-probs and entropy directly, so the [batch, seq, vocab] logits tensor is never built.
2. Oct 8, 2026 · Research paper · 1 source
   Where to Adapt Matters: Layer-Selective Fine-Tuning for Capability Retention
   Parameter-efficient fine-tuning (PEFT) enables large language models (LLMs) to adapt to specialized tasks, but often at the cost of degrading general capabilities acquired during pretraining.
3. Oct 8, 2026 · Research paper · 1 source
   Stability-Plasticity Balance via Singular-Vector Selection in LLM Continual Learning
   Domain-specific continual adaptation of LLMs risks catastrophic forgetting, creating a fundamental tension between acquiring new capabilities and preserving those learned during pretraining.
4. Oct 6, 2026 · Research paper · 1 source
   Are Parameter-Efficient Fine-tuning Methods Really Different?
   Parameter-efficient fine-tuning (PEFT) offers many parameterizations, yet their methodological and functional differences remain unclear.
5. Oct 6, 2026 · Research paper · 1 source
   MemFLoRA: Memory-Floor LoRA for CNN Adaptation at the Edge
   This paper introduces Memory-Floor LoRA (MemFLoRA), a low-rank CNN adapter built around a memory-first design principle rather than a direct application of transformer-oriented LoRA.
6. Oct 6, 2026 · Open-source release · 1 source
   huggingface/transformers v5.19.0: Release v5.19.0
   EmbeddingGemma 2 is a multimodal embedding model from Google built on the Gemma 4 architecture.
7. Oct 6, 2026 · Research paper · 1 source
   Dynamic Positional Attention Modulation for Parameter-Efficient Fine-Tuning of Large Language Models
   In this work, we propose DyPAM (Dynamic Positional Attention Modulation), a PEFT method that adapts how positional information contributes to attention by operating directly on the query and key representations.
8. Oct 1, 2026 · Product / feature launch · 1 source
   unslothai/unsloth v0.1.902-beta: Command Palette + Desktop UI/UX
   This release brings faster navigation, shareable run settings, and clearer errors to Unsloth Desktop.
9. Oct 1, 2026 · Open-source release · 1 source
   huggingface/peft v0.21.2
   This is a PEFT release fixes an issue that prevented encoder-decoder models to work when using Transformers ≥ 5.18.0.
10. Sep 29, 2026 · Open-source release · 1 source
    huggingface/peft v0.21.1
    This is a small PEFT release to enable Tensor Parallel (TP) to work properly with PEFT.
11. Jun 15, 2026 · Open-source release · 1 source
    huggingface/transformers v5.12.1: Patch release v5.12.1
    Updated the lower bound for PEFT and a fix for auto tokenizer to properly resolve the mistral tokenizer (when mistral-common is installed).

Often appears with