Skip to content

PEFT Overview

About

The d9d.peft package fine-tunes models with parameter-efficient methods, such as LoRA, or with full fine-tuning of selected modules.

Apply Before State Loading

This package is built on the model state mapping framework.

Methods like LoRA change the model structure. For example, an nn.Linear layer becomes a LoRALinear wrapper. The keys in the original checkpoint (e.g. layers.0.linear.weight) then no longer match the model keys (e.g. layers.0.linear.base.weight). d9d.peft returns a ModelStateMapper that renames only the keys the method changes. Chain it after your checkpoint mapper with ModelStateMapperSequential, so all other keys load unchanged.

So you can apply a PEFT method before the model is initialized or horizontally distributed. Other PEFT frameworks usually require initialized weights before they apply PEFT. That can break your horizontal parallelism setup or make it harder to reuse.

Configuration

Every PEFT method has a Pydantic configuration. Pydantic validates the hyperparameters and serializes the configuration. d9d.peft.all.peft_method_from_config builds the method for any configuration (see Method Stacking).

The Injection Lifecycle

Every PEFT method implements PeftMethod and follows an inject, train, merge lifecycle:

  1. Inject (inject_peft_and_freeze): The method finds the target layers in the nn.Module and replaces them with adapter layers if needed. All parameters are frozen except those the method trains.
  2. State mapping: The injection returns a ModelStateMapper. It maps the original keys of the changed layers to the new model structure. It does not cover the other keys.
  3. Train: You train the model.
  4. Merge (merge_peft): After training, the method merges the adapters into the base weights and restores the original architecture.

Usage

See LoRA, Full Fine-Tuning and Method Stacking for examples.

API Reference

d9d.peft

Core logic and base definitions for parameter-efficient fine-tuning (PEFT).

PeftInjectionResult dataclass

The result of injecting a PEFT method into a model.

Attributes:

Name Type Description
parameters_to_train list[Parameter]

The parameters that must stay trainable.
load_state_mappers list[ModelStateMapper]

The mappers that rename the original keys of the changed layers to the modified structure. They do not cover keys that the method leaves unchanged.

PeftMethod

Bases: ABC, Generic[TConfig]

Abstract base class for all parameter-efficient fine-tuning (PEFT) methods.

from_config(config) abstractmethod classmethod

Creates the method from a configuration object.

Parameters:

Name Type Description Default
config TConfig

The configuration object.

 required

Returns:

Type Description
Self

The method instance.
inject(module) abstractmethod

Modifies the module in place to apply the PEFT method.

Parameters:

Name Type Description Default
module Module

The PyTorch module to modify.

 required

Returns:

Type Description
PeftInjectionResult

The trainable parameters and the mappers for loading weights into the new structure.
merge(module) abstractmethod

Merges the trained adapters back into the base model parameters.

Parameters:

Name Type Description Default
module Module

The PyTorch module to update.

 required

inject_peft_and_freeze(method, module)

Applies a PEFT method to a module and freezes all parameters it does not train.

  1. Sets requires_grad=False for all parameters in the module.
  2. Calls method.inject to modify the module structure.
  3. Sets requires_grad=True for the parameters returned by the injection.

Parameters:

Name Type Description Default
method PeftMethod

The PEFT method to apply.

 required
module Module

The PyTorch module to modify.

 required

Returns:

Type Description
ModelStateMapper

A ModelStateMapper that renames only the keys the method changes. Chain it after the checkpoint
ModelStateMapper

mapper with ModelStateMapperSequential to load a full checkpoint.

merge_peft(method, module)

Merges PEFT adaptations back into the base model weights.

Parameters:

Name Type Description Default
method PeftMethod

The PEFT method that was applied.

 required
module Module

The PyTorch module to merge.

 required