Skip to content

Method Stacking

About

Fine-tuning can combine several methods. The d9d.peft.all package groups multiple PEFT configurations into a single PeftStack. The stack injects its methods in order and merges them in reverse order.

Usage

This example applies LoRA to attention layers and fully fine-tunes normalization layers.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
import re

from d9d.model_state.mapper.adapters import identity_mapper_from_module
from d9d.model_state.mapper.compose import ModelStateMapperSequential
from d9d.peft import inject_peft_and_freeze, merge_peft
from d9d.peft.all import PeftStackConfig, peft_method_from_config
from d9d.peft.full_tune import FullTuneConfig
from d9d.peft.lora import LoRAConfig, LoRAParameters

model = ...  # Your model, before its weights are loaded.

# Maps the checkpoint keys to the original model structure.
checkpoint_mapper = identity_mapper_from_module(model)

# 1. Define the methods.
config = PeftStackConfig(
    methods=[
        # LoRA on attention projections.
        LoRAConfig(
            module_name_pattern=re.compile(r".*self_attn\..*_proj"),
            params=LoRAParameters(r=8, alpha=16, dropout=0.05)
        ),
        # Full fine-tuning of normalization layers.
        FullTuneConfig(
            module_name_pattern=re.compile(r".*norm.*")
        )
    ]
)

# 2. Build a PeftStack that contains both methods.
method = peft_method_from_config(config)

# 3. Inject.
# The PEFT mapper only renames the keys of the LoRA layers.
peft_mapper = inject_peft_and_freeze(method, model)
mapper = ModelStateMapperSequential([checkpoint_mapper, peft_mapper])

# ... return the mapper from your ModelProvider, or load a checkpoint with it ...

# ... train the model ...

# 4. Merge the adapters into the base weights before exporting the model.
merge_peft(method, model)

API Reference

d9d.peft.all

Stacking of several PEFT methods.

PeftStack

Bases: PeftMethod[PeftStackConfig]

PEFT method that applies a list of PEFT methods in order.

__init__(methods)

Constructs the PeftStack object.

Parameters:

Name Type Description Default
methods list[PeftMethod]

The PEFT methods to apply in order. merge runs them in reverse order.

 required

PeftStackConfig

Bases: BaseModel

Configuration for applying a stack of multiple PEFT methods sequentially.

Attributes:

Name Type Description
kind Literal['stack']

Discriminator field. Always "stack".
methods list[AnyPeftConfig]

The PEFT configurations (e.g. LoRA, full tune) to apply in order.

peft_method_from_config(config)

Builds the PEFT method that matches the configuration type.

Parameters:

Name Type Description Default
config TConfig

A PEFT configuration object, e.g. LoRAConfig.

 required

Returns:

Type Description
PeftMethod[TConfig]

The corresponding method instance.