Skip to content

Schedule Visualization

About

It is often difficult to mentally visualize complex multiphase learning rate schedules.

To address this, d9d allows you to visualize the resulting learning rate curve interactively using plotly.

Usage Example

The visualize_lr_scheduler function takes a factory function that constructs your scheduler, simulates a training run, and plots the learning rate history.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
import torch

from d9d.lr_scheduler.visualizer import visualize_lr_scheduler
from d9d.lr_scheduler.piecewise import piecewise_schedule, CurveLinear, CurveCosine


def create_scheduler(optimizer: torch.optim.Optimizer):
    return (
        piecewise_schedule(initial_multiplier=0.0, total_steps=100)
        .for_steps(10, 1.0, CurveLinear())
        .fill_rest(0.0, CurveCosine())
        .build(optimizer)
    )


# Opens an interactive plot in browser/notebook
visualize_lr_scheduler(
    factory=create_scheduler,
    num_steps=100,  # Duration to simulate
    init_lr=1e-3  # Base LR to visualize
)

API Reference

d9d.lr_scheduler.visualizer

visualize_lr_scheduler(factory, num_steps, init_lr=1.0)

Visualizes the learning rate schedule using Plotly.

This function simulates the training process for num_steps to record the LR changes and generates an interactive plot.

Parameters:

Name Type Description Default
factory SchedulerFactory

A callable that accepts an Optimizer and returns an LRScheduler.

 required
num_steps int

The number of steps to simulate.

 required
init_lr float

The initial learning rate to set on the dummy optimizer.

 1.0

Raises:

Type Description
ImportError

If the plotly library is not installed.