Skip to content

Container Metrics

About

Managing many metrics one by one means repeating the sync, compute and reset calls for each of them. Container metrics bundle several metrics and manage them together.

Compose Metric

ComposeMetric wraps a mapping from string keys to Metric instances into a single metric.

You cannot call .update() on a ComposeMetric, because its children can take different arguments. Update the children directly instead. The lifecycle methods .sync(), .compute(), .reset(), .to(), .state_dict() and .load_state_dict() apply to all children.

Usage

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
import torch
from d9d.metric.impl.aggregation import SumMetric, WeightedMeanMetric
from d9d.metric.impl.container import ComposeMetric

dist_context = ...  # The DistributedContext of the job.

# 1. Group metrics together.
metrics = ComposeMetric({
    "loss": WeightedMeanMetric(),
    "total_samples": SumMetric(),
})

# 2. Update each child with its own arguments.
metrics["loss"].update(torch.tensor(0.5), torch.tensor(32.0))
metrics["total_samples"].update(torch.tensor(32.0))

# 3. Lifecycle methods apply to all children.
metrics.to("cuda")
metrics.sync(dist_context)

# 4. compute() returns a dictionary from metric name to result.
results = metrics.compute()

# 5. Reset all metrics for the next epoch or evaluation.
metrics.reset()

API Reference

d9d.metric.impl.container.ComposeMetric

Bases: Metric[dict[str, Any]]

Container metric that groups named child metrics into one metric.

Synchronization, computation, reset, device moves and checkpointing apply to all children. compute() returns a dictionary with the result of each child. Update the children directly.

children property

The child metrics by name.

__getitem__(item)

Returns the child metric with the given name.

__init__(children)

Constructs the ComposeMetric object.

Parameters:

Name Type Description Default
children Mapping[str, Metric]

The child metrics by name.

 required

update(*args, **kwargs)

Always raises, because the children must be updated directly.

Raises:

Type Description
ValueError

If called.