@tensorleap_custom_metric

code_loader.inner_leap_binder.leapbinder_decorators.tensorleap_preprocess.tensorleap_custom_metric

The purpose of the tensorleap_custom_metric decorator is to add a custom metric suited to your needs to the tensorleap platform.

@tensorleap_custom_metric(name='metrics',
                          direction=MetricDirection.Downward,
                          compute_insights=None)
def metrics(
    tensor_1: npt.NDArray[np.float32],
    tensor_2: npt.NDArray[np.float32],
    ...
) -> Union[npt.NDArray[np.float32], Dict[str, npt.NDArray[np.float32]]]:
    pass

Args
name	The name of the the metric
direction	(optional, defaults toMetricDirection.Downward) For a metric where lower values are better use MetricDirection.Downward For a metric where higher values are better use MetricDirection.Upwards
compute_insights	(optional, defaults to None). Specify metrics that should not have insights computed on them: in case the metric function returns a dictionary, compute_insights should return a dictionary as well, mapping metric names to booleans. otherwise, compute_insights=False if this metric should be ommited when computing insights

Metric Function inputs:

np.ndarray tensors with a batch dimension

Metric Function outputs:

np.ndarray batched metric values

Examples

Basic Usage

import numpy as np
import numpy.typing as npt
from code_loader.inner_leap_binder.leapbinder_decorators import tensorleap_custom_metric


@tensorleap_custom_metric(name='metrics',
                          direction=MetricDirection.Downward,
                          compute_insights=None)
def custom_metric_distances(
    y_true: npt.NDArray[np.float32],
    y_pred: npt.NDArray[np.float32]
) -> dict[str, npt.NDArray[np.float32]]:
    diff = y_true - y_pred
    axis = tuple(range(1, diff.ndim))

    return {
        "mean_difference": np.mean(diff, axis=axis),
        "mean_absolute_difference": np.mean(np.abs(diff), axis=axis)
    }

Full script usage can be found at Integration Script.