Metric Node

The Metric Node computes the metric for your model

The Tensorleap platform allows you to add metrics to be computed in each model evaluation

These Metrics are divided into two types:

• Default metrics

  • MeanSquaredError

  • MeanSquaredLogarithmicError

  • MeanAbsoluteError

  • MeanAbsolutePercentageError

  • Accuracy

  • BinaryAccuracy

  • MeanIOU

• Custom metrics

It is highly recommended to use your own custom metrics in the integration script. This would allow you to test results in your local development environment before you upload the model to Tensorleap. This helps to ensure the metrics return the expected values before the integration

Custom Metric Example

Adding the following metric in your code:

@tensorleap_custom_metric("cost")
def cost(pred80,pred40,pred20,gt):
    gt=np.squeeze(gt,axis=0)
    d={}
    d["bboxes"] = torch.from_numpy(gt[...,:4])
    d["cls"] = torch.from_numpy(gt[...,4])
    d["batch_idx"] = torch.zeros_like(d['cls'])
    y_pred_torch = [torch.from_numpy(s) for s in [pred80,pred40,pred20]]
    _,loss_parts= criterion(y_pred_torch, d)
    return {"box":loss_parts[0].unsqueeze(0).numpy(),"cls":loss_parts[1].unsqueeze(0).numpy(),"dfl":loss_parts[2].unsqueeze(0).numpy()}

Would result in the following metric in selectable from the menu:

Metric Block

In order to add a Custom metric it must be added to the dataset script and parsed first as described in the

Setup

The Metric node have several properties:

• Selected Metric: a dropdown from which a custom or default metric can be selected. The list would include all registered custom metrics from your integration script.

• Name: The name of the selected visualizer.

• Labels: The expected arguments for your visualizer.