Loss and Optimizer

Ease training by adding a Loss function and Optimizer to your network

The goal of neural networks is generally to minimize error. This is why the objective function is commonly referred to as a cost function or a loss function, and the value calculated by the loss function is simply referred to as "loss."

The optimizer is a function or algorithm that modifies the neural network's parameters in such a way that it tries to reduce the overall loss, thereby increasing the estimation accuracy.

This section describes how to set up the loss function and optimizer at the output (prediction) of our model.

After adding layers, the final steps in setting up a network are:

• Adding a Loss Function and connecting it to the prediction from the model's last layer(s) in the network. The Loss function is also connected to the target value that is defined by the dataset's ground truth.

• Adding an Optimizer and connecting the Loss function to it.

• Adding a Visualizer (see Visualizers)

Typical setup of Ground Truth, Loss, and Optimizer in a network

Add a Loss Node

You may add the ground truth before the Loss function.

The order in which the Ground Truth and Loss function are added is irrelevant to the network, as long as the Ground Truth passes its labeled data to the Loss function.

To add the ground truth:

1. On the Network view, right-click anywhere and select GroundTruth from the menu.

2. When the GroundTruth node appears in the Network view, click it to open a panel to the right. From here, you can view and edit the GroundTruth node's properties.

To add a Loss function:

1. On the Network view, right-click anywhere and select the preferred Loss function from the menu.

2. When the Loss node appears in the Network view, click it to open a panel to the right. From here, you can view and edit the Loss node's properties.

3. Connect the last layer on the network to the prediction input of the Loss node.

4. Connect the GroundTruth node to the ground_truth input of the Loss node.

Adding the Loss function and connecting it to the GroundTruth

Add Custom Loss

Adding a CustomLoss node let you select your own Custom Loss Function and use it as your loss function.

To add a Custom Loss:

1. In the Integration Script write your own Custom Loss Function, use the add_custom_loss function to add the function to the custom losses, and Save.

2. On the Network view, right-click anywhere and select the preferred CustomLoss from the menu.

3. When the CustomLoss node appears in the Network view, click it to open a panel to the right. From here, you can set the Selected Custom Loss to the Custom Loss Function.

4. Connect the CustomLoss node as described here.

Add an Optimizer node

An Optimizer is added after the Loss function.

To add an Optimizer:

1. On the Network view, right-click anywhere and select the preferred Optimizer from the menu.

2. When the Optimizer node appears in the Network view, click it to open a panel to the right. From here, you can view and edit the Optimizer node's properties.

3. Connect the Loss function to the Optimizer.

Adding the Optimzer and connecting it to the Loss function