Validate mapping

How to ensure you created a valid mapping

It is highly recommended to utilize the integration test locally before validating your assets in the platform. This allows you to ensure a smooth code flow, and to fix any issues within your local debugging environment

When should assets be validated?

Assets in the platform should be validated after

• A new mapping was created or modified in the platform

• Changes was made to the linked code integration script causing data flow to change

• A new model was uploaded to the platform, with a different architecture than previous models

If you're unsure whether the assets for this specific configuration was tested, check the color of the Validate Assets button in the code integration view of the Network.

If it's blue - no validation was tested on this configuration yet

If it's green - the assets in this configuration are valid

If it's red - The assets currently used in the platform are not valid

How to validate assets in the platform

An example of a mapping created but not validated

To validate assets in the platform, we go to the Network Tab, and ensure:

• (optional) A fully completed Network Wizard indication, with all green markings.

• A green "Validate Assets" Button within the Code mapping tab in the right of the Network tab.

The Network Wizard ensures you would have all the assets needed for a complete experience of the Tensorleap platform. Since some of the assets (metrics) are not mendatory for the analysis to begin, it does not have to be completed before validating the ,apping

Validate Assets

The "validate assets" process in Tensorleap aims to ensure current mapping that exists in the network graph is valid. Upon a click on the "Validate Assets" button - Tensorleap starts a process that would use your integration script to load data, run it throught the model, and visualize the inputs. A successful execution of this process means that your model and data is fully integrated & configured in Tensorleap, and you can begin the evaluation process. The validate assets does the following:

• It loads the code you set up in the code integration tab

• It initializes your PreProcessing Function

• It runs the first sample from your dataset and:

  • Use the Input Encoder to load the input

  • Use the Ground Truth Encoder (if present) to load the ground truth

  • Use the Metadata Encoders (if present) to load the metadata

• It then utilize the Network Mapping to:

  • Feed the input to your model and infer it

  • Run connected metrics, loss, and visualizers

A successful assets validation indication in the platform.

Once the validate assets process is successfully finished it is recommended to save it to file to ensure a re-use upon repeating uploads of the same model.

Assets validation mostly fail due to these reasons:

• A bug or unexpected use of the integration script. Can be mostly prevented with integration testing.

• Connecting an asset to the wrong input. It is important to notice that the mapping connection are as the integration script expect them to be.

A successful validate assets does not ensure the loss, metrics, or visualizers would behave as you might have expected, especially, for custom defined encoders. It only ensures the Tensorleap platform is able to propagate data through your model and into these functions and get a valid reponse. We highly recommend utilizing the integration test ensure your custom defined properties return the expected result.

Upon dataset parse (i.e. uploading a codebase to the model) inputs, ground truth, and metadata are also being run by Tensorleap. The process of "Validate Assets" is to validate the rest of the assets, defined by your mapping: i.e. model inference validity, metrics, losses, and visualizers.