Analysis

Tensorleap provides several innovative tools for model analysis and debugging.

Layout Overview

Model analysis in Tensorleap is performed via the Analyzer panel, which is shown when you click the tab at the top. From this same panel, you can view the results of the analyses performed on your models.

To start analyzing a model, go to the Analyzer panel and select Analyzer from the list at the top.

Population Exploration

Tensorleap tracks how each learned feature, within each layer, responds to each sample. From this information, a vector can be constructed capturing how the model perceives each sample. This allows Tensorleap to perform a similarity map between different samples, as they are perceived by the model. Thus, similar samples would activate similar learned features within a model.

Population Exploration Analysis takes the vector and runs dimension reduction algorithms to 2D representation of the data. This analysis reveals how the model perceives each sample.

Population exploration analysis is performed automatically after each epoch during training (more info in Evaluate / Train Model). An epoch is a complete pass of the training dataset through an algorithm.

For example, if the number of epochs for a model is set at 10, population exploration analysis is also performed 10 times, with each analysis being performed soon after completion of an epoch.

Each dot represents a sample. A preview of the sample is shown when you position your mouse over the dot.

By default, the dot size and color represent each sample's loss. This can be changed to fit your preferences.

Dot size represents the loss (error). Therefore, large dots represent samples that failed prediction.

When a sample dot is clicked, its details are displayed on the right.

Sample Analysis

Analysis of a sample returns results from a variety of explainability and error-analysis algorithms, inn addition to all its visualizations defined by the Visualizers.

You can run Sample Analysis from the Population Exploration view by:

1. Click a dot within the Population Exploration view and selecting a sample.

2. Click on the right panel.

Alternatively, you can run Sample Analysis on a specific sample by:

1. Click on the top of the Analyzer view.

2. Make sure that a target model is selected on the Versions view and click .

3. Set the Data-Slice and Sample index.

4. Click .

After the analysis is complete, you can further explore the model's response to the sample. The list of Visualizers and their outputs can be found on the left, and the error-analysis visualizations can be found on the right.

Fetch Similar

Since Tensorleap tracks the response of every feature in a model, it identifies samples that the model considers as similar to one another.

To request samples that are similar to a selected sample, on the Analyzer panel, click Add New Analysis, then click the button on the bottom right.

This action starts a machine and returns a cluster of similar samples.

In the example above, it returns a cluster contains similar writings of the number 6.

Unlabeled Data

You can use the Fetch Similar analysis to help prioritizing samples for labeling. This can be done by looking for similar samples in an unlabeled data.

In order to run the analysis on unlabeled data, you need to add it to the Integration Script, and then perform the analysis on that data. More info at Unlabeled Data.