Model Perception Analysis
Last updated
Last updated
To perform this analysis, go to the Dashboard view on the right, and select Analyzer from the list at the top.
The Tensorleap platform tracks how each learned feature, within each layer, responds to each sample. From that information, it constructs a vector that captures how the model perceives each sample. This allows the platform to create a similarity map between samples as they are interpreted by the model. A more intuitive explanation would be that similar samples would activate similar learned features within the model.
For more information on model evaluation and training, see Evaluate/Train Model.
Population Exploration Analysis takes these vectors and runs dimension reduction algorithms in order to visualize the similarities in the UI and provide helpful insights on how various samples are perceived by the model.
In this section, we will perform a Population Exploration analysis. For more information, see Population Exploration.
After each epoch in the training process, a Population Exploration analysis is performed automatically, the results of which can be found under the Analysis panel in the Dashboard view.
Each dot represents a sample (hover your mouse over it to see the sample's preview), while by default the dot's size and color represent the sample's loss. This can be changed to fit your preferences. For example, we can change the color to represent the ground truth label - metadata_label. By doing that, we can easily see the clusters formed for each label. In addition, we can see which label cluster is perceived to be similar to another, and which failing samples are not located within the right cluster.
Further exploration reveals that the cluster that represents the digit 7 is close to the cluster that represents the number 1. This means that the network finds these clusters similar relative to the other clusters.
In the default view, the dot size represents the loss (error). Large dot size is highly correlated with a failed prediction for that sample.
Exploring some of the samples with high loss reveals possible ambiguities. Once a sample dot is clicked, its details are displayed on the right. For example:
As the above image shows, ground truth is 8 (metadata_label: 8), but it can be easily mistaken for a 9.
Note that due to the randomness of the initial weights, your model could converge to a different state, thus rendering different results.
Analysis of a sample returns results from a variety of explainability algorithms. Details about these algorithms can be found at Sample Analysis.
Once the analysis completes, we can explore the model's response to the sample. In the example below, we can see that the model's prediction was 9 while the ground truth is 8.
Additionally, we can see which features contribute to the loss by clicking the LOSS ANALYSIS on the right.
By clicking on the feature_map heat-map on the right, we can analyze and see which features activates for each output. Let's explore the heat maps for output classes 8 and 9.
Click the 8 tab below to see the heat map highlights features correlated with class 8. There is a red spot around the upper loop of the digit 8, but it is fairly subtle
Click the 9 tab below to see the heat map for class 9. The red spots are what the model finds to be highly correlated with class 9.
Below are heat maps for the different classes, each highlights areas that correspond to features in the class:
As explained, Tensorleap tracks how each learned feature responds to each sample. This allows finding samples that the model considers similar to one another, as they activate similar learned features.
When this task completes, it returns a cluster of similar samples:
As can be seen above, the cluster contains various similarly written versions of the number 6.
The model does not have enough samples that are written in this way, and it is recommended to either add such samples to the dataset or label them differently.
Thus far, we've shown how Tensorleap performs population exploration analysis, sample analysis, and analysis using similar samples.
Next, we will add custom metadata to help us find more correlations in our samples and model. Let's now move on to Advanced Metrics.
Click a sample dot to select it, then click at the bottom right to analyze the selected sample.
Another interesting example is this sample - which contains features that can correspond to various digits. In this sample, the model's predictions are non-zero probabilities for various classes.
To present similar samples, select a sample and click the button at the bottom right.
The MNIST dataset includes two ways of writing the number four: and .
Below are samples that are similar to . The presented heat maps highlight the areas in the input that define those similarities.
The model detected 4 samples that are similar to , while the other samples are mostly 9. This means two things:
The model finds to be very similar to 9, and there is a chance that some will be confused by this similarity.
There are also two ways of writing the number seven: and . You can try and analyze this digit in the same manner as above.
89012389
