LogoLogo
  • Tensorleap
  • Examples
    • Semantic Segmentation
    • Image Analysis
    • Sentiment Analysis
    • MNIST Project Walkthrough
    • IMDB Project Walkthrough
  • Quickstart using CLI
  • Guides
    • Full Guides
      • MNIST Guide
        • Dataset Integration
        • Model Integration
        • Model Perception Analysis
        • Advanced Metrics
      • IMDB Guide
        • Dataset Integration
        • Model Integration
        • Model Perception Analysis
        • Advanced Metrics
    • Integration Script
      • Preprocess Function
      • Input Encoder
      • Ground Truth Encoder
      • Metadata Function
      • Visualizer Function
      • Prediction
      • Custom Metrics
      • Custom Loss Function
      • Custom Layers
      • Unlabeled Data
      • Examples
        • CelebA Object Detection (YoloV7)
        • Wikipedia Toxicity (using Tensorflow Datasets)
        • Confusion Matrix
        • CelebA Classification (using GCS)
  • Platform
    • Resources Management
    • Project
    • Dataset
    • Secret Manager
    • Network
      • Dataset Node
      • Layers
      • Loss and Optimizer
      • Visualizers
      • Import Model
      • Metrics
    • Evaluate / Train Model
    • Metrics Dashboard
    • Versions
    • Issues
    • Tests
    • Analysis
      • helpers
        • detection
          • YOLO
    • Team management
    • Insights
  • API
    • code_loader
      • leap_binder
        • add_custom_metric
        • set_preprocess
        • set_unlabeled_data_preprocess
        • set_input
        • set_ground_truth
        • set_metadata
        • add_prediction
        • add_custom_loss
        • set_visualizer
      • enums
        • DatasetMetadataType
        • LeapDataType
      • datasetclasses
        • PreprocessResponse
      • visualizer_classes
        • LeapImage
        • LeapImageWithBBox
        • LeapGraph
        • LeapText
        • LeapHorizontalBar
        • LeapImageMask
        • LeapTextMask
  • Tips & Tricks
    • Import External Code
  • Legal
    • Terms of Use
    • Privacy Policy
Powered by GitBook
On this page
  • Population Exploration Analysis
  • Sample Analysis
  • Fetch Similar
  • Up Next - Advanced Metrics

Was this helpful?

  1. Guides
  2. Full Guides
  3. MNIST Guide

Model Perception Analysis

PreviousModel IntegrationNextAdvanced Metrics

Last updated 3 years ago

Was this helpful?

To perform this analysis, go to the Dashboard view on the right, and select Analyzer from the list at the top.

Population Exploration Analysis

The Tensorleap platform tracks how each learned feature, within each layer, responds to each sample. From that information, it constructs a vector that captures how the model perceives each sample. This allows the platform to create a similarity map between samples as they are interpreted by the model. A more intuitive explanation would be that similar samples would activate similar learned features within the model.

For more information on model evaluation and training, see .

Population Exploration Analysis takes these vectors and runs dimension reduction algorithms in order to visualize the similarities in the UI and provide helpful insights on how various samples are perceived by the model.

In this section, we will perform a Population Exploration analysis. For more information, see .

After each epoch in the training process, a Population Exploration analysis is performed automatically, the results of which can be found under the Analysis panel in the Dashboard view.

Each dot represents a sample (hover your mouse over it to see the sample's preview), while by default the dot's size and color represent the sample's loss. This can be changed to fit your preferences. For example, we can change the color to represent the ground truth label - metadata_label. By doing that, we can easily see the clusters formed for each label. In addition, we can see which label cluster is perceived to be similar to another, and which failing samples are not located within the right cluster.

Further exploration reveals that the cluster that represents the digit 7 is close to the cluster that represents the number 1. This means that the network finds these clusters similar relative to the other clusters.

In the default view, the dot size represents the loss (error). Large dot size is highly correlated with a failed prediction for that sample.

Exploring some of the samples with high loss reveals possible ambiguities. Once a sample dot is clicked, its details are displayed on the right. For example:

As the above image shows, ground truth is 8 (metadata_label: 8), but it can be easily mistaken for a 9.

Note that due to the randomness of the initial weights, your model could converge to a different state, thus rendering different results.

Sample Analysis

Once the analysis completes, we can explore the model's response to the sample. In the example below, we can see that the model's prediction was 9 while the ground truth is 8.

Additionally, we can see which features contribute to the loss by clicking the LOSS ANALYSIS on the right.

By clicking on the feature_map heat-map on the right, we can analyze and see which features activates for each output. Let's explore the heat maps for output classes 8 and 9.

Click the 8 tab below to see the heat map highlights features correlated with class 8. There is a red spot around the upper loop of the digit 8, but it is fairly subtle

Click the 9 tab below to see the heat map for class 9. The red spots are what the model finds to be highly correlated with class 9.

Below are heat maps for the different classes, each highlights areas that correspond to features in the class:

Fetch Similar

As explained, Tensorleap tracks how each learned feature responds to each sample. This allows finding samples that the model considers similar to one another, as they activate similar learned features.

When this task completes, it returns a cluster of similar samples:

As can be seen above, the cluster contains various similarly written versions of the number 6.

Investigating the Closed 4 Digit

  • The model does not have enough samples that are written in this way, and it is recommended to either add such samples to the dataset or label them differently.

Up Next - Advanced Metrics

Thus far, we've shown how Tensorleap performs population exploration analysis, sample analysis, and analysis using similar samples.

Analysis of a sample returns results from a variety of explainability algorithms. Details about these algorithms can be found at .

Click a sample dot to select it, then click at the bottom right to analyze the selected sample.

Another interesting example is this sample - which contains features that can correspond to various digits. In this sample, the model's predictions are non-zero probabilities for various classes.

To present similar samples, select a sample and click the button at the bottom right.

The MNIST dataset includes two ways of writing the number four: and .

Below are samples that are similar to . The presented heat maps highlight the areas in the input that define those similarities.

The model detected 4 samples that are similar to , while the other samples are mostly 9. This means two things:

The model finds to be very similar to 9, and there is a chance that some will be confused by this similarity.

There are also two ways of writing the number seven: and . You can try and analyze this digit in the same manner as above.

Next, we will add custom metadata to help us find more correlations in our samples and model. Let's now move on to .

Advanced Metrics
Evaluate/Train Model
Sample Analysis
Population Exploration
Color Dots by Label
Sample's Details
Sample Ground Truth vs Prediction
Heatmaps Correlated for the Loss
Input Heat Map for 8
Input Heat Map for 9
Heat Map for Class 0
Heat Map for Class 1
Heat Map for Class 2
Heat Map for Class 3
Heat Map for Class 4
Heat Map for Class 5
Heat Map for Class 6
Heat Map for Class 7
Heat Map for Class 8
Heat Map for Class 9
Fetch Similar
Fetch Similar Results
Similar Samples
Similar Samples Heat Map