MNIST Project Walkthrough
The MNIST project, included in the Free Trial, uses the MNIST database (handwritten digits) with a simple classification model.
In the following steps, you will train the model, view the analytics, and perform basic analyses. For a more in-depth guide to the MNIST use-case, see the MNIST Guide.
To open the project, in the Welcome screen, go to Projects and click
MNIST:
The Network tab displays the model's nodes and connections in a simple convolutional neural network (CNN) model:
You can zoom in and out using the scroll wheel, and pan by dragging the background. Click on a node to view its details.
The orange node on the left relates to the MNIST dataset (the script can be viewed in the Resources Management).
The light blue nodes seen in the center of the model represent the model's layers, and the colored nodes at the end of the model represent the Loss and Optimizer.
The model includes Tensorleap Visualizer nodes (dark blue) to extract visualizations from different outputs.
The pre-saved projects provided are not yet trained. They must be trained before viewing the analytics and analyses.
To train the model for one epoch, on the top bar, click 
and 
in the training dialog:
Training (click-to-zoom)
Once training has initiated, a
PENDING notification will appear indicating that the training process is initializing. This could take a minute or so. Once the training begins you will see a STARTED notification.Training will take somewhere between 10 to 45 minutes, depending on your system resources. You can track the status by clicking 
.
To display the model's analytics on the dashboard, on the top left of the dashboard, click 
to open the Versions view. Expand the version and make sure that the current model is selected, as shown in the video below:
Metrics and Analytics
- Loss (error) vs Batch - decreases as training progresses
- Accuracy vs Batch - increases as training progresses
- List of samples ordered by loss, from high to low
- Image Mean (a.k.a brightness) vs Loss - validation samples with high loss values tend to fail
- Distance from the Class Centroid vs Loss (the class centroid is the averaged image of all the images within the class) - the more unique the samples are, the more they tend to fail
Tensorleap creates a latent space that is relatively close to the entire model's latent space. This latent space is composed of feature activations from all of the model's layers in order to distribute the data in the most informative way.
Using this information, Tensorleap creates a similarity map between samples as they are interpreted by the model. A more intuitive explanation would be that similar samples would activate similar learned features within the model.
This similarity map is called a Population Exploration analysis, and it is performed automatically after each epoch.
To find patterns in the similarity map that correlate with selected metrics and metadata, perform the following steps (also shown in the video below):
Perform the following steps only after training has finished
- 1.On the top left click.
- 2.Resize the Population Exploration analysis panel, by dragging the bottom right corner.
- 3.Color the dots by their ground-truth label, clicking, where the
lossis currently selected, and change it to(ground truth label).
Population Exploration (click-to-zoom)
Notice that clusters are formed when projecting the samples onto the latent space. The clusters represent how the model perceives the data. When coloring the dots according to the ground truth label, we can see that each cluster correlates to a class. Large dots represent high loss values for these samples. Hover over the dots to see a preview of the sample.
Note how the model misinterpreted some of the samples - shown by the large dots and multi-colored samples within clusters.
Click each sample to show its preview, metadata, and metrics. To analyze a sample, select it and click 
on the right panel, as shown below:
Tensorleap analyzes the sample and displays the results in the Analyzer panel. This process should take a few moments. The next section describes the Sample Analysis.
The Sample Analysis tool runs explainability algorithms on selected samples and displays the visualizations correlated with the Visualizer blocks.
In the example shown below, a sample from the
validation subset is analyzed. The horizontal bar graph reveals that the ground truth for the sample is 8, while the model interprets it as 6.
Sample Analysis
Ground Truth
Prediction
Next, the
Loss Analysis shows a heat-map of the features that were correlated with the wrong prediction. In this case, a corner on the top left.
Sample Loss Analysis
Moreover, you can access the heat-maps corresponding to each of the output classes by clicking the heat-map on the right.
Congratulations on completing this short MNIST walkthrough for the Free Trial. This is a rather simple use-case and model, which achieves more than
98% accuracy on the Validation subset.Next, you can follow the full guide at MNIST Guide, which takes you through dataset integration, model building, and importing, as well as reviewing and analyzing additional metrics.
Last modified 7mo ago