MNIST Project Walkthrough
The MNIST project, included in the Free Trial, uses the MNIST database (handwritten digits) with a simple classification model.
In the following steps, you will train the model, view the analytics, and perform basic analyses. For a more in-depth guide to the MNIST use-case, see the MNIST Guide.
To open the project, in the Welcome screen, go to Projects and click MNIST:
The Network tab displays the model's nodes and connections in a simple convolutional neural network (CNN) model:
You can zoom in and out using the scroll wheel, and pan by dragging the background. Click on a node to view its details.
The orange node on the left relates to the MNIST dataset (the script can be viewed in the Resources Management).
The light blue nodes seen in the center of the model represent the model's layers, and the colored nodes at the end of the model represent the Loss and Optimizer.
The model includes Tensorleap Visualizer nodes (dark blue) to extract visualizations from different outputs.
Training
The pre-saved projects provided are not yet trained. They must be trained before viewing the analytics and analyses.
Once training has initiated, a PENDING notification will appear indicating that the training process is initializing. This could take a minute or so. Once the training begins you will see a STARTED notification.
Metrics and Analytics
This Dashboard includes the following Dashlets:
Loss (error) vs Batch - decreases as training progresses
Accuracy vs Batch - increases as training progresses
List of samples ordered by loss, from high to low
Image Mean (a.k.a brightness) vs Loss - validation samples with high loss values tend to fail
Distance from the Class Centroid vs Loss (the class centroid is the averaged image of all the images within the class) - the more unique the samples are, the more they tend to fail
Population Exploration
Tensorleap creates a latent space that is relatively close to the entire model's latent space. This latent space is composed of feature activations from all of the model's layers in order to distribute the data in the most informative way.
Using this information, Tensorleap creates a similarity map between samples as they are interpreted by the model. A more intuitive explanation would be that similar samples would activate similar learned features within the model.
This similarity map is called a Population Exploration analysis, and it is performed automatically after each epoch.
To find patterns in the similarity map that correlate with selected metrics and metadata, perform the following steps (also shown in the video below):
Perform the following steps only after training has finished
Resize the Population Exploration analysis panel, by dragging the bottom right corner.
Notice that clusters are formed when projecting the samples onto the latent space. The clusters represent how the model perceives the data. When coloring the dots according to the ground truth label, we can see that each cluster correlates to a class. Large dots represent high loss values for these samples. Hover over the dots to see a preview of the sample.
Note how the model misinterpreted some of the samples - shown by the large dots and multi-colored samples within clusters.
To learn more about the Population Exploration analysis, see the Full Guides.
Tensorleap analyzes the sample and displays the results in the Analyzer panel. This process should take a few moments. The next section describes the Sample Analysis.
Sample Root Cause Analysis
The Sample Analysis tool runs explainability algorithms on selected samples and displays the visualizations correlated with the Visualizer blocks.
In the example shown below, a sample from the validation subset is analyzed. The horizontal bar graph reveals that the ground truth for the sample is 8, while the model interprets it as 6.
Next, the Loss Analysis shows a heat-map of the features that were correlated with the wrong prediction. In this case, a corner on the top left.
Moreover, you can access the heat-maps corresponding to each of the output classes by clicking the heat-map on the right.
Summary
Congratulations on completing this short MNIST walkthrough for the Free Trial. This is a rather simple use-case and model, which achieves more than 98% accuracy on the Validation subset.
Next, you can follow the full guide at MNIST Guide, which takes you through dataset integration, model building, and importing, as well as reviewing and analyzing additional metrics.
You can also check our Full Guides for more advanced use cases.
To learn about integrating your custom data into the platform, see the Integration Script.
For more information about the Standalone Trial, see Quickstart Standalone Trial.
Last updated
