LogoLogo
  • Tensorleap
  • Examples
    • Semantic Segmentation
    • Image Analysis
    • Sentiment Analysis
    • MNIST Project Walkthrough
    • IMDB Project Walkthrough
  • Quickstart using CLI
  • Guides
    • Full Guides
      • MNIST Guide
        • Dataset Integration
        • Model Integration
        • Model Perception Analysis
        • Advanced Metrics
      • IMDB Guide
        • Dataset Integration
        • Model Integration
        • Model Perception Analysis
        • Advanced Metrics
    • Integration Script
      • Preprocess Function
      • Input Encoder
      • Ground Truth Encoder
      • Metadata Function
      • Visualizer Function
      • Prediction
      • Custom Metrics
      • Custom Loss Function
      • Custom Layers
      • Unlabeled Data
      • Examples
        • CelebA Object Detection (YoloV7)
        • Wikipedia Toxicity (using Tensorflow Datasets)
        • Confusion Matrix
        • CelebA Classification (using GCS)
  • Platform
    • Resources Management
    • Project
    • Dataset
    • Secret Manager
    • Network
      • Dataset Node
      • Layers
      • Loss and Optimizer
      • Visualizers
      • Import Model
      • Metrics
    • Evaluate / Train Model
    • Metrics Dashboard
    • Versions
    • Issues
    • Tests
    • Analysis
      • helpers
        • detection
          • YOLO
    • Team management
    • Insights
  • API
    • code_loader
      • leap_binder
        • add_custom_metric
        • set_preprocess
        • set_unlabeled_data_preprocess
        • set_input
        • set_ground_truth
        • set_metadata
        • add_prediction
        • add_custom_loss
        • set_visualizer
      • enums
        • DatasetMetadataType
        • LeapDataType
      • datasetclasses
        • PreprocessResponse
      • visualizer_classes
        • LeapImage
        • LeapImageWithBBox
        • LeapGraph
        • LeapText
        • LeapHorizontalBar
        • LeapImageMask
        • LeapTextMask
  • Tips & Tricks
    • Import External Code
  • Legal
    • Terms of Use
    • Privacy Policy
Powered by GitBook
On this page
  • Project Setup
  • Dataset Block Setup
  • Set Model Layers
  • Add Loss and Optimizer
  • Add Visualizers
  • Save Network Version
  • Training
  • Metrics
  • Up Next - Model Perception Analysis

Was this helpful?

  1. Guides
  2. Full Guides
  3. MNIST Guide

Model Integration

PreviousDataset IntegrationNextModel Perception Analysis

Last updated 2 years ago

Was this helpful?

In this section, we will set up our classification model by either importing or building it.

Project Setup

First step is to with the nameMNIST, and the description Digit Classifier.

The project name, MNIST, is set with the leap init command, as discussed in above.

The project page contains the Network and Dashboard views. To toggle them, click the buttons at the top.

Dataset Block Setup

We'll start by pointing the model's dataset block to our mnist dataset. This will update the Dataset Block with the relevant input.

In the Network view, click the Dataset Block to display the Dataset Details panel on the right, then click Connect Dataset and select the mnist dataset from the list.

A more detailed explanation about this step can be found at .

Set Model Layers

Importing a Model from Tensorflow or PyTorch

Tensorleap can import a model saved by Tensorflow (PB/H5/JSON_TF2) and PyTorch (ONNX). For more information, see .

In this sample code, a simple CNN model is created using Tensorflow and then saved as a file named mnist-cnn.h5.

import tensorflow as tf

input = tf.keras.layers.Input(shape=(28, 28, 1))
layer = tf.keras.layers.Conv2D(32, [3, 3], activation='relu')(input)
layer = tf.keras.layers.MaxPooling2D(pool_size=(2, 2))(layer)
layer = tf.keras.layers.Conv2D(64, [3, 3], activation='relu')(layer)
layer = tf.keras.layers.MaxPooling2D(pool_size=(2, 2))(layer)
layer = tf.keras.layers.Flatten()(layer)
layer = tf.keras.layers.Dropout(0.5)(layer)
output = tf.keras.layers.Dense(10, activation='softmax')(layer)

model = tf.keras.Model(inputs=input, outputs=output)
model.save('mnist-cnn.h5')

For your convenience, you can also download this file here:

Import Model

  1. Set the Revision Name to mnist-cnn and the Model Name to pre-trained.

  2. For File Type, select H5_TF2, then choose mnist-cnn.h5 in the Upload File field.

  3. Click the Import button.

  4. Back in the Network view, set the Dataset Block to point to the mnist Dataset.

Building our Classification Model using UI

In this section, we will add layers, update their properties, and connect them together to form the model.

The model we'll build is based on a small CNN (Convolutional Neural Network):

Layers
Properties

Convo2D

filters=32, kernel_size=(3, 3), activation="relu"

MaxPooling2D

pool_size=(2, 2)

Conv2D

filters=64, kernel_size=(3, 3), activation="relu"

MaxPooling2D

pool_size=(2, 2)

Flatten

Dropout

rate=0.5

Dense

units=10, activation="softmax"

Adding Layers and Their Properties

In the Network view, follow these steps for each layer:

All steps are illustrated below.

Build Model

Tensorleap Model Integration Script

Next, edit the .tensorleap/model.py file and point it to the model defined in mnist_cnn.py:

from pathlib import Path
from mnist_cnn import build_model


def leap_save_model(target_file_path: Path):
    # Load your model
    model = build_model()
    # Save it to the path supplied as an arugment (has a .h5 suffix)
    model.save(target_file_path)

The function leap_save_model gets called by the CLI on leap push, with the argument target_file_path, which is the location where the model is to be saved.

To validate the correctness of our code, run the following command:

leap check --all

Upon a successful check, we can continue to pushing the model:

leap push --model --description=mnist-cnn --model-name=pre-trained --branch-name=master

Opening the Model Version in the UI and Set the Dataset Block

  • Back on the Network view, set the Dataset Block to point to the mnist Dataset Instance, and connect it to the first layer.

Once all layers have been connected to each other, the model should look like this:

Each block shows the calculated output shape affected by the preceding layers.

Add Loss and Optimizer

After completing this section, our model will be ready for training.

  1. Right-click and add the following:

    • Loss -> CategoricalCrossentropy

    • GroundTruth, and set it to Ground Truth - classes

    • Optimizer -> Adam

  2. Connect the last Dense layer and the GroundTruth to the CategoricalCrossentropy block. Additionally, connect the Loss block to the Adam optimizer.

All steps are illustrated below.

Add Visualizers

Dataset Input Visualizer

There must be at least one visualizer connected to the model's input for analysis. To add the Visualizer for the input, follow the steps below.

Within the Network view:

  1. Right-click and choose Visualizer to add it.

  2. Click the Visualizer node to open up the Visualizer Details on the right.

  3. Choose Image from the Selected Visualizer list.

  4. Connect the Dataset node output to the input of the Visualizer node.

Prediction and Ground Truth Visualizer

Additional Visualizers will be connected to the prediction and ground truth, in order to visualize the model's prediction output. Follow the steps below:

Within the Network view:

  1. Right-click and choose Visualizer to add it.

  2. Click the Visualizer node to open up the Visualizer Details on the right.

  3. Choose HorizontalBar from the Selected Visualizer list.

  4. Connect the last Dense layer output to the input of the Visualizer node.

  5. Repeat the steps, and connect the second visualizer to the GroundTruth node's output.

Save Network Version

Great! Your first version is ready to be saved.

Training

Tensorleap can import trained and untrained models. In our case, the model was created from scratch and needs to be trained.

Metrics

Once training begins, you can start tracking metrics in real-time.

Add a Dashboard and Dashlets

Overview

As training progresses, you should see loss values declining and accuracy values increasing. When training is completed, the model achieved an accuracy of 98%.

Up Next - Model Perception Analysis

Tensorleap provides a host of innovative tools for model analysis and debugging. It tracks how each sample flows through each layer, and how each learned feature flows through the model. It also stores metrics in a big dataset to analyze the model's response to data.

In the Network view, click the button to open the Import Model panel.

Once completed, the imported model, mnist-cnn, is added to the view. Position your cursor over that version, click to Open Commit.

Right-click and add the corresponding layer. More info at .

Update the corresponding properties in the Layer Properties view on the right. More info at .

Connect the Dataset to the first layer, and then connect all the layers in order. More info at .

Copy the mnist_cnn.py file to the mnist folder we created in . This file defines the CNN model, and can be found here:

Once leap push completes, open the view in the UI, where you should now see the imported model, mnist_cnn.

Position your cursor over mnist_cnn and click to Open Commit.

In this section, we will set the Categorical Crossentropy loss function, and connect it to both the dataset's ground truth and the last layer in our model. We'll then add an Adam Optimizer block and connect it to the loss block. For more information, see .

Within the view:

Visualizers define how to display tensors within the model graph. For more info, see .

Click the button and set the Revision Name to cnn-2 (Convolution Neural Network with 2 convolutional layers). This adds the new version to the view. For more information, see .

To train the model, click from the top bar. Let's set the Number of Epochs to 10 and click . For more information, see .

To add a new , click and fill in the dashboard name - mnist.

Make sure that the model is selected from the view, and add a new Dashlet.

Click for a , set the name to Loss, and turn on Split series by subset to separate training and validation metrics.

Next, add another for the accuracy. Follow all the previous steps to add it and set the Dashlet Name to Accuracy and set the Y-Axis to metrics.Accuracy. Do not forget turn on Split series by subset.

We've discussed integrating and training our model. It's now time to analyze it. The part of this tutorial will demonstrate various analyses of the model and data.

Once you are ready, proceed to .

Add Layers
Versions
Loss and Optimizer
Network
Visualizers
Versions
next
Model Perception Analysis
Dataset Integration
Create a Project
Dataset Block
Import Model
Dataset Integration
Versions
Versions
Evaluate/Train Model
Metrics Dashboard
Save a Version
Layer Properties
Connections
Line Dashlet
Line Dashlet
612B
mnist_cnn.py
156KB
mnist-cnn.h5
Importing a Model and Dataset Block Setup (click-to-zoom)
Add Layers and Connect them
Connected Layers and Dataset
Add Loss and Optimizer
Add Visualizers
Full Model with Dataset, Visualizers, Layers, Loss and an Optimizer
The newly-saved version appears on the Versions view
Train Model Dialog
Add a Dashboard and a Loss Dashlet
Loss vs Batch
Accuracy vs Batch - Reaching 98% Accuracy