LogoLogo
  • Tensorleap
  • Examples
    • Semantic Segmentation
    • Image Analysis
    • Sentiment Analysis
    • MNIST Project Walkthrough
    • IMDB Project Walkthrough
  • Quickstart using CLI
  • Guides
    • Full Guides
      • MNIST Guide
        • Dataset Integration
        • Model Integration
        • Model Perception Analysis
        • Advanced Metrics
      • IMDB Guide
        • Dataset Integration
        • Model Integration
        • Model Perception Analysis
        • Advanced Metrics
    • Integration Script
      • Preprocess Function
      • Input Encoder
      • Ground Truth Encoder
      • Metadata Function
      • Visualizer Function
      • Prediction
      • Custom Metrics
      • Custom Loss Function
      • Custom Layers
      • Unlabeled Data
      • Examples
        • CelebA Object Detection (YoloV7)
        • Wikipedia Toxicity (using Tensorflow Datasets)
        • Confusion Matrix
        • CelebA Classification (using GCS)
  • Platform
    • Resources Management
    • Project
    • Dataset
    • Secret Manager
    • Network
      • Dataset Node
      • Layers
      • Loss and Optimizer
      • Visualizers
      • Import Model
      • Metrics
    • Evaluate / Train Model
    • Metrics Dashboard
    • Versions
    • Issues
    • Tests
    • Analysis
      • helpers
        • detection
          • YOLO
    • Team management
    • Insights
  • API
    • code_loader
      • leap_binder
        • add_custom_metric
        • set_preprocess
        • set_unlabeled_data_preprocess
        • set_input
        • set_ground_truth
        • set_metadata
        • add_prediction
        • add_custom_loss
        • set_visualizer
      • enums
        • DatasetMetadataType
        • LeapDataType
      • datasetclasses
        • PreprocessResponse
      • visualizer_classes
        • LeapImage
        • LeapImageWithBBox
        • LeapGraph
        • LeapText
        • LeapHorizontalBar
        • LeapImageMask
        • LeapTextMask
  • Tips & Tricks
    • Import External Code
  • Legal
    • Terms of Use
    • Privacy Policy
Powered by GitBook
On this page

Was this helpful?

  1. Guides
  2. Integration Script

Preprocess Function

PreviousIntegration ScriptNextInput Encoder

Last updated 3 years ago

Was this helpful?

The preprocessing_func (custom name) is a preprocess function that is called just once before the training/evaluating process. It prepares the data for later use in input encoders, output encoders, and metadata functions.

from code_loader.contract.datasetclasses import PreprocessResponse

def preprocessing_func() -> List[PreprocessResponse]:
...
    train = PreprocessResponse(length=len(train_X), data=train_df)
    val = PreprocessResponse(length=len(val_X), data=val_df)
    test = PreprocessResponse(length=len(test_X), data=test_df)
    return [train, val, test]

This function returns a List of objects. The elements on that list correspond with the train , validation, and test data slices.

Usage within the full script can be found at the .

Guides

Full examples can be found at the Dataset Integration section of the following guides:

PreprocessResponse
MNIST Guide
IMDB Guide
Dataset Script