Preprocess Function

The preprocessing_func (custom name) is a preprocess function that is called just once before the training/evaluating process. It prepares the data for later use in input encoders, output encoders, and metadata functions.

from code_loader.contract.datasetclasses import PreprocessResponse

def preprocessing_func() -> List[PreprocessResponse]:
...
    train = PreprocessResponse(length=len(train_X), data=train_df)
    val = PreprocessResponse(length=len(val_X), data=val_df)
    test = PreprocessResponse(length=len(test_X), data=test_df)
    return [train, val, test]

This function returns a List of PreprocessResponse objects. The elements on that list correspond with the train , validation, and test data slices.

Usage within the full script can be found at the Dataset Script.

Guides

Full examples can be found at the Dataset Integration section of the following guides:

• MNIST Guide

• IMDB Guide