> For the complete documentation index, see [llms.txt](https://docs.tensorleap.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.tensorleap.ai/tensorleap-integration/python-api/code_loader/datasetclasses/preprocessresponse.md).

# PreprocessResponse

code\_loader.contract.datasetclasses.PreprocessResponse

A `PreprocessResponse` holds one data split: the ids of its samples and whatever data your encoders need to load them. The [preprocess function](https://docs.tensorleap.ai/tensorleap-integration/python-api/code_loader/decorators/tensorleap_preprocess.md) returns one per split. The platform then passes it, with a sample id, to every input encoder, ground truth encoder and metadata function.

```python
from code_loader.contract.enums import DataStateType

@dataclass
class PreprocessResponse:
    length: Optional[int] = None          # deprecated
    data: Any = None
    sample_ids: Optional[Union[List[str], List[int], List[List[str]], List[List[int]]]] = None
    state: Optional[DataStateType] = None
    sample_id_type: Optional[Union[Type[str], Type[int]]] = None
```

| Args             |                                                                                                                                                                                                                                                                                                                                       |
| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `sample_ids`     | The id of each sample in the split, as a `list` (a `tuple` or `range` also works). Ids are `str` by default; for `int` ids, also pass `sample_id_type=int`. NumPy strings (`np.str_`) are accepted as `str`. Other NumPy values are not: convert them with `.tolist()`. A list of lists makes a [grouped](#grouped-samples) response. |
| `data`           | *(Any)* Any object your encoders need, e.g. a dictionary, a `pandas.DataFrame` or a list of file paths.                                                                                                                                                                                                                               |
| `state`          | *(*[*DataStateType*](https://docs.tensorleap.ai/tensorleap-integration/python-api/code_loader/enums/datastatetype.md)*, optional)* Which split this is. Set it explicitly: without it, the split is taken from the response's position in the preprocess list, and the local run warns.                                               |
| `sample_id_type` | *(`str` or `int`, optional)* The type of the ids. Defaults to `str`. Inferred for grouped responses.                                                                                                                                                                                                                                  |
| `length`         | *(int, deprecated)* The number of samples. Passing only `length` creates the int ids `0 … length-1`. Use `sample_ids` instead.                                                                                                                                                                                                        |

Creating a response checks its ids: a wrong container type, a wrong id type, or a mix of types raises an error that says how to fix it.

### Grouped samples

When several samples must be fed to the model together (e.g. the frames of one video clip), pass `sample_ids` as a list of non-empty lists. Each inner list is a **group**. Encoders and metadata functions then receive a whole group as `idx` (a list of ids), and return one result per sample in the group.

In one integration, either every response is grouped or none is. Grouped responses do not support model-computed custom latent spaces, and simulations must return a flat (not grouped) response.

Useful read-only properties: `is_grouped`, `groups` (the list of groups, or `None`), `num_groups`, and `flat_sample_ids` (all ids in order).

## Examples

#### Basic Usage

```python
from code_loader.contract.datasetclasses import PreprocessResponse
from code_loader.contract.enums import DataStateType

# string ids
train = PreprocessResponse(sample_ids=train_df['image_name'].tolist(), data=train_df,
                           state=DataStateType.training)

# integer ids
val = PreprocessResponse(sample_ids=list(range(len(val_df))), sample_id_type=int, data=val_df,
                         state=DataStateType.validation)
```

#### Within the Preprocess Function

Full examples can be found at the **Dataset Integration** section of the following guides:

* [**MNIST Guide**](https://docs.tensorleap.ai/guides/full-guides/mnist-guide.md)
* [**IMDB Guide**](https://docs.tensorleap.ai/guides/full-guides/imdb-guide.md)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.tensorleap.ai/tensorleap-integration/python-api/code_loader/datasetclasses/preprocessresponse.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
