Secret Manager

To add, update, retrieve, and delete Secrets, use the Secret Manager

A Secret is a string that contains sensitive data and needs to be stored securely. Usually, this string contains sensitive credentials, such as login information or certificate keys, for accessing data on a cloud or a service. Common use-cases are secrets to AWS or GCP buckets that contain data.

Tensorleap provides a Secret Manager to manage these secrets. Each secret is stored securely, and only a machine running a task can access it.

Add a Secret

A common use of a Secret is to store credentials that allow a Dataset Instance to access the storage and fetch the data. A Secret can later be retrieved by a dataset script (see Retrieve a Secret).

To add a Secret:

1. Click the side menu, then Resources Management.

2. On the Resources Management window, under the Secrets section, click +.

3. On the Secret Manager, enter a name for the secret, then either drag and drop a json file that contain the secret OR copy paste the secret into the appropriate location

5. Click Upload File. The record is then added to the list under the Secrets section.

Update a Secret

Since Secrets are stored securely in our servers, only a machine running a dataset script can access a Secret normally. If you need to update a Secret, you can do so via the Tensorleap UI, but you cannot view the content of the Secret you previously entered.

To update a Secret:

1. Click the side menu, then Resources Management.

2. On the Resources Management window, under the Secrets section, look for the Secret to be retrieved from the list.

3. When you find the Secret, position your cursor over the record, then click and Update Secret to display the Secret's details on the right.

4. Either drag and upload the updated secret key, or copy and paste it to the Secret Manager, then click Save.

Use a Secret

Secrets can be securely used by the Integration script, allowing it to access sensitive data stored in your project. After creating a secret, set it in your project by running the following command in the project directory:

leap secrets set

You’ll be prompted to select the secret you created. Once set, the secret is exposed to your script as the environment variable AUTH_SECRET.

You can access and parse this secret in your code as follows:

auth_secret_string = os.environ['AUTH_SECRET']
auth_secret = json.loads(auth_secret_string)

This ensures your script securely reads and uses the secret value at runtime without hardcoding sensitive information.

Note that when running an integration test locally, the AUTH_SECRET environment variable is not set and you should set it manually.

Delete a Secret

If you do not need a Secret anymore, you may delete it to make your list of Secrets more manageable.

Only Secrets that are not in use by any of the dataset instances can be deleted.

To delete a Secret:

1. Click the side menu, then Resources Management.

2. On the Resources Management window, under the Secrets section, look for the Secret to be deleted from the list.

3. When you find the Secret, position your cursor over the record, then click and Delete Secret.