Environment Variables

Environmental variables in running blocks

When a block runs on the platform, several environmental variables are set. These variables provide information on the workflow context and any run-time parameters that the block should be aware of.

The current variables are:

• UP42_TASK_PARAMETERS – the parameters for the task (i.e. the running block). These can be query details as well as processing parameters, depending on if it is a data block or a processing block.

• UP42_JOB_MODE – the mode to run a job with (DEFAULT or DRY_RUN).

UP42_TASK_PARAMETERS

The value of the task parameters variable is a JSON payload structured as follows:

{
    "parameter1": "value1",
    "parameter2": "value2"
    //...
}

The content of the JSON payload is variable, though for data blocks it needs to adhere to the STAC query standard. Be aware that the parameters intersects, contains, bbox, limit, and time are special parameters that MUST adhere to their definitions in the STAC query specifications.

See also the allowed parameters for the builtin blocks.

How a block interprets the parameters is up to the block itself, but it should specify what parameters it expects via the manifest parameters.

In Python, the code to read the parameters for a task is simply the following

import os
import json

def fetch_params():
    return json.loads(os.env["UP42_TASK_PARAMETERS"])

UP42_JOB_MODE

The value of the job mode variable is either DEFAULT or DRY_RUN (case-sensitive!).

• DRY_RUN mode allows to fetch only metadata information for a query and is only available for data blocks.

• DEFAULT mode refers to a regular block run.

Using credentials and secrets with blocks

When building custom blocks, you may want to call out to a seperate API, or utilize a custom storage solution for managing stateful data, and for this you will need to give the running block credentials or secrets that it can authorize with.

We do not recommend using block parameters to manage secrets, because these are passed to the running blocks in plain text, and therefore not completely secure.

While a more permanent solution is on the platform roadmap, in the meantime you can use the following technique to include secrets within a block at build-time, so that the secrets themselves do not need to be stored as part of the block code or checked into source control:

• Specify a Docker build arg in your Dockerfile to supply the secret to the containerized environment

• Utilize the value of this build arg to either set an environmental variable, or store the data in a file at build time

For example, in the case of a simple API key, you would set up your Dockerfile along these lines:

FROM my-base-image

ARG manifest
ARG api_key

ENV API_KEY $api_key
LABEL "up42_manifest"=$manifest

# ...

At build time, you would then run the following:

$ docker build . -t <tag-or-url> \
    --build-arg api_key="YOUR_API_KEY_HERE" \
    --build-arg manifest="$(cat UP42Manifest.json)"

to set both the API ey and the manifest values.

Within your block code, accessing the value of this API key is as simple as reading an environmental variable. A Python example would be as follows:

import os

API_KEY = os.environ.get("API_KEY")