Storing data

Storing and accessing data in blocks.

The default storage mechanism for UP42 is stateless, in that each job that runs does not have access to the results of previous jobs.

Additionally, each block only has the output data from the previous block available, meaning that if you want to combine results from the previous block with results from another block several steps back in the workflow, you must either pass that data forward, or store it in another location for later access.

While a more permanent solution is on the platform roadmap, in the meantime you can use the following technique to store data for access outside the existing mechanism:

For example, given the following steps:

  1. Fetch a Sentinel-2 full scene.

  2. Create a cloud-optimized GeoTIFF from the SAFE archive.

  3. Store the COG for later use in another workflow.