Download outputs

Introduction

After the live job has been successfully run, the geospatial outputs are ready for download. Currently, UP42 does not support the direct visualization of geospatial outputs and users need to download and save the outputs separately for further rendering in third-party GIS software.

The outputs can be downloaded for the entire job or for each individual job task. UP42 provides 3 result types (see table below): TAR archive, Quicklooks and GeoJSON file.

Job result type	Description
TAR archive	Results (e.g. raster, vector data) delivered as a TAR archive.
Quicklooks	Low resolution, preview images of the actual images. These are only available if the upstream API of the data provider supports them.
GeoJSON file	A file `data.json` that contains geometry information and metadata associated with the job output. The content of this file depends on the workflow and its blocks.

Download TAR archive

Final output

First define a variable for the results endpoint:

RESULTS="https://api.up42.com/projects/$PROJ/jobs/$JOB_ID/downloads/results"

Define a variable for the signed URL that contains the results:

RESULTS_URL=$(curl -s GET -H "Authorization: Bearer $PTOKEN" \
              $RESULTS | jq -j '.data.url')

Download and save the archived result of this job run in your desired folder location. Please note that the output is associated with the last block.

curl -s -L $RESULTS_URL -H "Authorization: Bearer $PTOKEN" \
-o final_output.tar.gz

Individual outputs

Define a variable for the tasks endpoint:

TASKS="https://api.up42.com/projects/$PROJ/jobs/$JOB_ID/tasks"

Extract all the task IDs and define them as separate variables. In this example, we previously added 3 blocks, so the job consists of 3 tasks.

curl -s GET -H "Authorization: Bearer $PTOKEN" $TASKS | \
jq -r '.data[].id'

TASK1=b5c33b12-aa62-4809-94b8-5352bdfecf4a
TASK2=8e390f0b-49ec-4b6b-8e71-6ba3ec3cc13b
TASK3=f0dc96cb-c60a-49bd-aafa-0f6e261908b2

Get the archived results from each task (i.e. the result of each block) by defining one variable for each signed URL:

TASK1_URL=$(curl -s GET -H "Authorization: Bearer $PTOKEN" \
https://api.up42.com/projects/$PROJ/jobs/$JOB_ID/tasks/$TASK1/downloads/results \
| jq -j '.data.url')

TASK2_URL=$(curl -s GET -H "Authorization: Bearer $PTOKEN" \
https://api.up42.com/projects/$PROJ/jobs/$JOB_ID/tasks/$TASK2/downloads/results \
| jq -j '.data.url')

TASK3_URL=$(curl -s GET -H "Authorization: Bearer $PTOKEN" \
https://api.up42.com/projects/$PROJ/jobs/$JOB_ID/tasks/$TASK3/downloads/results \
| jq -j '.data.url')

Download and save the archived results in your desired folder location:

curl -s -L $TASK1_URL -H "Authorization: Bearer $PTOKEN" \
-o task1_output.tar.gz

curl -s -L $TASK2_URL -H "Authorization: Bearer $PTOKEN" \
-o task2_output.tar.gz

curl -s -L $TASK3_URL -H "Authorization: Bearer $PTOKEN" \
-o task3_output.tar.gz

Download quicklooks

Display the job tasks and save them in a separate file (job_tasks.json).

curl -s GET -H "Authorization: Bearer $PTOKEN" \
https://api.up42.com/projects/$PROJ/jobs/$JOB_ID/tasks \
| jq '.' > job_tasks.json

Use the previous file, extract the corresponding task IDs and save them in a new file (task_ids.json).

cat job_tasks.json | jq -r '.data[] | .id' | uniq > task_ids.json

Get the quicklook information for all job tasks and save it into new files.

while read -r taskId; do curl -s -L -H "Authorization: Bearer $PTOKEN" \
https://api.up42.com/projects/$PROJ/jobs/$JOB_ID/tasks/$taskId/outputs/quicklooks \
| jq '.' > task_quicklook_$taskId.json; done < task_ids.json

While the job tasks for the processing blocks do not contain any quicklook information, the data blocks usually generate a quicklook filename. Example:

{
  "data": ["2cb2469a-8b79-42d8-be08-2b0e647c8376.jpeg"],
  "error": null
}

In order to get the quicklooks for one task, we first need to define the job and task IDs. Then we save the quicklooks filenames in a separate file and extract the filenames to download the quicklooks:

JOB_ID=023694d4-1b52-46e8-a9a0-05c9ee23f15c
TASK_ID=299f950b-da2f-45f0-b0a7-a7898d449c75

curl -s -L -H "Authorization: Bearer $PTOKEN" \
https://api.up42.com/projects/$PROJ/jobs/$JOB_ID/tasks/$TASK_ID/outputs/quicklooks \
| jq '.' > task_quicklook_$TASK_ID.json

Use the previously saved file (in this example: task_quicklook_299f950b-da2f-45f0-b0a7-a7898d449c75.json) to download the quicklooks:

for i in $(cat task_quicklook_299f950b-da2f-45f0-b0a7-a7898d449c75.json \
| jq -r '.data[]'); do curl -s -L -O -H "Authorization: Bearer $PTOKEN" \
"https://api.up42.com/projects/$PROJ/jobs/$JOB_ID/tasks/$TASK_ID/outputs/quicklooks/$i"; \
done

Download GeoJSON file

Define a variable for the outputs endpoint:

OUTPUTS="https://api.up42.com/projects/$PROJ/jobs/$JOB_ID/outputs/data-json"

Get the GeoJSON output of this job run (i.e. the output of the last block) and save it in a separate file (job_output.json):

curl -s GET -H "Authorization: Bearer $PTOKEN" $OUTPUTS \
| jq '.' > job_output.json

Get the GeoJSON outputs for all job tasks (i.e. all the intermediate and final outputs) and save them in separate files.

while read -r taskId; do curl -s GET -H "Authorization: Bearer $PTOKEN" \
https://api.up42.com/projects/$PROJ/jobs/$JOB_ID/tasks/$taskId/outputs/data-json \
| jq '.' > task_output_$taskId.json; done < task_ids.json

Congratulations! You managed to create a workflow, add data and processing blocks as workflow tasks, configure, estimate and run a job.