Download outputs

Download the results generated by a job run using the API.


Introduction

After the live job has been successfully run, the geospatial outputs are ready for download. Currently, UP42 does not support the direct visualization of geospatial outputs and users need to download and save the outputs separately for further rendering in third-party GIS software.

The outputs can be downloaded for the entire job or for each individual job task. UP42 provides 3 result types (see table below): TAR archive, Quicklooks and GeoJSON file.

Job result typeDescription
TAR archiveResults (e.g. raster, vector data) delivered as a TAR archive.
QuicklooksLow-resolution, preview images of the actual images. These are only available if the upstream API of the data provider supports them.
GeoJSON fileA file data.json that contains geometry information and metadata associated with the job output. The content of this file depends on the workflow and its blocks.

Download TAR archive

Final output

  1. First define a variable for the results endpoint:
RESULTS="https://api.up42.com/projects/$PROJ/jobs/$JOB_ID/downloads/results"
  1. Define a variable for the signed URL that contains the results:
RESULTS_URL=$(curl -s GET -H "Authorization: Bearer $PTOKEN" \
              $RESULTS | jq -j '.data.url')
  1. Download and save the archived result of this job run in your desired folder location. Please note that the output is associated with the last block.
curl -s -L $RESULTS_URL -H "Authorization: Bearer $PTOKEN" \
-o final_output.tar.gz

Individual outputs

  1. Define a variable for the tasks endpoint:
TASKS="https://api.up42.com/projects/$PROJ/jobs/$JOB_ID/tasks"
  1. Extract all the task IDs and define them as separate variables. In this example, we previously added 3 blocks, so the job consists of 3 tasks.
curl -s GET -H "Authorization: Bearer $PTOKEN" $TASKS | \
jq -r '.data[].id'
TASK1=b5c33b12-aa62-4809-94b8-5352bdfecf4a
TASK2=8e390f0b-49ec-4b6b-8e71-6ba3ec3cc13b
TASK3=f0dc96cb-c60a-49bd-aafa-0f6e261908b2
  1. Get the archived results from each task (i.e. the result of each block) by defining one variable for each signed URL:
TASK1_URL=$(curl -s GET -H "Authorization: Bearer $PTOKEN" \
https://api.up42.com/projects/$PROJ/jobs/$JOB_ID/tasks/$TASK1/downloads/results \
| jq -j '.data.url')

TASK2_URL=$(curl -s GET -H "Authorization: Bearer $PTOKEN" \
https://api.up42.com/projects/$PROJ/jobs/$JOB_ID/tasks/$TASK2/downloads/results \
| jq -j '.data.url')

TASK3_URL=$(curl -s GET -H "Authorization: Bearer $PTOKEN" \
https://api.up42.com/projects/$PROJ/jobs/$JOB_ID/tasks/$TASK3/downloads/results \
| jq -j '.data.url')
  1. Download and save the archived results in your desired folder location:
curl -s -L $TASK1_URL -H "Authorization: Bearer $PTOKEN" \
-o task1_output.tar.gz

curl -s -L $TASK2_URL -H "Authorization: Bearer $PTOKEN" \
-o task2_output.tar.gz

curl -s -L $TASK3_URL -H "Authorization: Bearer $PTOKEN" \
-o task3_output.tar.gz

Download quicklooks

  1. Display the job tasks and save them in a separate file (job_tasks.json).
curl -s GET -H "Authorization: Bearer $PTOKEN" \
https://api.up42.com/projects/$PROJ/jobs/$JOB_ID/tasks \
| jq '.' > job_tasks.json
  1. Use the previous file, extract the corresponding task IDs and save them in a new file (task_ids.json).
cat job_tasks.json | jq -r '.data[] | .id' | uniq > task_ids.json
  1. Get the quicklook information for all job tasks and save it into new files.
while read -r taskId; do curl -s -L -H "Authorization: Bearer $PTOKEN" \
https://api.up42.com/projects/$PROJ/jobs/$JOB_ID/tasks/$taskId/outputs/quicklooks \
| jq '.' > task_quicklook_$taskId.json; done < task_ids.json

While the job tasks for the processing blocks do not contain any quicklook information, the data blocks usually generate a quicklook filename. Example:

{
  "data": ["2cb2469a-8b79-42d8-be08-2b0e647c8376.jpeg"],
  "error": null
}
  1. In order to get the quicklooks for one task, we first need to define the job and task IDs. Then we save the quicklooks filenames in a separate file and extract the filenames to download the quicklooks:
JOB_ID=023694d4-1b52-46e8-a9a0-05c9ee23f15c
TASK_ID=299f950b-da2f-45f0-b0a7-a7898d449c75
curl -s -L -H "Authorization: Bearer $PTOKEN" \
https://api.up42.com/projects/$PROJ/jobs/$JOB_ID/tasks/$TASK_ID/outputs/quicklooks \
| jq '.' > task_quicklook_$TASK_ID.json
  1. Use the previously saved file (in this example: task_quicklook_299f950b-da2f-45f0-b0a7-a7898d449c75.json) to download the quicklooks:
for i in $(cat task_quicklook_299f950b-da2f-45f0-b0a7-a7898d449c75.json \
| jq -r '.data[]'); do curl -s -L -O -H "Authorization: Bearer $PTOKEN" \
"https://api.up42.com/projects/$PROJ/jobs/$JOB_ID/tasks/$TASK_ID/outputs/quicklooks/$i"; \
done

Download GeoJSON file

  1. Define a variable for the outputs endpoint:
OUTPUTS="https://api.up42.com/projects/$PROJ/jobs/$JOB_ID/outputs/data-json"
  1. Get the GeoJSON output of this job run (i.e. the output of the last block) and save it in a separate file (job_output.json):
curl -s GET -H "Authorization: Bearer $PTOKEN" $OUTPUTS \
| jq '.' > job_output.json
  1. Get the GeoJSON outputs for all job tasks (i.e. all the intermediate and final outputs) and save them in separate files.
while read -r taskId; do curl -s GET -H "Authorization: Bearer $PTOKEN" \
https://api.up42.com/projects/$PROJ/jobs/$JOB_ID/tasks/$taskId/outputs/data-json \
| jq '.' > task_output_$taskId.json; done < task_ids.json

Congratulations! You managed to create a workflow, add data and processing blocks as workflow tasks, configure, estimate and run a job.