Introduction
In this article, you will learn how to further process geospatial datasets that you previously purchased either through the data platform (collections, catalog or tasking orders) or the processing platform (data blocks) and are delivered to storage.
Currently, UP42 provides 2 methods for processing datasets from storage: the data block Processing from Storage and the OneAtlas data blocks.
The difference between the 2 methods is explained in the table below.
Actions | Processing from Storage | OneAtlas Data Blocks |
---|---|---|
Process previously ordered datasets (collections, catalog) | ✅ | |
Process previously ordered tasking datasets | ✅ | |
Process datasets previously retrieved through data blocks | ✅ | ✅ |
This feature is intended for advanced users and it requires basic familiarity with JavaScript Object Notation.
Asset Identifiers
An asset is a unique item generated from a successful job run using a OneAtlas data block, a completed order or a fresh image captured through sensor tasking and it contains the downloadable geospatial dataset.
Every asset gets a unique asset identifier (ID). The newly-generated asset ID is displayed on the storage dashboard, where each asset can be downloaded.
These asset IDs can be used to further process your previously delivered datasets on the UP42 platform.
Processing from Storage
Requirements
The objective of this method is to further process geospatial datasets that have been delivered to storage through job runs of workflows or ordering (collections, catalog or tasking orders).
Steps
Please follow the steps below:
- Go to Storage.
- In the dashboard, locate the asset name, then scroll to the right.
- Click on the first grey circle and click Image Information.
- Scroll to the right and copy the asset ID under Asset Identifier.
- Create a new workflow and add the block Processing from Storage. For more information about building workflows, please refer to Create a Workflow.
- In the job configuration window, find the parameter
asset_ids
and paste the asset ID in an array. For more information about jobs, please refer to Configure Job. - You don’t need to adjust the JSON parameters of the processing block that follows the data block.
- Click Run Job.
- Done.
Example
A workflow that contains the data block Processing from Storage, the processing block Pansharpening and multiple asset IDs:
Please note that you can add more asset IDs inside an array, where each item is separated by a comma. Example: ["3475e76d-4e04-420e-a683-bc4831df824f", "d266389c-798c-42b5-a164-721caed9b0c5"]
.
Please note that the capabilities between this block and the following processing blocks cannot be verified. We recommend exploring the capabilities of the corresponding dataset. For instance, the SPOT 6/7 Display or SPOT 6/7 Reflectance product will have different compatible processing blocks.
OneAtlas Data Blocks
Requirements
This section assumes that you already created a workflow that contains a OneAtlas data block and ran a job of the workflow. For more information, please refer to Create a Workflow.
If you run a job of this workflow for the first time, a new dataset is retrieved and an asset ID is generated.
Currently, the OneAtlas data blocks that generate asset IDs are shown in the table below.
OneAtlas Data Blocks | Asset IDs |
---|---|
Pléiades Display (Download) | ✅ |
Pléiades Reflectance (Download) | ✅ |
SPOT 6/7 Display (Download) | ✅ |
SPOT 6/7 Reflectance (Download) | ✅ |
To process the datasets that were previously retrieved using OneAtlas data blocks, you need to use the same OneAtlas data block that was used the first time. For example, if you previously ran a job of a workflow that contains a Pléiades Reflectance (Download) data block, the newly-generated asset ID can only be used with the same Pléiades Reflectance (Download) data block.
Steps
Please follow the steps below:
- Go to Storage.
- In the dashboard, locate the asset name, then scroll to the right.
- Click on the rightmost grey circle to copy the asset ID. Please note that only the assets with metadata can be processed with OneAtlas data blocks. Assets with metadata are indicated by this rightmost grey circle.
- Go to your previously-created workflow and click on Update & Configure Job.
- In the job configuration window, find the parameter
asset_ids
and paste the asset ID in an array. For more information about jobs, please refer to Configure Job. - You don’t need to adjust the JSON parameters of the processing block that follows the data block.
- Click Run Job.
- Done.
Example
A workflow that contains the data block Pleiades Display (Download) and the processing block DIMAP→ GeoTIFF Conversion:
This functionality relies on the presence of the file data.json
, which contains the asset metadata. This file is generated during data retrieval through data blocks.
Please note that you can also add more asset IDs in the form of an array, where each item is separated by a comma. Example: ["3475e76d-4e04-420e-a683-bc4831df824f", "d266389c-798c-42b5-a164-721caed9b0c5"]
.