Block manifest

Overview

The block manifest defines block metadata and is used to update the block information every time a new version is published.

Step 1. Create a manifest file

Create a JSON file named UP42Manifest.json in the custom block folder and populate it with manifest parameters. For more information, see manifest schema.

{
  "_up42_specification_version": 2,
  "name": "processing-block-1",
  "type": "processing",
  "display_name": "Processing block template",
  "description": "This is a template processing block.",
  "machine": {
    "type": "small"
  },
  "parameters": {},
  "input_capabilities": {},
  "output_capabilities": {}
}

Step 2. Choose a machine type

An example of setting NVIDIA Tesla K80 GPU as a machine type:

{
  "machine": {
    "type": "gpu_nvidia_tesla_k80"
  }
}

Different use cases require different machine types:

• small
  • Use with a data block that downloads images.
  • Use with a processing block with a very basic algorithm.
• medium
  • Use with a processing block with a basic algorithm. For example, it's used with NDVI.
• large
  • Use with multi-band imagery analysis that can't be parallelized with raster tiling.
• xlarge
  • Use with advanced algorithms. For example, it's used with K-means clustering.
• gpu_nvidia_tesla_k80
  • Use with GPU-intensive ML algorithms. For example, it's used with Super-resolution Sentinel-2.

The chosen machine type affects the infrastructure costs for running a job with this block. For more information on infrastructure costs, see Pricing.

Step 3. Specify additional parameters

You can specify a schema with additional parameters that modify block behavior.

There are no restrictions on the names of the parameters, except for the reserved parameters that should be of a specific type:

An example of parameters:

{
  "parameters": {
    "bbox": { "type": "array", "default": null },
    "intersects": { "type": "geometry", "default": null },
    "contains": { "type": "geometry", "default": null },
    "polarisations": {
      "type": "array",
      "required": false,
      "description": "Requested polarisations for the output",
      "default": ["VV"],
      "items": {
        "type": "string",
        "enum": ["VV", "VH"]
      }
    },
    "calibration_band": { "type": "array", "default": ["sigma"] },
    "speckle_filter": { "type": "boolean", "default": true },
    "mask": { "type": "array", "default": null, "items": { "type": "string", "enum": ["land", "sea"] } },
    "tcorrection": { "type": "boolean", "default": true },
    "clip_to_aoi": { "type": "boolean", "default": false },
    "linear_to_db": { "type": "boolean", "default": true }
  }
}

An example of how to use the example parameters in a job configuration:

{
  "snap-polarimetric:1": {
    "bbox": [13.35, 52.48, 13.39, 52.5],
    "mask": null,
    "contains": null,
    "intersects": null,
    "clip_to_aoi": true,
    "tcorrection": true,
    "linear_to_db": true,
    "polarisations": ["VV"],
    "speckle_filter": true,
    "calibration_band": ["sigma"]
  }
}

Step 4. Set input and output capabilities

Combinations of blocks are limited, which means that a block can only be connected to certain other blocks. Block capabilities dictate the specific input and output types and parameters of the block. For example, a block that outputs a GeoTIFF is compatible with the next block that accepts GeoTIFF as input.

• Input capabilities define what data types are accepted as inputs.
• Output capabilities define what kind of data types are generated by a block. Can be left empty for the last block of a workflow.

Operators

{
  "input_capabilities": {
    "raster": {
      "up42_standard": {
        "format": "DIMAP",
        "sensor": {
          "or": ["Pleiades", "SPOT"] // Uses the OR operator
        },
        "bands": ["red", "green", "blue", "nir", "pan"]
      }
    }
  },
  "output_capabilities": {
    "raster": {
      "up42_standard": {
        "format": "GTiff",
        "bands": {
          "or": [
            ["red", "green", "blue", "nir"],
            ["red", "green", "blue", "nir", "pan"]
          ]
        },
        "sensor": ">", // Forwards data to the next block
        "resolution": ">",
        "processing_level": ">",
        "dtype": ">"
      }
    }
  }
}

{
  "input_capabilities": {
    "raster": {
      "up42_standard": {
        "format": "GTiff"
      }
    }
  },
  "output_capabilities": {
    "raster": {
      "up42_standard": {
        "format": "GTiff",
        "bands": ">",
        "resolution": ">",
        "sensor": ">",
        "dtype": ">",
        "processing_level": ">",
        "tile_width": "${tile_width}", // Inserts values from job parameters
        "tile_height": "${tile_height}"
      },
      "custom": {
        "match_extents": "${match_extents}"
      }
    }
  }
}

Data types

Data types precede other parameters in a capability object and correspond to the types of data that can be created or consumed inside a workflow.

A data type object needs to contain at least one of the following objects: up42_standard or custom.

{
  "input_capabilities": {
    "raster": {
      "up42_standard": {
        "format": "GTiff",
        "dtype": "uint8"
      }
    }
  }
}

{
  "output_capabilities": {
    "vector": {
      "custom": {
        "object_type": "ships"
      }
    }
  }
}

{
  "input_capabilities": {
    "raster": {
      "up42_standard": {
        "format": "GTiff",
        "dtype": "uint8",
        "resolution": 1.5,
        "tile_width": 768,
        "tile_height": 768
      }
    }
  },
  "output_capabilities": {
    "vector": {
      "up42_standard": {
        "format": "GeoJSON",
        "geometry_type": "Polygon"
      },
      "custom": {
        "object_type": "ships"
      }
    }
  }
}

{
  "input_capabilities": {
    "raster": {
      "up42_standard": {
        "format": "GTiff",
        "dtype": "uint8"
      }
    }
  },
  "output_capabilities": {
    "raster": {
      "custom": {
        "format": "PNG",
        "depth": 8,
        "has_alpha": true,
        "dtype": "uint8"
      }
    }
  }
}

Standard capabilities

Step 5. Validate your manifest file

To validate the block manifest, run the following code in your terminal from the block folder:

curl -X POST -H 'Content-Type: application/json' -d @UP42Manifest.json https://api.up42.com/validate-schema/block

If you get this response, the block manifest is valid:

{
  "data": {
    "valid": true,
    "errors": []
  },
  "error": null
}

Otherwise, the endpoint will provide a detailed error message.

If you use an IDE, you can configure it to provide automatic hinting and validation by fetching the manifest schema.

Step 6. Add your manifest to the Dockerfile

Add the following lines below the FROM directive that sets the base image:

ARG manifest

LABEL "up42_manifest"=$manifest