Processing in SDK

Run processing jobs using the SDK.


Overview

Apply advanced processing to purchased geospatial data.

View repository

Class: ValidationError

A data class that represents errors encountered during job validation.

Attributes

AttributeDescription
message

str

The message of the validation error.

name

str

The name of the validation error.

Class: JobStatus

An enumeration class that defines possible statuses for a job.

Constants

ConstantDescriptionValueTerminal status
CREATEDThe job has been created, but not yet accepted.createdNot terminal inline-icon-large
LICENSEDThe EULA has been accepted for this process. The job will move on to the validation stage.licensedNot terminal inline-icon-large
UNLICENSEDThe EULA hasn’t been accepted for this process. The job will fail. Accept the EULA through the console or the API and try again.unlicensedTerminal inline-icon-large
VALIDThe job has been validated.validNot terminal inline-icon-large
INVALIDThe job has failed validation. Fix your input parameters and try again.invalidTerminal inline-icon-large
ACCEPTEDThe job has been validated and accepted. The credits corresponding to the price are temporarily withheld until the job is completed.acceptedNot terminal inline-icon-large
REJECTEDThe job has been canceled due to an insufficient account balance. Top up your balance and try again.rejectedTerminal inline-icon-large
RUNNINGThe job is in progress.runningNot terminal inline-icon-large
SUCCESSFULThe job has been completed, and the results have been delivered to your storage.successfulNot terminal inline-icon-large
FAILEDThe job has failed.failedNot terminal inline-icon-large
CAPTUREDThe job has been completed, the results have been delivered to your storage, and any withheld credits have been deducted from your account balance.capturedTerminal inline-icon-large
RELEASEDThe job has failed. Any withheld credits have been released. Try again, and if the issue persists, contact support.releasedTerminal inline-icon-large

Class: JobSorting

A class that provides sorting options for jobs.

Attributes

AttributeDescription
process_id

utils.SortingField

Sorts by process ID. The default order is ascending.

status

utils.SortingField

Sorts by job status. The default order is descending.

created

utils.SortingField

Sorts by creation date. The default order is descending.

credits

utils.SortingField

Sorts by credit consumption. The default order is descending.

Python
from itertools import islice
# Sort by cost, from the least expensive to the most expensive
jobs_sorted = up42.Job.all(sort_by=up42.JobSorting.credits.asc)
10 collapsed lines
# Sort by creation date, from the most recent to the earliest
# jobs_sorted = up42.Job.all(sort_by=up42.JobSorting.created.desc)
# Sort by job status, in descending alphabetical order
# jobs_sorted = up42.Job.all(sort_by=up42.JobSorting.status.desc)
# Sort by process ID in ascending alphabetical order
# jobs_sorted = up42.Job.all(sort_by=up42.JobSorting.process_id.asc)
# Define output for one of the sorting examples
for job in islice(jobs_sorted, 0, 5): # Print first 5 results
print(f"- Job ID: {job.id}")
print(f" Price: {job.credits} credits")
print(f" Status: {job.status.value}")
print(f" Finished at: {job.finished or 'Not finished'}\n")

Class: Job

A data class that represents a job in the system.

Attributes

AttributeDescription
process_id

str

The process ID in a name format.

id

str

The job ID.

account_id

str

The account ID.

workspace_id

Optional[str]

The workspace ID.

definition

dict

The inputs and other parameters used to execute the job.

status

JobStatus

The job status.

created

datetime.datetime

The timestamp when the job was created.

updated

datetime.datetime

The timestamp when the job was last updated.

collection_url

Optional[str] = None

The URL of the resulting STAC collection. Available only if the job status is terminal.

errors

Optional[List[ValidationError]]

Job errors. Available only if the job status is INVALID.

credits

Optional[int]

The job cost, in credits.

started

Optional[datetime.datetime]

The timestamp when the job was started.

finished

Optional[datetime.datetime]

The timestamp when the job was finished.

Python
# Select a job
job_id = "a0d443a2-41e8-4995-8b54-a5cc4c448227"
# Fetch job info
job = up42.Job.get(job_id=job_id)
# Define output
print(f"Job details for ID: {job.id}")
print(f"Process ID: {job.process_id}")
print(f"Price: {job.credits} credits")
print(f"Status: {job.status.value}")
print(f"Created at: {job.created}")
print(f"Started at: {job.started}")
print(f"Finished at: {job.finished or 'Not finished'}")

Properties

collection

Contains the STAC collection associated with the job. Available only if the job status is terminal. Returns pystac.Collection.

Python
# Select a job
job_id = "a0d443a2-41e8-4995-8b54-a5cc4c448227"
# Fetch job info
job = up42.Job.get(job_id=job_id)
# Define output
print(f"STAC collection ID: {job.collection.id}")

Methods

track

Tracks the job status by retrying until the status changes to a terminal status.

ParameterDescription
wait

int

An interval between queries, in seconds. The default value is 60.

retries

int

The duration of querying. The default value is 60 * 24 * 3.

Python
# Select a job
job_id = "a0d443a2-41e8-4995-8b54-a5cc4c448227"
# Fetch job info
job = up42.Job.get(job_id=job_id)
print(f"Initial job status is '{job.status.value}'. Starting to track...\n")
# This will print a status log to your console every 20 seconds
# and will PAUSE the script here until the job is done
job.track(wait=20)
# This line will only run AFTER the job has finished tracking
print("\nTracking complete!")
print(f"The final job status is: {job.status.value}")

get

Fetches a specific job by its ID. Returns Job.

ParameterDescription
job_id

str

The job ID.

Python
# Select a job
job_id = "a0d443a2-41e8-4995-8b54-a5cc4c448227"
# Fetch job info
job = up42.Job.get(job_id=job_id)
# Define output
print(f"Job details for ID: {job.id}")
print(f"Process ID: {job.process_id}")
print(f"Price: {job.credits} credits")
print(f"Status: {job.status.value}")
print(f"Created at: {job.created}")
print(f"Started at: {job.started}")
print(f"Finished at: {job.finished or 'Not finished'}")

all

Retrieves all jobs, with optional filtering. Returns Iterator["Job"]. Use itertools.islice to offset and limit the results.

ParameterDescription
process_id

Optional[List[str]]

Process IDs. Use to search for jobs running any of the provided processes.

workspace_id

Optional[str]

The workspace ID. Use to get jobs from a specific workspace. Otherwise, jobs from the entire account will be returned.

status

Optional[List[JobStatus]]

Job statuses. Use to search for jobs with any of the provided statuses.

min_duration

Optional[int]

The minimum duration of a job. Use to get jobs with runtimes greater than or equal to a specific number of seconds. Only jobs with the following statuses will be displayed:

  • RUNNING
  • SUCCESSFUL
  • CAPTURED
  • RELEASED
  • FAILED
max_duration

Optional[int]

The maximum duration of a job. Use to get jobs with runtimes less than or equal to a specific number of seconds. Only jobs with the following statuses will be displayed:

  • RUNNING
  • SUCCESSFUL
  • CAPTURED
  • RELEASED
  • FAILED
sort_by

Optional[utils.SortingField]

The results sorting method that arranges elements in ascending or descending order based on a chosen field. To view the list of possible values, see JobSorting.

Python
from itertools import islice
# Search for jobs
jobs = up42.Job.all(
max_duration=500,
sort_by=up42.JobSorting.process_id,
)
# Define output
for job in islice(jobs, 0, 5): # Print first 5 results
print(f"- Job ID: {job.id}")
print(f" Process ID: {job.process_id}")
print(f" Price: {job.credits} credits")
print(f" Status: {job.status.value}")
print(f" Created at: {job.created}")
print(f" Started at: {job.started}")
print(f" Finished at: {job.finished or 'Not finished'}\n")

Class: Cost

A data class that compares the cost of a process with custom values. Supported comparison operators:

  • Greater than: <
  • Greater than or equal to: >
  • Less than: <=
  • Less than or equal to: =>

Attributes

AttributeDescription
strategy

str

The pricing strategy used to calculate the cost.

credits

int

The process cost, in credits.

size

Optional[int]

The size of the input STAC item.

unit

Optional[str]

The unit of measurement used to calculate the size.

Python
from up42 import processing_templates
# Select an item
11 collapsed lines
stac_item_id = "68567134-27ad-7bd7-4b65-d61adb11fc78"
# Get the item from the STAC client
stac_client = up42.stac_client()
stac_item = next(stac_client.get_items(stac_item_id))
# Instanitate a DetectionTreesSpacept template
job_template = processing_templates.DetectionTreesSpacept(
title="Detect trees over UK",
item=stac_item,
)
# Verify that the template cost is less than or equal to 100 credits
acceptable_cost = 100
if job_template.cost <= acceptable_cost:
print(f"Template cost: {job_template.cost.credits} credits")
print(f"Template strategy: {job_template.cost.strategy}")
print(f"STAC item size: {job_template.cost.size}")
print(f"Unit of measurement: {job_template.cost.unit}")
else:
print(f"Template cost is higher than {acceptable_cost} credits")

Class: JobTemplate

A class for job templates that define the process ID and input validation logic.

Attributes

AttributeDescription
process_id

ClassVar[str]

The process ID in a name format.

workspace_id

Union[str, base.WorkspaceId]

The workspace ID.

errors

set[ValidationError]

A list of errors populated if the job isn’t valid.


Properties

is_valid

Checks if the job is valid. Returns bool:

  • True: The job is valid.
  • False: The job isn’t valid.

Python
from up42 import processing_templates
# Select an item
stac_item_id = "68567134-27ad-7bd7-4b65-d61adb11fc78"
# Get the item from the STAC client
stac_client = up42.stac_client()
stac_item = next(stac_client.get_items(stac_item_id))
# Instanitate a DetectionTreesSpacept template
job_template = processing_templates.DetectionTreesSpacept(
title="Detect trees over UK",
item=stac_item,
)
# Assert that the job is valid, print errors if not
if not job_template.is_valid:
for error in job_template.errors:
print(f"{error}\n")

Methods

execute

Executes the job. Returns Job.

Python
from up42 import processing_templates
# Select an item
11 collapsed lines
stac_item_id = "68567134-27ad-7bd7-4b65-d61adb11fc78"
# Get the item from the STAC client
stac_client = up42.stac_client()
stac_item = next(stac_client.get_items(stac_item_id))
# Instanitate a DetectionTreesSpacept template
job_template = processing_templates.DetectionTreesSpacept(
title="Detect trees over UK",
item=stac_item,
)
# Execute the templated job
job = job_template.execute()
# Define output
print(f"Job details for ID: {job.id}")
print(f"Process ID: {job.process_id}")
print(f"Status: {job.status.value}")
print(f"Price: {job.credits or 'N/A'} credits")
print(f"Created at: {job.created}")

Class: SingleItemJobTemplate

A data class for the process templates that require one STAC item as input:

Atributes

AttributeDescription
title

str

The title of the resulting objects: STAC item and STAC collection.

item

pystac.Item

The STAC item to process.

Class: MultiItemJobTemplate

A data class for the process templates that require multiple STAC items as input:

Atributes

AttributeDescription
title

str

The title of the resulting objects: STAC item and STAC collection.

items

List[pystac.Item]

The STAC items to process.

Learn more


Last updated: