Metail Scanatar Service

In ADR 12 we note that the Metail Scanatar Creation service will be invoked by starting an AWS Step Function execution.

The StartExecution API endpoint is documented here. The parameters of interest are:

Field Description
input JSON input data for the execution
name The name of the execution (a random UUID)
stateMachineArn ARN of the step function - will be known once the endpoint is in place

Input

The step function input JSON (serialized to a string in the input field above) must conform to the following schema:

{
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "$id": "https://etryon-h2020.eu/schema/metail/scanatar-service.json",
    "title": "Metail Scanatar Service",
    "description": "Step function input for Metail Scanatar creation",
    "type": "object",
    "properties": {
        "inputScan": {
            "description": "Presigned GET URL for the Quantacorp scan in Google Cloud Storage",
            "type": "string",
            "format": "uri"
        },
        "outputAvatar": {
            "description": "Presigned PUT URL for the output scanatar in Google Cloud Storage",
            "type": "string",
            "format": "uri"
        },
        "outputError": {
            "description": "Presigned PUT URL for pipeline error metadata in Google Cloud Storage",
            "type": "string",
            "format": "uri"
        },
        "gender": {
            "description": "Gender of the subject",
            "type": "string",
            "enum": ["male", "female"]
        }
    },
    "required": ["inputScan", "outputAvatar", "outputError", "gender"],
    "examples": [
        {
	        "inputScan": "https://storage.googleapis.com/example-bucket/scan/user/enK0PQ8YY0hXDICJm1MKfjYTTvu1/d072dffc-1aeb-11ec-a8f1-ffa30997c493.obi?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=example%40example-project.iam.gserviceaccount.com%2F20181026%2Fus-central-1%2Fstorage%2Fgoog4_request&X-Goog-Date=20181026T181309Z&X-Goog-Expires=900&X-Goog-SignedHeaders=host&X-Goog-Signature=247a2aa45f169edf4d187d54e7cc46e4731b1e6273242c4f4c39a1d2507a0e58706e25e3a85a7dbb891d62afa8496def8e260c1db863d9ace85ff0a184b894b117fe46d1225c82f2aa19efd52cf21d3e2022b3b868dcc1aca2741951ed5bf3bb25a34f5e9316a2841e8ff4c530b22ceaa1c5ce09c7cbb5732631510c20580e61723f5594de3aea497f195456a2ff2bdd0d13bad47289d8611b6f9cfeef0c46c91a455b94e90a66924f722292d21e24d31dcfb38ce0c0f353ffa5a9756fc2a9f2b40bc2113206a81e324fc4fd6823a29163fa845c8ae7eca1fcf6e5bb48b3200983c56c5ca81fffb151cca7402beddfc4a76b133447032ea7abedc098d2eb14a7",
            "outputAvatar": "https://storage.googleapis.com/example-bucket/avatar/user/enK0PQ8YY0hXDICJm1MKfjYTTvu1/d072dffc-1aeb-11ec-a8f1-ffa30997c493.fbx?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=example%40example-project.iam.gserviceaccount.com%2F20181026%2Fus-central-1%2Fstorage%2Fgoog4_request&X-Goog-Date=20181026T181309Z&X-Goog-Expires=900&X-Goog-SignedHeaders=host&X-Goog-Signature=247a2aa45f169edf4d187d54e7cc46e4731b1e6273242c4f4c39a1d2507a0e58706e25e3a85a7dbb891d62afa8496def8e260c1db863d9ace85ff0a184b894b117fe46d1225c82f2aa19efd52cf21d3e2022b3b868dcc1aca2741951ed5bf3bb25a34f5e9316a2841e8ff4c530b22ceaa1c5ce09c7cbb5732631510c20580e61723f5594de3aea497f195456a2ff2bdd0d13bad47289d8611b6f9cfeef0c46c91a455b94e90a66924f722292d21e24d31dcfb38ce0c0f353ffa5a9756fc2a9f2b40bc2113206a81e324fc4fd6823a29163fa845c8ae7eca1fcf6e5bb48b3200983c56c5ca81fffb151cca7402beddfc4a76b133447032ea7abedc098d2eb14a7",
            "outputError": "https://storage.googleapis.com/example-bucket/avatar/user/enK0PQ8YY0hXDICJm1MKfjYTTvu1/d072dffc-1aeb-11ec-a8f1-ffa30997c493.fbx?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=example%40example-project.iam.gserviceaccount.com%2F20181026%2Fus-central-1%2Fstorage%2Fgoog4_request&X-Goog-Date=20181026T181309Z&X-Goog-Expires=900&X-Goog-SignedHeaders=host&X-Goog-Signature=247a2aa45f169edf4d187d54e7cc46e4731b1e6273242c4f4c39a1d2507a0e58706e25e3a85a7dbb891d62afa8496def8e260c1db863d9ace85ff0a184b894b117fe46d1225c82f2aa19efd52cf21d3e2022b3b868dcc1aca2741951ed5bf3bb25a34f5e9316a2841e8ff4c530b22ceaa1c5ce09c7cbb5732631510c20580e61723f5594de3aea497f195456a2ff2bdd0d13bad47289d8611b6f9cfeef0c46c91a455b94e90a66924f722292d21e24d31dcfb38ce0c0f353ffa5a9756fc2a9f2b40bc2113206a81e324fc4fd6823a29163fa845c8ae7eca1fcf6e5bb48b3200983c56c5ca81fffb151cca7402beddfc4a76b133447032ea7abedc098d2eb14a7",
            "gender": "female"
        }
    ]
}

Error Handling

If the pipeline fails with an error, a JSON payload is written to the outputError URL (a pre-signed PUT URL in the input) with an error message and the id of the failed Step Function execution. For example:

{
    "Execution":"5a190979-8436-43ce-88f3-5a065b3f1d2c",
    "Message":"Mesh pipeline failed"
}

Note on implementation

Developers rarely need to be concerned with the underlying REST API for AWS services - it is much more common to use one of the AWS SDKs. For example, to invoke the scanatar step function from a Python program you would use the boto3 step function client and your code would look something like:

import json
import uuid
import boto3

SFN_ARN = 'arn:aws:states:{region}:{accountId}:stateMachine:etryon-scanatar-creation'
INPUT_URL = 'https://storage.googleapis.com/example-bucket/scan/user/...'
OUTPUT_URL = 'https://storage.googleapis.com/example-bucket/avatar/user/...'
ERROR_URL = 'https://storage.googleapis.com/example-bucket/avatar/user/...'

client = boto3.client('stepfunctions')
response = client.start_execution(
    stateMachineArn=SFN_ARN,
    name=str(uuid.uuid4()),
    input=json.dumps({
	    "inputScan": INPUT_URL,
        "outputAvatar": OUTPUT_URL,
        "outputError": ERROR_URL,
        "gender": "female"
    })
)
print(f'Started execution {response["executionArn"]}')