UC2 DressMeUp Metail Composition Service

The Metail Composition service will be invoked by starting an AWS Step Function execution.

The StartExecution API endpoint is documented here. The parameters of interest are:

Field Description
input JSON input data for the execution
name The name of the execution (a random UUID)
stateMachineArn ARN of the step function - will be known once the endpoint is in place

Input

The step function input JSON (serialized to a string in the input field above) must conform to the following schema:

{
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "$id": "https://etryon-h2020.eu/schema/metail/uc2-composition-service.json",
    "title": "Metail Composition Service",
    "description": "Step function input for composing 3D garments into 2D user photos",
    "type": "object",
    "properties": {
        "userAvatar": {
            "description": "Presigned GET URL for the Metail avatar in Google Cloud Storage",
            "type": "string",
            "format": "uri"
        },
        "userPhoto": {
            "description": "Presigned GET URL for the photo of the user that the garment will be composed into, in Google Cloud Storage",
            "type": "string",
            "format": "uri"
        },
        "garment": {
            "description": "Presigned GET URL for the browzwear file containing the garment to be composed, in Google Cloud Storage",
            "type": "string",
            "format": "uri"
        },
        "size": {
            "description": "Garment size selected by the user",
            "type": "string",
            "format": "string"
        },
        "avatarGender": {
            "description": "Gender of the user avatar",
            "type": "string",
            "format": "string"
        },
        "outputImage": {
            "description": "Presigned PUT URL for the output image in Google Cloud Storage",
            "type": "string",
            "format": "uri"
        }
    },
    "required": ["userAvatar", "userPhoto", "garment", "size", "outputImage"],
    "examples": [
        {
	        "userAvatar": "https://storage.googleapis.com/see-adr-0021",
            "userPhoto": "https://storage.googleapis.com/see-adr-0021",
            "garment": "https://storage.googleapis.com/see-adr-0021",
            "size": "LXXL",
            "outputImage": "https://storage.googleapis.com/see-adr-0021"
        }
    ]
}

Note on implementation

Developers rarely need to be concerned with the underlying REST API for AWS services - it is much more common to use one of the AWS SDKs. For example, to invoke the scanatar step function from a Python program you would use the boto3 step function client and your code would look something like:

import json
import uuid
import boto3

SFN_ARN = 'arn:aws:states:{region}:{accountId}:stateMachine:etryon-dress-me-up-composition'

client = boto3.client('stepfunctions')
response = client.start_execution(
    stateMachineArn=SFN_ARN,
    name=str(uuid.uuid4()),
    input=json.dumps({
        "userAvatar": "https://storage.googleapis.com/...",
        "userPhoto": "https://storage.googleapis.com/...",
        "garment": "https://storage.googleapis.com/...",
        "size": "LXXL",
        "avatarGender": "male",
        "outputImage": "https://storage.googleapis.com/..."        
    })
)
print(f'Started execution {response["executionArn"]}')

Operation

See the sequence diagram for more details. A high level view of the operation of this service is as follows: -

  1. Download inputs
  2. Estimates user pose in photo.
  3. Compute pose offsets for VStitcher.
  4. Create avatar FBX file with 3D pose & VStitcher anchor points.
  5. Import avatar file in VStitcher & select pose
  6. Load garment file & select size
  7. Dress avatar pose
  8. Export V-Ray scene file
  9. Set camera, lights, and avatar offset.
  10. Render garment using V-Ray
  11. Morph rendered garment sprite to match user photo
  12. Compose morphed sprite & photo
  13. Upload result to presigned URL