UC2 DressMeUp Metail Composition Service
The Metail Composition service will be invoked by starting an AWS Step Function execution.
The StartExecution
API endpoint is documented here. The parameters of interest are:
Field | Description |
---|---|
input |
JSON input data for the execution |
name |
The name of the execution (a random UUID) |
stateMachineArn |
ARN of the step function - will be known once the endpoint is in place |
Input
The step function input JSON (serialized to a string in the input
field above) must conform to the following schema:
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://etryon-h2020.eu/schema/metail/uc2-composition-service.json",
"title": "Metail Composition Service",
"description": "Step function input for composing 3D garments into 2D user photos",
"type": "object",
"properties": {
"userAvatar": {
"description": "Presigned GET URL for the Metail avatar in Google Cloud Storage",
"type": "string",
"format": "uri"
},
"userPhoto": {
"description": "Presigned GET URL for the photo of the user that the garment will be composed into, in Google Cloud Storage",
"type": "string",
"format": "uri"
},
"garment": {
"description": "Presigned GET URL for the browzwear file containing the garment to be composed, in Google Cloud Storage",
"type": "string",
"format": "uri"
},
"size": {
"description": "Garment size selected by the user",
"type": "string",
"format": "string"
},
"warp_region": {
"description": "Warp region for the garment type",
"type": "string",
"format": "string"
},
"colorway": {
"description": "Colorway of the garment",
"type": "string",
"format": "string"
},
"avatarGender": {
"description": "Gender of the user avatar",
"type": "string",
"format": "string"
},
"outputImage": {
"description": "Presigned PUT URL for the output image in Google Cloud Storage",
"type": "string",
"format": "uri"
},
"outputError": {
"description": "Presigned PUT URL for pipeline error metadata in Google Cloud Storage",
"type": "string",
"format": "uri"
}
},
"required": ["userAvatar", "userPhoto", "garment", "size", "outputImage", "outputError"],
"examples": [
{
"userAvatar": "https://storage.googleapis.com/see-adr-0021",
"userPhoto": "https://storage.googleapis.com/see-adr-0021",
"garment": "https://storage.googleapis.com/see-adr-0021",
"warp_region": "full",
"size": "XL",
"outputImage": "https://storage.googleapis.com/see-adr-0021",
"outputError": "https://storage.googleapis.com/see-adr-0021"
}
]
}
Error Reporting
If the pipeline encounters an error and cannot complete the request, a JSON payload will be saved to Cloud Storage using the pre-signed outputURL
in the request. This will be a map containing the following fields:
Field | Description |
---|---|
Execution |
Step function Execution Id, can be used for detailed debugging by Metail |
ErrorCode |
An error code indicating which step of the pipeline failed (see table below) |
Message |
A human-readable error message (high level, not informative) |
The ErrorCode
indicates which step of the pipeline failed, possible values are:
ErrorCode | Description |
---|---|
ErrInit | Could not read inputs from Cloud Storage (may work on retry) |
ErrCreateAvatar | Could not process user’s photo to create an aligned avatar |
ErrGarmentRender | VStitcher garment render failed, e.g. requested size/colour not available in Browzwear file |
ErrComposeImages | Error composing garment image onto user’s photo |
ErrSaveOutput | Error saving the completed image back to Cloud Storage (may work on retry) |
Note on implementation
Developers rarely need to be concerned with the underlying REST API for AWS services - it is much more common to use one of the AWS SDKs. For example, to invoke the scanatar step function from a Python program you would use the boto3 step function client and your code would look something like:
import json
import uuid
import boto3
SFN_ARN = 'arn:aws:states:{region}:{accountId}:stateMachine:etryon-dress-me-up-composition'
client = boto3.client('stepfunctions')
response = client.start_execution(
stateMachineArn=SFN_ARN,
name=str(uuid.uuid4()),
input=json.dumps({
"userAvatar": "https://storage.googleapis.com/...",
"userPhoto": "https://storage.googleapis.com/...",
"garment": "https://storage.googleapis.com/...",
"warp_region": "full",
"size": "XL",
"colorway": "red",
"avatarGender": "male",
"outputImage": "https://storage.googleapis.com/...",
"outputError": "https://storage.googleapis.com/..."
})
)
print(f'Started execution {response["executionArn"]}')
Operation
See the sequence diagram for more details. A high level view of the operation of this service is as follows: -
- Download inputs
- Estimates user pose in photo.
- Compute pose offsets for VStitcher.
- Create avatar FBX file with 3D pose & VStitcher anchor points.
- Import avatar file in VStitcher & select pose
- Load garment file & select size
- Dress avatar pose
- Export V-Ray scene file
- Set camera, lights, and avatar offset.
- Render garment using V-Ray
- Morph rendered garment sprite to match user photo
- Compose morphed sprite & photo
- Upload result to presigned URL