ADR-0027 Manual creation of avatar & garment models for UC3
| Publication Date | 2021-11-17 |
|---|---|
| Last Update | 2021-11-17 |
| Status | Accepted |
| References | ADR-0012 Metail Scanatar Creation |
| References | ADR-0013 Metail VStitcher Headless Service |
| References | ADR-0026 Use Browzwear parametric avatars for UC3 |
Context
For Use Case 3, the Magic Mirror app, a designer needs to prepare each garment in multiple sizes and style a different avatar for each size. In ADR-0026 Use Browzwear parametric avatars for UC3 we outlined the creation of those avatars and how they need to be dressed.
The overall architecture is described in Magic Mirror System. The process is divided in 2 steps: avatar creation, and garment creation.
For the avatar creation:
- Odlo receives 10 avatars from Browzwear. These avatars need an A-pose that will be used for dressing.
- Odlo sends Metail the avatar files in VSA format.
- Metail export the A-pose of those avatars as a mesh file.
- Metail runs the same mesh pipeline described in ADR-0012 Metail Scanatar Creation on those mesh files and create 10 doppelgängers of the parametric avatars that use Genesis 2 skeletons, and the same A-pose. We need to make sure the avatars are aligned with theirs, by importing them in VStitcher and checking if we need any offsets.
- Metail bakes 1 Mixamo animation per avatar, modified so every animation starts with the same A-pose.
- Metail store the FBX files (with A-pose & with animations) in the asset store.
For the garment creation:
- Odlo prepares the 5 sizes of a garment in each of the corresponding avatars, and stores the styled results in snapshots in a BW file.
- Metail receives the BW file and a request to process it.
- Metail swaps the avatar in BW file by its corresponding doppelgänger and exports an Alembic animation file.
- Metail export runs their automatic skinning tool. This produces an FBX with a Genesis 2 skeleton, another FBX with an ARKit skeleton, and material textures (it may need to be repeated for every colour variant if it can’t be automated).
- Metail takes the Alembic file, the corresponding avatar FBX with animation from the avatar creation step, the Genesis2-skinned garment file, and computes the Obi parameters texture and an Obi blueprint.
- Metail uploads the garment ARKit FBX file, 1 albedo per colour variant, other material textures, the Obi blueprint, the Obi parameters texture, and a metadata file describing all these files to the asset store.
In ADR-0013 Metail VStitcher Headless Service we said we would set up a headless instance of VStitcher to help us automate different pipelines. That is still true for Use Case 2, but for UC3 VStitcher headless is no longer relevant because a person needs to manually run some of the tools. The reasons are:
- VStitcher API does not support all the necessary functions for importing and exporting animations automatically.
- The avatar creation needs to be supervised by a human to make sure there are no issues with the doppelgängers, in particular:
- Did the mesh pipeline succeed? The mesh pipeline has only be tested on scans of real people, not on idealised bodies created from average measurements.
- Is the A-pose aligned with its corresponding avatar in VStitcher?
Decision
We won’t use VStitcher Headless in Use Case 3. Instead, we have created the figure of the “Avatar Model Maker” and the “Garment Model Maker” – see architecture diagram in Magic Mirror System. These persons will use Metail tools to aid in the automatic skinning and export of files, but they will be in charge of orchestrating which scripts to run.
Consequences
- Odlo will have to communicate to Metail when an avatar or a garment needs processing.
- Metail will process the requests manually, aided by tools.
- The admin / back office CLI will no longer add value and won’t be developed