Approach to Material Capture

Mission

The primary mission of M-XR is to enable the most realistic capture of 3D & 2D assets for a variety of render engines - Film, AAA Games & Fashion all demand a ‘true’ capture solution. Secondary to this, the wide variety of 3D stakeholders have equally varying requirements when it comes to scalability, capture requirements and budgets. A true solution towards capture has to service these goals

• Accurate materials: measured and not generalised

• Simple Hardware: allow more user to capture

• Scalability: accommodate for future renderings & representations

• Generative / Procedural: A better data format for Machine Learning (ML)

New to 3D? Read more about the current challenges surrounding capture.

Which material definition?

The realism of CGI materials relies heavily on the selected material representation - namely the shader and engine used. Deciding on ‘the best’ representation isn't straightforward, as each approach is optimised for different specifications such as speed, memory usage, and GPU/CPU performance. Moreover, many of these solutions struggle to render some of the more obscure and complex characteristics of the real world.

Take the use of textures, for instance. The real-world scale of a texture's pixel size can vary dramatically, from sub-millimeters to multiple centimeters, depending on the resolution and the mapping. Characteristics that may be represented through normals or displacement at one-pixel scale might transition into roughness and anisotropy at a different pixel scale to achieve the same visual effect. Essentially, these properties are scattering light in the same manner but at various scales. Infinite 3D resolution isn't practical, which is why solutions like textures and materials serve as efficient alternatives.

The current state of 3D rendering isn't an objectively perfect way to describe reality. Its purpose is threefold: to generate convincing visuals on a screen at runtime, to be easily understood by a creator, and to be memory-efficient. Attempting to translate reality into these compressed and lossy formats - it's simply the wrong tool for the job. Capturing reality requires a more comprehensive approach, akin to the RAW (of photography) file for a real-world object. Such a format wouldn't be bound by delivery specifications like shader and render kernel, texture resolution, or UV wrapping.

Introducing .mxr - RAW 3D capture

RAW formats are prevalent in photographic data capture. '.mxr' marks M-XRs inaugural RAW 3D capture for material acquisition. This format is unbiased, lossless, and unrestricted by material definitions, UV mapping, or resolution - the core of M-XRs tech stack.

The '.mxr' file preserves data in an unbiased form, enabling us to sample it according to user specifications and export the material properties for a chosen engine/shader. This ensures optimal material properties for the selected engine and output resolutions, offering unparalleled flexibility for current and future engines and texture mapping.

Workflow & Capture

The '.mxr' format simplifies and automates the capture-to-delivery workflow, accommodating for various scanner configurations. It eradicates the need for specialised components like polarisation filters, working with any mix of single or multiple cameras, and single or multiple lights. This format is versatile & robust, being suitable for both high-end expensive film industry scanners right the way down to point-and-shoot hobbyists. Most scanners can be broken down to these three types listed bellow.

Left : Camera-Priority often used in full-body scanning
Middle : Light-Priority often used in swatch scanners and turn table scans
Right : Omni-Directional, handheld scans

To date, this technology has been applied to both 2D & 3D scanners built by M-XR, and has also been used by many compatible third-party systems. These third-party systems were already being used for cross-polarisation capture, but were unable to capture the full gamut of material properties. By removing the polarisation filters, the system was ready to capture into '.mxr' with minimal calibration required.

Upon capture, the source photos, 3D data and camera & light positions are converted into an '.mxr' file. This file can then be exported to a conventional 3D file with materials, as per a user’s specifications. The benefit of '.mxr' gives users the ability to export into multiple formats, resolutions and render engines without losing quality and realism along the way - a big problem in 3D these days.

Results

With the '.mxr' format, we've successfully scanned a variety of materials, stress-testing the approach. Each scan, rich in detail, enables hyper-realistic renders for real-time or pre-rendered content. See our full swatch showcase on our vimeo.

It is common for other solutions to have ‘cross-contamination’ of material-properties. Features within the diffuse bleed into normals. Our technology ensures accurate and untainted results.

Quality is crucial for film-grade assets. Hence, all material predictions are color balanced and in 32-bit linear floating point, even from 8-bit capture. This high dynamic range texture retention allows even the subtlest material nuances to be preserved.

As this solution is flexible & robust across different 3D scanners, the same quality and type of materials that have been showcased with swatch scans is also present for 3D scans. Delivering the best-in-class 3D objects that look photoreal within engine. See our full Boxing Glove showcase on our vimeo.

Key Features

To recap, the key features of M-XR's solution and the '.mxr' format

• Accurate materials

• Render Engine impartial

• Automatic pipeline

• Better data for ML

• Dynamic Materials

• Future Proof

photography - CGI