Matte objects and matte lights

Introduction

Matte objects and matte lights are advanced techniques to aid including synthetic objects under matching lighting (captured for example via lightprobes) onto a real world photograph (a virtual backplate or environment map). In a classic workflow, one needs to assign special custom shaders to objects to receive a separate shadow pass along with additional global illumination information that must then be combined with the original rendering pass in external compositing steps.

Iray Photoreal simplifies this classic workflow by avoiding the separate postprocessing steps and allowing for a fully progressive and physically plausible rendering of the complete scene at once. In addition, matte objects do not need to rely on secondary buffers and setting up the additional shaders that must usually be tweaked on a per scene basis, as all of the necessary logic is directly incorporated into the actual core rendering architecture.

To guide the interaction of backplate and rendered objects though, additional matte objects have to be provided that should (roughly) match the photographed real world objects using the same camera settings. In addition, these objects should be assigned (roughly) matching materials to further perfect the illusion of the rendered objects being part of the environment and especially interacting closely with the backplate.

Attributes

The following attributes on the mi::neuraylib::IAttribute_set class controls matte objects:

bool matte = false

If set to true, this attribute flags objects and lights as matte objects or matte lights, respectively.

mi::Float32 matte_shadow_intensity = 1.0

If the matte flag is set for an object, this controls the intensity of the shadow cast on the matte object with a value greater or equal to 0.0. A value of 0.0 denotes no shadow at all and a value of 1.0 denotes regular shadow intensity, thus values between 0.0 and 1.0 denote a lighter shadow. Values above 1.0 increase the shadow intensity. Note that the range of useful values above 1.0 is rather limited and exceeding this range may lead to an unnatural dark shadow that can influence other additive lighting effects, such as reflections of the objects in the scene. The exact useful range is scene dependent.

Scene Option

bool matte_remap_uv_to_dome_in_aux_canvas = false

If set to true, all matte objects (including the implicit dome ground plane) will feature special texture coordinates that are written into the texture_coordinate canvas. These can be used for the actual uv-mapping into a spherical environment texture map.

bool matte_visible_in_aux_canvas = false

When set to true, all matte objects (including the implicit dome ground plane) will show up in any auxiliary render canvas like texture_coordinate, object_id or distance.

Example

A simple example would be to add a synthetic cube onto a backplate of a photographed wooden table.

• The scene must be set up to include both the cube and the top of the table (which can be a very rough approximation, so a simple rectangle the size of the table can already be sufficient).
• The virtual camera must be set up to match the original real world camera position and lens settings (to have the objects in scene and photograph align).
• An environment map with similar lighting characteristics (best would be a matching lightprobe of the same real world scene) must be provided.
• Approximately match the material of the original table, including textures (or even a bump or normal map) and enabling the matte flag of the table object.

Instead of rendering the table object directly into the output buffer like any other standard object, Iray Photoreal now triggers a specialized handling of the matte object that directly combines backplate information, environment lighting, interactions of both matte and synthetic objects (global illumination) and the artificial shadows cast onto the matte objects, in a single progressive rendering step.

Additional notes and limitations

An environment map or a backplate (or both) should be specified to get the full benefit of matte objects. These define the appearance of matte objects when they interact (through reflections and refractions) with synthetic objects. The backplate is projected onto the matte object from the camera. If this projection is impossible, the environment map is used instead. If a hemispherical environment dome of sufficient resolution and quality is available, this can also be used alone. But these are difficult to generate, so in most cases a 2D photograph will be used as the backplate, which can be seen as a high-quality complement to the environment map.

It is possible to use the matte objects feature with neither a backplate nor an environment map. In such a case, matte lights should be used to approximate the scene lighting. But many interesting effects will then be missing from the rendered image, such as reflections of matte objects onto synthetic objects, because the backplate/environment defines how matte objects appear in those reflections.

Alpha channel

In addition to the interactions between matte objects and LPEs and LPE control of the alpha channel, matte objects may also catch shadows. The shadows cast by synthetic objects upon matte objects do not by default affect the alpha channel. This can be changed by setting the following option:

bool matte_shadow_affects_alpha = true

If set to true (the default), shadows received by matte objects will make the alpha channel of the output canvas opaque. Otherwise, matte shadows will not affect the alpha channel.