Translucent vs Masked Rendering in Real-Time Applications

Rendering translucent materials is a difficult problem in real-time applications, especially as many existing methods are too costly for virtual reality. Oculus Avatars in Home 2.0 for Rift and Rift S are meant to look translucent which introduces rendering challenges when using conventional translucent methods. Today we will provide solutions for some of these challenges, and show how masked rendering can be used as an alternative. This article provides implementation details and properties of translucent and masked rendering.

Home 2.0, implemented with Unreal Engine 4, uses clustered forward rendering instead of deferred rendering for materials and geometry, enabling the ability to use multisample anti-aliasing (MSAA), along with lower bandwidth costs at higher resolutions. Previously, Avatars in Home were rendered translucent, using a two-pass rendering technique. This provided a good quality solution when the Avatars did not have lighting affect them. For higher visual quality, Avatars in Home now have lighting affect their look, leading to performance and quality challenges when using translucent rendering. As a result, we switched to rendering masked Avatars, allowing you to control visibility of objects in a binary manner. This method works well with MSAA, but without MSAA, artifacts from masked rendering are apparent.

Throughout this post, the term “translucent” specifies the rendering method for transparent objects to be consistent with Unreal Engine terminology. Unreal Engine uses the term translucent for materials of transparent objects. For example, when choosing the “Blend Mode” and “Lighting Mode” for a material of a transparent object, we must choose the “Translucent” and “Surface Translucency Volume” options, respectively.

Due to potential confusion between the two terms, it is necessary to define “translucent”, and elaborate on why we decided to choose one option over the other. The main difference between transparent and translucent materials is the physical properties that affect the amount of light that is allowed to pass through them. Transparent materials allow the full passage of light, while objects that are on the other side of a transparent object are clearly visible with no distortion. Translucent materials allow light to enter, and then scatter based upon the material's physical properties. It allows some light to pass through them, all while distorting and diffusing light paths along with objects located behind them.

Avatars in Home with translucent rendering are implemented with alpha blending. The image above shows an example of depth issues that occur with this implementation: the hand on the left shows incorrect depth sorting and the hand on the right shows correct depth sorting. For correct results, objects must be blended from back to front which is not commutative; therefore, triangles must be sorted, which is neither cheap nor sufficient for certain edge cases. The order in which surfaces are blended affects the total visibility of each surface. We could use order-independent transparency which allows for per-pixel geometry sorting, but it is a fairly costly technique that is not suitable for VR.

We were able to combat the sorting issue by leveraging Unreal Engine's Custom Depth, providing occlusion similar to that from a Z-Buffer, while achieving correct translucent sorting order. Custom Depth is a feature in Unreal Engine that uses a second depth buffer which can be used for effects like custom culling for translucency. For Avatars, we use Custom Depth for per-pixel depth occlusion.

When rendering the mesh into this depth buffer, pixels that are behind the outer shell of the mesh can be culled. We run two passes, as follows:

The image above depicts what would be rendered to the Custom Depth buffer in that scene on the left and what would be the resulting scene after the Translucency pass on the right. The Avatar is the only object seen on the left since it is the only mesh in the scene rendering to Custom Depth. The Custom Depth buffer used for Avatars in Home also has the same resolution and MSAA samples as the Scene Depth buffer and resulting texture.

Instead of doing a hard comparison between Scene Depth and Custom Depth, a small depth bias should be added to prevent too many pixels from being culled, including the outer shell. The image below illustrates the effects of adding a depth bias before and after. This will be discussed in-depth under the “MSAA Artifacts” section.

Our solution to minimize “bleeding” artifacts involves adding multiple depth biases, ensuring depth errors are minimized, especially with MSAA. This is done during depth rejection when determining opacity of a pixel, with no depth bias added, many fragments are culled out.

The images below portray varying levels of depth biases added during depth rejection. The first image depicts depth comparison with a small, constant depth bias, which doesn't take into account varying steepness levels around edges. The second image depicts depth comparison with a larger, constant depth bias. The third image depicts depth comparison with both a constant and gradient depth bias, which provides the best solution. (Note: this is considered a soft comparison).

The following steps outline the soft comparison process between Scene Depth and Custom Depth.

Unfortunately, this method does not fix all of the “bleeding” artifacts that are caused from per-object sorting, as there are certain worst case scenarios when these artifacts are even more apparent. One such example is if a translucent Avatar is in front of a High Dynamic Range (HDR) light source. HDR light sources have a high saturation value that is not clamped before the translucency pass runs and the “bleeding” artifacts appear. Therefore, the high color values are “bleeding” through to the translucent material before the color is remapped, causing artifacts to be more conspicuous.

One method to fix “bleeding” artifacts is to increase the two different depth biases mentioned above; however, this can cause other depth-related issues. Depth biases are similar to shadow biases in their behavior. Shadow biases fix shadow “acne”, which is similar to “bleeding” artifacts, but a shadow bias that is too large can also make shadows disconnected. Similarly, a depth bias that is too large can cause translucent materials to have intersection-related artifacts.

The images below depict intersection-related artifacts with an increasing depth bias, leading to more artifacts.

It is necessary to find a good balance between minimizing “bleeding” artifacts with intersection-related artifacts in order to get the best possible results, and we've found that adding depth biases is a great way to do so.

While Home is implemented with clustered forward rendering for materials and geometry, deferred shadows are still used for opaque and masked rendering. Deferred shadows are implemented by running an Early Z pass to compute depth, which is later used to create a shadow mask used within the Base pass. In other words, deferred shadows are computed in screen space and read by all non-translucent objects in the Base pass.

The image below depicts what is rendered in the Early Z pass with masked rendering for Avatars and what is later rendered in the Base Pass.

Ultimately, shadow receiving is guaranteed for masked rendering because opaque portions of a masked material respect lighting. Sorting of objects is also correct so we are not required to use a second depth buffer.

Masked rendering does not encounter the same MSAA artifacts as translucent rendering, it uses alpha-to-coverage which “maps the alpha value output by the pixel shader to the coverage mask of MSAA.” With MSAA and alpha-to-coverage combined in masked rendering, there are plenty of sub-samples to average, which lead to higher visual quality. There is also no depth bias needed during the depth comparison for masked rendering. The images below compare masked and translucent rendering with MSAA. The image on the right shows the remaining “bleeding” artifacts from MSAA for translucent rendering that were not eliminated with the depth biases.

Masked rendering solves many of the issues that come up with translucent materials, including the ability to allow for deterministic occlusion and deferred shadows in Unreal Engine.

CPU performance is similar for both rendering paths, as translucent rendering requires approximately 750 draw calls per frame and masked rendering requires 720 draw calls per frame. GPU performance is evaluated by measuring relevant passes for rendering masked and translucent Avatars. The numbers in the table indicate that translucent rendering adds almost 80% more GPU time per frame when compared to masked rendering.

Masked rendering runs an Early Z pass to obtain the correct depth, it then renders in the Base Pass. Translucent rendering skips the Early Z pass and instead runs a Custom Depth pass after the Base pass has run. It then renders in the Translucency pass. The Resolve Scene Depth pass is needed for translucent rendering in order to resolve Custom Depth to the final Z when submitted to OVR to be used as a texture.

The items listed in the table above, along with the performance impacts discussed in this article should help to evaluate whether to use translucent or masked rendering in real-time applications, especially when using forward rendering. Based on these findings, we updated the avatar docs for Unity and Unreal and the Unity and UE4 integrations to include an option for masked rendering.