Graphics Showcase: Using Vulkan Subpasses in UE4 for Performant Tone Mapping on Quest

Tone mapping can provide a variety of fullscreen effects useful in game development. These effects include gamma correction, color grading lookup tables (LUTs), color tint, vignette, fullscreen fades, and many others. Tone mapping effects utilize a global shader applied to the color buffer after the scene has been rendered. Typically this requires an additional render pass. As a consequence, most Quest developers don’t use tone mapping as the performance cost of another render pass is too high. UE4 also requires Mobile HDR (which is not supported on Quest or Quest 2) to use tone mapping effects thus taking an even larger performance hit.

We believe that tone mapping is an important artistic tool, so we built a Graphics Showcase (if you don't already have a free Unreal Engine account, sign up for one here to access the link) featuring a subpass-based solution that is much more performant than traditional techniques. In some configurations the tone map subpass adds as little as 0.6 ms, comparable to the time needed for the store color operation alone in a second render pass. (Please note that these engine changes are only available in the Oculus fork of UE 4.26. More info on building UE4 from Oculus’s github found here.)

Vulkan provides subpasses as a way of performing additional rendering operations without a full render pass. While subpasses have some limitations, they are well suited to tone mapping. If we were to use a separate render pass, color data would be written to RAM at the end of the first render pass and then read back from RAM for our tone map pass. Using a subpass, we can access the color data from the previous subpass that is still in tile memory and avoid the delay of the write and read operations. A limitation of the subpass fetch is that we only have data for the current pixel coordinate, so we can only apply effects using the current pixel’s data. Effects like bloom, blur, or depth of field are not suitable for subpass based tone mapping as they require color data from surrounding pixels. More info on subpasses can be found here.

In the Graphics Showcase, you can see three examples of the tone mapping subpass in action. For reference, here is what the scene looks like with no tone map applied.

If this scene looks familiar, it’s because it was part of the Dreamdeck app released on Oculus Rift back in 2016.

Here is a quick video showing off the effects.

To view the scene without tone mapping, press A to toggle it off or back on. This triggers blueprint logic to set “r.Mobile.TonemapSubpass=1” or “r.Mobile.TonemapSubpass=0”. You can also create your own logic to disable tone mapping at runtime and regain performance! For example, if you are only using tone map for fades at beginning and end of a level, you can disable the subpass the rest of the time. Or, if you want it always on, just set “r.Mobile.TonemapSubpass=1” in DefaultEngine.ini.

Each of our effects make use of Post Process Volumes to control their parameters. Looking at the properties for a Post Process Volume, you will see a long list of features. Not all of these can be supported by Vulkan subpasses, and we have only implemented a few. However, with the framework in place it shouldn’t take too much effort to implement additional features or use your own custom tone map shader to get the look you want. For the Graphics Showcase we utilize multiple Post Process Volumes to allow switching effects on the fly. If you aren’t switching dynamically, a single volume will be sufficient. Make sure the volume is enabled and set to infinite.

The first effect you see running in the Graphics Showcase is fade in / fade out. It is implemented via the fade track in the level sequence. As the scene starts, it fades in from black and later fades back out. Without tone mapping, the fade track is non-functional and you must resort to camera attached polys with transparency textures to reproduce this type of fade. With tone mapping, it’s as simple as adding a fade track and a few key points to your level sequence.

The next effect uses the level sequence again to modify color tint values simulating a day-night cycle. Different color keys on the sequence set the tint value. As the sequence plays the tint blends between these values giving the impression of dawn and dusk. You can also set the color tint value directly instead of using the level sequence. Select PostProcess_DayNightCycle and look under Color Grading > Misc > Scene Color Tint.

The last effect is color grading using a LUT. This allows us to completely change the color palette of the game. LUTs can be created by an artist following Unreal’s documentation or you can find hundreds of premade LUTs on Unreal Marketplace. Add the LUT texture to your project content and then set it in the Post Process Volume under Color Grading > Misc > Color Grading LUT.

If you’re curious about the engine changes needed to get this working, here’s a high level overview.

UE4 uses subpass hints to tell the RHI which subpasses are enabled. The forward renderer typically has ESubpassHint::DepthReadSubpass already enabled. We added a new subpass hint ESubpassHint::MobileTonemapSubpass. Though ESubpassHint is an enum, it was not setup to support multiple hints. We changed ESubpassHint to a bit mask to allow running both subpasses together.

SubpassFetch is used to retrieve the color data from tile memory of the previous subpass. While this functionality already existed in UE4, it did not have support for MSAA. We expanded this by adding SubpassFetchMS which takes an index for which sample to fetch. SubpassFetch and SubpassFetchMS are both intrinsic functions that take advantage of the subpassInput and subpassInputMS uniform types. Next, we set up a preprocessor directive so the pixel shader knows how many samples to ask for. Inside the shader we average the sampled scene color values performing our own resolve. This led to the next issue.

Normally the resolve stage will handle MSAA. Since we are resolving inside the shader, we need the render pass to start with 2 or 4 samples for the main pass and then switch to 1 sample on the output from the tone map subpass. This is handled by checking if the tone map subpass is enabled and setting the rasterization samples to 1 in the Vulkan pipeline. This allows our subpass to take in 2 or 4 samples and output just 1.

When not using MSAA there is no resolve texture to output to. Our subpass can’t use the scene color texture as both an input attachment and color attachment, so we modified the logic to create the resolve texture for nonMSAA when the tone map subpass is enabled. The ERenderTargetActions for the color texture and resolve texture are Clear_DontStore and Clear_Store respectively. Only the resolve texture is written to RAM. We disabled the call to RHICopyToResolveTarget in RHIEndRenderPass as this will overwrite the frame buffer our subpass just output to. By using the resolve texture for nonMSAA, we avoid creating a second color texture attachment and keep the code path nearly the same as the MSAA path.

We created a new pixel shader based on MainPS_Mobile in PostProcessTonemap.usf. This provided the parameters from the Post Process Volume which allow us to control our effects from the editor GUI. Most of the logic is copied from MainPS_Mobile except the SubpassFetch and resolve logic (averaging the samples).

The other big difference in the shader is the color LUT logic. This was not supported in the mobile tone map shader, so was copied from the non-mobile version. Before the tone map shader can use the LUT texture, it is run through a Combine LUT shader. This is added as an additional render pass, but this pass is much smaller (1024x32) than our full display and has minimal impact on performance. The color grading texture, output from the combine LUT pass, is then passed to our tone map subpass. Note that the LUT texture is the same for both left and right displays, so must only be rendered once per frame.

You must be very careful with subpass dependencies, color actions, and input attachments. If they’re not set just right your image may render correctly, but run as an additional render pass instead of as a subpass. This loses all the performance benefits. Always verify color data is only stored once. You can check by running “ovrgupprofiler -t” from adb shell and ensure that you have a single Render followed by a single StoreColor.

Bad (Render, Store, Load, Render, Store)

Good (Render, Store)

Rémi Palandri provides much more detail in his post on Loads, Stores, Passes, and Advanced GPU Pipelines.

Now let’s see how the additional subpass is impacting performance. The table below shows GPU timing data from VRAPI logcat output (“logcat | grep FPS”) on Quest 2.

The 2x MSAA FFR3 is the sweet spot for our subpass adding less than 1 millisecond to the frame duration. The GPU is able to perform 2 concurrent texture fetches without stalling, but not 4. Thus the loss in efficiency when switching to 4x MSAA.

As you can see from the data, using Vulkan subpasses provides a performant option to apply tone maps on the Quest. With these tone maps you can change the entire look of your game and give it a unique feel. We hope that this Graphics Showcase has inspired you to add tone mapping to your next game.