Multiview WebGL Rendering

Warning: Multiview is an experimental feature and you may see behavior that is different from what is described in this document.

To render VR content, you need to draw the same 3D scene twice; once for the left eye, and again for the right eye. There is usually only a slight difference between the two rendered views, but the difference is what enables the stereoscopic effect that makes VR work. By default in WebGL, the only option available is to render to the two eye buffers sequentially — essentially incurring double the application and driver overhead — despite the GPU command streams and render states typically being almost identical.

The WebGL multiview extension addresses this inefficiency by enabling simultaneous rendering to multiple elements of a 2D texture array.

Note: Only CPU-bound experiences will benefit from multi-view. Often, a CPU usage reduction of 25% - 50% is possible.

Multiview Design

With the multiview extension draw calls are instanced into each corresponding element of the texture array. The vertex program uses a new ViewID variable to compute per-view values — typically the vertex position and view-dependent variables like reflection.

The formulation of the multiview extension is purposely high-level to allow implementation freedom. On existing hardware, applications and drivers can realize the benefits of a single scene traversal, even if all GPU work is fully duplicated per view.

In WebGL on Quest hardware, multiview can be enabled with the OCULUS_multiview extension. Only WebGL 2.0 supports this extension; WebGL 1.0 cannot use multiview.

OCULUS_multiview is a upgraded version of the OVR_multiview2 extension. OCULUS_multiview operates in the same way as OVR_multiview2, but includes support for multisampled antialiasing (MSAA).

The differences from the OVR_multiview2 are as follows:

The OCULUS_multiview is available out-of-the-box in the Browser, while the OVR_multiview2 extension is behind the flag (chrome://flags). The latter extension is not enabled by default because it is in the ‘Draft’ state;

The OCULUS_multiview extension includes all the functionality of the OVR_multiview2, plus multisampling support:

  void framebufferTextureMultisampleMultiviewOVR(GLenum target, GLenum attachment,
                                        WebGLTexture? texture, GLint level,
                                        GLsizei samples,
                                        GLint baseViewIndex,
                                        GLsizei numViews);

Using OCULUS_Multiview in WebGL 2.0

A WebGL app can be relatively easily modified to benefit from the extension.

Sample code which implements multiview rendering into a WebXR layer can be found at Cubes (WebGL 2.0) - Source code

This is the recommended way to render a scene on Quest hardware.

First of all, the OCULUS_multiview or OVR_multiview2 extension should be requested:

var is_multiview, is_multisampled = false;
var ext = gl.getExtension('OCULUS_multiview');
if (ext) {
  console.log("OCULUS_multiview extension is supported");
  is_multiview = true;
  is_multisampled = true;
}
else {
  console.log("OCULUS_multiview extension is NOT supported");
  ext = gl.getExtension('OVR_multiview2');
  if (ext) {
    console.log("OVR_multiview2 extension is supported");
    is_multiview = true;
  }
  else {
    console.log("Neither OCULUS_multiview nor OVR_multiview2 extensions are supported");
    is_multiview = false;
  }
}

Then create a projection layer with a texture-array textureType and add it to your xrSession

    function onSessionStarted(session) {
      xrFramebuffer = gl.createFramebuffer();
      // ...
      let layer = xrGLFactory.createProjectionLayer({
        textureType: "texture-array",
        depthFormat: gl.DEPTH_COMPONENT24
      });
      session.updateRenderState({ layers: [layer] });
      // ...
    }

Render loop

When you get an XRFrame, bind the ViewSubImage (retrieved with getViewSubImage) from each layer to the framebuffer.

    function onXRFrame(t, frame) {
      // some code removed for clarity
      let session = frame.session;
      let pose = frame.getViewerPose(refSpace);

      if (pose) {
        let glLayer = null;

        gl.bindFramebuffer(gl.FRAMEBUFFER, xrFramebuffer);

        let views = [];
        for (let view of pose.views) {
          glLayer = xrGLFactory.getViewSubImage(session.renderState.layers[0], view);
          glLayer.framebuffer = xrFramebuffer;
          gl.bindFramebuffer(gl.FRAMEBUFFER, xrFramebuffer);
          let viewport = glLayer.viewport;

          if (views.length == 0) { // for multiview we need to set fbo only once, so only do this for the first view
            if (!is_multisampled || !do_antialias.checked)
              mv_ext.framebufferTextureMultiviewOVR(gl.DRAW_FRAMEBUFFER, gl.COLOR_ATTACHMENT0, glLayer.colorTexture, 0, 0, 2);
            else
              mv_ext.framebufferTextureMultisampleMultiviewOVR(gl.DRAW_FRAMEBUFFER, gl.COLOR_ATTACHMENT0, glLayer.colorTexture, 0, samples, 0, 2);

            if (glLayer.depthStencilTexture === null) {
              if (depthStencilTex === null) {
                console.log("MaxViews = " + gl.getParameter(mv_ext.MAX_VIEWS_OVR));
                depthStencilTex = gl.createTexture();
                gl.bindTexture(gl.TEXTURE_2D_ARRAY, depthStencilTex);
                gl.texStorage3D(gl.TEXTURE_2D_ARRAY, 1, gl.DEPTH_COMPONENT24, viewport.width, viewport.height, 2);
              }
            } else {
              depthStencilTex = glLayer.depthStencilTexture;
            }
            if (!is_multisampled || !do_antialias.checked)
              mv_ext.framebufferTextureMultiviewOVR(gl.DRAW_FRAMEBUFFER, gl.DEPTH_ATTACHMENT, depthStencilTex, 0, 0, 2);
            else
              mv_ext.framebufferTextureMultisampleMultiviewOVR(gl.DRAW_FRAMEBUFFER, gl.DEPTH_ATTACHMENT, depthStencilTex, 0, samples, 0, 2);

            gl.disable(gl.SCISSOR_TEST);
            gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);
          }

          views.push(new WebXRView(view, glLayer, viewport));
        }

        scene.drawViewArray(views);
      }

      scene.endFrame();
    }

Again, refer to the Code sample section for fully functional multiview example(s).

Note, the multiview extension may be used withoutWebXR; you just need to provide correct view and projection matrices and setup proper viewports.

Changes in shaders

If you are converting WebGL 1.0 to WebGL 2.0, you should use ES 3.00 shaders: only those support multiview.

The following changes might be necessary for vertex shaders in a multiview-enabled experience:

#version 300 es should be added at the top of the shader code;

GL_OVR_multiview extension should be requested on the second line: #extension GL_OVR_multiview : require

layout(num_views=2) in; must be provided on the following line;

In order to convert a WebGL 1.0 shader to ES 3.00, all attribute keywords must be replaced with in, and all varying keywords must be replaced with out:

in vec3 position;

in vec2 texCoord;

out vec2 vTexCoord;

Both left and right projection / model matrices must be provided as uniforms:

uniform mat4 leftProjectionMat;

uniform mat4 leftModelViewMat;

uniform mat4 rightProjectionMat;

uniform mat4 rightModelViewMat;

A built-in view identifier - gl_ViewID_OVR - should be used to determine which matrix set - left or right to use:

mat4 m = gl_ViewID_OVR == 0u ? (leftProjectionMat * leftModelViewMat) : (rightProjectionMat * rightModelViewMat);

The gl_ViewID_OVR is of unsigned int type.

An example WebGL 1.0 vertex shader...

uniform mat4 projectionMat;
uniform mat4 modelViewMat;
attribute vec3 position;
attribute vec2 texCoord;
varying vec2 vTexCoord;

void main() {
  vTexCoord = texCoord;
  gl_Position = projectionMat * modelViewMat * vec4( position, 1.0 );
}

...and the equivalent multiview ES 3.00 shader:

#version 300 es
#extension GL_OVR_multiview : require
layout(num_views=2) in;
uniform mat4 leftProjectionMat;
uniform mat4 leftModelViewMat;
uniform mat4 rightProjectionMat;
uniform mat4 rightModelViewMat;
in vec3 position;
in vec2 texCoord;
out vec2 vTexCoord;

void main() {
  vTexCoord = texCoord;
  mat4 m = gl_ViewID_OVR == 0u ? (leftProjectionMat * leftModelViewMat) :
                                 (rightProjectionMat * rightModelViewMat);
  gl_Position = m * vec4( position, 1.0 );
}

The fragment (pixel) shader should be modified to comply with ES 3.00 spec as well, even though the shader’s logic remains untouched. (Both vertex and fragment shaders must be written using the same specification, otherwise shaders won’t link.)

The main difference is absence of gl_FragColor and necessity to use in and out modifiers. Use explicit out declaration instead of gl_FragColor.

An example WebGL 1.0 fragment shader...

precision mediump float;
uniform sampler2D diffuse;
varying vec2 vTexCoord;

void main() {
  vec4 color = texture2D(diffuse, vTexCoord);
  gl_FragColor = color;
}

...and the equivalent multiview ES 3.00 shader:

#version 300 es
precision mediump float;
uniform sampler2D diffuse;
in vec2 vTexCoord;
out vec4 color;

void main() {
  color = texture(diffuse, vTexCoord);
}

Note: After the conversion, please see console output in the browser developer tools: there will be a detailed error message if the converted shaders have issues.