Mixed reality capture (MRC) and casting deliver a heightened sense of engagement and immersion for users outside of the VR headset, and there are numerous ways for you to design and build your app for an improved spectator experience.

To begin, think about your reasons for using MRC. Are you producing a high fidelity trailer to improve app awareness, empowering influencers to stream your game, or all of the above? Note your target audience, who are they, and how will their unique preferences impact the way your app is viewed? Once these top line decisions have been made, then you can begin to align the design of your game to maximize user engagement for those outside the headset.

How items occlude the VR user and vice versa is critical to an engaging spectator experience. You will want to find a balance between the environment, objects and characters included within the scene, especially those in the foreground.

The perspective of the virtual camera is a major factor. If the action is taking place away from the VR user, you may want a wider FOV + camera lens. Wider angles also give you more opportunity to tell your story with the larger environment and the objects that make up your world. The opposite is also true, as large, empty spaces can be interpreted as hollow and generally less exciting.

For small, tight rooms, the angle is especially important, as you want to make sure that walls do not obstruct the spectator’s view. If you design your app for MRC and streamers, you may want to provide them a few guidelines for where they should place the virtual camera if their space allows for this sort of flexibility.

Starting with the VR player avatar, you will want to assure that there is a balance between those items you display for added embodiment like arms, hands, etc., and whether you want to display these to the spectator. Here’s a note from the team at Kluge Interactive:

We ship our game with built-in avatars, support for custom avatars and a third person camera system built into the game. Avatar gameplay [for casting] is incredibly popular with content creators, so having this immediately available in game also helps them generate their content with those engaging, third person perspectives of gameplay which show our game in the best light. - AnnMarie Bartholomaeus, Product Manager

Also, consider those objects that are acquired throughout the VR experience, or additional clothing like a watch, hat, or holster. Will these add to the spectator experience if they occlude the MR capture? See below for an example of Reel VR fishing and how they display the glove + rod throughout.

Locomotion should be a core consideration if you plan to use MRC. If your app has a great deal of movement around the scene, locomotion mechanics like teleporting and smooth locomotion can make the user look as though they are skating.

While it is specific to more stationary experiences, one method of resolving this design challenge is to display a platform under the user. Feel free to be creative with how to “ground” the user in the environment, as this “skating” effect can sometimes detract from the spectator experience.

One of the more substantial opportunities for an improved spectator experience is with UI overlay elements, especially those that are unique to the spectator. Whether it’s the score of the game, inventory amounts, or a real-time map, we’re excited to see how VR designers leverage this opportunity to share unique, contextual information and further elevate the spectator experience.

Designing your app for watchability via MRC or casting brings new opportunities, challenges, and potentially new modes of thinking about VR design. See below for a few of the new opportunities and considerations you should keep in mind as you look to enable an enhanced spectator experience.

What does the spectator see and would it drive interest in your app: Whether you are sharing MRC footage or casting your app, think about what the viewer is experiencing, especially if this is their first time viewing the app. Would they be able to understand what’s happening and want to check out the experience first hand?

This is especially important for casting, when you are capturing one of the eye buffers. Make sure that this vantage point isn’t overly confusing for a first time spectator.

The importance of interactions to elevate the spectating experience: Interactions are a core element of any VR app, and this goes for the spectator experience as well (if the above considerations are also taken into account).

Audio is just as important for MRC + casting: As with general VR design recommendations, spatial audio is essential to an engaging experience. This is also the case with MRC as the soundscape you provide will only enhance the experience for the VR player and spectator.

Live streaming MRC content: In regards to distribution, remember that there are numerous social channels available to host your MRC/streaming content, including Facebook Watch. These further increase the reach of your content, as well as the ability to target your specific audience.

Shader layering and the environment: As with objects and avatars, be cognizant of how you layer on shaders throughout your environment, especially with how it’s arranged as compared to the MRC object plane.

Recommend using a Rift S for picture in picture mode: If you or your streaming partner is looking to use some sort of picture-in-picture mode, it is recommended that you use a Rift S variant of the game, as visuals will naturally display at a higher quality, putting less strain on the wireless network.

Adding the opportunity for viewers to select more than one perspective: You might offer the opportunity for multiple spectator perspectives, including first person, a “birds eye view” from above the main character, or some other angle of the scene. Coupled with the opportunity for the spectator to switch between cameras can make for an especially exciting experience.

A well produced, thoughtfully designed MRC production will help you to communicate your experience in new ways, publishing these on your product detail page (PDP) can lead to increased conversation and user engagement. Here’s a note from the team at Miragesoft:

I strongly believe that Mixed Reality Capture is a valuable tool and is a “Must-use” for VR Developers for their success in publishing their game. - Mark Choi, COO

If you’re ready to begin developing with MRC tools, and producing your next MRC-enabled trailer, check out the following guide for more best practices, tips and recommendations: