Jul 20, 2021

Why Mixed Reality Capture is Effective: Q&A with Scout House

Caitlin Bigham, Ecosystem Manager - Oculus Blog Hero Image

What is Mixed Reality Capture?

Mixed Reality Capture (MRC) technology allows you to capture video that shows both VR gameplay and the player in the same frame. The end result is an immersive video form that more accurately conveys to an audience what a VR experience is truly like, as compared to traditional 2D gameplay footage which lacks the immersive qualities of VR. Because of its realistic nature, MRC can serve as a great marketing tool.

Last November I set out to measure exactly how much MRC moves the needle in marketing and sales of VR content. I worked with the developers of OhShape (Odders Lab) and Holopoint (Alzan Studios) to integrate the Oculus Mixed Reality Capture Tool⁠ into their Quest titles, and then partnered with Scout House⁠, a mixed reality studio based in LA, to produce MRC trailers for OhShape and Holopoint.

MRC vs POV A/B Test

To truly measure the impact of MRC trailers on converting Quest owners to purchase content, we decided to run A/B tests on the Oculus Store pages for OhShape and Holopoint. The tests ran during the ‘20-21 holiday timeframe (12/21/20-01/11/21).

During the A/B test, 50% of Quest owners saw an MRC trailer on the games’ Oculus Store pages and the other 50% saw a first person point-of-view gameplay (POV) trailer. The gameplay featured in each trailer was identical, except in the MRC version the player was visible as well. Our hypothesis was that the Quest owners who saw the MRC trailers would be more likely to purchase the game than were the Quest owners who saw the POV trailers — seeing someone play and have fun in the game is more compelling to a potential buyer.

MRC Trailers Outperform Gameplay Trailers in Converting People to Purchase Apps

The results are in! The MRC trailers significantly outperformed the POV trailers. OhShape’s Oculus Store page featuring an MRC trailer saw a 30% increase in purchase conversions when shown to Quest 2 owners in headset, as opposed to their Oculus Store page featuring a POV trailer. Holopoint’s PDP featuring an MRC trailer saw a 25% increase in purchase conversions when shown to Quest 2 owners in headset, as opposed to their Oculus Store page featuring a POV trailer.*

While MRC is clearly an invaluable marketing tool for VR games, it does have some nuances when it comes to live capture. I sat down with the Scout House team to learn the tips and tricks of creating Mixed Reality Capture content for Quest games.

Q&A with Eric Chevalier, Jonathan Nafarrette and Zane Parker from Scout House: Mixed Reality Capture Tips & Tricks

Q: Tell us about Scout House and your experience shooting mixed reality video?

Scout House is a mixed reality studio based in LA. Since 2015, we’ve been deeply immersed in the VR industry, working closely with developers and partners like Oculus to bring games to market with high quality mixed reality video content.

Over the past couple years, we have helped launch over 20 titles for the Oculus Quest. We have also worked hand-in-hand with the team at Oculus to help test and refine the MRC product line.

Q: Why mixed reality?

We pride ourselves on giving viewers a real sense of the action in VR. Instead of relying solely on CG and VFX to recreate the feeling of a game, we put a lot of care into authentic mixed reality capture. Real players. Playing in real time. And as the technology evolves, we are constantly pushing what is possible on the Quest 2 and Oculus MRC tools.

MRC Integration & Optimization: Setting your application up for MRC greatness

Hiding and showing specific assets

Q: If a game utilizes avatars and player models like hands, is it still possible to capture MRC?

A: We shoot mixed reality video to show human interaction in a virtual world. To make these interactions clear, we often hide assets like avatars and modeled hands from the MRC camera to seamlessly blend our human talent into the environment and create a more immersive viewing experience without the confusion of duplicate bodies and hands waving about. We always leave them visible to the headset camera so that our talent can operate the game.

Some assets can look compelling when left activated, so try toggling certain assets on and off to get a sense of what looks best. You can even add a toggle menu into your game to allow users to hide assets from the camera that they may or may not want visible when shooting MRC.

Clearing an environment of anything that would block the camera

Q: When it comes to game environments, what should be taken into consideration when there’s an additional MRC camera in the scene?

The MRC camera captures the action from a 3rd person perspective. This means that you’ll have another camera in the scene that will need line of sight to the player. Because of this, it's best to set up the walls and objects in your game to cull out (stop rendering) when the MRC camera intersects them or to make sure that walls and large structures in the game don’t obstruct the camera's view.

Make sure to test your game in MR to figure out what assets should be removed. Some assets like barrels, crates, railings, and other interesting objects your player moves behind can be shown in the foreground of the MRC capture feed, lending more depth and dimension to your captured video.

Production Do's and Don'ts

Lighting and environment

Q: Say someone has never really worked with a greenscreen before, what are some considerations they should take into account?

A: The most important aspect of shooting green screen capture is lighting. Light your green screen background as evenly as possible. Also make sure that the player is evenly lit and stands at least 6-8 feet away from the background. Be cognizant of shadows or areas that will make it more difficult to remove the green screen during the edit.

Standing further from the green screen will also help avoid green “spill”, which is when the green light bounces off the backdrop and onto your subject. Again, this will save you time (and money) spent in post production manually separating the background from the player.

You’ll usually want to avoid lighter colors and highly reflective fabrics and materials. They may cause green screen light to spill on your subject’s outfit, so always run tests before you shoot. And of course, don’t let your talent wear green.

The internet is full of information about shooting and lighting on a green screen. These tried and true techniques are a major factor in capturing high quality mixed reality footage.

Connecting camera through capture card

Q: What's the best way to capture the player on camera and see them in the game?

A: We suggest using a camera with the highest fidelity that fits within your budget. At the very least, we recommend shooting with a DSLR, or even a GoPro, with the ability to preview via HDMI and connect to a capture card, like an Elgato Cam Link. This will let you stream into OBS with the Oculus MRC plugin.

Q: How do you capture footage to use when editing in post-production?

A: Using OBS and the MRC plugin provided by Oculus, you’ll want to separate the layers of the MRC feed and capture card so that each layer can be edited separately in a post production program like Adobe Premier. This will allow you to make adjustments to each separate layer and add different visual effects to the game or player without affecting the other.

Q: What are ways to optimize the footage and make it more engaging?

A: Start your video off with mixed reality footage. A combination of POV, MRC, and in-engine pickups and fly-throughs all work together to tell a story, but we’ve found starting off with a compelling MRC shot to grab the viewer within the first few seconds to increase performance.

When it comes to visual effects, since the MRC camera is locked off on a tripod, you can use some simulated camera movements in post-production to make the shots even more dynamic.

Extra Polish: Hair, makeup, and wardrobe

Q: Any final tips when filming that can plus up mixed reality capture?

A: Budget permitting, having a hair and makeup artist on set will keep talent looking their best from shot to shot. A wardrobe stylist with green screen experience is also helpful.

If you’re interested in creating an MRC trailer to leverage in your game’s marketing campaign, get started today by integrating the Oculus Mixed Reality Capture Tool⁠ into your Quest title.

*This data is based off of A/B tests run 12/21/20-01/11/21. The findings reported reflect A/B test results where 50% of Quest 2 owners saw an MRC trailer and the other 50% saw a POV trailer upon landing on the Oculus Store pages for OhShape and Holopoint. All data referenced is statistically significant.

Apps

Optimization

Quest

Did you find this page helpful?

Explore more