Public VR Critique #3: Playhead

This is a short, free audio experience that was one of the first apps available for Gear VR.

One of the things I took away from judging the VR Jam entries was how much high quality audio stood out when using headphones. Practically every graphics scene has something I can nit-pick, but there were a number of times that I thought “Wow, that sounds really good!”. It kills me that Gear VR is so often used without headphones, making the audio much lower quality. I’m as guilty of it as anyone; the big win for Gear VR is how convenient it is to just pop it on your head anywhere, and plugging in and donning headphones seems to double the effort to get into it. I very much hope that a future Gear VR can get integrated audio of some sort, but it turns out to be sort of tricky – little ear wings like the consumer Rift has would probably not survive the toss-it-in-a-backpack transport model that Gear VR gets.

In one of the internal multi-user apps we are working on, I explicitly put up a “Plug in headphones!” message on the main screen when headphones aren’t attached, and I think this is a good suggestion for applications where audio is really important. Oculus task: make a Unity demo of this.

In the initial game scene, the emissive control system lights show a lot of aliasing when activated. As a graphics guy, I know I over-index on this, and I want to make sure everyone keeps it in perspective – perfect filtering won’t help a bad experience, and you can still have an awesome experience with terrible filtering, but it does make a difference with first impressions and reduces eye strain.

It looks like the line art may not have mip maps, and is slightly undersampling, but even with mip maps, fine line art will still tend to look poor with most mobile renderers for a couple reasons. A lot of games can get away with not being “gamma correct” if their art style doesn’t have a lot of high frequency elements in it, but line art and star fields make a dramatic case for it.

Line art that is rendered without proper gamma correction will have a “ropey” look to it – there will be brighter and dimmer bands along the lines that move around as the lines are rendered at different sub-pixel offsets.

This is a poorly understood topic, so a brief diversion:

Imagine a texture with a single white dot in the middle, aligned with the screen so that each screen pixel lines up with a texel.The screen will be all 0.0 except for the one pixel at 1.0.

With bilinear filtering, if you shift the view half a pixel over, the pixels will now be between two texels, and there will be two pixels with a value of 0.5.

If you then shift the view half a pixel down, there will be four pixels with a value of 0.25. In general, except for the times you land exactly on a texture row or column, there will be four pixels lit with values that sum to 1.0. This makes intuitive sense; there should be a “conservation of light energy” regardless of where on the screen the textured dot winds up.

This would be what we want if our displays had a linear response, but they don’t. Most displays today theoretically use the (mostly) exponential sRGB color space⁠. This curve makes the four x 0.25 pixel dot only 20% the intensity of the single 1.0 pixel dot – the dot will be changing intensity by up to 5x as you make tiny head motions! It isn’t quite as bad as it sounds, because “perceived intensity” is also exponential (that’s why sRGB works as data compression in the first place), but it isn’t good.

For a longer discussion about this, see: Linear-Space Lighting (i.e. Gamma)⁠. Color calibration on displays in the real world is all over the place, but Gear VR is much better than most monitors.

GPUs today have dedicated hardware to do the conversions between linear space and sRGB space, but you have to explicitly enable it on the framebuffer before you draw to it, and on every texture that should be sRGB (basically anything that comes from an artist, but not things like lightmaps). It is a bit of a pain, and it does have a small performance cost in some cases, so it can be hard to retrofit onto existing titles. The hack fix is to just reduce contrast on anything that is showing color space aliasing – blur it by a pixel or two, and the error between linear and exponential gets much smaller.

New titles should consider going with gamma correct rendering from the start. Oculus task: convert at least some of our Unity demos to gamma correct rendering.

With current generation VR headsets, there is a final problem for line art: the resolution is low enough that you can still pretty clearly discern individual pixels, so a sharp line that has its width changing between one and two pixels wide will still be perceived as “lumpy”, even if there is perfect conservation of light energy. Making the lines thicker resolves it – the change from 2 to 3 pixels wide, or 3 to 4, is much less noticeable than 1 to 2.

It doesn’t matter much for Playhead, since the touch pad is only used to start and for optional timeline controls, but I encourage developers to always allow a joypad button press to perform the same action as a touchpad press, even if the game doesn’t need any other joypad functionality.

The gaze cursor is always being drawn over the scene without regard to depth testing; you can put it behind some of the near field geometry, but it still draws on top of it, which causes eye strain. It doesn’t hurt as much here because the near field geometry has fairly flat texturing so your eyes don’t get a good depth fix on it, but physically implausible depth errors are never a good idea. In the Oculus first party apps, I have the cursor turn into a broader random scatter when it is occluded (you can see this easiest in Cinema by looking down at the chairs when the menu is up), but I’m not sure that is any better than just letting it get knifed off by the depth test. Arranging things so you can fade the cursor out before it can be put behind geometry, or dynamically adjusting its depth, is probably best.

The title text is 3D rendered, but I didn’t even notice that the first time I saw it, I just thought it was badly aliasing text. Unless you are going to club the user over the head with the fact that text is actually 3D models with thickness, it is usually best to just build a high quality image with Illustrator, render it out at a high enough resolution that it never gets stretched, and generate a full mip map chain for trilinear filtering. A blended texture is like 256x MSAA on geometry.

The use of a blocky, low resolution, 8 bit-ish font in a few places seemed an off stylistic choice.

The button image on your vehicle is a play / pause status indicator, but it looks almost exactly like the action buttons on the pop-in menu; I tried tapping it a couple times with no response.

It would be nice to be able to fast forward through the intro on later launches.

The technical comments are basically little nits that I would point out on almost every title; overall I thought most of the visuals were pleasant and tasteful. The big issue is that my first run through the experience, I had no idea what I was supposed to be doing. I hit the end of the track without activating all the instruments and got sent back, which wasn’t a positive feeling. I went through the entire thing a few times, and after you know what to do it is hard to recapture the initial feeling of ignorance, but watching more “tissue testers” (use fresh once, then discard) would probably show it to be fairly common.

In a mostly static environment, everyone knows to look at the floating thing, but in a complex and moving environment, gaze targets need to draw attention to themselves in very blatant ways. Especially in a linear “experience” where you aren’t trying to challenge the user, your job is to make them always feel like they know what to do.

Motion changes on the surface of an object are usually the most effective way to draw attention, but lighting / glow changes are often easier to integrate. For both, sharper numeric changes are usually more effective – pulses rather than sine waves.

I usually recommend audio cues with the transition to gaze-on, but that might get lost in an audio focused experience.

When the gaze does hit an active object, the object should visibly react. If there is a chance that the gaze-on might be brief, the feedback should have a very sharp attack, rather than slowly fading in. The pop-over quads definitely let you know that you are activating something, but it is still possible to not correlate it with exactly what you were looking at.

In a situation like this where you need to make fairly large gaze changes, there might be a useful intermediate step where there is a reaction as the object is approached, before it is actually activated. If there was a synchronized change in the gaze cursor color/shape/motion with the effects on the object, it would probably tie them together and give a feel of “attraction” as it strengthened.