How Frame and Oculus Brought Trello’s Virtual Offsite to Life

As the COVID-19 pandemic dragged on into its second year, the team at Trello⁠ was ready to organize their annual offsite to facilitate team-building and boost morale: Trello Together 2021. While they hoped to hold the event in person, safety precautions and a desire to slow the spread of the virus dictated otherwise. Some members of the team had joined Trello and onboarded remotely—never having stepped foot inside the company’s New York City office space, while others hadn’t been back to the office in over a year. What better way to bring everyone together than with a digital recreation of the Trello office that the team could gather inside of and explore?

That’s when FRAME ⁠from Virbela ⁠came into play. FRAME is an immersive meeting tool built with WebXR that runs on desktop, mobile, and the Oculus Quest Platform via the Oculus Browser. FRAME makes it easy for people to meet across different devices, to collaborate, and to set up persistent, immersive offices, classes, galleries, event spaces, and more. While there are plenty of environments FRAME users can choose from and use “out-of-the-box,” this experience for Trello required something special.

For this use case, the goal was to recreate the Trello office in a way that could be experienced using an Oculus Quest headset or through the web on desktop and mobile. The result was an immersive and realistic digital twin that surprised and delighted Trello’s offsite attendees. Newer employees got an accurate sense of the office’s look and feel, while those who had been around pre-pandemic said it felt like going home.

This came with some challenges. One was the design of the environment and ensuring that it was optimized, built, and designed according to Trello’s needs and the reality of the physical space. Their office is actually a fairly large space with a lot of different rooms, textures, and objects, and we needed the virtual experience to be performant on a mobile chipset.

We had to compress our textures, for starters. We used basis compression—an open-source compression method that creates textures you can load right into the GPU. This leads to more efficient memory utilization and spares the CPU cycles typically required to decompress JPEGs or PNGs.

We used instancing for the geometry. A few different offices had the same geometric shape, so rather than recreate each of those shapes, we just built one of them and then instanced it a few times—also more efficient for the GPU. That helped us keep the file size manageable by avoiding duplicate geometry. We also used baked lighting rather than dynamic lighting to help improve performance.

We developed a sophisticated avatar system that lets the avatars be instanced, too. Even though there are elements to the avatars that you can customize (like picking a different hat, skin color, or earrings), all of the elements of the avatar are instances. With this system, we were able to provide a range of customization while still comfortably accommodating 40 avatars in each of these Trello offices. We’re already working on an iteration of this system that will comfortably allow for hundreds of simultaneous, customizable avatars.

We took some shortcuts on certain design elements, like the office plants. We didn’t need to model them in hyper-realistic detail (plants are complex!), but we still needed them to be there and look decent. So, the plants were essentially just 2D images, strategically positioned to give the illusion of being 3D. But we splurged in other areas, like their distinct office mural, which we made sure looked fantastic. The performance ended up being so good that, aside from people joining from their Oculus Quest headsets or on a desktop, some people came in right from mobile.

Another thing that helped this event come together was that while we designed the Trello environment, the Trello teammates who were putting the event together were able to use FRAME on their own to customize a lot of the other aspects of their experience. FRAME provides a simple drag-and-drop interface for adding other assets to the 3D scene like images, 360° photos, screen shares, whiteboards, 3D models, audio, video, and more. Trello wanted many of these assets (particularly whiteboards and 360° photos), and we had to keep an eye on the total asset count to make sure performance was still up to snuff. It was really neat seeing the Trello team drawing together using those whiteboards and checking out images from previous Trello Together events that their teammates had put up on the walls.

We used a browser protocol called WebRTC for our communication between the avatars, so the audio, video, avatar position, and rotation synchronization was all done through WebRTC. WebRTC is peer-to-peer by default, but that wasn’t good enough for interaction at this scale. So we had to bring servers into the mix with an SFU, a Selective Forwarding Unit. We’ve done a lot of our own SFU architecture so that our backend can flex the way we need it to—and we’re already building the next iteration of this architecture to handle even larger events.

We used the Opus audio codec to help reduce the load of the audio streams, but more importantly, we progressively limited the amount of data that people were getting from other peers based on the performance of their device. So if you’re in the experience and you’re at 60 FPS or higher, we’d just keep feeding as much data to you as we could. You’d get really quick updates from other avatars and things like that. But if we detected that your device was struggling a bit and you were at 30 FPS or lower, then we actually restricted the frequency of the updates that you received. That way, you’d still be able to get those updates, see people moving, and hear people talking, but those updates didn’t impact performance as much. We have a lot of work to do on these kinds of dynamic performance improvements, but the ones we built for this event were enough to help it enormously.

FRAME uses the WebXR API, and we’re grateful that the Oculus Browser team puts so much effort into ensuring that the Oculus Browser is a powerful vehicle for this kind of WebXR experience. Rather than having to download an app and then navigate to a particular location, with FRAME you just have the link that you put in the browser and then you’re there. It reduces a lot of friction. It also means that people can come in from a variety of devices, so you don’t need to have a VR headset to come in and have a great experience—this was important to Trello for this event.

On the topic of VR, we entirely redid our VR user interface in preparation for this event. This was the first big event in FRAME where most people were going to be on Quest, which was special. We’ve always built for VR, but never before had there been an event of this scale in FRAME where so many people were on Quest. It really prodded us to expand on and enhance our user interface for VR users. Before this event, you could join FRAME in VR, but you really just had an avatar and you could talk and move around. But for this event, and for the product generally, we built a whole new VR UI. Now you have a little icon on your wrist, and if you pop it open you can do things like customize your avatar, adjust a lot of your settings, and blast out emojis.

One thing that helped make this experience so successful was that we did a little bit of FRAME training with the Trello team before they actually came into the virtual office. The office was a new world that really surprised and delighted them when it was time for the event, but by the time they came into their office for the event, they were already familiar with FRAME and knew how to move around, talk, and more. Many of them had already created their avatars, and it was clear that the orientation sessions they joined before the event to get acquainted with FRAME and the controls paid off immensely.

At its heart, this was a matter of listening closely to the client. This was a case where we were building something personal to Trello—a core part of the company’s culture. We had weekly meetings with Trello for months until the project was done. This kind of iterative approach was key. You’re not going to be able to recreate every aspect of a real-world location exactly, but we were able to really hone in on the super important bits with them. There were some parts of the office that were deeply important for their culture and company history—little things you wouldn’t expect. It’s not always the flashy things like the murals—although yeah, the murals are unique and epic.

For example, there’s this little motto in one of the small offices that pays homage to a former Trello employee. That’s the kind of thing we wouldn’t have even picked up on unless we had those close meetings—you probably would have missed it if you’d only seen a photograph of the room. It was neat to see people who had seen the New York City office before showing around the people who had been hired in the past year and pointing these things out to them.

If I had it to do over again, I would have been a little bit less cautious with our graphics budget. We might have erred too much on the side of caution—we ended up getting the model down to 4MB. This is great, but we could have had it a bit higher and gotten some huge resolution boosts, without sacrificing performance too much. There were some textures that got a little bit more love than others.

We mastered some of these compression techniques right at the tail end of the project, so we didn’t have time to round it out throughout the whole space. That’s why some of the walls and ceiling look a little bit gritty. I’m sure our artists would’ve appreciated a little more juice to play with. I was always telling them, “Squeeze it down,” but the Quest is actually so powerful (yes, even the Oculus Browser), it can comfortably handle much more than what we threw at it.

We’re excited to see how Trello continues to use their virtual office in FRAME, and we’re already improving FRAME based on lessons learned from Trello Together 2021. This extraordinary event showed that WebXR is viable, that the web browser is a powerful vehicle for cross-platform experiences, that Trello is a forward-looking company, and that Quest is a transformative piece of hardware. Most importantly, it showed that connection matters more than ever.

This is exactly what we built FRAME to do: help people come together and feel together online no matter where they are in the world.