Pre-test planning and the documentation of your plan may be one of the most undervalued steps in the testing process. A well-prepared plan will ensure a smooth test process throughout and enable you to achieve internal alignment early in the testing process. Be sure to document any and all resource and timing constraints. Your plan should also include the general goals, target audience, logistical considerations/requirements, and any other helpful details you can document before moving forward.

What is it that you’re hoping to learn from your playtest? Your goals will be driven by your position in the development lifecycle, app type, and target audience, and they will fall into one of the following categories:

Find the Fun: Test your game mechanics to discover what elements of the game will drive engagement and adoption. This can be the most difficult step in your development process, as it requires a great deal of creativity and thoughtful iteration. Thankfully this can also take place with your internal team, leveraging WIP assets so that you can quickly iterate on your initial game mechanics, and enable you to evolve your concept into an experience that has testers excited for more.

Usability: While your game does need to be generally enjoyable, you must also test the many design, interaction, and UI choices to ensure that your experience is generally usable. For your app to be successful, the user must understand the rules, player objectives and how the objectives can be met.

Core Gameplay Loop: You have a fun experience, and nothing about your UX is confusing to the user. Now, you need to be sure to test the full game loop. Difficulty balancing will be an essential step in this process, while it’s obviously preferred that your experience keeps the user’s attention from start to finish.

Technical QA + Bug Reports: While you will find technical bugs throughout many of the steps above, you should also find time to test your app specifically to assure technical stability. Keep in mind the VRCs for Meta Quest and Rift, as testing these early will help your odds of a smooth app submission process.

Like many product-focused efforts, we recommend a test/research plan to help ensure team alignment and guide the execution of your test. The depth of this document will be determined by the complexity of the questions you’re looking to answer, with this in mind, see below for a number of items you may want to include in your test plan.

Your target audience should be determined early in your development process, and if you’re new to this step, check out Module 2: The Essentials of Strategy of the Unity + Oculus course for more detail on how to determine the ideal audience for your app.

Once you’ve determined your target audience, you will want to devise a recruitment strategy. While your goals, timing, and resources will impact your recruitment process, it is important to recruit members of your target audience, as recruiting the wrong audience can drastically diminish the legitimacy of your entire test.

This is not always an easy process, so you may need to get creative. Does your app have board game elements? Maybe check out a local table top game meetup or online community forums. How about a basketball app? Can you set up at a local rec center?

Depending on your test, this might also be a good time to leverage your existing communities via your marketing email list, or the communities you’ve garnered on forums + social networks. Here’s a note from AnnMarie Bartholomaeus, Product Manager at Kluge Interactive:

Whether you’re hosting the playtest at your office, a local event, or remotely, every VR playtest has logistic considerations to keep in mind. Planning and documenting your logistics may be the difference between a fragmented list of comments and a well-organized set of actionable feedback. See below for some of the key logistical items to keep in mind:

Many playtests are managed by a single researcher, and with this guide, you should feel empowered to manage the entire process from planning to results analysis. However, if you have teammates available during your playtest, it can be helpful for them to provide support in the following areas:

Taking into account each of the roles and responsibilities above, it can be helpful in the long term to recruit members of your design and/or engineering team for any of these assignments. This can help them to gain empathy for the end user, and become an advocate as you look to create actionable changes to your app based on these results.

Hardware can be especially challenging when testing outside of your workspace. Bring extra cables, batteries, and power supplies, an extra wi-fi hotspot can also be a life saver. Be sure to update all of your software/drivers with plenty of time for the playtest. Some teams prefer to lock down their build 3+ days before the playtest to ensure that no unplanned features or bugs creep up during a playtest. Lastly, this should come as no surprise, document your hardware and PC specs whenever possible.

No matter the location of your test, you will want to test varying playspaces to ensure that your app performs well for the maximum number of users. This includes testing your app in stationary vs seated mode, along with playspaces that might be long but narrow, short but wide, or irregularly shaped. Like your hardware/software, be sure to document the playspace size/shape and make this part of your survey questionnaire if it is a remote playtest.

For the setup, it might help to fill out a check list with items like area is clear, boundary is set, etc. We hope that setup is not a major point of complexity, but it doesn’t hurt to note it in your test plan in case there is anything unique about your setup process.

From a technical perspective, we do want to remind you of the hardware testing requirements to access the Meta Horizon store. Be sure to keep these in mind as they are a part of the VRCs which you must pass before going live on the store: VRCs for Meta Quest.

While resources and time can certainly impact your ability to record a playtest, we recommend recording with what hardware you have available. This could be with the audio recorder or video camera on your mobile phone, or 2 video cameras on either side of the tester.

Be sure to record the in-game footage for future reference, and review the section below on casting to a mobile device. Lastly, it is worth emphasizing that if you are recording, get written or verbal approval from the tester prior to starting your test.

To add a bit of automation to your testing process, adding data hooks to enable telemetry throughout your application can help to provide a number of additional data points to your test. While this data will be helpful in the analysis phase, prioritization is key, as it will require time and resources to review and report on this data.

See below for a few examples of helpful telemetry to record:

While testing in-person can streamline much of the effort and provide added opportunities for observation, it can also make for a few logistical challenges.

Our partners at Schell Games have a great deal of experience with playtesting VR and 2D games, see below for a few of their key considerations when planning for in-person testing:

Like in-person testing, playtesting your app remotely can present its own set of unique challenges. As you will more than likely have less control over the environment when the tester is remote, be sure to keep the following in mind:

Hardware + playspace size considerations: Be sure that your tester clearly communicates their available hardware specs and playspace to be sure that it aligns with your research goals.

Ensure your tester is aware of these environmental considerations: There are many environmental variables to take into account for remote testing. Keep the below in mind as you analyze certain challenges with tracking, or your own testing environment:

(With the above in mind) Enable your audience to test in their natural environment: A key component of remote testing is that it takes place in the user’s natural environment. This will reveal all sorts of things you weren’t aware of - like the tester may not turn on their lights all the way when they play your game.

In the recruiting phase, you should determine if they have enough space to experience your type of app, especially if yours requires more than what is allotted for a stationary playspace. Ultimately the natural environment is something you want to collect data on, not change.

Moderate your remote tests for early iterations: For early iterations of your build that may lack UX/design polish, it’s strongly recommended that you moderate your remote tests whenever possible. This ensures you can provide verbal instructions and troubleshooting guidance should the user become stuck in some way.

Custom release channels for streamlined sharing of your build: Custom release channels are a secure, user friendly solution for distributing your pre-release app for the purpose of playtesting. Every organization comes with existing release channels, and has the ability to create more. See below for a number of ways this feature can support your testing efforts:

See the following technical documentation for how to get started with custom release channels:

Casting your experience + recommended instructions: While casting is a great way to generally improve user engagement, it can also be an essential tool for remote testing. If you instruct your testers to cast their experience, we recommended you communicate the following:

You may also want to share the following video walkthrough to cast from a Meta Quest device to the Meta Quest Mobile App.

For remote, moderated testing some of our developer partners have found that instructing the tester to cast their phone while sharing their screen through a video sharing app has been especially effective.

Further considerations for unmoderated, remote tests: If your remote test will be unmoderated, there are a number of unique items you should keep in mind, once again our friends at Schell Games have plenty of experience in this area and recommend the following:

With your research goals documented, this should streamline the writing process for your script + survey. See below for a number of best practices, as well as example documentation to help you get started.

While you may have the urge to wing it, we strongly recommend that you document your full script prior to the test. Be sure to practice the script, preferably with a teammate or friend, and include the recommendations below:

The types of questions you might ask all depend on your goals + research plan. Example areas to test include:

We recommend reading the following post by the team at Schell Games as it provides a number of insights for building game playtest survey questions. The Defin­i­tive Guide to Playtest Questions

Be sure to check out the next guide on VR playtesting which covers facilitation, moderation and post-test data analysis. In this guide we will share the many techniques to guide your tester through the test, and analyze the resulting data to ensure that you acquire actionable insights to further improve your VR app.