How to Run an Unmoderated Remote Usability Test (URUT)

Key Takeaways

Hi, my name is Chris Gray and today I'm going to be talking about how to run an unmodderated remote usability test or URUT for short. So, this is a technique that evaluates the usability of an interface or product via an online testing platform. It's really similar to in-person usability testing. However, participants complete tasks in their own environment without a facilitator present. Now, there are two broad methods for collecting participant behavior and this is important to understand. The first is via video recordings of participants interacting with an interface and the second is via clickstream data much like web analytics. So why use unmodderated remote usability testing? Firstly, it's valuable when you need to obtain a large sample andor need a high degree of confidence with your study. URU can also be used where your audience is geographically dispersed or hard to access because it's conducted online much like a survey. It can be completed in the participants own time and in a location of their own choosing. It's also valuable where speed is crucial. A UR study can be run entirely in a couple of days. It's valuable where a specific environment or context is critical and also where budgets are tight. URU can be relatively inexpensive. It's also helpful in cases where you need to compare two or more products or interfaces. URU is perfect for benchmarking studies, comparing either competitor products or different iterations of your product. The ability to capture large sample sizes means that statistically significant differences between interfaces can be identified. So how do we actually run an unmodderated remote usability test? So there are a couple of things to consider before you start. Firstly, define the project objectives and identify your research questions. This will help with designing the study and picking the right tool. Identify your sample. Ideally, participants are representative of the product's audience. Source your participants through either a database of existing customers, running an intercept on a website, utilizing your social media presence, or paying for a sample via a pre-existing panel. Tasks developed for URU need to be clear and provide enough detail for participants to complete the task on their own. Try to include any information they would require to complete the task. For example, if a task requires credit card details, providing fictitious card details will be necessary. Also, look at using questions in addition to tasks to collect further information. Questions can be used to verify that a task has been completed correctly. Include questions after each individual task to measure ease of task completion. Questions can also be provided after the test as a whole to gauge an overall assessment of the experience. And finally, open-ended questions allow participants to expand on their answers and give you more detail. Piloting the study with either a subset of participants or in a preview mode will allow issues with the prototype, technology, tasks, or questions to be ironed out. And while you're testing, it's important to monitor the data and be available for offering help to participants. This will ensure everything is working as planned and that you're able to receive data that actually meets the objectives of your study. Once you've collected your results, it's time for analysis. Start by looking at some overarching metrics such as overall task completion, the system usability scale, and customer satisfaction. These will give you a sense of the overall performance of the product. Next, look into the individual tasks and identify those that are causing issues and why. Watch video of specific tasks to observe behavior and identify the elements of the interface that are causing the issues. For clickstream services, focus in on the combination of the pages visited during the tasks to identify behavior and the pages where the issues have occurred. Now that you have an idea of how to run an unmodderated remote usability test, here are some final tips. Choose the testing platform after you've identified the objectives of your study. It is crucial to select a tool that is fit for purpose and will support your study's objectives. Set clear expectations for participants. Obtaining useful data is dependent on participants understanding what is expected of them. Remember that participants won't receive any assistance during the study from a facilitator. It is crucial to ensure that tasks are clear, userfriendly, and that help is available. Avoid bias. Randomize the order of tasks and pay attention to task wording to avoid bias. Keep participants engaged. Avoid participants quitting your study by keeping it interesting and short. Unmodderated remote usability testing is a technique that can offer quick, inexpensive, and robust usability testing. Of particular value can be the ability to use the technique for benchmarking and context sensitive studies. It's a great tool to have in your bag of research techniques. Exploring the different tools on offer and experimenting with technique is the best way to learn and develop expertise. Good luck. [Music]