A Blueprint For Mobile Usability Testing

The idea of mobility usability testing for apps has only been around a couple of years, but it is a critical procedure for those people involved in the mobile app game. While the testing process does share some similarities with traditional desktop usability testing, it is different enough to cause concern for people who haven’t done it before which is where our blueprint for mobility usability testing comes in.

As there are many differences in the app development game the overall process might need to be adjusted as you go; but for now this blueprint provides a good starting point with our own insights as to what we have found effective.

The MUT Blueprint

• Timing – An important aspect of testing to be aware of is the timeframe and planning that needs to go into testing. Realistically you need to start prep work on testing at least six weeks, if not eight weeks, from the target test date. The testing dates always seem to arrive faster than expected and there are a number of steps that need to be taken that have a time element to them that is hard to circumvent without shortcutting the process.
• Test Options – There are few methods of testing we recommend; moderated or unmoderated. What this means is that you can ‘moderate’ the testing by having the users and the testing facilitator utilize the same space at the same time with direct communication between the two. Unmoderated simply means the tester does the testing on their own, often recording the session, which the testing facilitator (or project manager) will review later. Then with either of these options you can do them in-person or remotely.
  • Benefits of Moderated – Moderated testing allows for real-time support, questions, clarification and the like. One of the biggest benefits is being actively involved from start to finish; you can learn a lot from testers during the window of testing because of the ability to ask questions and gain clarification.
  • Benefits of Unmoderated – More flexibility on timeframes and costs. This type of testing can be done remotely by people overnight if needed. Ideally this type of testing works best for a study of only a few things, such as an update or the change of a few features.
  • In-person vs. Remote – This also ends up being a timing and expense issue. In-person is often far superior to remote monitoring because you have the chance to read body language, gauge the true time between actions and notice the right time for follow-up questions. However, remote testing is good for people on a budget or when your testers are a further distance away and/or spread out.
• Pre-Test Meeting – Four to six weeks out from the testing date a meeting between the person in charge of usability testing and the apps design and development team should take place. The goal is to get everyone on the same page about when testing will take place, what is going to be tested, and what will be available from a prototype aspect at that time for people to test on.
• Usability Test Plan Creation – It is always important to write out your plan. In this case you need to document the 5 W’s (who, what, where, when and why) of your test which helps confirm you are testing the right features using the correct metrics.
• Target User Group – Rather than just select random people, you need to ensure your target user group is the pool from which you draw your testers as they are the ones whose usability you are looking to ensure is met. If your demographic is women ages 25 to 30 just out of college then you need testers near or in that demographic.
• Recruit Test Users – You want to recruit test users using the parameters from your target user group which can take time. Recruiters and online user groups are a great way to connect with potential testers.
• Locate a Testing Environment – For in-person tests if your office does not have the proper space you can find corporate office space that has rooms which can be rented for a day. Ideally you want at least two rooms, one for testing and one for waiting. Rooms need proper electric outlets and Wi-Fi. Remote testing environments are available as well and should be reserved ahead of time.
• Review Testing Plan – This should be done with the app Project Manager to ensure everything that needs to be tested will and for both of you to look for holes in the plan.
• Confirm Recording Methods – Consider how you will record, devices or software that will be used, whether people testing on mobile devices will block camera views, and which method of recording will work best for your needs.
• Walkthrough Prototype – Sit down with a designer and walkthrough the prototype so that you have all the information needed to answer questions and conduct the testing.
• Practice Run – A practice run should include setting up, testing and recording along with post test procedures. Always ensure there are extra testing devices available, notifications are turned off and extra power cables are available.
• Actual Testing – This should actually be the easiest step after all the prep work.

From there it is a matter of thanking (and paying) the testers before moving to post-testing review and evaluation by the appropriate parties in your group.

The bottom line is that mobile usability testing is very important to ensure your app is competitive from the first day of its release. While it might seem a bit overwhelming of a process, our blueprint for mobile usability testing provides an excellent starting point from which to build your own method.