How to write tasks for an unmoderated usability test

In an unmoderated test nobody is there to explain what you meant. People read your task, do what they think it asks, and move on. If a task is unclear, you get a confused, silent recording and a guess at an answer.

So the quality of an unmoderated test is mostly decided before anyone takes it, by how you write the scenario, the tasks and the questions.

A scenario gives people a reason to use your product, so they act as they would in real life instead of clicking around. Keep it to one or two sentences, specific and believable for the people taking part.

• Weak: “Imagine you're a user of our app.”
• Better: “You've just been paid and want to send money to your supplier.”

Give people every detail they'll need, like a pretend amount or a name to search for. Never ask for a real PIN, password or real money.

On myUsersInsight, the Scenario field shows before the first task.

Tell people what to achieve, not where to tap. If you list the steps, you only learn whether people can follow instructions.

• Weak: “Tap Send, then choose Fees.”
• Better: “Find out how much it costs to send GHS 200 to a friend.”

• Don't use the words on your buttons or menus. If the button says Transfer, don't write “transfer”, or people just hunt for the matching word.
• One goal per task. Two goals in one task make it hard to tell which part went wrong.
• Give every task a clear end, so people know when they're done: “…and stop when you see the fee.”

Three to six tasks is plenty for one test. Put them in the order people would really do them, so each task starts where the last one ended. Long tests get tired, rushed answers at the end.

On myUsersInsight a test can have up to 20 steps. Each task pays the participant GHS 10 and each question GHS 5, and How long it takes tells people how much time to set aside. Be generous with it: people who rush give thinner answers.

The recording only shows what people did. Their voice tells you why. Most people go quiet when they concentrate, so ask for it more than once.

In your scenario, add a line such as “Please say what you're looking for and what you expect as you go. It helps us more than getting it right.” On myUsersInsight each task also reminds people to think aloud, and their screen and voice are recorded once they agree.

A question straight after a task catches what people felt while it's fresh. Use a choice question for something you'll compare across people, and a written one to learn why.

• Single choice for quick measures: “How easy was that?” from Very easy to Very hard.
• Written response for reasons: “What, if anything, was confusing?”
• Ask about what happened, not opinions. “Did you like it?” mostly gets “yes”.
• Keep written questions few. Typing long answers is slow, especially on a phone.

On myUsersInsight people can also press I couldn't do it on a task. That's a finding, not a failure: watch the recording to see where they got stuck.

Take your own test from start to finish, then ask a colleague who hasn't seen the product to take it too. You'll find the task that can't be done, the link that asks for a login and the question you forgot.

• Check every link opens without a password or special access. On myUsersInsight links must start with https://, and each task can open its own link.
• Try it on a cheaper phone and a slower connection, if that's what your customers use.
• Time it, and set How long it takes from what you saw.

1. Write a one- or two-sentence scenario with every detail people need.
2. Write each task as a goal with a clear end.
3. Avoid the words on your buttons and menus.
4. Keep it to three to six tasks, in a natural order.
5. Ask people to think aloud, and say why it helps.
6. Follow tasks with a choice question and one written why.
7. Take the test yourself, then with a colleague.
8. Check every link opens without a login.