7.8 KiB
Module 17: Usability Testing
Lesson 2: Conducting the Test
The session, the silence, and the replay
Consistency Is Everything
Every participant must get the same test. Same introduction. Same tasks. Same conditions.
If you explain the interface to participant 2 but not participant 1, you can't compare their results. If you give participant 3 a hint because they're struggling, their "success" is meaningless.
The test scenario from Lesson 1 is your script. Follow it.
Arrive and Set Up
At Their Location (Preferred)
You go to them. Their home, their office, their desk. They use their own device, their own browser, their own chair.
Set up your recording before they sit down:
- Position your phone camera where it captures the screen and their hands
- Start screen recording on their device (with permission)
- Make sure everything is working before they start
Remote (When Necessary)
Start the video call. Ask them to share their screen. Begin recording.
Remote is less ideal — you miss body language, environmental context, and the natural pace of their interaction. But it's infinitely better than not testing.
Read the Introduction
Read your introduction script exactly as written. Same words every time.
"Thanks for helping me with this. A few things before we start:
I'm testing the design, not you. There are no wrong answers
and you can't do anything wrong.
I'll give you a few tasks to try. Please think out loud as you
go — tell me what you're looking at, what you're thinking,
what you expect to happen.
I won't be able to help you during the tasks. If you get stuck,
that's valuable information for me. Just do what you would do
if I weren't here.
Any questions before we start?"
Answer any questions about the process. Don't answer questions about the product.
Present the Task
Give them one task at a time. Read it aloud. Hand them the card or send the message.
"You've heard about [product name] and want to try it. Go ahead and create an account."
Then stop talking.
The Silence
This is the hardest part of usability testing. You must not help.
Don't Help
The user clicks the wrong thing. You know the right answer. You want to say "try the button on the left." Don't.
Their wrong click is your data. It tells you the design failed to guide them. If you help, you lose that information.
Don't Discuss
They say "is this where I'm supposed to go?" You want to confirm or redirect. Don't.
Say: "What do you think?" or "What would you do if I weren't here?"
Don't Explain
They're confused by a label. You want to explain what it means. Don't.
If the label needs explaining, the label is wrong. That's a finding.
Deflect Questions
They will ask you questions. It's natural. They want to do well. Deflect every one:
| They say | You say |
|---|---|
| "Is this right?" | "There's no right or wrong. What do you think?" |
| "Should I click here?" | "Go ahead and do whatever feels natural." |
| "What does this mean?" | "What do you expect it to mean?" |
| "Am I doing this correctly?" | "You're doing great. Just keep going." |
| "I'm stuck, can you help?" | "What would you try if I weren't here?" |
When It's Painful
You will watch someone struggle for 30 seconds on something that takes 2 seconds if you explain it. It will feel wrong. It will feel rude. Do it anyway.
That 30-second struggle is the most valuable data you'll collect. It reveals exactly where the design fails.
The only exception: If the user is genuinely distressed — frustrated to the point of being upset — move to the next task. Say: "Let's skip this one and try the next task." Note it as a critical finding: task abandoned due to user frustration.
What to Observe
While they work, take notes on your observation sheet. Use timestamps.
Behavioral Signals
- Hesitation — They pause before clicking. What are they deciding between?
- Scanning — Their eyes search the page. What are they looking for that isn't obvious?
- Backtracking — They go back. Their expectation didn't match reality.
- Misclicks — They click the wrong element. What did they think it would do?
- Speed — Fast and confident, or slow and uncertain?
Verbal Signals
Listen for think-aloud comments:
- "Oh, I expected..." — Their mental model doesn't match the design
- "Where is the..." — Navigation or visual hierarchy problem
- "What does this mean?" — Label or messaging failure
- "I think I need to..." — They're guessing, not knowing
- "Oh!" — Surprise, positive or negative
Completion
For each task, note:
- Did they complete it? Yes / No
- How long did it take?
- How many wrong paths did they try?
- Did they seem confident or uncertain at the end?
Between Tasks
Brief pause. "How was that?" Listen to any immediate reaction. Keep it to 30 seconds.
Don't explain what they "should have done." Don't reveal the intended path. Don't discuss the design.
Present the next task card.
After All Tasks
Quick Debrief (2 minutes)
Ask open questions:
- "What stood out to you?"
- "Was anything confusing or unexpected?"
- "If you could change one thing, what would it be?"
These questions sometimes surface things you didn't observe. But keep it brief — the real value was in watching their behavior.
The Retrospective Replay
This is the most powerful part of the session.
How It Works
After the tasks and debrief, play the recording back to the participant. Watch it together.
At key moments — hesitations, wrong clicks, confusion — pause and ask:
- "What were you thinking here?"
- "I noticed you paused — what was going on?"
- "You went back here — what did you expect to find?"
- "You clicked this — what did you think it would do?"
Why It Works
During the tasks, the user is focused on performing. They can't fully explain their thinking in the moment, even with think-aloud.
In the replay, the pressure is off. They see their own behavior from the outside and can articulate what was actually happening in their head. This produces insights you can't get any other way.
"Oh yeah, here I was looking for a 'Sign Up' link in the menu because that's where I usually find it on other sites. I didn't realize the big button was for registration."
That explanation is worth more than 10 minutes of guessing at what went wrong.
When to Use the Replay
- Always for formal sessions on critical flows
- Optional for guerrilla tests or quick checks
- Skip if the participant is short on time — ask if they have 10 more minutes
Close the Session
Thank them sincerely. They gave you their time and their honesty.
A coffee, a gift card, or a genuine thank you — acknowledge their contribution. They helped make the product better.
Guerrilla Testing
Not every test needs the full process.
The 5-minute test: Grab someone nearby. Show them the prototype. "Can you sign up for an account?" Watch what happens. No recording, no replay, just observation.
Guerrilla testing is imperfect. The participant probably doesn't match your persona. There's no recording to review. But it catches obvious problems you've become blind to.
Use it: During development, between build steps, for quick sanity checks.
Don't use it: For critical flows, when persona match matters, when you need evidence for stakeholders.
What's Next
You have recordings and observation notes. In the next lesson, you'll turn them into findings that connect back to your specifications and drive real design improvements.
Continue to Lesson 3: Acting on Findings →
← Back to Lesson 1 | Back to Module Overview
Part of Module 17: Usability Testing