Study // Knowledge card 5.1
Spend an expert's hour before a user's
How to use Jakob Nielsen and Rolf Molich's heuristic evaluation to find a screen's obvious problems before you spend a user's time.
- Date
- rev.
- Aim
- When is an expert review worth running before a usability test?
- Topic
- Evaluation & Testingmethod · core
- Built with
- Claude + NotebookLMUpdated · 6 min read
Notes
5.1aTen questions, asked of every screensrc 02
A heuristic evaluation · đánh giá heuristic checks an interface against Nielsen's ten usability heuristics · nguyên tắc khả dụng. Each heuristic is a question a reviewer asks of every screen:
| # | Heuristic | The question |
|---|---|---|
| 1 | Visibility of system status | Does it show what just happened? |
| 2 | Match with the real world | Are these the person's words, or ours? |
| 3 | User control and freedom | Is there an exit, an undo? |
| 4 | Consistency and standards | Does the same thing look and act the same? |
| 5 | Error prevention | Could this mistake be made impossible? |
| 6 | Recognition over recall | Is what they need visible, or must they remember? |
| 7 | Flexibility and efficiency | Is there a faster path for the experienced? |
| 8 | Aesthetic and minimalist design | Does every element earn its place? |
| 9 | Recognize, diagnose, recover | Does the error say what to do next, in plain words? |
| 10 | Help and documentation | When they are stuck, is help one step away? |
The rules turn a review from taste into Confused: If the ten are rules of thumb, who decides whether a finding is a violation or a preference?. A thermostat that gives no sign it heard you fails heuristic 1, and a reviewer can say so with no user in the room.
In my words
Each heuristic names a kind of failure people keep running into, so a reviewer can spot one before anyone runs into it.
5.1bWhy five reviewers, each alonesrc 04
No single reviewer finds most of the problems. Over six of Nielsen's projects, Surprising: I expected one careful reviewer to catch most of them. One finds 35 percent, so a solo review only samples the problems.. He recommends about five, and at least three. Three reasons:
- Each person misses different things. Reviewers bring different habits and blind spots, so their lists overlap only in part.
- Independence keeps the misses different. If reviewers talk first, the first opinion anchors the rest, and five sets of eyes collapse into one.
- Returns fall after five. Each new reviewer mostly finds what someone already found, so three to five is where the effort pays.
Critique
Goal: find usability problems before users do. Element: the designer reviewing their own screens. Works? Only with others, because the author knows what every element means, so the gaps are invisible to them. Pair with two or more reviewers working alone.
5.1cFour steps from screen to fix listsrc 04
- Pick three to five evaluators. Fewer leaves most problems unfound; more mostly repeats what others found.
- Each reviews alone, screen by screen. Write down every problem and the heuristic it breaks. Actionable: Before my next design review, give each reviewer ten minutes alone with the ten questions, before anyone speaks.
- Rate severity 0–4, then merge. Nielsen's scale runs from 1, cosmetic, to 4, a usability catastrophe. Merge duplicates. The rating turns findings into an order of work.
- Fix the 3s and 4s before anyone recruits a participant. The test sessions then go to what experts cannot see.
Why
Why clear only the severe ones before a test? Because those are the problems that stop a participant, and a stopped participant shows you nothing past that point.
5.1dAt work: four situations
- "Can we book usability tests for next week?" Testing an unreviewed screen spends users' hours on what a reviewer would catch. I put an expert pass ahead of the booking.
- A design review keeps turning into taste. "I don't like it" has no answer. Ask which heuristic it breaks and how severe it is. A named rule gives the room something to argue over.
- There is no budget to recruit anyone this month. You can still run an evaluation. Actionable: Put 'prediction' in the status column of my next findings sheet until a session confirms each one. and test the 3s and 4s when you can.
- "We all reviewed it together in the meeting." That was one review with several signatures. Ask for solo passes first, then merge.
Link
What people say is a hypothesis — a heuristic finding is a prediction about behavior, so it ranks below watching people use it.
Examples
One case of the same shape, kept for reference.
5.1eGmail, 2008src 03
Gmail Labs added a forgotten-attachment check: when the message says "attached" and nothing is, it asks before sending. One check at that moment removes a common embarrassment.
Link
5.1a — heuristic 5, error prevention: make the mistake hard to make.
Meaning
Sketch (from my case study · Dash Cam LINK)
- One reviewer: the sole designer
- 100+ existing screens auditedheuristics not recorded
- Severity ratingsnot recorded
- Merge independent evaluatorsnot recorded
- Fix by severitynot recorded
Dash Cam LINK was a legacy app, built up over years of incremental updates. I audited more than 100 of its existing screens, as the product's sole UI/UX designer. I would call that an expert review. Nothing records named heuristics, a second evaluator working alone or severity ratings, and those are the parts this note says make a heuristic evaluation work.
Paraphrase (my own words)
- An expert pass clears the obvious problems, so test sessions reach the subtle ones.
- Citing the broken heuristic turns a finding from opinion into a claim others can check.
- A lone reviewer averages 35 percent, so pool several who work apart.
- Work solo, score severity, merge, and clear the worst before a test.
Critique (disagree, edge cases, trade-offs)
- Treating it as a substitute for users. It finds violations of rules. Demand, and whether a predicted problem shows up in use, are beyond it. Run a usability test after it to find out.
- A list without severity. Unranked, a long list gives the team no place to start. Tell-tale sign: findings sorted by screen, with no number beside them.
Remember
The note in a few cards: the idea, the words, the steps.
Concept
Expert review first
Review against the heuristics before testing, since rule-breaking problems need no participant to find.
Concept
Five reviewers, each alone
One reviewer finds 35 percent on average; three to five working alone find more, because their misses differ.
Term
Heuristic evaluation · đánh giá heuristic
An expert review that checks every screen against named usability rules, such as Nielsen's ten.
Not A usability test with users.
Term
Usability heuristics · nguyên tắc khả dụng
Nielsen's ten rules of thumb — system status, error prevention, user control and seven more — each a screen-level question.
Term
Severity rating
A 0–4 score per finding, from cosmetic to catastrophe, that decides what gets fixed first.
Workflow
Run a heuristic evaluation
- Pick 3–5 evaluators
- Review alone, name heuristics
- Rate severity, then merge
- Fix 3s and 4s
Retrieval
Cover the notes. Answer each question from memory, then turn the card to check.
Q1
Why do five reviewers working alone find more than five reviewing together, even when each is equally skilled?
Tap to answer
Q1Answer
Why do five reviewers working alone find more than five reviewing together, even when each is equally skilled?
Their blind spots differ, and working apart keeps them different, so the lists add up. Talking first lets one view set the frame for everyone.
Tap to turn back
Q2
Put the four steps of a heuristic evaluation in order, and name the one the method depends on.
Tap to answer
Q2Answer
Put the four steps of a heuristic evaluation in order, and name the one the method depends on.
Choose the reviewers, review solo with the heuristic named, score and merge, then fix the severe ones. Solo review carries the method.
Tap to turn back
Q3
Your team has two days before user sessions on a new checkout flow nobody outside the team has seen. What do you do before the first participant arrives?
Tap to answer
Q3Answer
Your team has two days before user sessions on a new checkout flow nobody outside the team has seen. What do you do before the first participant arrives?
Have three colleagues walk it alone against the ten questions, then clear the severe findings, so participants spend their time on what reviewers cannot predict.
Tap to turn back
Q4
Your review produced forty findings and none of them got fixed. How do you fix it, and why does the fix work?
Tap to answer
Q4Answer
Your review produced forty findings and none of them got fixed. How do you fix it, and why does the fix work?
Score each finding 0–4 and start at the top. A ranking tells the team where its next hour goes.
Tap to turn back
Q5
When is a heuristic evaluation the wrong tool, even though it is cheaper than a test?
Tap to answer
Q5Answer
When is a heuristic evaluation the wrong tool, even though it is cheaper than a test?
When the open question is demand or real behavior. Rules can only predict a problem; watching people is what confirms it.
Tap to turn back
Q1 / 5
Open question
The ten heuristics assume a designer chose every status message and every error. When the system's reply is generated by a model instead, which of the ten still hold?
Sources
- 01Heuristic Evaluation of User Interfaces (opens in a new tab) — Jakob Nielsen, Rolf Molich — CHI 1990, 1990
- 0210 Usability Heuristics for User Interface Design (opens in a new tab) — Jakob Nielsen, Nielsen Norman Group, 1994, updated 2024
- 03New in Labs: Forgotten attachment detector (opens in a new tab) — Official Gmail Blog, 2008
- 04The Theory Behind Heuristic Evaluations (opens in a new tab) — Jakob Nielsen, Nielsen Norman Group
- 05Severity Ratings for Usability Problems (opens in a new tab) — Jakob Nielsen, Nielsen Norman Group
Related
Next note