Glossary · Accessibility term
User testing with disabled users
User testing with disabled users means watching real people with disabilities try real tasks on your site. Not a developer imitating a screen reader, but somebody who uses one every day, working through your checkout. Watching them finds usability problems that a conformance check on its own does not, and that is the whole case for doing it. The reverse holds just as firmly, and it is the half most people selling this leave out. A set of sessions cannot tell you whether a site is accessible, because no realistic number of them covers the range of disabilities, strategies and tools a standard exists to cover. That is what the standard is for. So it is the third pass of our audits, not a substitute for the first two.
In practice
It answers a different question from an audit. An audit asks whether the site meets the standard. A session asks whether somebody can actually get something done. What it turns up is usually a problem in the flow rather than in any single element. Plenty of those do have a criterion behind them, since complete processes and a good deal of the interaction guidance reach exactly that ground. What a session gives you is the thing a row on a checklist cannot, which is somebody stuck in front of you.
Read one session carefully, because it does not generalize. Input from one person with a disability tells you about that person, not about all people with disabilities, and W3C says so directly. A daily screen reader user also carries years of workarounds for badly built sites, so they get through things you would call broken, which can hide a barrier that would stop somebody newer cold.
Five users is the number every client already knows, and it comes from general usability research that says nothing about disability at all. Its own author's caveat cuts against applying it flatly here, because the rule weakens when a product has several distinct user groups. Screen reader users, magnifier users, switch users and people with cognitive disabilities are four distinct groups, which points at a larger sample rather than a smaller one.
Run the session on the participant's own machine, with their own assistive technology and their own settings. That is where their speech rate, their verbosity settings and their workarounds live, and a borrowed laptop tests somebody else's configuration rather than theirs.
One number to refuse. A figure claiming user testing finds 45% more problems than an audit circulates widely and is usually attributed to an organization whose own pages do not contain it. We went looking for the study behind it and found no sample size, no method and no publication. If a vendor quotes it at you, ask them for the source before you use it in a deck of your own.
Why it matters
A site can pass every criterion, including at the highest level, and still not be accessible to everybody, and the standard says that about itself, naming cognitive, language and learning areas in particular. Its own supporting material goes further, describing some advisory techniques as possibly the only way certain users can reach certain content. That is the standards body conceding that conformance and usability are two different achievements, and a session is how you find out which one you have. One practical bonus. Sessions turn up general usability problems that affect every visitor, so this is not a disability-only line in a budget.
Why it persuades
Nothing in a slide deck matches watching somebody stop at a button announced as "button" on your own checkout, try three more, and give up. Ten minutes of that changes a roadmap in a way a spreadsheet of findings never does.
Where this shows up on the site
Related terms
- Manual testingManual testing is a person working through a page against the rules a machine cannot decide.
- Automated testingAutomated testing is software that checks a page's code for the accessibility failures a machine can prove.
- Accessibility auditAn accessibility audit is a structured evaluation of a website against WCAG, ending in a documented pass or fail for every applicable rule with the evidence attached.
- Screen readerA screen reader is software that reads the screen out loud, or sends it to a braille display, for people who cannot see it.
Knowing the word is the easy part.
Find out where your own site stands. The free scan checks 10 pages in a real browser against all 90 supported automated rules, separates 27 best-practice checks from its WCAG findings, and names the rule behind every result.