Skip to main content
WCAGrules
Quick navigation

How we test

Three passes. Every problem screenshotted.

An audit is not a score. It is evidence. Here is exactly how yours gets made, in three passes. A machine sweep, then an expert review, then a real blind screen-reader user working through your site and capturing a screenshot at every point where it fails them.

1

The automated sweep

Every audited page goes through a full automated pass first, because it is the cheapest hour of the three and it clears the flat facts. An image with no alt attribute. A page that declares no language. The pass runs before a person opens the site, so nobody spends billable time on something a machine had already settled.

Two of the checks everyone assumes are settled are not. W3C's own contrast rule records that passing it does not mean the criterion is met, because nothing decides legibility when text sits over a photograph. And a field labelled by nothing but a placeholder is handed an accessible name by that placeholder, so it sails through the label check while the label itself disappears the moment somebody types. No rule in the set decides whether a persistent visible label and its instructions are there. That is the mechanical reason your scanner report and our auditor will disagree about the same page, and it is worth knowing before you assume one of us is wrong. We graded 356 of the W3C's 432 techniques and documented failures, and by that grading of ours this pass can fully verify 10 of them, which is exactly why the next two passes exist.

You do not have to take our word for that ceiling, because W3C publishes the same conclusion from the other side. There are 90 test rules written against WCAG, 87 of them live, and each states what its own result means. On 59 of them a clean pass is recorded as leaving the criterion still needing further testing. Exactly four can settle a criterion outright. And 49 of WCAG's 86 criteria have no published test rule of any kind, which is the whole automated ceiling stated by the body that writes the standard rather than by the firm selling you the next two passes.

2

The expert review

A trained reviewer then works every page against all 55 WCAG 2.2 Level A and AA rules by hand. That means reading the accessibility tree rather than the design, pulling the heading outline and the link list the way a screen reader user would, and grading whether each piece of alt text actually says what the image is doing there. A scanner can confirm alt text exists. Only a person can tell you it describes the wrong product.

Where something fails, the reviewer captures the screenshot, the exact element, the state the page was in, and the markup that proves it. The point of writing all four down is that somebody who disagrees with us can go and check, which is the only thing that makes a judgement call worth paying for.

3

The blind tester's journey

Then the pass no tool can imitate. A professional blind screen-reader user takes on your real tasks. Find the product, add it to the cart, reach checkout, submit the form. All keyboard, Tab by Tab and then with whatever keys each component actually expects, listening to what the screen reader announces, on NVDA, JAWS or VoiceOver depending on which fits your audience. Federal regulators have said the same thing in writing, because the FTC's complaint against an overlay vendor states that no automated tool alone can determine whether a site meets accessibility standards and that manual human testing is required.

The moment something fails them, focus vanishes, a button announces nothing, the cart stays silent, they capture a screenshot right there at the exact step where the journey broke. Each screenshot lands in the report pinned to its element and its criterion, so your developer sees precisely what the tester hit and where, and nobody has to reproduce it from a description.

What lands in your report

  • Coded findings (WR-001 and up) mapped to their WCAG 2.2 criterion and severity
  • A screenshot for every problem, captured at the moment it was found
  • The exact element, page state, and markup excerpt behind each finding
  • The date of the evaluation, and the browser and screen reader it was carried out with
  • Each finding linked to its fix guide, from the free library of 432

The format is public: read a sample report before you spend a cent.

Three things this method does not cover

Every audit firm has a boundary. Most of them let you find it after you have paid, so here is ours in advance.

  1. It covers the pages in your scope, not your site. W3C's evaluation methodology states that a conformance claim cannot be made for a whole website on the strength of a selected sample, and conformance in the standard is defined for a web page rather than for a site. So what you get is a true account of the pages we tested. Anyone who sells you a site-wide certificate from a ten-page sample is selling you something the standard does not define. Which pages should be in that scope is a question with a method behind it, and the scope worksheet walks through it in five passes.
  2. Testing is done at desktop widths. A full page includes every variation it presents for different screen sizes, and each of those has to conform before the page conforms at all, so an evaluation at one width cannot settle how your site behaves on a phone. The limit is the width we tested at rather than the hardware we tested on. That is not a small carve-out, because the people this work is for are already there. In WebAIM's tenth screen reader user survey, 91.3% of the people who answered said they use a screen reader on a phone. Findings we raise will usually be true at every width, and the ones that only appear on a narrow screen are outside what we looked at, so a site whose traffic is overwhelmingly mobile should hear that from us before it buys rather than after.
  3. A Rapid audit runs one screen reader and browser pairing. Accessibility support is a property of a combination rather than of a page, so the report names the pairing we used and you know exactly what the evidence covers. A Standard audit runs two, NVDA and VoiceOver, which is what the extra sessions in that tier are. Pairings are not evenly spread either. The same survey put JAWS with Chrome top at 24.7% of respondents and NVDA with Chrome next at 21.3%, so if your audience runs mostly on JAWS, tell us when we confirm scope.

Why screenshots, not scores

A score tells you how worried to be. A screenshot of a real blind user stuck at your checkout tells you what to fix, where, and why it matters. And it holds up when a developer, an agency, or a regulator asks "prove it." WCAG defines no rating scheme at all, so the severity on any finding is the auditor's judgment rather than a measurement. How to read an audit report walks a finished one line by line, including whose severity that is and what the header can never say.

Go somewhere useful

Find tools, resources and your workspace.

29 destinations