By role · QA and testers
What to Automate, and What to Do by Hand
By our own count, a machine fully settles 10 of the 356 WCAG techniques and failures we graded. Knowing which 10, and covering the rest properly, is the whole job.
You already test against a written specification for a living. That is most of what accessibility testing is, which is why it tends to sit well in QA.
Accessibility testing is organised testing against a written specification, which is the thing QA already does every day. Our own technique-by-technique count puts 10 of the 356 WCAG techniques and failures we graded fully inside a scanner's reach. Separately, 24 of the 55 Level A and AA criteria have no rule at all in W3C's ACT collection. That collection is a published catalogue of test rules, not a census of what every tool can do. Everything else needs a person to look and decide. Most teams get this the wrong way round. They automate hard, read the green result as an answer, and never build the judgement half at all.
That split is the job. Put the machine checks in CI so the regressions you configured for cannot ship, then spend your actual testing time on the rules a machine cannot reach. The clearest published measurement of that gap is still a 2017 government study. It built a page carrying 143 deliberate failures and pointed ten tools at it. Counting manual-inspection prompts as well as errors, the best single tool reached 41%, and all ten together reached 71%. The tools have moved on since, and the shape of the answer has not: a large minority of failures waits for a person, and finding them is what this role is for.
What you own
- Automated checks running in CI on every build, so a regression the rules cover cannot ship twice. Be honest about the gate you set, though. Of the 70 rules in axe-core 4.13 carrying a WCAG A or AA tag, 63 run once the experimental and deprecated ones drop out. Of those 63, 58 can report a serious or critical result. Failing the build on serious and critical is not a cautious start, it is nearly the whole rule set at once.
- Manual passes over every rule a scanner sees only partly or not at all, which by our own count is the large majority.
- Findings written with an exact location, steps that reproduce, and a description of what the fixed version looks like.
- Severity ranked by what happens to a person, not by which letter the rule carries. A Level AAA failure that blocks checkout gets worked before a Level A failure in the footer. Rank the work that way and keep the levels straight separately, because the footer one still blocks an AA claim.
- A regression pass after every fix lands, because a fix that quietly rearranges the tab order has traded one finding for a worse one.
What is not yours
Deciding which level the company commits to, and signing the claim that the site meets it. Those belong to whoever carries the legal obligation. You produce the evidence they decide on, and the decision stays theirs.
Where to start, in order
- The QA test pass20 checks in a fixed order, and how to write each finding so a developer can act on it without coming back to ask you anything.
- Automated vs. manual testingRule by rule, whether a machine can settle it or a person has to judge it.
- How to test a website yourselfFour manual passes, ordered so the ones that find the most come first.
- Screen reader testingSix keystrokes that tell you whether the page announces itself correctly to assistive technology.
- The WCAG 2.2 checklistAll 55 Level A and AA criteria in plain language, which is the raw material for your test cases.
- Templates, journeys, states and layoutsThe five-pass scope worksheet an audit is drawn from. Pass three is the states inventory, which is the same list a release test pass should already be working from.
- The audit report templateThe eight fields every finding needs, and how to rank one by what it does to a person.
- Reading somebody else's audit reportA finished report annotated line by line, including the difference between a finding that is one example and a count of every occurrence.
- Retest or new auditWhich changes in a release make old evidence stop describing the product, so you can raise the question before somebody else does.
- What a free scan cannot seeWhere a scanner's coverage actually stops, ours included, so you know what your CI gate is not testing.
- Free tools comparedWhat WAVE, axe and Lighthouse each find, and what all three of them miss together.
The mistakes we see most from this role
Not a criticism. These are the failures that recur across audits, and knowing them is most of avoiding them.
- Reading a tool's score as a pass, when WCAG defines no score at all. It has three levels, and each one is met in full or it is not met.
- Testing only the state the page loads in, so closed menus, unopened dialogs and collapsed content never get looked at once.
- Ranking findings by WCAG level instead of by impact, so a footer issue nobody hits gets fixed before a checkout nobody can finish.
- Filing a finding with no selector, no screenshot and no description of the fix, which turns your work into somebody else's investigation.
- Testing in one screen reader and assuming the rest behave the same way. In WebAIM's survey of screen reader users, no single one is the primary desktop choice of much more than 40% of respondents, and which one leads shifts by region.
- Reading an automated pass as conformance. Of the 87 live ACT rules, 59 record an outcome where a pass leaves the criterion still needing further testing, and exactly four clear one outright.
Tools you will use
Other roles
Find out what your site actually needs.
The free scan checks 10 pages in a real browser against all 90 supported automated rules, keeps its 27 best-practice checks separate from WCAG findings, and names the rule behind every finding.