Ask for the conformance report and for a second document explaining how it was produced. The report on its own tells you what a vendor concluded. The second one tells you what they did to reach that conclusion, and which parts of their product a disabled person cannot use at all.
Where this comes from
US federal buyers are told to require both, for every off-the-shelf item, and to consider reserving the right to test the product themselves before award. That is the agency that helped create the conformance template in the first place.
The Report, and the Document That Explains It
The first document is the ACR, one answer per criterion with remarks alongside. The second is called a Supplemental Accessibility Report in federal language, and you can call it whatever you like, because what matters is the five things it carries.
- How the report was produced. The evaluation methods, stated to show the conformance answers were reached with due diligence rather than typed in.
- The features that help. What the product actually does for disabled users, which the criterion rows tend to bury.
- The core functions that cannot be used. Named, plainly. This is the item buyers most often forget to ask for and the one that changes decisions.
- How to configure and install it accessibly. Defaults are frequently not the accessible setting.
- For anything that authors content, how the tool lets your people produce accessible output.
That third item is the one to hold on to. A conformance report has no way to say a whole function is unusable; it has a criterion row and a remarks box. Asking the question directly is how you find out before the contract, rather than when somebody on your team cannot do their job.
Reserve the Right to Test
Federal guidance suggests stating that the agency may test some or all of the proposed items before an award decision, to check the accuracy of the response. If you do reserve that right, two conditions come with it. The vendor supplies the real release rather than a trial build, because trial versions are missing features and produce results that are not about the product you would buy. And they supply a test plan describing typical user scenarios, so both sides are testing the same thing.
You do not have to be a government to do this. It is simply the only mechanism in any of these documents that connects the vendor's claim to something you can check yourself.
A Configured Product Is Not the Base Product
If you are buying something that will be themed, configured, extended or filled with your own content, the base product's report answers a narrower question than the one you have. Federal guidance treats customized items separately for exactly this reason, requiring a report that addresses how the item will meet the requirements in your solicitation, plus the methods that will be used to validate it.
The practical version: ask which configuration was evaluated. A platform can be reported honestly and still hand you a tenant that fails, because the theme, the plugins and the content are yours. That is not the vendor being dishonest. It is the report being about their product rather than about your instance.
Can an Automated Report Satisfy Your RFP?
That is decided by your solicitation, and only by your solicitation. Federal practice describes two routes: a pass or fail check that the required statement arrived at all, and a trade-off evaluation where your technical team sets the standard, decides which tools will be used and decides how consistency is held across responses. GSA's own page says sample evaluation criteria for accessibility are still being developed, which is an unusually honest thing for a guidance page to admit.
So the question to answer internally is what you will accept and why. If you will accept a scanner export, say so and know what you are getting. On our own classification of W3C's 432 techniques and documented failures we graded 356, and 10 of those can be settled by a machine outright. And 49 of the 86 success criteria in WCAG 2.2 have no automated rule written against them, so a clean scan is silent about more than half the standard rather than approving it.
Wording worth stealing
Federal sample language asks vendors to clarify whether conformance is achieved throughout the entire item, covering user, administrator and reporting functions, or only in limited areas. That is the closest thing to a scope question anywhere in procurement, and it exists in the instructions because the template has no field for it.
Reading What Comes Back
When the report lands, the useful questions are about the document's own rules rather than about the product. Which edition is it, because that decides which WCAG version it answers to. Does it name a product version. Does the methods section say anything, or does it name a proprietary process and stop. And has the word Not Evaluated appeared anywhere outside the AAA section, which the template does not allow.
Our ACR review checklist walks through those in order and gives you the questions to send back. It runs in your browser and never sees the report, which matters when the report arrived under a confidentiality term.
Having a Report Is Not Being Accessible
One vendor has a conformance report and the other does not. That tells you which vendor filled in a form. It does not tell you which product a screen reader user can check out of, because nobody reviews these documents, there is no certification behind them, and the product owner writes their own answers.
You will read the opposite claim in circulation, sometimes on official-looking pages. Treat the report as the start of a conversation and the supplementary questions as the conversation. The evidence you actually want is somebody having tested the journeys your people will use, and being willing to tell you what broke.