Testing & Tools

Automated vs Manual Accessibility Testing

No automated accessibility scanner — including AllyProof's — can test 100% of WCAG on its own, and any tool claiming otherwise should be treated skeptically. The honest picture is a three-way split: some criteria are fully automatable with high confidence, some are partially automatable (a scanner can flag a likely issue but a human has to confirm it), and roughly half require manual testing entirely.

Fully automatable (~30% of criteria)

These have a mechanically checkable, objective pass/fail condition — a missing alt attribute, a contrast ratio below 4.5:1, a missing lang attribute. Tools like axe-core can flag these with zero false positives, meaning a reported violation here is essentially always a real issue. See axe-core Explained.

Examples: Missing Alt Text (1.1.1), Insufficient Color Contrast (1.4.3), Missing Page Language (3.1.1), Missing Form Labels (3.3.2).

Partially automatable (~18% of criteria)

A scanner can detect a pattern that's often (but not always) a real problem, needing human confirmation. An alt attribute being present is fully automatable to check; whether the alt text is actually a good description requires a person to read it and judge. Heading structure existing is checkable; whether headings are descriptive (per Headings and Labels) needs human judgment.

Manual only (~52% of criteria)

Roughly half of WCAG genuinely requires a human to evaluate — no current automated approach can reliably assess these at all:

• Whether audio description accurately conveys a video's visual content
• Whether a page's reading order is genuinely logical (versus just structurally consistent)
• Whether keyboard focus order matches a page's intended logical flow
• Whether error messages are genuinely helpful, not just technically present

Why disclosure matters more than the automation percentage itself

The actual percentage automatable is less important than being transparent about which criteria fall into which bucket for a given scan — a report that clearly separates "confirmed violations," "needs manual review," and "not automatically testable, requires manual audit" is more trustworthy and more useful than one that either overclaims full coverage or vaguely bundles everything into a single "score" with no methodology disclosed.

What this means practically

Treat an automated scan as a strong first pass that reliably catches the most commonly cited, highest-litigation-risk issues (see Web Accessibility Lawsuit Trends — the top violations are almost all fully-automatable ones) — but not as a substitute for a full manual audit if genuine, comprehensive WCAG conformance is the goal, particularly for anything customer-facing at real legal or reputational stakes.

Common questions

What percentage of WCAG can automated tools test?

About 30% of success criteria are fully automatable with high confidence, roughly 18% are partially automatable (a scanner flags a likely issue for a human to confirm), and about 52% require manual testing entirely.

Can an automated scanner make a site fully WCAG compliant?

No. No automated tool can test all of WCAG, and any tool claiming full compliance should be treated skeptically. Roughly half of WCAG needs human judgment.

What is automated accessibility testing good for?

As a reliable first pass that catches the most common, highest-litigation-risk issues — missing alt text, low contrast, missing labels — before a manual audit covers the rest.