WCAG Coverage

WCAG 2.2 Level AA includes 56 success criteria. AllyProof's multi-engine scanner can automatically test approximately 57-70% of them. This page explains what is covered, what is not, and why the gap exists.

The 57% Baseline

The widely cited "57%" figure comes from research on axe-core's rule coverage against WCAG 2.x success criteria. With axe-core alone, approximately 57% of WCAG 2.2 AA criteria have at least one automated rule that can detect failures.

By adding HTML_CodeSniffer as a second engine, AllyProof extends coverage to approximately 67-70%. The additional rules catch issues that axe-core's strict zero-false-positive policy causes it to skip.

Coverage by WCAG Principle

Fully Automatable Criteria

These criteria can be tested with high confidence using automated rules. A passing result from the scanner is a strong indicator of conformance:

• 1.1.1 Non-text Content — Detects images without alt attributes, empty alt on non-decorative images
• 1.3.1 Info and Relationships — Checks for proper heading structure, form labels, table markup
• 1.4.3 Contrast (Minimum) — Computes foreground/background color contrast ratios
• 1.4.11 Non-text Contrast — Checks UI component and graphical object contrast
• 2.1.1 Keyboard — Detects interactive elements not reachable via keyboard
• 2.4.1 Bypass Blocks — Checks for skip navigation links or landmarks
• 2.4.2 Page Titled — Verifies pages have descriptive titles
• 2.4.4 Link Purpose (In Context) — Detects empty links and links with non-descriptive text
• 3.1.1 Language of Page — Checks for lang attribute on HTML element
• 3.1.2 Language of Parts — Detects content in different languages missing lang attributes
• 4.1.1 Parsing — Detects duplicate IDs and other markup errors
• 4.1.2 Name, Role, Value — Checks ARIA roles, states, and properties

And approximately 8 more with strong automated coverage.

Partially Automatable Criteria

These criteria have automated rules that catch some failures but not all. A clean scan result does not guarantee conformance — manual review is still recommended:

• 1.3.2 Meaningful Sequence — Can detect some CSS reordering issues, but cannot assess whether reading order is meaningful
• 1.3.5 Identify Input Purpose — Can check for autocomplete attributes on forms, but cannot verify they are correct
• 1.4.4 Resize Text — Can detect fixed font sizes, but cannot verify layout at 200% zoom
• 2.4.6 Headings and Labels — Can detect empty headings and labels, but cannot assess whether they are descriptive
• 2.4.7 Focus Visible — Can detect outline: none without replacement, but cannot assess focus indicator visibility in all states
• 3.3.2 Labels or Instructions — Can detect missing labels, but cannot assess whether instructions are adequate

And approximately 9 more with partial automated coverage.

Manual-Only Criteria

These criteria cannot be meaningfully tested by automated tools. They require human judgment, assistive technology testing, or evaluation of content meaning:

• 1.2.1-1.2.5 Time-based Media — Require human evaluation of captions, audio descriptions, and transcripts
• 1.3.3 Sensory Characteristics — Instructions that rely solely on shape, color, size, or location
• 2.1.2 No Keyboard Trap — Requires interactive testing of all focusable elements
• 2.2.1 Timing Adjustable — Requires testing session timeouts and timed interactions
• 2.4.5 Multiple Ways — Requires assessment of site navigation alternatives
• 3.2.3 Consistent Navigation — Requires cross-page comparison of navigation patterns
• 3.3.3 Error Suggestion — Requires testing form validation messages for helpfulness
• 3.3.4 Error Prevention (Legal, Financial, Data) — Requires testing of confirmation and review steps

And approximately 13 more that require purely manual assessment.

What "Not Evaluated" Means in VPATs

When AllyProof generates a draft VPAT, criteria that fall into the "manual only" category are marked as Not Evaluated. This conformance level means:

• The criterion was not tested because no automated rule covers it
• This is not the same as "Does Not Support" — it means the result is unknown
• A human accessibility tester should evaluate these criteria and update the VPAT to reflect actual conformance

The VPAT remarks column for Not Evaluated criteria includes a note explaining which type of manual testing is recommended (e.g. "Screen reader testing required" or "Keyboard-only navigation testing required").

Why Not 100%?

Many WCAG criteria require understanding intent and meaning, which automated tools fundamentally cannot assess:

• Is this alt text accurate? (Tools can verify it exists, not that it is correct.)
• Are these instructions clear? (Requires human comprehension.)
• Does the reading order make sense? (Requires understanding content structure.)
• Are captions synchronized and accurate? (Requires watching the video.)
• Can a user complete a task using only a keyboard? (Requires interactive testing.)

This is an inherent limitation of all automated accessibility testing tools, not specific to AllyProof. The 57-70% figure is consistent across the industry. AllyProof maximizes automated coverage and clearly identifies where manual testing is needed.