Why voluntasOS

What automated testing cannot do

Automated accessibility testing — ours and everyone else's — reaches roughly 20–40% of WCAG. The rest needs a person. A clean result from us means no automated check found a barrier. It is not an audit, and we will not describe it as one.

Of what we do find, we measure how often we are wrong — and, just as importantly, how often we miss. Against the W3C ACT Rules Community Group corpus, which we did not write, on 410 published test cases covering twenty-eight of their rules:

• Precision 0.96. When we report a barrier, it is almost always really there — 5 false positives in that run. All five are places where we knowingly disagree with ACT, each written up in our test harness with the argument on both sides; they still count against this number, because a disagreement we are confident about is a barrier the customer did not have.
• Recall 0.91. We miss roughly one in eleven of the barriers those rules describe — 12 false negatives in that run.

Both numbers went down on 1 September 2026, because we widened the corpus rather than because the engine got worse. We added five more of their rules to what we score against — 23 to 28 — and two of the five we detect nothing on. One is a keyboard shortcut registered with addEventListener, which our scan cannot see because it runs after the page's own scripts. The other is audio that starts on its own: the engine has that rule, we run it, and on those cases it answers “I cannot tell” because the media file is not there to be measured. We drop that answer instead of passing it on, and that part is ours to fix. We claim both criteria, so both misses belong inside the published number instead of outside it. Removing the two rules would put recall back near 0.94 and would tell you less.

Precision went up on 2 September 2026, and the engine did not change. Our catalogue tells you “an ARIA attribute on this element is not valid” and keeps which of four internal rules noticed to itself. That is right for a report and wrong for scoring: ACT asks each of those as a separate question, and our harness matched on the shared name — so one rule firing counted as an answer to all four. It manufactured a false positive we were not making, on a role attribute that is perfectly valid. The harness now scores each question against the rule that actually answers it, which took the false positives from 6 to 5. The five that remain are real disagreements we argue for, not measurement mistakes, and they still count against precision. Recall and the twelve misses are unchanged; nothing was removed to make the number look better.

Three of the twelve are the same thing: text sitting on a background image or a gradient. Contrast cannot be computed there without sampling the pixels, so we report it as unchecked rather than guess. If your hero banner puts pale text over a photograph, we will tell you we did not measure it.

Recall is the number that matters to you and it is the weaker one, so it is the one we lead with. A scan that comes back clean has not established that your site is clean; it has established that the checks we run did not fire.

We also test against a corpus of our own — 26 fixtures with deliberate traps — and score higher on it. We are not publishing that number as an accuracy claim, because measuring ourselves against our own judgement is not a measurement. It is useful for catching regressions and that is all we use it for.

What we can actually fix

16 barrier types. Six we can repair without you; ten need a sentence from someone who knows the page — a link's real destination, a field's real name. Everything else we report and leave alone.

Fixes set attributes and add CSS. Nothing reorders your DOM or invents markup, because a tool cannot know from the outside what your structure was meant to be. Anyone claiming otherwise is guessing on your site.