How We Test This Site
The six checks every change must pass, what we've caught, and what we test by hand.
What We Test Against, and When
This site is built to conform to WCAG 2.2 Level AA. Section 508 incorporates WCAG 2.0 Level AA, an earlier version of the same standard. We test the site two ways: an automated quality gate on every change, and manual passes before each release.
The quality gate runs in continuous integration (CI), the build service that checks every push and pull request that touches the site. If any check fails, the change can't deploy. A failed quality gate never ships.
- Standard
- WCAG 2.2 Level AA
- Quality gate
- Every push and pull request
- Automated
- axe-core 4.13 via Playwright 1.63, Lighthouse 13.5
- Manual
- Keyboard-only pass; VoiceOver with Safari
- Browsers
- Chromium (automated), Safari (manual)
The Quality Gate, Check by Check
Each check loads the built site in Chromium, the engine behind Chrome and Edge, and fails the build on any defect it finds.
| Check | What it does | What it finds |
|---|---|---|
| axe-core | Runs axe-core's WCAG 2.0 and 2.1 Level A and AA rules, its WCAG 2.2 Level AA rules and its best-practice rules on every page at desktop width, and again on each footer sheet, opened from its button, in both reading modes. | Defects a machine can read from the code: links, buttons and images with no name; text below the contrast minimum; invalid ARIA; skipped heading levels; touch targets that are too small. |
| Keyboard walk | Presses Tab through each page, up to 200 stops. Every stop must be visible, show a focus outline, and sit clear of the header and the pinned R&D bar. The walk must reach at least five stops, and it fails if focus cycles back to any stop but the first. | A keyboard trap (2.1.2 No Keyboard Trap), focus on hidden elements, a missing focus outline (2.4.7 Focus Visible), focus hidden under the header or the R&D bar (2.4.11 Focus Not Obscured), and a tab order that stops almost at once. |
| Reflow at 320 pixels | Narrows the window to 320 CSS pixels and checks that the page doesn't scroll sideways. A wide table may scroll inside its own box. At 320 by 256, the size of 400 percent zoom, the R&D bar must not stick to the bottom of the window. | Content that forces sideways scrolling on a phone or at 400 percent zoom, and a pinned bar that would cover a fifth of a zoomed window (1.4.10 Reflow). |
| No-JavaScript render | Loads each page with JavaScript turned off. The page must have exactly one h1 and at least 100 characters of text in its main region. | Content that only appears when scripts run, and a page with no main heading or more than one. |
| Lighthouse accessibility | Runs Google's Lighthouse on an emulated phone. The accessibility score must be 100. | The same kind of rule failures as axe-core, on a phone-sized layout, where the page is arranged differently. |
| Lighthouse performance | Scores the page on an emulated phone with a slowed network and processor. The score must be 95 or more. It gets up to three runs and keeps the best. | Pages that paint slowly or move after they first appear. Content that moves can make someone lose their place or tap the wrong thing. |
axe-core runs in both reading modes, Light and Dark; the keyboard walk, reflow and no-JavaScript checks run once per page. Lighthouse runs on every page except the 404 page.
What the Checks Caught
From our repository and CI history. Each item names the check that found it. Pull request numbers refer to our private repository.
A Performance Failure Blocked a Merge
2026-09-25 · PR #15 · Quality gate
The capability statement scored 92 to 94 for mobile performance, best of three runs, against a floor of 95. The quality gate failed six CI runs, and the change didn't ship. A first fix shortened the page's opening paragraph, which was painting late; it still failed. The real cause was layout shift: on the CI machine's wider fallback font, a long certification row wrapped to more lines, and every row below it moved when the web font loaded. A shorter row fixed it.
A Real Regression, Not a Slow Machine
2026-09-22 · Quality gate change
The home page scored 79 and 83 in CI on changes that didn't touch it, and 100 locally. Lighthouse performance now gets up to three runs and keeps the best. Accessibility stays a single run, since it gives the same answer every time. A real regression still fails; one slow machine doesn't. In the PR #15 runs, the home page's first run scored as low as 65 and passed on a retry, while the capability statement failed all three.
Testing the Test
2026-09-27 · PR #47 · Quality gate change
The keyboard walk used to stop when it reached a link whose text it had already seen, so two links with the same name could end the walk early. It now marks each element it visits and stops only when focus returns to one of them. With that fix, it reaches the footer and its controls. On 2026-10-02, while this page was being written, we found the walk still couldn't catch most keyboard traps: on a revisit it simply stopped. It now fails when focus comes back to anything but the first stop. We confirmed it by planting a trap in a test build; the quality gate failed on it.
A One-Off Test for a New Risk
2026-09-29 · PR #54 · One-off Playwright test
A help button was added, fixed to the bottom-right corner of every page. A fixed element can cover whatever has keyboard focus behind it. A one-off Playwright test, a script that drives a real browser, put focus on 31 controls under the button and confirmed that none was fully hidden, as 2.4.11 Focus Not Obscured requires. The ACR row was reworded to match what the test checked.
A Console Check Found a Blocked Font
2026-09-29 · PR #55 · Manual console check
While the help chat was being tested, the browser console showed the site's Content Security Policy blocking one font file. The policy lists what a page may load, and the build tool had embedded that font in the stylesheet, which the policy doesn't allow. It was already happening on the live site. The fix stopped embedding fonts and kept the policy strict. Pages then loaded under the real policy with no console errors.
What This Record Doesn't Show
Local runs · Not logged
No axe-core, keyboard walk or reflow failure has ever reached CI. Those checks also run locally, with npm test, before changes are pushed, and local failures aren't logged. So we can't say how many they caught, and we won't guess.
What Automation Can't Do, and What We Do by Hand
Automated checks read code. They can't tell whether alt text describes its image, whether a heading fits its section, whether the focus order makes sense, or how a screen reader actually announces a page.
Before each release, by hand:
- A keyboard-only pass of every page.
- A screen-reader pass with VoiceOver in Safari.
- An update to any ACR row the release touches.
Known Gaps
- Screen-reader review with NVDA in Firefox and JAWS in Chrome is not yet part of the release checklist. VoiceOver is.
- When the help chat was added on 2026-09-29 (PR #54), its keyboard and Safari VoiceOver passes had not been done before merge. The release checklist includes them.
We Update the ACR with the Code
Our Accessibility Conformance Report (ACR) lists every WCAG 2.2 Level A and AA success criterion with its conformance and remarks. 41 criteria are marked Supports. Each one is a promise about the built site, backed by the quality gate or a manual pass. When a markup change touches a criterion, its row changes in the same pull request.
For example, the help chat added the site's first form (PR #54). In that pull request, 3.3.1 Error Identification and 3.3.2 Labels or Instructions moved from Not applicable to Supports, and six other rows were rewritten for the chat and the help button, including the 2.4.11 row from the one-off test above.
Read the full conformance report on our accessibility page.
For Client Work
Our client work runs the same way: every deliverable is built to WCAG 2.2 AA, and Section 508 code remediation fixes issues mapped to success criteria (keyboard behavior, focus, semantics, contrast), in your repository, through your review process. For scope, read the capability statement.