Metadata-Version: 2.4
Name: a11y-fixer
Version: 0.1.0
Summary: LLM accessibility remediation agent for JSX/TSX with structural safety checks
Author: a11y-fixer authors
License: MIT
Project-URL: Homepage, https://github.com/arysarin/a11y-fixer
Project-URL: Repository, https://github.com/arysarin/a11y-fixer
Project-URL: Changelog, https://github.com/arysarin/a11y-fixer/blob/main/CHANGELOG.md
Project-URL: Issues, https://github.com/arysarin/a11y-fixer/issues
Keywords: accessibility,a11y,wcag,aria,jsx,react,llm,codemod
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Software Development :: Build Tools
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: groq>=0.11.0
Requires-Dist: python-dotenv>=1.0.0
Requires-Dist: tree-sitter>=0.21
Requires-Dist: tree-sitter-typescript>=0.23
Requires-Dist: tomli>=2.0; python_version < "3.11"
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Provides-Extra: openai
Requires-Dist: openai>=1.0; extra == "openai"
Dynamic: license-file

# a11y-fixer

An accessibility remediation agent for React JSX/TSX. It scans a file or
directory, sends each component through an LLM (Groq, OpenAI, or any
OpenAI-compatible endpoint) with a WCAG 2.0/2.2 + ARIA APG system prompt,
**validates the result structurally**, and then either rewrites the source,
emits a patch, or reports for CI.

The validation step is what makes the output safe to apply unattended: model
output is parsed as JSX/TSX and compared against the original, and a fix is
rejected if it fails to parse or drops an event handler or hook call.

## Install

```bash
pip install a11y-fixer
```

That gives you the core agent — structural gate, `--write` / `--patch` /
`--check` / `--fail-on`, all providers, the cache, `--since`. The optional
eslint-jsx-a11y feedback loop (`--lint`) and the eval harness's axe pass
(`--axe`) run a bundled Node package that isn't in the wheel; for those, work
from a clone:

```bash
git clone https://github.com/arysarin/a11y-fixer && cd a11y-fixer
pip install -e ".[dev]"        # editable install + pytest
npm --prefix lint install      # only needed for --lint / --axe
```

Set an API key via environment or a local `.env` (the variable depends on the
provider — `GROQ_API_KEY` by default, `OPENAI_API_KEY` for `--provider openai`):

```bash
echo "GROQ_API_KEY=gsk-..." > .env
```

## Usage

```bash
# dry run: print diffs, change nothing
a11y-fixer src/

# apply validated fixes in place (writes .bak alongside each changed file)
a11y-fixer src/ --write

# CI: check only what this PR changed
a11y-fixer src/ --since origin/main --check

# write validated fixes to a patch instead of touching the tree
a11y-fixer src/ --patch a11y-fixes.patch

# CI gate: exit non-zero if any file still has an accessibility issue
a11y-fixer src/ --check --json-summary a11y-summary.json
```

### Output modes

| Mode | Flag | Writes source? | Exit on findings |
| --- | --- | --- | --- |
| Dry run | *(default)* | no | 0 |
| Write | `--write` | yes (validated only) | 0 |
| Patch | `--patch FILE` | no (diff to `FILE`) | 0 |
| Check | `--check` | no | 1 |

`--write` is ignored when combined with `--check` or `--patch`.

### Exit codes

- `0` — clean, or fixes applied/emitted successfully
- `1` — `--check` (or `--fail-on`) found a gated outcome, **or** a file errored
  (API failure, truncated response, unreadable file)
- `2` — bad invocation (missing API key, path not found, `--workers < 1`)
- `130` — interrupted with Ctrl-C (partial results saved)

### Options

| Flag | Default | Purpose |
| --- | --- | --- |
| `--ext` | `jsx,tsx` | comma-separated extensions to scan |
| `--exclude` | `node_modules,.git,dist,build,.next` | directory names to skip |
| `--since REF` | — | only process files changed vs a git ref (e.g. `origin/main`) |
| `--model` | `openai/gpt-oss-120b` | Groq model id |
| `--write` | off | apply fixes in place |
| `--no-backup` | off | skip `.bak` files when writing |
| `--strict` | off | also block a fix that drops *any* identifier, not just handlers/hooks |
| `--repair-attempts N` | `1` | re-prompt a fix that fails the structural gate, up to N times (`0` disables) |
| `--lint` | off | run eslint-jsx-a11y, target its violations, verify the fix (needs Node) |
| `--lint-rounds N` | `2` | max model passes to clear lint violations |
| `--lint-config FILE` | bundled a11y config | ESLint config to lint with |
| `--config FILE` | auto-discover `a11y.toml` | project config (see above) |
| `--no-config` | off | skip config auto-discovery |
| `--provider` | `groq` | `groq`, `openai`, or `openai-compatible` |
| `--model` | provider default | model id (`openai/gpt-oss-120b` for groq, `gpt-4o-mini` for openai) |
| `--base-url` | — | API base URL; required for `openai-compatible` |
| `--check` | off | CI mode: never write, exit 1 on findings |
| `--fail-on STATUSES` | — | narrow the exit-1 gate to specific outcomes, e.g. `invalid` or `would-fix,invalid,residual-violations` (overrides `--check`'s gate; a file that errored still exits 1) |
| `--patch FILE` | — | write validated fixes to `FILE` as a unified diff |
| `--json-summary FILE` | — | write a machine-readable run summary |
| `--cache FILE` | `.a11y-cache.json` | cache file for skipping unchanged files |
| `--no-cache` | off | ignore and don't update the cache |
| `--api-key` | `$GROQ_API_KEY` | Groq key override |
| `--workers` | `1` | files processed concurrently (with `> 1`, all workers share one rate-limit cooldown, so a provider 429 pauses the pool instead of each thread backing off blind) |
| `--timeout SECONDS` | `120` | per model request / lint subprocess timeout (`0` = none) |
| `--log FILE` | — | append a JSONL event log |
| `--quiet` / `--verbose` | — | suppress per-file output / show lint+repair progress |

## Observability

`--log FILE` appends one JSON object per line as the run proceeds — it survives
Ctrl-C and crashes, since each line is flushed immediately:

```
{"event":"run_start","mode":"CHECK","model":"...","files":42, ...}
{"event":"file_start","path":"src/NavBar.jsx"}
{"event":"lint","path":"src/NavBar.jsx","phase":"initial","violations":2}
{"event":"model_call","path":"src/NavBar.jsx","kind":"lint_round","round":1,"prompt_tokens":970,"completion_tokens":734}
{"event":"file_done","path":"src/NavBar.jsx","status":"would-fix","seconds":7.6,"violations_before":2,"violations_after":0,"repair_attempts":1, ...}
{"event":"run_end","duration_seconds":8.6,"interrupted":false,"scanned":42, ...}
```

Ctrl-C lets in-flight files finish, saves the cache and prints the summary, then
exits `130`. A second Ctrl-C bails immediately. Parallel runs sharing a
`.a11y-cache.json` merge on save (each writes only the entries it touched) behind
a best-effort lock file, so they don't clobber each other.

## Linter feedback loop (`--lint`)

By default the agent trusts the model to find and fix accessibility issues.
`--lint` closes that loop with a real linter:

1. Run `eslint-plugin-jsx-a11y` over the file.
2. If it's clean, **skip the model entirely** (no tokens spent).
3. Otherwise pass the exact violations into the prompt, get a fix, and re-lint it.
4. Repeat up to `--lint-rounds` (default 2) until clean or give up.
5. The structural gate (parse, handlers/hooks) still runs on the final result.

The run summary reports `a11y lint  N -> M violations`, the JSON summary carries
per-file `violations_before` / `violations_after` / `lint_rounds`, and under
`--check` any file left with violations fails CI.

**Setup** — needs Node.js and a one-time install of the bundled ESLint:

```bash
cd lint && npm install        # installs eslint + jsx-a11y + the TS parser
a11y-fixer src/ --lint --check
```

`--lint-config FILE` points at your project's own ESLint config instead of the
bundled a11y-only one. The bundled rule set is folded into the cache key, so
changing it re-checks every file.

## Project config

Drop an `a11y.toml` (or `.a11y.toml`) in the directory you run from — it's
auto-discovered (`--config FILE` to point elsewhere, `--no-config` to skip). See
[a11y.toml.example](a11y.toml.example).

```toml
primitives = ["Button", "Link", "Modal"]   # already-accessible components; leave them alone
rules = ["Our <Icon> is decorative unless a title prop is set."]
ignore = ["**/*.stories.tsx", "src/legacy/**"]   # skip these files
ext = "jsx,tsx"                              # optional; CLI --ext still wins
exclude = "node_modules,dist,coverage"      # optional; CLI --exclude still wins
```

`primitives` and `rules` are injected into the system prompt so the model stops
adding redundant ARIA to your wrapped components. `ignore` patterns are matched
(with `fnmatch`) against both the working-dir-relative path and the basename.
Changing the config changes the prompt, which invalidates the cache.

## Providers

The model call goes through a small `LLMClient` adapter, so any OpenAI-shaped
API works:

```bash
# Groq (default)
a11y-fixer src/ --check

# OpenAI
a11y-fixer src/ --provider openai --model gpt-4o-mini --check      # needs OPENAI_API_KEY
pip install "a11y-fixer[openai]"                                    # installs the openai SDK

# Any OpenAI-compatible server (Ollama, vLLM, LiteLLM, LM Studio, ...)
a11y-fixer src/ --provider openai-compatible \
  --base-url http://localhost:11434/v1 --model qwen2.5-coder:7b --check
```

`--base-url` also lets the `openai-compatible` provider run without an API key
(for local servers that don't need one). Changing provider, model, or base URL
invalidates the cache.

## Caching

Every run records the content hash of each file it verified as **clean**
(`unchanged`) or **already fixed** (`fixed`) into `--cache` (default
`.a11y-cache.json`, keyed relative to the working directory). A later run whose
file content still matches skips the model call entirely and reports it as
`unchanged (from cache)` with zero tokens.

Only known-good outcomes are cached, so the cache can never suppress a pending
fix, a validation failure, or a diff you need to see — `would-fix`, `invalid`,
and `error` files are re-processed every run. The whole cache is discarded when
the model (`--model`) or the system prompt changes, since a stale "clean"
verdict would no longer be trustworthy.

Add `.a11y-cache.json` to `.gitignore` (already done here). Use `--no-cache` in
environments where you want every file re-checked unconditionally.

## How validation works

For each file the agent builds a structural fingerprint (via `tree-sitter`) of
the original and the model output:

1. **Parse gate** — if the original parsed cleanly and the fix does not, the fix
   is rejected (`invalid`) and never written.
2. **Handler / hook preservation** — any `on*` JSX attribute or `use*` hook call
   present in the original but gone from the fix rejects it.
3. **`--strict`** — additionally rejects if any other identifier drops to zero
   references (noisier: legitimate refactors like `window.location.href = …` →
   `<a href>` trip this).

If the original file does not parse, all checks are skipped (no trustworthy
baseline) and the fix is written as-is under `--write`.

**Repair pass** — when the structural gate rejects a fix, it is sent back once
(configurable with `--repair-attempts N`, default 1, `0` disables) with the exact
reason (`event handlers removed: onClick`, `does not parse`, …) and re-validated.
A recovered fix proceeds normally; the summary reports `repair  K/N recovered`.
This costs one extra model call per rejected file.

A truncated or empty model response is always an error, never a partial write.

File content is treated strictly as code to remediate; the system prompt tells
the model to ignore any instructions embedded in it (prompt-injection guard),
and the structural gate above is the real backstop regardless. The eval corpus
carries `prompt-injection` fixtures (override text in a JS comment and in
rendered JSX) that check this holds against the live model.

Files are read as UTF-8; a byte-order mark and CRLF line endings are detected
and preserved on write-back. A non-UTF-8 file is reported as an `error` rather
than guessed at.

`--since REF` narrows the run to files that differ from a git ref — tracked
modifications plus new untracked files, deletions excluded. A run with nothing
changed exits `0` ("No changed files to process"); an unknown ref or a
non-git directory exits `2`.

## CI

See [.github/workflows/a11y.yml](.github/workflows/a11y.yml): runs `--check` on
pull requests touching `.jsx`/`.tsx`, and a manual `workflow_dispatch` produces
a fix patch as a build artifact. Requires a `GROQ_API_KEY` repository secret
(or `OPENAI_API_KEY` if you switch `--provider`).

To stop the build only on a fix that failed the safety gate — while letting
merely-unremediated files through — use `--fail-on`:

```bash
a11y-fixer src/ --since origin/main --lint --fail-on invalid
```

## Opening a PR with the fixes

`a11y_pr.py` (console script `a11y-fixer-pr`) runs the agent and turns the result
into a pull request. Run it from the repo root with a clean tracked tree:

```bash
a11y-fixer-pr src/ --lint --label accessibility
```

It runs `a11y-fixer src/ --lint --patch … --json-summary …` (any extra args pass
through), and if there are validated fixes: branches from HEAD (`--branch`,
default `a11y-fixes`), applies the patch, commits (message carries the lint
delta), force-pushes, and creates or refreshes a PR through the `gh` CLI. The PR
body is a per-file table (status, `violations_before → after`, repair passes)
with token/cost totals and a note that rejected fixes were left out. `--dry-run`
stops after the local commit; without `gh` it pushes and prints the compare
instructions. [.github/workflows/a11y-pr.yml](.github/workflows/a11y-pr.yml) wires
it to `workflow_dispatch`.

## Limitations

- Without `--lint`, semantic correctness of a fix is not verified — only that it
  parses and keeps handlers/hooks. `--lint` verifies against
  `eslint-plugin-jsx-a11y`; `axe-core` on rendered output would be stronger still.
- The bundled `lint/` directory ships in source checkouts and editable installs
  (`pip install -e .`); a plain wheel install does not carry it, so run `--lint`
  from a checkout.
- Each file is processed in isolation; the agent has no knowledge of shared
  design-system components (an `a11y.toml` mitigates this).
- Cost figures in the summary are estimates from `MODEL_PRICING` in
  `a11y_fixer.py`; update the rates for your model.

## Tests

```bash
pytest
```

`.github/workflows/ci.yml` runs the suite on Python 3.10 / 3.11 / 3.12 on every
push and PR, plus a smoke check that the bundled `eslint-plugin-jsx-a11y` config
still flags a known violation and that the package builds with its runtime data
files (`lint/`, `a11y.toml.example`).

## Releasing

`a11y_fixer.__version__` is the single source of truth; `pyproject.toml` reads it
dynamically, so a release is: bump that string, move the `CHANGELOG.md`
`[Unreleased]` items under a dated version heading, tag `vX.Y.Z`. `a11y-fixer
--version` reports the installed version.

## Evals

[evals/](evals/) holds golden fixtures and a scorer for measuring a prompt /
model / validation change rather than eyeballing it:

```bash
GROQ_API_KEY=... python evals/run.py --lint --json after.json
GROQ_API_KEY=... python evals/run.py --lint --axe   # also render + run axe-core
```

Reports `fix_rate`, `false_positive_rate`, `regression_rate`, and (with `--axe`)
`axe_clean_rate`; exits non-zero below `--threshold` (default 1.0). `--axe`
renders each opted-in fixture's fixed component and runs axe-core against it in
jsdom, checking the fix actually helps assistive tech rather than just passing
the linter. See [evals/README.md](evals/README.md).
