leanest diffs your PR, judges each test's relevance in parallel, and hands the real subset to your existing runner — Playwright or Vitest, unmodified. One step replaces the one you have, and it fails open, so a broken judge never hides a test.
| @@ -10,8 +10,11 @@ | ||
| 10 | 10 | e2e: |
| 11 | 11 | runs-on: ubuntu-latest |
| 12 | 12 | steps: |
| 13 | 13 | - uses: actions/checkout@v7 |
| 14 | 14 | with: |
| 15 | 15 | fetch-depth: 0 |
| 16 | - - name: Browser smoke test | |
| 17 | - run: bunx playwright test --shard=${{ matrix.shard }}/3 | |
| 16 | + - name: Browser smoke test (leanest-selected specs) | |
| 17 | + uses: baronunread/leanest@v1 | |
| 18 | + with: | |
| 19 | + framework: playwright | |
| 20 | + # defaults to classifier-dev: free, no key needed |
Measured on a real PR against rdyrct 's Playwright suite, using the jev provider. A pure CI or config diff gives the judge no application code to reason about, so confidence drops and leanest runs closer to everything, on purpose. The number above is the representative case: a real feature change. Token cost and latency vary by provider; jev's numbers are shown here because that's what this run used.
git diff against your base branch. No API call yet, this step is free.
Finds your e2e specs by reading the framework's own config, not a guessed glob.
One semantic question per test, evaluated in parallel by your judge provider: could this diff affect it.
Low confidence always means run. A test whose own file changed always runs.
Hands the selected paths to your real playwright or vitest, unmodified.
leanest's selection judgment is pluggable. It never assumes one vendor, one hosted API, or
one model: pick a provider with LEANEST_PROVIDER, or leave it on the free default. Adding a new one is one adapter behind the same interface,
not a rewrite.
classifier-dev
default
classifier.dev, a free zero-shot classifier
key: none
jev TypeSafe's Jev, over HTTP
key: TYPESAFE_API_KEY
laya Laya, self-hosted, runs in-process via ONNX Runtime
key: none
leanest has no server of its own. Whichever provider you pick, the CLI calls it directly with your credentials, if it needs any. Nothing routes through leanest's infrastructure, because there isn't any.
No key, an expired one, a timeout, or an unknown provider name all resolve the same way: leanest runs the full suite and prints why, instead of silently skipping coverage.
⚠ Judge unavailable (...), running the full suite.
gh secret set TYPESAFE_API_KEY --repo you/your-repo
provider: jev
typesafe-api-key: ${{ secrets.TYPESAFE_API_KEY }} The default classifier-dev provider needs no secret at all, which also means forked-repo pull requests (no access to your repo's secrets) still get selection.
Replace your existing "run e2e tests" step. Same reporter, same exit code, fewer tests executed.