Dungbeetle vs Playwright's toHaveScreenshot

Playwright's built-in toHaveScreenshot() is a free, local visual regression assertion — and for a small suite on a single platform it is genuinely the right place to start. Dungbeetle is a snapshot and visual regression testing tool — a free CLI plus self-hostable cloud — built for AI agents and the humans they work for. The honest verdict up front: start with toHaveScreenshot; add Dungbeetle when reviewing baseline changes, managing them across a team, or letting coding agents run the loop starts to hurt. The two compose — Dungbeetle can ingest the very screenshots your Playwright suite already takes.

toHaveScreenshot, done right

If you're on Playwright, this is the built-in:

test("home page looks right", async ({ page }) => {
  await page.goto("/");
  await expect(page).toHaveScreenshot("home.png", {
    fullPage: true,
    animations: "disabled",
    maxDiffPixelRatio: 0.001
  });
});

• First run fails with "snapshot doesn't exist" — run npx playwright test --update-snapshots to mint baselines, commit them.
• animations: "disabled" and a small maxDiffPixelRatio kill most flake.
• Pin the rendering platform. Screenshots minted on macOS fail on Linux CI over font antialiasing alone. Either generate baselines in a container/CI and download them, or accept per-platform snapshot suffixes (home-darwin.png, home-linux.png) and double the maintenance.

That last bullet is not a Playwright flaw — it's physics; every pixel tool has it (Dungbeetle included — our own baselines are CI-minted for exactly this reason).

Where it stops scaling

Real limits we hit ourselves, in rough order of when they bite:

1. Reviewing a baseline change is reviewing a binary. A PR that updates home.png shows two images at best. No overlay, no onion-skin slider, no per-region diff — the reviewer eyeballs two full-page screenshots and hopes.
2. Approving means re-running. "This change is intentional" = --update-snapshots locally (wrong platform, see above) or downloading CI artifacts and committing them by hand. There's no approve button.
3. Pixels are the only language. A renamed button, a copy tweak, and a broken layout all look the same: a red blob. There's no structural diff to tell you what changed.
4. No memory. No history of who approved what, no flakiness analytics, no "this target failed 4 of the last 20 runs" — just the current PNG.
5. Agents have no identity. A coding agent that can run --update-snapshots can approve its own visual changes. There is no scope model to give an agent "run and report, but never touch baselines".

Keep Playwright — add a review workflow

Dungbeetle doesn't replace your Playwright suite; it sits after it. Point targets at the screenshots Playwright already writes (screenshotFile), or let Dungbeetle's own playwright driver capture structured DOM alongside pixels:

{
  "version": 1,
  "project": { "name": "my-app-ui" },
  "baselinesDir": "dungbeetle.snapshots",
  "lifecycle": {
    "capture": [
      { "kind": "web", "name": "home", "screenshotFile": "e2e/screenshots/home.png" },
      { "kind": "web", "name": "pricing", "screenshotFile": "e2e/screenshots/pricing.png" }
    ]
  },
  "comparison": {
    "pixelTolerance": { "maxChangedRatio": 0.001, "perChannelThreshold": 32 }
  }
}

npx playwright test          # your suite writes e2e/screenshots/*.png
npx dungbeetle update        # mint baselines from them (commit dungbeetle.snapshots/)
npx dungbeetle test          # exits non-zero on any visual change

What that buys over raw toHaveScreenshot:

• A review UI — onion-skin/side-by-side diffs, an approve/reject button, and promoted baselines, instead of eyeballing PNGs in a PR.
• Semantic diffs where they exist — DOM captures diff as structure ("one attribute changed on <button>"), with pixels as the backstop.
• History and flakiness analytics per target and branch.
• Agent auth — an agent gets a scoped, revocable token via a device flow a human approves; it can run tests and file reviews, but promoting a baseline requires the separate baselines:write scope. The approve click stays human. (See Docs for agents.)

The CI loop we run on ourselves

This site is gated by this exact setup — Playwright E2E writes screenshots, Dungbeetle pixel-compares them against committed baselines, and a red check blocks the deploy:

# .github/workflows/visual.yml (abridged from the real one)
name: Visual
on: [push, pull_request]
jobs:
  visual:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v6
        with: { node-version: "22.x", cache: npm }
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - run: npm run e2e                 # writes e2e/screenshots/
      - run: npx --yes dungbeetle test --config dungbeetle.config.json
      # optional: record the run centrally for the review UI
      - run: npx --yes dungbeetle push --report .dungbeetle/dogfood-report.json
        env:
          DUNGBEETLE_SERVER_URL: https://dungbeetle.dev
          DUNGBEETLE_CLIENT_ID: ${{ secrets.DUNGBEETLE_CLIENT_ID }}
          DUNGBEETLE_CLIENT_SECRET: ${{ secrets.DUNGBEETLE_CLIENT_SECRET }}

Baselines are Linux-minted in CI (the canonical platform), so a macOS laptop never fights the gate — intentional changes are reviewed and promoted in the UI, not re-rendered by hand.

Side by side

	Playwright toHaveScreenshot	Dungbeetle
Cost	Free, built-in	Free CLI (source-available); optional cloud, flat proposed tiers
Diff	Pixels	Semantic JSON (DOM/terminal/API/…) + pixels with tolerance
Baseline review	PNGs in the PR	PR-committed JSON/PNGs or cloud review UI with approve/reject
Approving a change	--update-snapshots re-run	One click (or dungbeetle update locally)
History / flakiness	—	Per-target analytics, run history
Agents	No identity — an agent can update baselines	Scoped tokens; baseline promotion stays human-gated
Surfaces	Web pages in Playwright	Web, terminal, API, performance, desktop, games

Try it in five minutes

No account needed to see a real diff:

npx dungbeetle anon https://your-site.example --compare https://staging.your-site.example --screenshot

Or wire your existing suite: npx dungbeetle init, add screenshotFile targets for the PNGs you already produce, npx dungbeetle update, done. More: Quick start, Web capture, Pricing.