l-agent-walk-test

Reproduce a user-reported bug by walking through the running dev:mock app with an observation subagent driven by the user's actual settings as a seed, then write a regression spec keyed to that fixtur...

l-agent-walk-test

Reproduction-first bug hunting via a bounded observation subagent.

Use the user's real settings as the seed, drive pnpm dev:mock with a headless browser, observe what the app does, write findings into a regression spec keyed to the same seed.

When to use

ALL of the following hold:

1. The user has reported a real behaviour bug (not a feature request).

2. The bug is in runtime behaviour the code-only reading cannot prove out — header pins, frameset switching, dialogs, persistence, route-sync, anything driven by AppSettings + saved framesets + react-router.

3. Either (a) prior PRs already attempted a fix and the user still says it's broken, OR (b) the bug has multiple plausible root causes and you would otherwise speculate.

If the bug is reproducible from code-reading alone (a typo, a missing prop, a clear off-by-one), skip this skill and just fix it. The skill exists to prevent speculative fixes against unproven hypotheses.

When NOT to use

• Pure CSS / layout issues — use /verify-ui (Level 5).

• Bugs the user has already reproduced and described in detail — fix directly.

• Mock-backend-only behaviour that doesn't actually run the Tauri-side code (file IO, PTY, watchers).

The pattern

Phase 1 — Seed the user's real settings as a fixture

The most expensive mistake is synthesizing a seed that doesn't match the user's actual data shape. Whatever corruption / customization / persisted state caused the bug, the user has it on disk; your synthetic seed almost certainly does not.

1. Locate the user's workspace settings — usually:

   <appname workspace>/.zudotext.settings.json

   The workspace path comes from ~/.config/zudotext/<appname>/config.json → workspace.

2. Copy it into the repo as a Playwright fixture:

   mkdir -p e2e/fixtures
   cp "<user-settings-path>" e2e/fixtures/<appname>-user-seed.json

3. Redact only what's strictly required. Path leaks: replace projectRoot with a generic mock path (/tmp/mock-<appname>-workspace). Tokens, emails, real URLs: scrub. Do NOT touch the bug-relevant fields (framesets, headerLeftPins, currentFramesetId, kanbanBoards, etc.) — the corruption you're chasing lives there.

4. Commit the fixture. It is the canonical "this user's broken state" artifact for every future fix attempt and regression guard against this report.

The existing e2e/fixtures/prompts-app-user-seed.json is a working example.

Phase 2 — Dispatch a bounded observation subagent

The subagent's job is observation, not diagnosis or fix. You (the parent) read the observations and decide what to do.

Server setup

pnpm dev:mock runs the frontend with the in-memory mock backend on port 1421. Check whether it's already running:

lsof -ti :1421

If not running, start it in the background. Do not block the subagent's setup with a server-start step that requires waiting.

Seeding the mock from a Playwright test

The mock entrypoint at tauri-app/renderer/main-mock.tsx reads localStorage["e2e:mock-settings-seed"] once at startup. To inject the fixture before the first navigation, use context.addInitScript (or page.addInitScript):

await context.addInitScript((data) => {
  window.localStorage.setItem("e2e:mock-settings-seed", JSON.stringify(data));
}, JSON.parse(fs.readFileSync("<path-to-fixture>", "utf8")));
await page.goto("http://localhost:1421/");

The seed is read-only at startup — to reseed mid-session, write the new value and page.reload().

Subagent brief shape

The subagent should be dispatched via the Agent tool (not claude -p), with subagent_type: "general-purpose". Its prompt must include:

• Goal — one sentence: "walk through the app with this seed, observe what happens at each step, report factually. Do not diagnose or fix."

• Setup — dev:mock URL, fixture path, seeding code snippet (verbatim from above).

• Browser — load /headless-browser. Prefer WebKit (per project CLAUDE.md "Prefer WebKit for verification") but fall back to Chromium if WebKit is unreliable.

• Scenario list — number 4-8 specific scenarios to walk: cold load, every header pin click, every relevant dialog open/save flow, every switcher row, any persistence round-trip the user mentioned. List them explicitly — don't hand-wave "explore the app."

• What to capture per scenario — URL after settle, aria snapshot of the body (cheap; ~50 lines), any console.error / console.warn (filter known-safe noise per e2e/helpers.ts:assertNoConsoleErrors).

• Output format — a markdown report with one section per scenario, ending with a ## Summary of likely bugs section that includes evidence (DOM excerpts, URLs, console errors) for each finding.

• Screenshots / driver scripts — save under $HOME/cclogs/zudo-text/ and __inbox/ respectively. Both are gitignored. Do not commit screenshots.

• Don'ts — no fixing, no diagnosing root causes, no exceeding ~10 minutes wall time, no exceeding ~1500 words.

A working example brief is in PR #1809 ("walkthrough subagent" Agent call).

Phase 3 — Read findings, decide fix scope

The parent (you) reads the subagent report and:

1. Maps each "likely bug" to a specific code location. Use Read + grep to trace the symptom to its source. Don't trust the subagent's hypothesis — it was told not to diagnose. You verify.

2. Prioritizes. Some findings may be lower-impact than others; some may be intentional. Apply advisor() if you're tempted to fix everything at once instead of the highest-impact one or two.

3. Decides what to defer. Mark out-of-scope findings as deferred and file GitHub issues for them so they don't get lost.

Phase 4 — Implement the fixes (small surgical changes)

Apply each fix in its own commit. Keep commits focused — one bug, one commit, one description of why. Do NOT bundle "while I'm here" cleanups into bug-fix commits; those mask the actual fix in the diff and make regressions harder to bisect.

Phase 5 — Lock the fix with a regression spec keyed to the same fixture

The spec lives at e2e/<topic-slug>.spec.ts and loads the SAME fixture from Phase 1. The exact selector / assertion shape that exposed the bug becomes the regression guard.

Spec template

import { test, expect, type Page } from "@playwright/test";
import { readFileSync } from "node:fs";
import { resolve } from "node:path";

function loadFixtureSeed(): Record<string, unknown> {
  const path = resolve(process.cwd(), "e2e/fixtures/<appname>-user-seed.json");
  return JSON.parse(readFileSync(path, "utf8")) as Record<string, unknown>;
}

async function seedAndOpen(page: Page, seed: Record<string, unknown>) {
  // Surface page errors and console.error lines so a failing test never
  // reduces to a mute "MOCK" snapshot.
  page.on("pageerror", (err) => console.log(`[page-error] ${err.message}`));
  page.on("console", (msg) => {
    if (msg.type() === "error") console.log(`[console.error] ${msg.text()}`);
  });
  await page.addInitScript((data) => {
    try {
      window.localStorage.setItem("e2e:mock-settings-seed", JSON.stringify(data));
    } catch {}
  }, seed);
  await page.goto("/");
  await page.waitForSelector('nav[role="tablist"]', { timeout: 10_000 });
}

/**
 * Read live currentFramesetId from the in-memory mock backend.
 * Avoids selector coupling to a particular toolbar / chrome data-testid.
 */
async function readCurrentFramesetId(page: Page): Promise<string | null> {
  return page.evaluate(async () => {
    const w = window as unknown as { __backend?: { settings: { get(): Promise<unknown> } } };
    const s = (await w.__backend?.settings.get()) as { currentFramesetId?: string } | null;
    return s?.currentFramesetId ?? null;
  });
}

test.describe("<topic> regression guards", () => {
  test("...", async ({ page }) => {
    await seedAndOpen(page, loadFixtureSeed());
    // ... reproduce the bug's user-facing action
    // ... assert the post-fix invariant
    // Use expect.poll for time-dependent state — NEVER waitForTimeout for
    // debounce windows (anti-pattern; flaky on slow CI, slow on fast machines).
    await expect.poll(() => readCurrentFramesetId(page), { timeout: 5_000 }).toBe("inbox");
  });
});

Spec correctness checklist

• Does the assertion actually exercise the bug code path? A "open and close dialog without mutation" test does NOT exercise the dialog's save flow, so it would pass on broken code. Toggle something net-zero (visibility on then off) to fire commit without changing visible state.

• Does the assertion poll the bug's observable, not just a UI-side proxy? e.g. read the live currentFramesetId via window.__backend.settings.get(), not just the pin's aria-selected attribute.

• Is the spec CI-safe (Chromium-headless, no @interactive keyboard shortcuts)? If a keyboard shortcut is genuinely required, tag the describe with @interactive so b4push and CI exclude it. See e2e/README.md.

• Does the failing-spec mode prove the fix is necessary? Before merging, revert the fix locally and re-run the spec. If it still passes, the spec is ornamental — fix the assertion until it fails on broken code.

Phase 6 — File issues for deferred findings

Anything the walkthrough surfaced that you decided not to fix in this PR — file a GitHub issue immediately. Don't trust the PR description to be searchable later; create the issue, link it from the PR body, and let it be tracked.

The walkthrough subagent's full report (saved under $HOME/cclogs/zudo-text/) is the source material for these issues. Quote the relevant scenario verbatim.

Reference

• Working example: PR #1809 "fix(frameset): prompts.app header pin click + add-pin reverts active page".

• Fixture: e2e/fixtures/prompts-app-user-seed.json.

• Regression spec: e2e/prompts-app-pin-frameset.spec.ts.

• Subagent brief shape: the Agent call in that PR's conversation log.

• Project-level docs: e2e/README.md (Two Test Categories — CI-safe vs @interactive).

Trade-offs and limitations

• The seed can hide bugs the migration normally corrects. If useSavedFramesets's normalizer auto-fixes the user's corruption before the buggy code path runs, the seed will not reproduce. Verify by reading the post-load state via window.__backend.settings.get() after seed apply — if the corruption was migrated away, you may need a different fixture or a less aggressive seed.

• WebKit vs Chromium. Per CLAUDE.md the Tauri runtime is WebKit, so WebKit is the more faithful repro target. Some headless WebKit configurations are flaky under Playwright; fall back to Chromium with a note in the report if WebKit doesn't behave. Regression specs should default to Chromium for CI portability.

• The walkthrough subagent is non-deterministic. Different runs may pick different DOM paths or miss edge cases. Once you've identified the bug, the regression spec — not the subagent — is the durable artifact.

• Don't commit the screenshots. They go under $HOME/cclogs/zudo-text/ and the spec uses screenshot: "only-on-failure" already. The fixture is the only artifact that needs to live in the repo.

When the walkthrough returns "no bug found"

Sometimes the seed + the same actions don't reproduce the user-reported bug. That is a signal, not a failure:

• The user's environment may have additional state (real backend file watcher, Tauri-only IPC) that the mock cannot reproduce. Try running against the real app instead of pnpm dev:mock (l-local-tauriapp-build for build + install).

• The bug may be intermittent / timing-dependent. Adjust the scenario to add delays or rapid-fire interactions.

• The user may have misremembered the steps. Ask them to walk through it themselves with screen recording, then map their recording to scenarios.

Do not invent a fix without a working reproduction. "Couldn't reproduce" is a valid report back to the user.