Testing Strategy

Two Axes: Testing Levels and Execution Tiers

Every test has a position on two independent axes.

Level answers "what can the test see?": pure logic (unit), DOM (component), built artifact, real browser (E2E), or real platform. The level is fixed by what the assertion needs to observe — it does not change based on how long the test takes or how hardware-constrained it is.

Tier answers "where and when does it run?": inner loop, PR gate, scheduled macOS job, or local heavy lane. The tier is chosen by cost, hardware constraints, and timing.

Keeping the axes separate dissolves the most common escalation mistake: when a test "feels too heavy for CI", that is a tier question, not a reason to rewrite it at a lower level. Fix the tier assignment; keep the assertion where it belongs.

This repo's tier mapping

No T2 (T1 is within its time budget). No unused tiers are scaffolded.

Tag Taxonomy

Tags appear in the Playwright test.describe name (e.g., test.describe("My Feature @interactive", () => { … })).

@flaky and pass-on-retry

playwright.config.ts sets retries: 2 for CI but the local lanes (b4push, exam) run with zero retries. A test that passes on its second or third CI attempt is reported green — but pass-on-retry is a triage signal, not a success. It belongs in the @flaky quarantine pipeline: tag with an inline issue URL, exclude from strict gates, run allowed-to-fail in T3, and fix or delete with a deadline.

Vitest quarantine: @flaky-vitest

Vitest has no equivalent allowed-to-fail execution lane. A quarantined file uses a greppable header with a full tracking-issue URL and skips its top-level describes:

// @flaky-vitest — quarantined (reason): https://github.com/zudolab/zudo-text/issues/NNNN
describe.skip("flaky suite", () => {
  // …
});

Unlike Playwright @flaky, which still runs allowed-to-fail in T3, describe.skip yields zero further failure data. Every @flaky-vitest issue therefore needs a re-triage deadline and one exit: fix the flake, replace it with a cheaper guard, or delete the test. Remove the marker and describe.skip in the same PR as the fix.

The CI workspace-package test step is wrapped by nick-fields/retry with max_attempts: 2, so a first-attempt Vitest failure may finish green. That pass-on-retry is still a triage signal; inspect and investigate the failed attempt rather than treating the final job status as success.

@verification and the graduation rule

Agent-authored verification specs are tagged @verification and excluded from all gates. Promotion to a regression tier is explicit: remove the tag, assign the appropriate tier (or leave untagged for T1), and confirm the spec fits within the target suite's time budget. An agent never promotes its own verification spec — it proposes promotion in the PR description and leaves the decision to the reviewer.

b4push vs exam Contract

b4push and exam are different jobs and must stay separate — one command slowly accreting both jobs is what breaks the pre-push gate. When b4push exceeds its budget, people skip it, and nothing runs before push. exam is allowed to be slow precisely because nobody sits waiting for it.

If a change touches code from the module-to-spec mapping table in e2e/README.md, pnpm b4push warns you to also run pnpm exam or dispatch gh workflow run e2e-webkit-macos.yml --ref <branch> before declaring done.

MCP lanes

The MCP checks use the same level/tier separation as the rest of the repository. The level says what the assertion can observe; the tier says when the cost is acceptable. A fake transport is useful deterministic evidence, but it is not deployed authenticated evidence.

Adding or changing an MCP tool

Use this checklist for a new tool, a schema or description change, or a route and skill contract change. Keep the package copy in packages/zudotext-mcp/CLAUDE.md verbatim with this checklist.

• Register the tool in its family, regenerate tool-inventory.json, and review the description diff with pnpm --filter @takazudo/zudotext-mcp exec vitest run src/server-inventory.test.ts -u.

• Add or adjust the MCP_FEATURES row or an explicit exemption, then run the route-parity test against workers/sync-server/test/agent-route-inventory.json.

• Update both shipped SKILL.md files and every counted document named by packages/zudotext-mcp/docs-inventory.test.mjs: the package README, doc/src/content/docs/packages/zudotext-mcp.mdx, doc/src/content/docs/guide/mcp-integration.mdx, and doc/src/content/docs/architecture/local-agent-authoring.mdx. Also check doc/src/content/docs/guide/agent-task-recipes.mdx: it is a subset-inventory document, so a renamed or removed tool it names in call form must be fixed there too, but it never needs a tool-count update.

• Extend workers/sync-server/test/mcp-journey only when the tool joins the core connected authoring story.

• Run pnpm --dir packages/zudotext-mcp release:verify -- --expected-tag "zudotext-mcp-v$(node -p \"require('./packages/zudotext-mcp/package.json').version\")".

• Note the tool, inventory, skill, or release impact in packages/zudotext-mcp/RELEASING.md.

Unit Tests — TypeScript Packages

Packages use Vitest for unit testing. Packages with test configurations:

• @takazudo/file-utils

• @takazudo/find-in-page

• @takazudo/shortcut-engine

• @takazudo/color-themes

• @takazudo/code-block

• @takazudo/command-palette

• @takazudo/kanban-parser

• @takazudo/kanban-board

• @takazudo/app-defaults

• @takazudo/backend-bridge

Run tests for a specific package:

cd packages/<package-name>
pnpm test

UI Component Tests

@takazudo/ui-components uses Vitest with jsdom and Testing Library for component tests:

• @testing-library/react — Component rendering and queries

• @testing-library/jest-dom — DOM assertions

• @testing-library/user-event — User interaction simulation

The test environment is configured in vitest.config.ts:

export default defineConfig({
  test: {
    environment: "jsdom",
    include: ["src/**/*.test.{ts,tsx}"],
    setupFiles: ["./vitest-setup.ts"],
  },
});

Backend Tests — Rust

The Rust backend unit tests live in the zudotext-core crate (tauri-app/core/):

tauri-app/core/src/helpers/ — Helper modules with #[cfg(test)] sections:

• filename.rs — Filename generation logic

• frontmatter.rs — Markdown frontmatter parsing

• path_utils.rs — Path utility functions

• path_containment.rs — Path containment checks

• pin_path.rs — Pin path resolution

• pin_tree.rs — Pin directory tree building

• safe_path.rs — Path sanitization

• watcher_dedup.rs — File watcher deduplication

tauri-app/src/helpers/ contains only thin re-export shims (pub use zudotext_core::helpers::…) with no #[cfg(test)] code of their own.

tauri-app/core/ — The zudotext-core crate mirrors business logic without Tauri dependencies, enabling cargo test on any platform (including WSL2 and CI without GTK libs):

cd tauri-app/core && cargo test

Run all Rust tests:

cd tauri-app
cargo test

Frontend Testing with MockAdapter

The @takazudo/backend-bridge package provides a MockAdapter that replaces the real Tauri backend in tests, Storybook, and the standalone mock dev mode (pnpm dev:mock). This enables frontend development and testing without a running Tauri process.

The BackendAPI Adapter Pattern

The frontend never imports @tauri-apps/api directly. Instead, all backend communication goes through a BackendAPI interface with swappable adapters. At app startup, exactly one adapter is initialized via initBackend(adapter). All components call getBackend() to access the backend. This means the same UI code runs in Tauri, in mock dev mode, in REST dev mode, and in tests — no conditional imports or branching needed.

See the Backend Bridge Pattern architecture page for the full adapter design.

Using the Mock Adapter

import { initBackend } from "@takazudo/backend-bridge";
import { createMockAdapter } from "@takazudo/backend-bridge/mock-adapter";

const { api, controls } = createMockAdapter();
initBackend(api);

// Use api just like the real backend
await api.messages.list();

// Simulate backend events
controls.triggerMessagesChanged("test.md");
controls.triggerNotesChanged("inbox");

MockControls

The controls object provides programmatic access to the mock backend's internal state:

interface MockControls {
  // Trigger event listeners (simulates backend events)
  triggerMessagesChanged: (filename?: string) => void;
  triggerPinFileChanged: (entryPath: string) => void;
  triggerDraftExternalChange: (draftNumber: number) => void;
  triggerIncomingWorkspaceConflict: (conflict: IncomingWorkspaceConflict) => void;
  triggerNotesChanged: (dir: string) => void;
  triggerSchemaChanged: () => void;
  triggerSubscriptionInfoChanged: (info: SubscriptionInfo) => void;

  // State setters
  setAuthState: (state: AuthState) => void;
  setSubscriptionInfo: (info: SubscriptionInfo) => void;
  addRemoteFile: (path: string, content: string) => void;

  // Direct access to in-memory stores
  files: Map<string, string>;       // message files
  pinFiles: Map<string, string>;    // pin files (keyed by "pinIndex:entryPath")
  remoteFiles: Map<string, string>; // simulated remote files for sync
}

Customizing Seed Data

To set up specific test scenarios, write to the internal stores before running your test:

const { api, controls } = createMockAdapter();
initBackend(api);

// Seed message files
controls.files.set("meeting-notes.md", "---\ntitle: Meeting Notes\n---\n\n# Notes");
controls.files.set("draft-email.md", "---\ntitle: Draft Email\n---\n\n# Email");

// Seed pin files
controls.pinFiles.set("0:readme.md", "# Project README");

// Trigger UI refresh
controls.triggerMessagesChanged();

Storybook Integration

Storybook stories for backend-dependent components use the mock adapter to provide realistic data without requiring Tauri. Initialize the adapter in the story setup:

import { initBackend } from "@takazudo/backend-bridge";
import { createMockAdapter } from "@takazudo/backend-bridge/mock-adapter";

const { api } = createMockAdapter();
initBackend(api);

The mock adapter provides:

• In-memory file stores for messages, pins, and drafts

• Event simulation via controls (message, note, schema, workspace-conflict, and subscription changes)

• Same BackendAPI interface as the real TauriAdapter

Testing with Real Data via REST Mode

When you need to test the frontend against the real Rust backend with full browser DevTools, use pnpm dev:rest (requires pnpm tauri:dev running in a separate terminal). See the REST Dev Mode setup guide for details.

REST mode uses the RestAdapter which talks to the same Rust backend over HTTP instead of Tauri IPC.

The filesystem-backed message and note paths in this mode are a retained local-engine development fallback. They do not represent the shipped cloud-workspace renderer's storage path.

This gives you:

• Real file system data — your actual messages, pins, and settings

• Chrome network tab — inspect every backend call, response times, and payloads

• Chrome performance profiler — identify rendering bottlenecks with real data volumes

• Accessibility auditing — use browser extensions on the live app

REST mode complements the MockAdapter approach: use mock mode for isolated component tests and rapid UI iteration, and REST mode when you need to verify behavior with real backend data and full browser debugging tools.

See the Backend Bridge Pattern docs for the adapter architecture.

Renderer Typecheck Gate

pnpm typecheck runs tsc --noEmit against the renderer production tree using tauri-app/tsconfig.typecheck.json. It resolves all workspace packages (@takazudo/*) from their TypeScript source (src/index.ts) rather than pre-built .d.ts files (Path B: "types" → source). This means a type error anywhere in a package's source that the renderer depends on will fail the gate — not just errors in pre-shipped .d.ts declarations.

The gate is zero-error and is enforced in two places:

• CI (.github/workflows/ci.yml, checks job): the "Renderer typecheck" step runs after "Build workspace packages".

• Pre-push (scripts/run-b4push.sh): pnpm typecheck runs after "Build workspace packages" as the "Renderer typecheck" step.

What it catches

Two bug classes that previously slipped through:

1. Missing barrel re-exports. A type defined in a package's source but missing from its index.ts barrel will resolve fine against the built .d.ts (which may include it transitively) but fail when tsc reads source directly. Example: removing LocalDirEntry from packages/backend-bridge/src/index.ts produces:

renderer/components/directory-tree-picker.tsx(3,15): error TS2724:
  '"@takazudo/backend-bridge"' has no exported member named 'LocalDirEntry'.

2. Feeding string[] into a string-typed CodeMirror keymap key. AppSettings.shortcuts.* values are string[] (multi-binding). CodeMirror's KeyBinding.key is typed string. Passing the array without taking [0] produces:

renderer/hooks/use-codemirror-editor.ts(157,7): error TS2322:
  Type 'string | string[]' is not assignable to type 'string'.
  Type 'string[]' is not assignable to type 'string'.

These two bugs are the regressions from #2142 that the gate was designed to prevent.

Reproducing the breaks

To verify the gate catches each bug class, apply the break, run pnpm typecheck, then revert:

Break A — barrel re-export:

1. In packages/backend-bridge/src/index.ts, remove the LocalDirEntry line from the export type { … } block.

2. Run pnpm typecheck — expect the TS2724 error above on directory-tree-picker.tsx:3.

3. Revert the line.

Break B — string[] into CodeMirror keymap:

1. In tauri-app/renderer/hooks/use-codemirror-editor.ts line 118, change appSettings?.shortcuts?.inlineAiCommandTrigger?.[0] ?? "Mod-j" to appSettings?.shortcuts?.inlineAiCommandTrigger ?? "Mod-j" (remove ?.[0]).

2. Run pnpm typecheck — expect the TS2322 error above on use-codemirror-editor.ts.

3. Revert the change.

App Lifecycle Tests

Beyond unit tests and component tests, zudo-text has a 3-phase integration testing strategy that validates the full application stack. Each phase targets a different layer, and together they catch issues that unit tests cannot.

Phase 1: REST API Smoke Test

Tests backend API correctness by exercising the REST endpoints directly.

What it tests: Settings retrieval, message CRUD operations, draft management, draft content handling.

What it catches: API regressions, data corruption, endpoint failures, serialization bugs.

# Requires pnpm tauri:dev running in another terminal
pnpm test:rest-smoke

The script sends HTTP requests against localhost:3001/api/* and validates response status codes, JSON structure, and basic data integrity.

Phase 2: macOS Launch Test

Tests the full app start/stop lifecycle on macOS — from open to quit to process cleanup.

What it tests: App launch, REST API readiness, graceful shutdown, orphaned process detection.

What it catches: Startup failures, orphaned processes, shutdown cleanup regressions, port binding issues.

# macOS only — requires a built app bundle
pnpm test:launch-macos

The test flow:

1. Opens the app via open /Applications/zudotext.app

2. Polls localhost:3001/api/settings until it responds (with timeout)

3. Runs the Phase 1 smoke tests against the live app

4. Quits the app via AppleScript

5. Verifies no orphaned processes remain and the port is freed

Set ZUDOTEXT_APP_PATH to test a different app bundle location.

Phase 3: Mock Headless Browser Test

Tests frontend page navigation and rendering using a headless browser with the mock backend.

What it tests: Page loading, navigation between views, console errors, component rendering.

What it catches: Frontend crashes, import errors, rendering failures, broken routes.

pnpm test:mock-headless

This phase starts the mock dev server (pnpm dev:mock) and uses Playwright to navigate through the app's pages, checking for JavaScript errors and verifying that key elements render correctly.

How the Adapter Pattern Enables Testing

The BackendAPI adapter pattern is what makes this 3-phase strategy possible. Each phase uses a different adapter:

• Phase 1 (REST Smoke) — Uses the REST adapter's HTTP endpoints. The Tauri app exposes the same backend logic over HTTP, so tests can exercise real Rust code without a GUI.

• Phase 2 (macOS Launch) — Tests the real Tauri app lifecycle while probing its development REST endpoint. This validates native IPC availability, Rust backend startup, filesystem fallback behavior, and process cleanup.

• Phase 3 (Mock Headless) — Uses the Mock adapter's in-memory backend. No Rust process needed — the frontend runs entirely in the browser with simulated data, isolating frontend-only issues.