QA Engine

AI-first bug reporting and QA automation, with a human in the loop and full manual testing. Playwright is the core engine. Outline v0.1 · 2026-11-22 · owner Greg · status: planning, nothing built yet.

1. What it is

One product with four parts that share one data model (projects, stories, test cases, runs, bugs):

QA Engine (the platform)

Test management in the style of Zoho QEngine: requirements and user stories, test cases (manual and automated), suites and plans, runs, bugs, reports. Automated cases run on Playwright. Most testing is AI-driven; people approve, edit and run manual cases.

Capture Embed (new)

A script tag or snippet that adds a bug-capture widget to any website. Opened by a tester or end user to report a bug with screenshots, video, voice and auto-collected context.

Capture Studio

The existing Capture Studio, extended to record test procedures: each recorded step becomes a manual test step and can be turned into a Playwright script. (Current state to be confirmed, see open questions.)

Mobile (later)

A mobile SDK / app version of the capture tool and mobile test runs. The bug and step schema is designed now so mobile is an addition, not a rewrite.

2. Capture Embed: bug reports from any page

Collected automatically

  • Page URL, title, referrer, timestamp, timezone
  • Browser name and version, OS, device, viewport, pixel ratio, language
  • Console errors and warnings, failed network requests (status, URL, timing)
  • Recent user actions (clicks, inputs, navigation) as draft steps to reproduce
  • App version / build id and logged-in user id, if the host page provides them

Entered by the reporter

  • Title, description / notes, severity, expected vs actual result
  • Steps to reproduce: auto-drafted, editable
  • Voice dictation: audio is transcribed into the notes (reuses re-libs Media Transcriber)
  • Attachments

Screenshots and annotation

  • Visible area, full page, or a selected element / region
  • Annotate: arrows, boxes, highlight, text, blur / redact
  • Element picker stores the selector, DOM snippet and computed styles

Video and annotation

  • Record the tab with the browser's screen capture, with optional microphone
  • Pin timestamped comments and drawn marks on the video
  • Voice track transcribed with timestamps; transcript links to the moment in the video
  • Console and network log is synced to the video timeline

Privacy and safety

  • Mask password fields and configurable selectors before capture
  • Per-project allowed origins and a public key; reports are signed
  • Reporter sees what will be sent before it is sent

Where reports go

  • QA Engine bug list; optional link to a test case or run
  • AI duplicate check and triage suggestion on arrival
  • Later: Jira, GitHub, Bitbucket, Slack sync

3. Capture Studio: recording test procedures

4. QA Engine: test management

AreaContents
Projects and environmentsApps under test, environments (dev, staging, prod), base URLs, test accounts (secrets stored encrypted, never shown to the AI), browsers and devices
RequirementsUser stories with acceptance criteria, tags, links to test cases, coverage view (stories without tests)
Test casesManual or automated, steps with expected results, preconditions, data sets, priority, version history, status (draft, approved, deprecated)
Suites and plansSmoke, regression, release plans; schedules; run on deploy via webhook / CI
RunsAutomated runs on Playwright workers (parallel, trace, video, screenshots); manual runs where a tester steps through and marks pass / fail / blocked, failing a step opens a pre-filled bug
BugsFrom Capture Embed, from failed runs, or by hand; lifecycle, assignee, severity, duplicates, link to story and test case
ReportsPass rate and trend, coverage, flaky tests, bugs by severity / age, release readiness, exportable and shareable by private link
AccessRE Auth sign-in; roles admin, QA lead, tester, developer, viewer

5. AI with a human in the loop

The AI drafts, a person approves. Nothing the AI writes becomes an approved test, closes a bug or changes a locator without a recorded decision.

AI doesHuman decides
Writes test cases from user stories and acceptance criteriaApprove, edit or reject each one
Turns recorded steps or a bug report into a Playwright scriptReview the script, run it once, then approve it into the suite
Explores an app (agent drives Playwright) and proposes new cases and bugsSet scope and budget; confirm findings
Analyses failures: product bug, test bug, environment, or flaky, with evidence from trace and screenshotsAccept the verdict or override it
Self-healing: proposes a new locator when a step breaksApprove the fix (auto-apply is an optional per-project setting, off by default)
Triage and dedupe incoming bugs, suggests severity and ownerConfirm or change
Visual comparison of screenshots against a baselineAccept as the new baseline or file a bug

Every AI call goes through the company LLM Router with a budget and a logged reason. Manual testing works fully with AI switched off.

6. The hard cases

Framed pages (iframes)

  • Playwright: frameLocator chains are recorded and replayed; frame path stored in each step.
  • Capture Embed runs in the top page and in each frame it is added to; cross-frame messages use postMessage with origin checks, so one report can include the parent and the framed app.
  • Cross-origin frames the owner does not control: capture through the browser extension / Studio, not the embed.

Drag-and-drop builders and canvases

  • Record drags as pointer sequences (down, moves, up) with positions relative to the source element and the drop target, not just screen pixels.
  • Replay with real mouse events and stepped moves so builder libraries fire their drag handlers.
  • Assert the result three ways: the DOM or block tree, the builder's saved state / JSON, and a screenshot of the canvas region.
  • Canvas elements (no DOM inside): coordinate steps plus visual and state checks; optional hooks the app can expose for test (window.__qa).
  • Test pages of this kind first with the AddieSaaS builders (popup, email, page).

Other

  • Shadow DOM (Playwright pierces open roots), file uploads, popups / new tabs, auth state reuse, responsive viewports.
  • Flaky controls: auto-wait, retries with trace, quarantine list.

Mobile (future)

  • Capture SDK for iOS / Android and React Native WebViews; same bug schema, device info and screen recording.
  • Test runs: Playwright against mobile web / emulated devices first, native later (Appium or equivalent, decided then).

7. Architecture (proposal)

8. Phases

PhaseDelivers
0Decisions on the open questions below; data model and API spec; this subdomain stays the plan page
1Capture Embed v1: context, screenshot + annotation, notes, steps, bug list in the platform
2Video recording and annotation, voice dictation, redaction; Capture Studio recorder to manual test cases
3Test management: stories, cases, suites, manual runs, reports
4Playwright runners, scheduled and CI runs, traces, failure to bug
5AI: generate cases and scripts, failure analysis, triage, self-healing proposals, approvals inbox
6Framed pages and drag-and-drop builder support hardened on the AddieSaaS builders; Jira / GitHub sync
7Mobile SDK and mobile runs

9. Open questions