QA Engine
AI-first bug reporting and QA automation, with a human in the loop and full manual testing. Playwright is the core engine. Outline v0.1 · 2026-11-22 · owner Greg · status: planning, nothing built yet.
1. What it is
One product with four parts that share one data model (projects, stories, test cases, runs, bugs):
QA Engine (the platform)
Test management in the style of Zoho QEngine: requirements and user stories, test cases (manual and automated), suites and plans, runs, bugs, reports. Automated cases run on Playwright. Most testing is AI-driven; people approve, edit and run manual cases.
Capture Embed (new)
A script tag or snippet that adds a bug-capture widget to any website. Opened by a tester or end user to report a bug with screenshots, video, voice and auto-collected context.
Capture Studio
The existing Capture Studio, extended to record test procedures: each recorded step becomes a manual test step and can be turned into a Playwright script. (Current state to be confirmed, see open questions.)
Mobile (later)
A mobile SDK / app version of the capture tool and mobile test runs. The bug and step schema is designed now so mobile is an addition, not a rewrite.
2. Capture Embed: bug reports from any page
Collected automatically
- Page URL, title, referrer, timestamp, timezone
- Browser name and version, OS, device, viewport, pixel ratio, language
- Console errors and warnings, failed network requests (status, URL, timing)
- Recent user actions (clicks, inputs, navigation) as draft steps to reproduce
- App version / build id and logged-in user id, if the host page provides them
Entered by the reporter
- Title, description / notes, severity, expected vs actual result
- Steps to reproduce: auto-drafted, editable
- Voice dictation: audio is transcribed into the notes (reuses re-libs Media Transcriber)
- Attachments
Screenshots and annotation
- Visible area, full page, or a selected element / region
- Annotate: arrows, boxes, highlight, text, blur / redact
- Element picker stores the selector, DOM snippet and computed styles
Video and annotation
- Record the tab with the browser's screen capture, with optional microphone
- Pin timestamped comments and drawn marks on the video
- Voice track transcribed with timestamps; transcript links to the moment in the video
- Console and network log is synced to the video timeline
Privacy and safety
- Mask password fields and configurable selectors before capture
- Per-project allowed origins and a public key; reports are signed
- Reporter sees what will be sent before it is sent
Where reports go
- QA Engine bug list; optional link to a test case or run
- AI duplicate check and triage suggestion on arrival
- Later: Jira, GitHub, Bitbucket, Slack sync
3. Capture Studio: recording test procedures
- Record a walkthrough: every click, type, select, navigation, drag and scroll becomes a numbered step with a screenshot and the element locator.
- Output 1: a manual test case (readable steps, expected results you add or AI proposes).
- Output 2: a Playwright script draft using role / test-id / text locators, reviewed by a person before it joins the suite.
- Voice notes during recording become step descriptions.
- Same recorder core as Capture Embed, so a bug report can be converted into a test case ("add a regression test for this bug").
4. QA Engine: test management
| Area | Contents |
|---|---|
| Projects and environments | Apps under test, environments (dev, staging, prod), base URLs, test accounts (secrets stored encrypted, never shown to the AI), browsers and devices |
| Requirements | User stories with acceptance criteria, tags, links to test cases, coverage view (stories without tests) |
| Test cases | Manual or automated, steps with expected results, preconditions, data sets, priority, version history, status (draft, approved, deprecated) |
| Suites and plans | Smoke, regression, release plans; schedules; run on deploy via webhook / CI |
| Runs | Automated runs on Playwright workers (parallel, trace, video, screenshots); manual runs where a tester steps through and marks pass / fail / blocked, failing a step opens a pre-filled bug |
| Bugs | From Capture Embed, from failed runs, or by hand; lifecycle, assignee, severity, duplicates, link to story and test case |
| Reports | Pass rate and trend, coverage, flaky tests, bugs by severity / age, release readiness, exportable and shareable by private link |
| Access | RE Auth sign-in; roles admin, QA lead, tester, developer, viewer |
5. AI with a human in the loop
The AI drafts, a person approves. Nothing the AI writes becomes an approved test, closes a bug or changes a locator without a recorded decision.
| AI does | Human decides |
|---|---|
| Writes test cases from user stories and acceptance criteria | Approve, edit or reject each one |
| Turns recorded steps or a bug report into a Playwright script | Review the script, run it once, then approve it into the suite |
| Explores an app (agent drives Playwright) and proposes new cases and bugs | Set scope and budget; confirm findings |
| Analyses failures: product bug, test bug, environment, or flaky, with evidence from trace and screenshots | Accept the verdict or override it |
| Self-healing: proposes a new locator when a step breaks | Approve the fix (auto-apply is an optional per-project setting, off by default) |
| Triage and dedupe incoming bugs, suggests severity and owner | Confirm or change |
| Visual comparison of screenshots against a baseline | Accept as the new baseline or file a bug |
Every AI call goes through the company LLM Router with a budget and a logged reason. Manual testing works fully with AI switched off.
6. The hard cases
Framed pages (iframes)
- Playwright:
frameLocatorchains are recorded and replayed; frame path stored in each step. - Capture Embed runs in the top page and in each frame it is added to; cross-frame messages use
postMessagewith origin checks, so one report can include the parent and the framed app. - Cross-origin frames the owner does not control: capture through the browser extension / Studio, not the embed.
Drag-and-drop builders and canvases
- Record drags as pointer sequences (down, moves, up) with positions relative to the source element and the drop target, not just screen pixels.
- Replay with real mouse events and stepped moves so builder libraries fire their drag handlers.
- Assert the result three ways: the DOM or block tree, the builder's saved state / JSON, and a screenshot of the canvas region.
- Canvas elements (no DOM inside): coordinate steps plus visual and state checks; optional hooks the app can expose for test (
window.__qa). - Test pages of this kind first with the AddieSaaS builders (popup, email, page).
Other
- Shadow DOM (Playwright pierces open roots), file uploads, popups / new tabs, auth state reuse, responsive viewports.
- Flaky controls: auto-wait, retries with trace, quarantine list.
Mobile (future)
- Capture SDK for iOS / Android and React Native WebViews; same bug schema, device info and screen recording.
- Test runs: Playwright against mobile web / emulated devices first, native later (Appium or equivalent, decided then).
7. Architecture (proposal)
- Platform: Nest-Next on
nest-next-coreand ui-kit, like the other company apps. Postgres, Redis, RE Auth. - Runners: a worker pool running Playwright in Docker (non-root, resource limits, no access to company networks), trace / video / screenshots stored in private object storage with signed links.
- Capture Embed: a small framework-free script served from the platform, loaded by project public key; sends reports to a public API with origin and rate checks.
- Shared libraries (reuse): Media Transcriber (dictation, video speech), LLM Router (all AI calls), Entity Dedupe (duplicate bugs), Suppressor (ignore lists for noisy console errors).
- Existing code to learn from: appdoc (Playwright screenshots in Docker) and block-detector (Playwright scanning); no new browser stack needed.
- Core tables: project, environment, requirement, test_case, test_step, suite, plan, run, run_result, bug, bug_capture (screenshots, video, transcript, context), recording, ai_proposal (status, decided_by), user.
8. Phases
| Phase | Delivers |
|---|---|
| 0 | Decisions on the open questions below; data model and API spec; this subdomain stays the plan page |
| 1 | Capture Embed v1: context, screenshot + annotation, notes, steps, bug list in the platform |
| 2 | Video recording and annotation, voice dictation, redaction; Capture Studio recorder to manual test cases |
| 3 | Test management: stories, cases, suites, manual runs, reports |
| 4 | Playwright runners, scheduled and CI runs, traces, failure to bug |
| 5 | AI: generate cases and scripts, failure analysis, triage, self-healing proposals, approvals inbox |
| 6 | Framed pages and drag-and-drop builder support hardened on the AddieSaaS builders; Jira / GitHub sync |
| 7 | Mobile SDK and mobile runs |
9. Open questions
- What exactly is the current Capture Studio (desktop app, extension, web)? Is it extended or does the new embed replace it?
- Who are the first users: internal team on our own products (AddieSaaS, FEC tools) or customers' sites too? This decides multi-tenant from day one.
- Embed delivery: script tag only, or also a browser extension for sites the owner does not control (needed for third-party frames)?
- Video: browser screen capture (needs user permission each time) is the plan; is a server-side session replay (DOM recording) also wanted?
- Where do bugs ultimately live: only here, or synced to Jira (RESP / MOR projects)?
- Name: "QA Engine" is the working name. Zoho's product is QEngine, so a distinct name is worth choosing before launch.
- AI budget and which model for exploration versus script writing.