✓ green builds
Practical writing on testing AI.
No think-pieces. Code-heavy, opinionated, written by someone who installs this for a living. Every post ends where your next step begins.
-
Setting up Allure reporting that people actually read
A practical Allure reporting setup for pytest and Playwright, install, meaningful annotations, attachments on failure, and trend history wired into CI.
Read → -
How to structure a pytest API testing framework
A clean pytest API testing framework structure, layered fixtures, a thin client wrapper, data builders, markers, and env-based config that scales past a few dozen tests.
Read → -
Manual QA vs automation vs evals: where each belongs in 2026
A clear-eyed map of when to use manual testing, automated tests, and evals, they solve different problems, and using the wrong one is why teams feel slow.
Read → -
Playwright vs Cypress in 2026: which one to actually pick
An opinionated 2026 take on Playwright vs Cypress, architecture, parallelism, language and browser support, API testing, and when each one still wins.
Read → -
What testing regulated gaming platforms for 300+ casinos taught me about shipping AI
Lessons from five years of zero-tolerance QA on a multi-jurisdictional gaming platform, applied to the move-fast world of shipping AI features.
Read → -
QA operating model for a 15-engineer team: the one-pager
A single-page QA operating model, roles, rituals, bug triage, release sign-off, and quality gates, sized for a 15-engineer team that hates process.
Read → -
RAG testing: how to know your retrieval is actually working
How to test a RAG pipeline end to end, retrieval quality separate from generation, grounding and citation checks, and catching silent retrieval rot.
Read → -
The 10 critical-path tests that catch 80% of embarrassing bugs
A concrete starter set of ten end-to-end tests covering the paths most likely to break in public, auth, payments, the core AI flow, and the quiet killers.
Read → -
When should a startup hire its first QA engineer?
An honest framework for when to make your first QA hire, the signals that say now, the ones that say not yet, and what to do in the meantime.
Read → -
Evals are unit tests for your prompts - wire them into CI
Treat evals like unit tests: fast, version-controlled, and blocking on regression. How to structure an eval suite and gate prompt changes in CI.
Read → -
Playwright + GitHub Actions: a CI test pipeline that doesn't flake
A reliable Playwright-on-GitHub-Actions setup, staged gates, parallelism, retries done right, and the flake-killing practices that keep the suite trusted.
Read → -
Your AI feature passed the demo. Here's why it'll fail in production.
The gap between a working AI demo and a reliable AI feature, the specific failure modes that live in that gap, and how to test for each one before users find them.
Read → -
Golden datasets: the unglamorous backbone of LLM testing
How to build and maintain a golden dataset for LLM evals, where the cases come from, how to label them, and how to keep the set honest as your product changes.
Read → -
The QA setup every seed-stage AI startup actually needs (and what to skip)
An opinionated, minimal QA stack for a 5–15 engineer AI startup, what to build now, what to skip, and what to deliberately defer until you're bigger.
Read → -
How I test LLM features that never return the same answer twice
A practical method for testing non-deterministic AI features, scoring against rubrics and golden sets instead of asserting exact strings, then gating it in CI.
Read →
✓ free checklist
Get Green Builds
One practical email a month on testing AI features: evals, CI gates, the stuff in these posts. Start with the free AI Startup QA Checklist.
One email a month. Unsubscribe anytime. Or read it right now.
One bad regression away from a lost week.
Book a 20-minute intro call. I’ll tell you honestly whether I can help and what the right next step is: audit, sprint, or nothing yet.