What testing regulated gaming platforms for 300+ casinos taught me about shipping AI
Lessons from five years of zero-tolerance QA on a multi-jurisdictional gaming platform, applied to the move-fast world of shipping AI features.
I spent five years testing real-money gaming systems, a multi-jurisdictional platform serving 300+ casino locations across 7+ jurisdictions, with a 100% compliance record across certifications. Real money moving, regulators watching, zero tolerance for defects. Then I moved into AI products, where the motto is “move fast and break things.” Here’s what carried over, and what didn’t.
”Move fast and break things” is half right
The half nobody quotes is which things. Regulated teams ship constantly, they’re not slow. What they don’t do is break the things that move money or violate a certification. Speed isn’t the enemy of quality; unmanaged speed is. You can move fast everywhere except the few paths where a failure is unbounded, and the skill is knowing which paths those are.
For an AI product, the “real money” equivalents are: an output that misleads a customer, a prompt-injection that exfiltrates data, a cost regression that triples your bill overnight. Move fast on the UI. Don’t move fast on those.
Auditability is the discipline AI teams are missing
In regulated work, “it passed” isn’t enough, you have to be able to reconstruct why it passed, months later, for an auditor. Every check, every sign-off, every result, recorded.
Most AI teams have the opposite: a prompt changed, someone eyeballed a few outputs, it shipped, and there’s no record of what “good” even meant that day. When it regresses, you can’t tell what changed. The fix is the same discipline I used on the gaming platform, version-control the prompts and the rubric, record eval scores per release, and make “safe to ship” a decision with criteria, not a feeling. That’s not bureaucracy; it’s how you debug six weeks later.
Risk-based testing: spend attention where failure is unbounded
You can’t test everything equally, and trying to is how teams burn out and still miss the important thing. On a regulated platform you map every path by blast radius and put your deepest coverage where a miss is catastrophic, financial transactions, regulatory validation, and lighter coverage on the cosmetic edges.
The same map works for AI. The risk is roughly untrusted input x model capability: where can a user (or a retrieved document) influence a prompt, and what can the model then do, read data, call tools, spend money, message a customer? Test the high-capability, high-exposure corners first. A cosmetic tone miss can wait; a tool-call the model shouldn’t have made cannot.
What actually transferred
The technical surface looks different, I went from a Playwright/pytest framework that cut manual cycles 40% and reached 75% automated coverage with 700+ scripts, to evals and prompt regression suites, but the judgment is identical:
- Decide what “good” and “safe” mean, explicitly, before you ship.
- Gate the things that matter; let the rest move fast.
- Make every release reconstructable.
- Turn every incident into a permanent check.
Non-determinism makes AI features feel untestable, so teams skip the discipline entirely. They’re not untestable, they just need the same rigor I learned where being wrong cost real money, applied with a different assertion style. That’s the whole argument of how LLM testing actually works.
The thing regulated work beats into you
Trust is built by the boring parts. Nobody celebrates the release that went out clean because the gates held, but that’s the entire job. The teams that win at AI won’t be the ones that shipped fastest; they’ll be the ones whose features still worked at 2am, for a real user, after a change nobody remembers making.
If you’re shipping AI features and the “is this safe?” call is currently a vibe, that’s exactly the judgment I install. Start with a QA Readiness Audit for a straight read on where the unbounded risks actually are.