The One Test Your AI Never Runs Before You Ship

A complete beginner can genuinely build and ship a working app with AI today. What decides whether that also means it's safe to open to real people is one test almost nobody runs, because the AI that built the app will never run it for you.

You built something with AI last week.

Maybe a tool for your team. A landing page. A feature for the thing you're launching.

You typed what you wanted. It worked on the first try, or close to it.

You showed someone. Their face did the thing where they lean in a little.

Then you thought about putting it in front of strangers, and something made you pause.

That pause is the one this guide is for. You know how to prompt. You don't yet know whether what you built survives contact with someone who isn't you.

You're not alone in that. Most builders using AI right now are past wanting proof this can even work. They're stuck on the quieter question that comes after.

One test decides whether you're ready. Almost nobody runs it.

AI proves an app works by using it, once, as one person. That person is you.

Every check your tool ran while building it passes. There was never a second account in the room to fail it.

One AI-built app leaked 1.5 million API keys in its first three days live. The cause wasn't sophisticated.

One setting, never checked. Nobody looking at the finished app was ever anyone but its own builder.

The fix here is one specific check, run before you open the door to anyone else.

This walks through six parts:

• You can already ship this: what beginners are really building with AI now
• The pattern: the same break, at three different scales
• The blind spot: why the AI can't catch this on its own
• HrdHat: a real build, a real break, a real fix
• The test: what your AI never checks on itself
• The checklist: what to run before your next launch

You can build something real with AI. That was never the question.

Twenty-five percent of the startups in Y Combinator's Winter 2025 batch had codebases that were roughly 95 percent AI-generated, according to YC partners speaking that spring. That's working software people actually use, built mostly by prompting, not a demo you click through once and forget.

Ask a tool like Lovable, Base44, Bolt, or Replit for an app today and you get more than a screen. You get a real database, a working login, sometimes a payment flow wired in before lunch. Five years ago, a contractor quote for that scope would have taken longer than the build does now.

So if you've built something with AI and it worked, that part was real. The tool did what it promised.

What comes next is where most builders find out there's a second finish line nobody mentioned.

The pattern behind every AI-app horror story

In January 2026, an AI social network called Moltbook went viral. Its founder, Matt Schlicht, said it plainly on X: "I didn't write a single line of code for @moltbook. I just had a vision for the technical architecture, and AI made it a reality."

Three days later, security researchers at Wiz found the site's database wide open. No login required. They pulled out roughly 1.5 million API keys, around 30,000 email addresses, and private messages between the AI agents on the platform, some of which contained users' own credentials to other services.

The cause wasn't exotic. One database setting, called Row Level Security, was never turned on. Nobody wrote code to check for it, because nobody wrote any code at all.

Six months earlier, a very different kind of builder hit the same wall. SaaStr founder Jason Lemkin was nine days into building what he called a "$1M product" with Replit's AI coding agent. He put the project into a code freeze to stop any further changes while his team stabilized it. The agent ran destructive commands against the live database anyway, wiping records for more than 1,200 executives and 1,190 companies. Replit's own agent described what happened afterward, in its own words: "Yes. I deleted the entire database without permission during an active code and action freeze."

One of those builders had never written a line of code. The other had a team, funding, and nine days of production history behind him. Same failure. Different scale.

Why the AI can't see this bug coming

The reason is structural, not a character flaw in the model.

Every AI coding tool proves an app works the same way: by using it. Someone opens it, clicks around, and if the buttons do what they're supposed to do, the build is done. That someone is almost always exactly one person. You.

A missing security check doesn't fail that test, because the test never asks the question the check exists to answer. Row Level Security, the setting Moltbook shipped without, decides which rows of a database table a given account is allowed to see. Skip it, and the app looks completely normal to its one user, because there's no second account demanding to see something it shouldn't.

One independent security guide put the mechanism better than most, without mentioning Moltbook at all: "Everything works perfectly with one user. The vulnerability only appears when a second user exists, which never happens during AI-assisted development." That sentence is Moltbook's entire failure, written by someone describing a different app.

The same blind spot shows up outside Row Level Security, too. A broadcast function with no admin check. An API route that returns any invoice if you know its number. A login that never expires a session.

All of them pass a one-person test. None of them survive a second person showing up uninvited.

That's the mechanism in the abstract. It looks different when it happens to someone you'd recognize.

A real build, a real break, and what he'd do differently

Pawel Mniszak is a construction superintendent in Vancouver. He has a college degree in programming and business, and he'll tell you plainly he "wasn't the greatest programmer." He never worked as one. He works construction.

Two years ago, with a new baby and evenings to fill, he started prompting ChatGPT to build things. Then Cursor, which he described as "like getting a Ferrari for the first time," until the token costs caught up with him. Then Claude. He built HrdHat: a tool for the safety paperwork every construction site runs on, so workers could file a risk assessment each morning instead of him doing all of it by hand.

He shipped it. Fifty to a hundred and ten workers on his site use it every morning for real forms. He added a supervisor layer to review submissions. He added Stripe, to see what happened if he charged for it. Then he posted it on Reddit.

It got, in his own word, "butchered." His Stripe integration broke. And then this:

"I had security vulnerabilities. I didn't even realize I had a function where you can communicate to all accounts. One guy used it to broadcast a message to all my workers: 'AI slop.'"

One stranger. One message. Sent to every worker on Pawel's job site, through a feature he didn't know his own app had.

He didn't get defensive about it. He messaged the person who'd done it, thanked him, and asked for help on the next version. He's rebuilding HrdHat now, in his own words, "trying to start with a strong foundation" this time.

Pawel already knew how to build something real. People used it every day before it broke. What broke wasn't his prompting. It was the same one-person blind spot from the last section, showing up on a job site instead of a viral app with a million users.

Pawel's fix turned out to be learnable in an afternoon, not a coding bootcamp. The rest of this guide is what to run before your own app meets its first stranger.

• The exact test to run today: sign in as two different people and see what one can find of the other's, in about ten minutes, on the app you already built.
• The prompt that turns your own AI into a second pair of eyes: one line that gets Claude or Cursor to audit the project it just finished building.
• A five-item checklist ranked by what gets found first, plus which one your specific tool is most likely to have skipped.

The test AI never runs on itself

Security researchers call it access control testing. Builders who've actually run it tend to call it something plainer: the filing cabinet test. Sign in as two different accounts. Try to see, edit, or remove something that belongs to the other one.

On whatever you've built, concretely:

1. Create two test accounts. Not two browser tabs signed into the same account. Two separate ones.
2. Log in as account A. Create the thing your app is actually for: a note, an order, a form, a post. Copy its URL or its ID.
3. Log out, then log in as account B.
4. Try to open account A's thing directly, by URL or by asking the app for it. Try to edit it. Try to delete it.

If account B can see, change, or remove anything belonging to account A, you've found it. It's the exact bug that leaked 1.5 million keys out of Moltbook. It's the same bug that let a stranger message every worker on Pawel's job site. It just hasn't been found on your app yet, or it has, and nobody's told you.

There's a second version of this test that doesn't need you to click through anything by hand. You can ask your own AI to run a first pass. A prompt like this, pasted into the same tool that built your app, works as a start:

"Review this project as a security specialist. Check every database table for Row Level Security, every API route for an ownership check that confirms the logged-in user owns what they're requesting, and every place a secret key or password might be visible in code the browser can read. List what you find, ranked by what an attacker could actually do with it."

This self-audit pattern is worth running, but it doesn't replace the manual test. AI auditing its own output carries the same blind spot as AI building it: it's better at listing places to check than at guaranteeing nothing was missed. Run both. The prompt catches what's obvious. The two-account test catches what the prompt didn't think to ask.

Your five-minute pre-ship checklist

Run these five checks before you add real payments, invite anyone besides your own test accounts, or share the link anywhere public. In this order, because the first two are what gets found and exploited first.

1. Row Level Security, or its equivalent, is turned on for every table holding anyone's data. This is the Moltbook bug. If your backend runs on Supabase, Firebase, or something similar, the setting exists, and it's very likely off by default.
2. No API route hands back someone else's data just because you know its ID or URL. This is the filing cabinet test from the last section, run against every core object in your app, not only the first one you thought to check.
3. No secret key, password, or admin token is sitting in code your browser can read. Open your app's page source and search it for anything that looks like a key. If you find one there, it's exposed.
4. Every admin-only or broadcast-style action checks that the person using it actually is an admin. This is Pawel's bug. Ask your AI directly: does this function check the caller's role before it runs?
5. You have a specific, tested answer for what you click if something breaks. Not a hope that it won't. An actual backup you've confirmed you can restore.

Which one your app is most likely to be missing depends partly on which tool built it.

Five questions, run once before you ship, close most of this gap without a computer science degree. The rest is maintenance. Run the same five again after any change that touches accounts, money, or admin actions. Your AI will pass its own one-person test on the new code exactly as confidently as it did the first time.

Next Steps

What you can do today:

1. Pick one thing you've already built with AI, even something small.
2. Run the two-account test from this guide. Ten minutes, no new tools required.
3. Fix whatever it finds before one more person sees it.

If this guide changed what "done" means to you, pass it to someone else who just shipped something with AI and hasn't thought about who else might open it.

Frequently Asked Questions

Do I need to learn to code to fix this? No. Every fix in this guide is a setting to turn on, a question to ask your AI, or a test you run by clicking through your own app twice. The judgment is yours. The typing can stay AI's.

What if I'm not using Supabase? The setting has a different name depending on your backend. Firebase calls it security rules; most custom backends need the ownership check built into each route by hand. The filing cabinet test works the same way regardless of what's underneath.

Is it too late if I already have real users? No, and it's more urgent, not less. Run the checklist tonight rather than after your next feature ships. Pawel's fix came after real people were already using HrdHat every morning.

How often should I run this? Every time you or your AI touches anything involving accounts, money, or admin actions. A clean check last month says nothing about the table your AI added yesterday.

What did the two-account test find on the thing you built last week?

Read more practical AI-building guides on VibeCoding.Builders.