The 15-minute security check for an AI-built app

Updated 2026-08-27 · MateValidate

You built an app with AI, people are starting to use it, and a quiet question arrives: is any of this leaking?

You do not need a security background for the first pass. AI-built apps fail in a short, repeatable list of ways, and the five below are checkable in about fifteen minutes with nothing but a browser. This is not an audit — it is the difference between "nobody ever looked" and "the obvious doors are shut".

1. Is a secret key shipped to the browser? (3 min)

Open your deployed site, press F12 → Network, reload, click the main JavaScript file (assets/index-….js), and search its content — then also search the page's own view-source — for:

• sk- and sk_live (OpenAI / Stripe secret keys)
• service_role (the Supabase key that bypasses all security)
• the words secret, apiKey, password near long random strings

Anything with real power found here is public. Every visitor can read the bundle. Revoke the key at its provider, move the call behind a backend or edge function, redeploy. The rule of which keys may be public — and which never — is in the env variables guide. (The Supabase anon key is designed to be visible; finding it is fine, if check 2 passes.)

2. Are your tables open to the anon key? (4 min)

With RLS enabled and no policies, reads return empty and you notice. The dangerous state is the opposite: RLS disabled — then your public anon key reads and writes the table for anyone who copies it out of your bundle.

In the Supabase dashboard, open Table Editor and look for the RLS disabled badge on each table; the Security Advisor (Advisors → Security) lists the same and more. Every table an app touches needs RLS on and explicit policies — writing them is covered in the RLS guide. Do not fix a red badge by deleting the table's data access from the app; fix it with policies.

3. Can one user see another user's data? (4 min)

The most common real leak in AI-built apps, and the one that ends up on social media. Test it exactly like this:

1. Sign up as account A in your normal window; create something — a project, a note, an order.
2. Sign up as account B in a private window.
3. As B: does anything of A's appear in lists, searches, or dashboards?
4. As B, paste the direct URL of A's item (/projects/abc123). Does it open?

Step 4 catches the classic miss: lists are filtered per-user, the detail route is not. The fix belongs in policies (auth.uid() = user_id), not in hiding links — a URL that only politeness protects is public.

4. Do admin routes exist, and who can open them? (2 min)

Generated apps love shipping an /admin. Try /admin, /dashboard/admin, /settings/users on your deployed site, signed out and signed in as a normal user. Anything that opens — or flashes real data before bouncing you — is running its check in the frontend only. Access must be refused by the server/database (a role checked in policies or on the API), and the frontend redirect kept only as decoration.

5. Does the password flow behave like one? (2 min)

Three quick behaviors, all visible from the sign-in page:

• Reset works end-to-end: request a reset, receive the email, set a new password, old one stops working. Half-wired resets are endemic in generated apps — the form exists, the email was never configured.
• Wrong email and wrong password answer identically. "No account with that email" hands a stranger your user list one guess at a time.
• The app talks to the site over HTTPS only — check the padlock, and check the Network tab for any http:// request (browsers flag these as mixed content; the CORS guide explains those messages).

Hand what you found to the AI

I ran a basic security pass on my deployed app and found:

[e.g. "sk_live key visible in the JS bundle", "orders table shows RLS
disabled", "account B can open account A's item by URL", "/admin opens for
normal users"]

Fix these in order of severity. Requirements: revoke and rotate any exposed
key and move its call server-side; enable RLS with proper auth.uid()-based
policies rather than hiding UI; enforce admin access server-side; make auth
errors identical for wrong-email and wrong-password. For each fix, tell me
how to verify it from the outside as a stranger would.

What this checklist is not

Fifteen minutes closes the doors that get AI-built apps burned in public. It does not cover dependency vulnerabilities, rate limiting, payment flows or anything an attacker with time and skill would try. When real money or real personal data arrives, pay for a real review — and until then, run this list again after every big generation session, because regenerated code happily reopens old doors. The broader everything-else pass — broken flows, dead buttons, error handling — is the testing guide.