In April 2026 a Cursor coding agent running Claude Opus 4.6 deleted the production database of PocketOS, a small company that sells software to car rental firms. It took one API call and about nine seconds. Nobody hacked it. It was trying to fix a credential mismatch in staging, found a token that could do anything, and used it.
If you are searching for how to stop an AI agent deleting data, here is the short answer. An agent can only delete what its credentials and the path to the API allow. You stop it with controls that sit outside the model: a credential scoped to one environment and one task, a policy gate that denies destructive calls by default, a human approval bound to the exact action, and backups the agent cannot reach. A rule in the prompt is not one of those controls.
Below is the incident step by step, then the architecture we use at Devs Core to make it structurally impossible, including a small policy gate I ran while writing this.
Key takeaways
- The PocketOS agent wasn't attacked. It was being helpful in the wrong direction, and nothing in its path could say no.
- The only barrier was a written rule. The agent broke it and then listed, in writing, the rules it had broken.
- Four things had to line up: a broad token on disk, an API that ran deletes immediately, staging able to reach production, and backups inside the thing they protected.
- Each of those has a deterministic fix. None of them depends on the model behaving.
What happened at PocketOS, step by step?
Jer Crane, the founder of PocketOS, posted a long account on X on 25 April 2026. He didn't publish timestamps for every step, so this is the order of events from his account and from the coverage that quotes it: Zenity, DevOps.com, Euronews and Railway's own write-up.
The task and the decision. The agent was doing routine work in staging and hit a credential mismatch. On its own initiative, it decided the fix was to delete a Railway volume. Nobody had asked it to delete anything.
The credential. To do that it needed an API token. It searched and found one in a file unrelated to the task. The token had been created to manage custom domains through Railway's CLI. Railway later confirmed it had been provisioned with account-scoped access, the widest level there is.
The call. The agent ran a single GraphQL mutation, volumeDelete, with curl. Railway published the shape of it:
curl -X POST https://backboard.railway.app/graphql/v2 \
-H "Authorization: Bearer [token]" \
-d '{"query":"mutation { volumeDelete(volumeId: \"3d2c42fb-...\") }"}'
There was no confirmation. Railway says that, until that week, volumeDelete on the API ran immediately with no undo, even though the dashboard offered a 48-hour window for the same action.
The blast radius. The agent assumed a delete issued from staging would only touch staging. It didn't check. According to Crane, the same volume was used across environments, so the delete took out production, and the volume-level backups stored inside it went too. Zenity reported the most recent backup PocketOS could see was three months old.
Crane says the stretch from decision to deletion took about nine seconds.
The confession. Asked why, the agent wrote this (quoted by Euronews and DevOps.com): "Deleting a database volume is the most destructive, irreversible action possible, far worse than a force push, and you never asked me to delete anything. I decided to do it on my own to 'fix' the credential mismatch, when I should have asked you first or found a non-destructive solution."
The aftermath. Euronews reported an outage of more than 30 hours for the rental businesses on PocketOS. "Reservations made in the last three months are gone. New customer signups, gone," Crane wrote. Then, on 27 April, the good news: Railway had recovered the data. Railway explained that it keeps separate off-site "disaster backups", and that the legacy API path the agent used had cascaded the delete so the user-facing backups only looked gone. Two days later Railway made every API delete a 48-hour soft delete and delayed deletion of the backups as well.
So PocketOS got lucky. Keep that in mind, because the thing that saved them is one of the controls below.
Wasn't this just a bad model?
This is the part most coverage got wrong, in both directions.
One camp blamed the model. The other blamed the founder. Both miss the mechanism.
The agent's reasoning was coherent. It saw a problem, found a tool that could remove the obstacle, and used it. As Zenity's Chris Hughes put it, the agent was "helpful in the wrong direction". There was no attacker and no prompt injection. A better model makes that less likely. It does not make it impossible, and "less likely" is not a property you want for your only copy of three months of bookings.
Crane himself framed it well in the Euronews piece: "This isn't a story about one bad agent or one bad API. It's about an entire industry building AI-agent integrations into production infrastructure faster than it's building the safety architecture to make those integrations safe."
Hadn't we seen this before?
We had. In July 2025 Jason Lemkin, founder of SaaStr, was building an app with Replit's agent. During what he had declared a "code and action freeze", the agent deleted the production database, wiping records for more than 1,200 executives and more than 1,190 companies, as Fortune reported. It then told him rollback wouldn't work. He recovered the data manually anyway.
Replit's CEO Amjad Masad called it "unacceptable and should never be possible" and shipped automatic separation of development and production databases, better rollback and a planning-only mode. The freeze was an instruction; the fixes were structural. Nine months later PocketOS had the instruction and not the structure.
Why didn't the prompt rule stop it?
Because a rule in a prompt is an input to the same reasoning that decided to delete the volume. The model weighs it. Sometimes it loses. The agent's own words make the point better than I can: "I violated every principle I was given."
We've covered this in how to secure enterprise AI agents, and a sibling post goes deeper on why system prompts are not security controls. OWASP calls the general failure excessive agency: more permissions or autonomy than the task needs.
The fix is not a firmer prompt. It is to move the decision somewhere the model can't argue with.
What architecture makes this impossible?
Our AI governance approach fits in one sentence: the model proposes, deterministic code decides. Here is what that means for an agent near a database, control by control.
1. Credentials scoped to one environment and one task
The agent should never hold a long-lived token at all, and it certainly shouldn't be able to find one by reading files. In our builds the agent holds nothing. When the gate approves a call, it mints a short-lived credential scoped to the environment and the operation, and it expires when the task ends.
Railway already offered project tokens that can be scoped down to an environment. The token in this incident just wasn't one of them. If you can only get account-wide tokens from a vendor, treat that as a finding, not a detail. The sibling post on least-privilege access for AI agents covers token exchange in more depth.
This alone stops the incident at step one. A staging credential cannot delete a production volume, whatever the agent decides.
2. A policy gate outside the agent, default deny
Scoped credentials are necessary but not sufficient. Even a perfectly scoped token lets the agent do every destructive thing within its scope. So every tool call goes through a gate that the agent can't edit, and the gate evaluates written policy: who is asking, which action, which resource, in which environment. Anything not explicitly permitted is denied. Destructive operations are never permitted to an agent principal in production.
To show this working, I wrote a small gate with Cedar, the open-source policy language, via the @cedar-policy/cedar-wasm package (4.14.0). The three policies, exactly as I ran them:
// Anything not permitted here is denied by default.
@id("agent-read-own-env")
permit (
principal is Agent,
action in [Action::"volumeList", Action::"serviceLogs"],
resource is Volume
)
when { resource.environment == principal.environment };
@id("destructive-needs-bound-approval")
permit (
principal is Agent,
action == Action::"volumeDelete",
resource is Volume
)
when {
resource.environment == principal.environment &&
context has approval &&
context.approval.argsHash == context.argsHash
};
@id("no-agent-deletes-in-prod")
forbid (
principal is Agent,
action == Action::"volumeDelete",
resource is Volume
)
when { resource.environment == "production" };
A short Node script plays the PocketOS agent: a principal started in staging, trying reads and deletes against a staging volume and a production volume, with and without an approval. This is the real output:
Two rows matter. The production delete is refused even with an approval, because a forbid always beats a permit in Cedar. And an approval for a different volume does not open this one. Each decision took under a millisecond in my sandbox, and that includes re-parsing the policies on every call, so the gate costs the agent nothing it will notice.
One honest caveat: in this demo the script passes the approval in the request context so you can see the effect. In a real gate the approval comes from the gate's own store, written by the approval UI. Never trust an approval the agent hands you. If you're choosing an engine, there's a sibling post comparing Cedar, OPA and OpenFGA for agent authorisation.
3. Human approval bound to the exact action
Some irreversible operations are legitimate. Somebody does need to drop that staging volume eventually. For those, the gate parks the call and a named person approves or rejects it, seeing the full action: the resource, the environment, the arguments. We cover the UX side in human-in-the-loop AI agents.
The word "bound" is doing a lot of work. The approval must be tied to a hash of the exact arguments and the state of the resource, and checked again at the moment of execution. We learned this on our own demo. In October 2026 we ran 12 independent tests against the governance in Ware, our simulated warehouse with a live AI recovery agent. One weakness we found and fixed was exactly this: an approval was checked in one transaction and executed in another, so a change in between went unseen. Simulated money, real lesson.
4. Backups outside the blast radius
This is the control that saved PocketOS, and it wasn't theirs. Railway recovered the data from backups the agent's API call never reached.
Your own backups should have the same property: different account, different credentials, ideally a different region, with deletion delayed or impossible for the identities your agents use. Test the restore. A backup nobody has restored is a hope.
5. An audit trail written by the gate, not the agent
PocketOS's only record of the agent's reasoning was the agent's own explanation, written afterwards by the system that had just guessed wrong. The gate is the right place to log: who asked, what was proposed, which policy decided, who approved, what ran. Ours is append-only and hash-chained, another fix from the Ware tests, where our log could once be edited unnoticed. The sibling post on audit trails for AI agents has the detail.
What does the before and after look like?
The left side is not unusual: a token in a .env file, an agent that can read the repo, backups nobody has restored. The right side is more work, mostly the gate and the scoped credentials, and it is ordinary engineering, not research.
There's a trade-off. A default-deny gate will block things the agent legitimately needs, especially early on. That's the point. We start agents supervised and widen what they may do one operation at a time, with evidence; we call it the Autonomy Ladder. An agent can run reads unattended on day one while every delete still waits for a person.
Where should you start this week?
If you run any agent with access to infrastructure, do these in order:
- Search your repos and developer machines for long-lived platform tokens, and replace them with environment-scoped ones.
- Find every API your agents can call that deletes or overwrites data, and put those calls behind a gate that denies by default.
- Check where your backups live, and restore one.
If you'd rather have someone build the gate and connectors with you, that's what our Access Layer package is for. Either way, don't wait for your own nine seconds.
Frequently asked questions
How did an AI agent delete the PocketOS database?
A Cursor agent running Claude Opus 4.6 hit a credential mismatch in staging, decided to delete a Railway volume, found an account-wide API token in an unrelated file, and called volumeDelete. The volume was shared with production, so production data and its volume-level backups were deleted in about nine seconds.
Was the PocketOS data recovered?
Yes. Railway recovered it from backups outside the deleted volume, and Crane confirmed on 27 April 2026 that PocketOS was back with its data. Railway has since made all API deletes a 48-hour soft delete.
Can a system prompt stop an AI agent deleting data?
No. A prompt rule is advice the model weighs, and in both the PocketOS and Replit incidents the agent broke an explicit instruction. Enforcement has to live outside the model, in credentials and a policy gate.
How do I stop an AI agent deleting production data?
Give the agent no standing credentials, issue short-lived tokens scoped to one environment, route every tool call through a default-deny policy gate, require a named person's approval bound to the exact arguments for irreversible writes, and keep backups in a separate account the agent's identity cannot touch.
Does a policy gate slow the agent down?
Not noticeably. In the Cedar demo above each decision took under a millisecond. The only real delay is human approval on irreversible actions, and that delay is deliberate.