Essay

Why fraud models should only propose. By Amir Shachar.

Read the essay
Skip to content
Riskernel

Prove your AI agents and models are safe

Nobody can say, in money terms, how bad a hijacked AI agent could get.

Banks put a limit on almost every exposure: credit, payments, daily payouts. The AI agent that issues refunds or changes account details is often the one actor whose worst day has no number. Saying it has guardrails describes the lock, not the size of the room behind it.

Built for Teams whose agents hold tools that move money or change records; CISO, CRO, Head of AI Risk, CTO at smaller firms

The problem in numbers

What the public record shows

Every figure links to the page it came from, with its date.

£450.7 million

lost to authorised push payment fraud in the UK in 2024: payments the account holder was tricked into approving themselves

ukfinance.org.uk, 2025-05

40 percent

more than this share of agentic AI projects Gartner expects to be cancelled by the end of 2027, with weak risk controls among the reasons

theregister.com, 2025-06

650.88

Canadian dollars in damages Air Canada had to pay for its chatbot's wrong advice: the tribunal held that the firm owns what its bot does

dentonsdata.com, 2024-02

What you get

The deliverables

You get a signed, independent statement of the worst-case loss, built on deterministic checks of the agent's tool scopes, limits and approval steps, with the assumptions that bound it written out. The fix is not sold in the same engagement, so the number stays independent.

A signed worst-case loss statement

One money figure for the most an agent could cause if fully hijacked, with the reasoning behind it, signed by Amir. It states its assumptions, for example that every allowed call is made at its limit until a person or control would stop it, so a board reader knows how the number was bounded.

A map of what the agent can touch

Every tool, the identity it runs as, each hard limit and each approval step, taken from code and configuration rather than design documents.

The paths that raise the number

The specific chains and gaps, such as split payments or one tool changing a destination while another moves money, that drive the worst case.

How it runs

1 to 3 weeks, set by how many tools the agent holds, start to finish

1 to 3 weeks, set by how many tools the agent holds

  1. Scope the agent
  2. Deterministic checks
  3. Statement and readout
Step 1

Scope the agent

We agree which agent is covered and you share its tool list, permissions, limits and approval rules.

Step 2

Deterministic checks

Amir checks each tool scope, limit and approval step, and the combinations of allowed calls that get around them.

Step 3

Statement and readout

You receive the signed statement and walk through it with your risk and engineering leads.

Consent. This runs only with written consent from your company before anything starts. It is a read-only review of tool definitions, permissions and limits: no customer data is needed and nothing is sent to anyone outside the agreed scope.

Method

How the answer is produced

Assume the model will slip

The statement does not depend on whether the model resists tricks. It asks what the agent could do once it does the wrong thing.

Only real limits count

A limit written in the prompt does not count. Only limits enforced in the downstream system go into the figure.

Independent by design

The fix is not sold in the same engagement, so nobody has a reason to make the number bigger or smaller than it is.

Arithmetic you can check

For each path, the figure is the per-call limit, times how many calls the agent can make per hour, times the hours until a person or control would notice and stop it. For example, a 500 refund limit, 60 calls an hour and 4 hours to detection gives 120,000. The statement shows each of the three inputs and where it came from.

Amir Shachar
Amir ShacharFounder, Riskernel

Why Amir

Designed and led by someone who has built fraud models in production

The patents closest to this problem:

Amir is a named inventor on EP 4744195 A2 (Methods and systems for threat assessment based on expected pathways of malicious actors, filed at Skyhawk Security; pending).

It rates a threat by the routes an attacker would take toward valuable assets, the same way the statement traces tool chains toward money. The statement is not an implementation of the patent.

View on Google Patents

Amir is a named inventor on US 12,634,298 B2 (Security posture management methods and systems using threat detection and response data, filed at Skyhawk Security; granted).

A related idea: it ranks weak settings by how they show up in real threats, the same way the statement picks which agent permissions drive the worst case. The statement is not an implementation of the patent.

View on Google Patents

These patents were filed at the companies named. They show the problems Amir has worked on; they are not Riskernel products.

Is this you

You probably need this if

  • Your agent can issue refunds, release payments, change payout details or call internal APIs.
  • A risk committee or board has asked how much an agent could lose, and the answer was a list of controls.
  • Some of the agent's limits live only in its prompt, not in the system it calls.
  • Each action is capped, but nothing caps how many actions happen per hour.
  • The agent runs under a service account built for an older process.

Questions

Before you reach out

Do you try to break the model?

No. The statement assumes the model can be tricked and checks what the agent could do afterwards: its tools, limits and approval steps.

What access do you need?

Read access to the agent's tool definitions, permissions, limits and approval rules, or exports of them. No customer data is needed.

Will you fix what you find?

Not in this engagement. Keeping the fix separate keeps the number independent. Your team, or anyone you choose, can work from the findings.

Who reads the statement?

It is written for a risk committee or board: one figure, the reasons behind it and the gaps that drive it.

How is the money figure calculated?

Per-call limit times the rate of calls the system allows times the time to detection, for each path that reaches money. The largest result, after hard caps such as daily limits, is the figure. Every input is shown so a board reader can redo the sum.