Essay

Why fraud models should only propose. By Amir Shachar.

Read the essay
Skip to content
Riskernel

Algogens

Algogens: models only propose.

The accuracy of algorithms plus the intelligence of generative AI.

Exhibit A · Comparison

Algogens vs agent-in-the-loop

Most AI in fraud operations today lets a model act and checks it afterwards: a person signs off, a log records it, a second model grades it. Algogens reverse the order. The model only proposes, and a deterministic validator decides what runs before anything executes. The table compares the two patterns as designs, not as products.

Algogens vs agent-in-the-loop
Agent in the loop (typical)Riskernel Algogens
Who proposesThe model, which also actsThe model, which can only propose
Who verifiesA person after the fact, or another modelA deterministic validator, before anything runs
What runs before executionNothing, or a sign-off queueOn AI actions: schema integrity, telemetry grounding, regulatory policy consistency, blast-radius solvability. On generated fraud rules: future information leaking in, spurious patterns, proxies for protected attributes, and how many past decisions the rule would change. A failed proposal goes to bounded repair or is rejected.
What is loggedThe agent’s output and actionsEvery proposal, every validator verdict, every repair
What a regulator can replayThe outcome, and the agent’s account of itThe full decision path, step by step
What happens when a proposal failsIt may already have run; a reviewer or a rollback catches itBounded repair, or rejection. It never runs as proposed
Where a person sitsInside each decision, or reviewing samples after the factSetting the checks and budgets in advance, and reviewing the queue the engine routes to them

Scope: the Algogens column describes flows where a validator really runs. Where a flow has no validator in front of it, Riskernel does not claim this column for it.

Exhibit B · Validation

Every proposal is checked before it acts.

There are two sets of checks, and each one covers a different thing. Neither uses a model to judge.

Checks on AI actions

Four stages before an action runs

  1. Schema integrity. The proposed action is well formed and uses only allowed fields.
  2. Telemetry grounding. The action is backed by signals actually seen on the flow.
  3. Regulatory policy consistency. The action fits the policy that applies to it.
  4. Blast-radius solvability. The action’s reach stays within what can be handled.

A failed proposal gets a bounded repair, or it is rejected. It does not run as proposed.

Checks on generated fraud rules

Four questions before a rule is admitted

  1. Future information. Does the rule use information that was not available at decision time?
  2. Spurious patterns. Is the pattern real, or an accident of the data?
  3. Protected-attribute proxies. Is the rule acting as a stand-in for a protected attribute?
  4. Decision-change count. How many past decisions would the rule change?

These are the checks described in the Algogens paper, which reports synthetic tests only.

Exhibit C · Purple Algogen Not shipped

The Purple Algogen: a purple team that learns across clients

At Skyhawk Security, Amir was one of the leaders of an AI purple team: red agents attacked, blue agents defended, and an AI overseeing both kept improving each side. Riskernel takes the idea into fraud and adds a level. The Purple Algogen learns from every client which attacks get through and which defenses hold, as patterns and outcomes, never records, and uses them to train each client’s red and blue agents. Because it is an Algogen, nothing it proposes reaches a client until deterministic checks pass it.

  1. 1 Propose

    Red agents invent attacks on your flows. Blue agents propose the rule, feature or threshold that stops each one.

  2. 2 Validate

    Each change replays on your own history and must pass the four fixed checks (schema, grounding, policy, blast radius). If it fails, it gets a bounded repair or is dropped.

  3. 3 Learn across clients

    Pass and fail verdicts, never records, train the Purple Algogen, so each client’s agents start from what held up everywhere else.

The Purple Algogen is a design Riskernel is building. It is not shipped. The checks above describe the Algogens design it would use.

Essay · Algogens

Why fraud models should only propose

For years my job was to put fraud models into production at large banks. The hard part was never getting a model to score well on a test set. The hard part was the meeting afterwards, when someone from model risk asked a simple question: "Show me why it did that, and show me it will do the same thing tomorrow."

That question is about to get much harder to answer. Generative AI is moving into fraud and payments operations quickly. It writes rules, drafts alert narratives, maps new data feeds and, more and more, takes actions: it freezes an account, holds a payout, closes an alert. The way most teams are wiring it in has a common shape, and I think the shape is wrong for this domain. Not wrong for every domain. Wrong for money.

The pattern most teams are building

Call it agent-in-the-loop. A model, often an agent with tools, looks at a case and acts. Around it sit the safety measures: a person who reviews a sample of decisions, or signs off on certain classes of action; detailed logs of what the agent saw and did; a kill switch if something goes badly; and increasingly a second model that grades the first one’s work.

I want to be fair to this pattern, because the people building it are not careless. Human review catches real mistakes. Logs matter. A kill switch is better than no kill switch. A second model can spot things a rule never would. In customer support, in research, in drafting, this is a reasonable design. If the agent gets something wrong, a person fixes it and little is lost.

The trouble is the order of operations. The model decides, the action runs, and the checking happens afterwards, or in parallel, or on a sample. Every one of those safety measures is about finding out. None of them is about preventing.

Why the order matters in fraud and payments

Three things make fraud different from most places agents are being deployed.

First, actions are hard to reverse. A payout that left is gone. A legitimate customer whose account was frozen on a Friday night does not experience the Monday correction as a fix; they experience it as the weekend their rent bounced. "We reviewed it later and reversed it" is a real cost, not a footnote.

Second, the adversary reads your system. Fraud is the one place where someone is actively probing your decision logic for the cheapest way through. A model that reads free text (a memo field, an invoice, a chat message) is a model that an attacker can write to. A reviewer who checks a sample after the fact is reviewing the cases the attacker chose to let you see.

Third, you will be asked to explain. Regulators and auditors do not want to know what the system usually does. They want to know what it did on this account, on this date, and why. "The model judged it, and a second model agreed" is not an answer a model-risk team can defend. It is two opinions where they asked for a reason.

Put those together and the agent-in-the-loop order looks backwards. You want the checking to happen before anything executes, not after.

Models propose. A validator decides what runs.

This is the idea behind Algogens, which I first set out in a 2024 paper (arXiv:2403.01426) and which Riskernel’s engine is built on. The short version: the accuracy of algorithms plus the intelligence of generative AI, with each doing only the part it is good at. The method Riskernel’s engine uses is patent pending (US provisional application 64/197,364, filed in November 2026).

The generative model is kept out of the decision path. It works offline, where it does what it is genuinely good at: proposing. It suggests a new feature, a new rule, a mapping for a data feed nobody has seen before. That is creative work, and generative models are good at it.

Every proposal then goes through a set of deterministic validators. Deterministic means the same input always gets the same verdict; there is no model inside the check and no sampling. There are two sets, and they check different things. The first set checks the fraud rules and features a model generates, and it is the one in the paper. Does this feature use information that would not have been available at decision time? Is this pattern spurious? Is it acting as a proxy for a protected attribute? And how many past decisions would this rule change if we turned it on? The second set checks the actions an AI proposes to take, such as freezing an account: schema integrity, telemetry grounding, regulatory policy consistency and blast-radius solvability. Neither set uses a model to judge.

On generated rules, that last check is the one I find operators understand immediately. Before a rule is admitted, it is replayed against history, and the share of past decisions it would have changed is measured against a fixed budget. A rule that would have flipped too many decisions is not simply accepted or refused.

Repair within bounds, or reject

This is the part that separates the approach from a plain approve or deny gate. When a proposal fails, there are two outcomes, and only two.

It can be repaired within bounds. If a proposed rule wants to block, but blocking would change too many past decisions, it is cut back to the strongest milder action that fits the budget: a hold or a review instead of a block. The repair itself is deterministic, so the same failed proposal always gets the same repair.

Or it is rejected. If no action above "allow" fits the budget, the proposal is dropped. A rule whose false-positive rate is too high is dropped. There is no third path where the model argues its case, and no path where a failed proposal runs "just this once".

Only proposals that pass are admitted. Nothing that failed runs as proposed.

What this buys a CRO

A chief risk officer, in my experience, wants to know three things: what can this system do, what has it done, and what is the worst it can do.

With Algogens the first answer is short. The system can only take actions from a fixed list, and only through rules and features that were checked before they were loaded. The model’s creativity is real, but it lives upstream of a gate it cannot talk its way through.

The second answer is a log, but a different kind of log. Not just the agent’s output, but every proposal, every validator verdict and every repair. You can see the rules that were rejected, not only the ones that won.

The third answer is a bound. Each admitted rule’s effect on past decisions was measured and capped at admission. That is a number someone chose in advance, written down, not a hope.

What it buys a regulator

Replay. A reviewer can take a decision from last quarter and walk the path step by step: which checks it passed, whether it was repaired, which action followed.

That is the difference between showing an outcome and showing a decision path. Agent-in-the-loop systems can usually show the outcome and the agent’s account of itself. An Algogen can show the path.

What it does not buy

I would rather say this myself than have someone find it. A validator only checks what it was written to check. The paper reports synthetic tests only, and I would not claim more for the design than that. A deterministic gate does not make a system correct. It makes the system’s behavior bounded, inspectable and repeatable, which is the precondition for making it correct and for proving it.

That is also why the architecture matters more than any single model. Models will keep getting better, and they will keep being wrong in new ways. The question is not whether your model is good enough to trust. It is whether your system is built so that it does not need to be.

Models should propose. Something you can read, test and replay should decide what runs.

Amir Shachar

See how this would run on your own flows.

Runs beside your current system. Nothing to rip out.