Prove your AI agents and models are safe
Your LLM may already have seen the test.
Large language models (LLMs, the ChatGPT-style AI now used in fraud and AML work) learn from huge amounts of public text, and public benchmarks, the standard tests vendors quote scores on, are public text too. A high vendor score can mean the model memorised the exam, not that it can handle your cases. The fix is held-out testing: cases the model has never seen. A validator signing off without that has little to show an examiner who asks how the model was tested. The same gap hits lenders whose credit and fraud models run for partner banks and credit unions: each new partner brings customers the model has never seen, and the partner's review asks how it was tested on them.
Built for Bank model-risk teams validating an LLM or fraud model before go-live; Head of Model Risk, Head of Data Science, Head of Credit Risk
The problem in numbers
What the public record shows
Every figure links to the page it came from, with its date.
8%
accuracy drop, up to, when leading LLMs took a fresh, never-published set of grade school maths problems instead of the public benchmark they are usually scored on
arxiv.org, 2024-05
65%
largest performance drop across leading LLMs when one irrelevant sentence was added to a maths question, the kind of trick case this exam uses
arxiv.org, 2024-10
8-18%
of the HumanEval coding benchmark found overlapping with public pre-training sets such as RedPajama-Data-1T and StarCoder-Data
arxiv.org, 2023-11
What you get
The deliverables
You get held-out test packs (fresh cases the model has never seen) generated from templates, with the right answer computed by code wherever it can be calculated, plus trick cases that look for blind spots and exploitable edges, a written test method signed by its author, and, for lenders whose models serve several partner institutions, a per-partner evidence pack.
Held-out test packs
Fresh test cases drawn each run from templates, so the model cannot have memorised them. Where the right answer can be calculated, code computes it.
Trick cases
Near-identical variants that should not change the answer, and variants built to push the model wrong, to find blind spots and exploitable edges.
A signed written test method
A document signed by its author, explaining what was tested, how cases were made and how scoring works, ready to attach to your own validation review. It is evidence for your validators, not a regulatory validation or approval.
A per-partner evidence pack
For lenders whose models run for several banks or credit unions: an out-of-sample test of the credit and first-party fraud models on generated cases shaped like each new partner's base (built from the segments and ranges you describe, not from customer records), with the method written up for that partner's third-party and model-risk review.
How it runs
Agreed on a short call, set by the model and the use case, start to finish
Agreed on a short call, set by the model and the use case
Scope the use case
We agree what the model is for and which rules, thresholds and outcomes it must get right.
Build and run the packs
Amir writes the templates, generates fresh cases and probes, and runs them against the model.
Findings and method
You receive scores, the edges where the model fails, and the signed test method.
Method
How the answer is produced
Answers computed by code
Where the right answer can be calculated, such as a threshold breach or a rule outcome, code computes it. Human grading drifts; code does not.
New cases every run
The logic stays the same but the values change each time, so the test cannot leak into prompt tuning.
Supports your validators
The exam gives your model-risk team independent evidence. It does not replace their judgement on whether the model fits its purpose.
Why Amir
Designed and led by someone who has built fraud models in production
The patents closest to this problem:
Amir is a named inventor on US 12,626,151 B2 (Method and system for testing a classification machine learning (ML) model of a tenant of a service provider, in a cloud-based environment, filed at NICE Actimize; granted).
Amir is a named inventor on US 12,626,151 (Method and system for testing a classification machine learning (ML) model of a tenant of a service provider, in a cloud-based environment, filed at NICE Actimize), which covers testing a client's classification model inside a shared cloud service before go-live, close to what this exam does for your model.
View on Google PatentsAmir is a named inventor on US 2023/0237494 A1 (Systems and methods for automatically creating machine learned fraud detection models, filed at NICE Actimize; pending).
Amir is a named inventor on US 2023/0237494 (Systems and methods for automatically creating machine learned fraud detection models, filed at NICE Actimize), which covers building a fraud model for a new client from what other clients learned, the situation a lender faces with each new partner bank.
View on Google PatentsThese patents were filed at the companies named. They show the problems Amir has worked on; they are not Riskernel products.
Is this you
You probably need this if
- You must validate an LLM or fraud model before it goes live, and the main evidence is the vendor's benchmark.
- Your team reuses the same fixed test set for every model version.
- You report one overall accuracy score but have not tested narrow edge cases an attacker could find.
- There is no written test method you could hand to an examiner.
- An examiner or auditor has asked how your LLM was tested.
- Your credit or first-party fraud model decides limits or approvals for several partner banks or credit unions, and each new partner brings a customer base the model has never seen.
- A partner institution's third-party or model-risk review asks how your model was tested on its own customers.
Questions
Before you reach out
Does this replace our model validation?
No. It gives your validators held-out evidence and a written method. The decision on whether the model is fit for use stays with your model-risk team.
Which models does it work for?
LLMs used in fraud or AML work, and classic fraud models, as long as the right answer for a case can be worked out by code.
Do you need our customer data?
No. Test cases are generated, including cases shaped like a partner's customers, which are built from the segments and ranges you describe, not from customer records. You may share the rules and thresholds the model must apply so the templates match your setting.
Does it work for credit models, not just fraud models and LLMs?
Yes. Credit and first-party fraud models are tested out of sample in the same way, including on cases shaped like a new partner institution's customers, and the written method can go into that partner's own review.
Scope Held-Out Model Validation Exam with Amir
Tell Amir what you run and what worries you. He replies with a short scope outline.