Free tool · Runs in your browser
Alert budget calculator
A model’s accuracy or AUC does not tell you how many analysts you need. Enter your daily volume, fraud rate and review capacity, add up to three operating points, and see what each one does to a day of alerts.
Everything is worked out in your browser. Nothing you type is sent or stored. The link button only builds an address that carries your inputs.
How this is calculated
Write V for events per day, r for the fraud rate, C for the alerts your team can review in a day, and F and L for fraud and legitimate events per day. Each operating point is either a true positive rate (TPR) with a false positive rate (FPR), or a precision with a recall.
F = V x r L = V - F TPR and FPR: TP = F x TPR FP = L x FPR precision, recall: TP = F x recall FP = TP / precision - TP alerts raised A = TP + FP alerts reviewed R = min(A, C) share nobody reviews (A - R) / A fraud caught TP x R / A fraud missed F - caught precision at capacity caught / R
Assumptions: alerts reach analysts in no particular order, so the reviewed share has the same precision as all alerts (a queue ranked by score would do better, so the figures are cautious); each alert takes one review; rates hold across the day; an alert nobody reviews is treated as a missed case. Precision and recall must be possible at your volume: the false positives they imply cannot exceed the legitimate events you have.
Why one accuracy or AUC number does not answer the staffing question
AUC summarises how well a model ranks fraud above legitimate events across every threshold. Staffing depends on one threshold: the number of alerts it raises each day. Two models with the same AUC can raise very different numbers of alerts at the same recall, and a model that finds more fraud can still catch less once its alerts pass what the team can review. Try two operating points with the same inputs and compare the fraud caught, not the rate.
Frequently asked questions
What is an operating point?
A threshold on the model’s score, described by the rates it produces: either true positive rate with false positive rate, or precision with recall. You can read them off a validation set or a shadow run.
Why can a higher detection rate catch less fraud?
A higher detection rate usually costs more false positives. When the extra alerts pass your review capacity, part of the queue is never looked at, and the fraud inside it is missed. The table shows the share of alerts nobody reviews for that reason.
Does the calculator send my numbers anywhere?
No. The figures are worked out in your browser. The “Copy link” button puts your inputs in the page address so you can share a result, and nothing else.
How do I get operating points for a new model?
Score the same traffic with the new model next to the current system without acting on its decisions. That is a shadow test, and the sample size calculator next to this one tells you how long it has to run.
Can I put the calculator on my own site?
Yes. Copy the embed code on this page. The embedded version keeps one credit link back to this page.
Go deeper
- Alert budgets, not AUC: why the staffing question needs an alert count.
- Shadow test sample size calculator: how many fraud cases a comparison needs.
- Shadow test: run a second system beside your current one and compare, with no change to production.