Transaction
real time
- feeds
- Feature storefeatures
All case studies
ML SystemsMLOpsMonitoring5 months
Fraud Detection Pipeline + Monitoring
Analysts spent their day on false alarms. The review queue is now a third of the size.
False positive rate
1.6%
Cost-calibrated model, same recall
Was
4.1%
Static rules at target recall
In plain terms
What this actually was.
No jargon in this section. The technical write-up is further down.
What was going wrong
A fixed set of rules caught the fraud it already knew about and nothing else. Analysts waded through false alarms all day while genuinely new attack patterns went unnoticed for weeks at a time.
What we built
We built real-time scoring that weighs the actual cost of each kind of mistake, added monitoring that raises a flag when behaviour shifts, and rolled it out gradually alongside the existing rules rather than replacing them overnight.
What changed
Catching the same amount of fraud, false alarms fell from 4.1% to 1.6% — cutting the review queue by nearly two thirds. New patterns now surface within days, and every declined transaction comes with a reason attached.
Architecture
How it fits together.
Scroll through the stages. Anything marked as added is a component that did not exist before this project.
Feature store
online
- receives from
- Transactionfeatures
- feeds
- GBDT scorer
GBDT scorer
241 feats
- receives from
- Feature store
- feeds
- Cost matrix · Drift monitorlog
Cost matrix
thresholds
- receives from
- GBDT scorer
- feeds
- Analyst queuereview
Analyst queue
explained
- receives from
- Cost matrixreview
Drift monitor
psi + ks
- receives from
- GBDT scorerlog
Transaction
real time
- feeds
- Feature storefeatures
Feature store
online
We added this- receives from
- Transactionfeatures
- feeds
- GBDT scorer
GBDT scorer
241 feats
We added this- receives from
- Feature store
- feeds
- Cost matrix · Drift monitorlog
Cost matrix
thresholds
We added this- receives from
- GBDT scorer
- feeds
- Analyst queuereview
Analyst queue
explained
- receives from
- Cost matrixreview
Drift monitor
psi + ks
We added this- receives from
- GBDT scorerlog
The write-up
How it was actually built.
Dataset, approach, measured results and the stack. Written for whoever has to review it.
ML SystemsMLOpsMonitoring5 months
Fraud Detection Pipeline + Monitoring
Real-time scoring with feature pipelines, monitoring, and controlled rollouts.
Problem
A static rule set caught known fraud patterns and nothing else. Analysts drowned in false positives while novel attack patterns went unflagged for weeks.
Dataset
- rows
- 186M transactions
- features
- 241
- sources
- Payments, device, behavioural
- window
- 28 months
Approach
- 01Built a real-time feature store with strict offline/online parity
- 02Trained a gradient-boosted model with heavy class imbalance handling
- 03Set thresholds from an explicit cost matrix rather than F1
- 04Added drift monitoring and a champion/challenger retraining loop
Metrics
- 0.79
- PR-AUC +0.34
- -61%
- false positives at equal recall
- 28ms
- scoring p95 inline
- 0.041
- drift PSI monitored
Business impact
- Analyst review queue cut by nearly two thirds
- Novel patterns surfaced in days instead of weeks
- Every decline explainable with feature attributions
Tech stack
- Python
- Kafka
- Flink
- LightGBM
- Feast
- Prometheus
- Grafana
Next
Need a system like this?
Answer six questions and we will tell you whether this shape fits your problem — including when it does not.