Design: Fraud Detection

The Problem

Design a system that scores every payment transaction in real time (under 50ms) to decide whether it is fraudulent. The system processes 10,000 transactions per second. Only about 1 in 10,000 transactions is actually fraud.

High-Level Architecture

  Incoming Transaction Event
              │
              ▼
  ┌───────────────────────────────┐
  │ LAYER 1: RULE ENGINE          │  (~1ms)
  │ Hard Rules & Velocity Checks  │
  │ Block known bad IPs, stolen   │
  │ card lists, impossible speeds │
  └───────────────────────────────┘
              │ (passes rules)
              ▼
  ┌───────────────────────────────┐
  │ LAYER 2: ML SCORING           │  (~10ms)
  │ Gradient Boosted Trees or     │
  │ Neural Network                │
  │ Output: Fraud Probability     │
  └───────────────────────────────┘
              │
     ┌────────┼────────┐
     ▼        ▼        ▼
  APPROVE   REVIEW    BLOCK
  (p<0.1)  (0.1<p<0.7) (p>0.7)
              │
              ▼
  ┌───────────────────────────────┐
  │ LAYER 3: HUMAN REVIEW QUEUE   │
  │ Analysts investigate medium   │
  │ risk transactions              │
  └───────────────────────────────┘

Layer 1: Rule Engine

Before running expensive ML models, apply simple deterministic rules that catch obvious fraud instantly:

• Blocklists: Known stolen card numbers, flagged IP addresses, sanctioned countries.
• Velocity Checks: More than 5 transactions in 1 minute from the same card.
• Impossible Travel: Card used in New York and then London 10 minutes later.
• Amount Thresholds: First-ever transaction on a new account exceeding $5,000.

Rules are fast, interpretable, and easy to update without model retraining.

Layer 2: ML Scoring

For transactions that pass the rule engine, a machine learning model assigns a fraud probability score.

Handling Class Imbalance

With only 0.01% positive (fraud) labels, standard training fails. Solutions:

1. Cost-Sensitive Loss: Assign 100x higher loss weight to missed fraud (false negatives) compared to false alarms.
2. Downsampling Negatives: Train on all fraud examples plus a random sample of legitimate transactions (e.g. 1:10 ratio).
3. SMOTE or Augmentation: Generate synthetic minority samples to balance the training distribution.

Features

• Transaction Features: Amount, currency, merchant category, time of day, device fingerprint.
• User Behavior Features: Average transaction amount over 30 days, typical merchant categories, usual login locations.
• Real-Time Aggregates: Number of transactions in last 5 minutes, total spend in last hour (computed via streaming from the Feature Store).
• Network/Graph Features: How many other flagged accounts share the same device or shipping address.

Model Choice

Gradient Boosted Trees (XGBoost, LightGBM) are the industry standard for fraud because they handle tabular data well, are fast at inference, and produce interpretable feature importances for regulatory compliance.

Layer 3: Human Review

Transactions with medium-confidence scores go to a human review queue. Analyst decisions feed back as new labeled training data, creating a human-in-the-loop cycle.

Key Design Challenges

1. Adversarial Drift: Fraudsters change tactics constantly. The model that works today may fail in 2 weeks. Retrain weekly or bi-weekly and monitor precision/recall drift daily.
2. False Positive Cost: Blocking a legitimate customer's $2,000 purchase creates a terrible user experience. Tune the decision threshold based on the business cost ratio of false positives versus false negatives.
3. Evaluation Metric: Use Precision-Recall AUC rather than ROC-AUC because ROC-AUC can look inflated on heavily imbalanced data.

Say this out loud

A fraud detection system uses three layers. A rule engine catches obvious fraud instantly. An ML model scores remaining transactions, handling extreme class imbalance with cost-sensitive loss and real-time aggregation features. Medium-confidence cases go to human review. The model requires frequent retraining because fraudsters adapt their tactics.

Followups to expect

1. How do you get labels for fraud? Chargebacks from banks arrive days or weeks later. Use early signals (customer dispute reports within 24 hours) as proxy labels and backfill with chargeback labels later.
2. How do you explain model decisions to regulators? Use SHAP values to show which features drove each individual fraud score, providing per-transaction explainability for compliance audits.