A Framework for Any ML Design Round

The 6-Step Whiteboard Framework

Never jump straight into model building. Structure your 45-minute whiteboard session into clear phases:

1. Scope & Goals   ─► 2. Data & Features   ─► 3. Model & Baseline
    (5 min)              (10 min)                (10 min)
                                                     │
6. Operations      ◄─ 5. Serving & Infra   ◄─ 4. Evaluation
    (5 min)              (10 min)                (5 min)

Step 1: Problem Framing & Constraints (5 mins)

• Business Goal: Increase conversion, reduce churn, improve engagement.
• ML Task Formulation: Binary classification? Pointwise / pairwise ranking? Regression?
• Key Scale & SLAs: DAU, QPS (queries/sec), latency budget (e.g. p99 < 50ms), throughput, hardware limits.

Step 2: Data Pipeline & Feature Engineering (10 mins)

• Data Sources: Click logs, user profiles, item metadata, context (time, device).
• Features: Static vs dynamic features. Point-in-time joins to avoid leakage.
• Storage: Feature Store (Offline for training: Parquet/S3; Online for serving: Redis/Cassandra).

Step 3: Model Architecture & Baselines (10 mins)

• Baseline: Rule-based heuristic (e.g. top trending items) or simple Linear/Logistic Regression.
• Primary Model: Two-stage architecture for RecSys/Search (Retrieval → Heavy Ranking).
• Loss Function & Optimization: Cross-Entropy, Focal Loss, Pairwise ranking loss (BPR).

Step 4: Evaluation Strategy (5 mins)

• Offline Metrics: ROC-AUC, PR-AUC, NDCG@k, MRR, RMSE.
• Online Metrics (A/B Test): Click-Through Rate (CTR), Conversion Rate, Revenue per User, Latency.
• Metrics Alignment: How offline metric gains map to online business metrics.

Step 5: Serving & Infrastructure (10 mins)

• Inference Pattern: Batch scoring vs Real-time online scoring.
• Caching & Fallbacks: Fallback to popular items if feature lookup times out or model service degrades.
• Quantization & Acceleration: ONNX, TensorRT, vLLM, GPU batching.

Step 6: Operations & Continuous Learning (5 mins)

• Monitoring: Prediction drift (PSI), data drift, latency, error rates.
• Feedback Loops: Position bias, click logs delayed labels.
• Retraining Strategy: Daily/weekly batch retraining vs online streaming updates.

Framing: Offline Metrics vs Online Business KPIs

Say this out loud

"I structure ML system design into six phases: first, clarifying business goals, QPS scale, and p99 latency SLAs. Next, defining real-time vs batch features and setting a simple heuristic baseline. For the core architecture, I use a multi-stage approach — fast candidate retrieval followed by a heavy ranking model. Finally, I define offline/online metric alignment, low-latency serving with Redis feature stores, and drift monitoring."

Follow-ups to expect

• How do you prevent training-serving skew? Use a unified Feature Store for both offline batch ETL and online point-in-time feature serving, ensuring identical feature transformations across training and inference.
• What happens if the feature store lookup times out in production? Fall back gracefully to static item-level default features or pre-computed batch recommendations to meet the strict SLA without dropping the user request.
• How do you handle cold-start items in a recommendation system? Use content-based embeddings (text/image features via two-tower networks), exploit bandit algorithms (Epsilon-greedy, Thompson sampling) to explore new items, or leverage graph embeddings.