Design: Recommendation Feed

The Problem

You are asked to design a recommendation feed for a platform with 100 million users and 10 million items. The feed must return personalized results in under 200ms.

Scoring every item with a deep neural model is impossible. At 2ms per item, scoring 10 million items takes over 5 hours. You need a multi-stage funnel.

High-Level Architecture

  10,000,000 Items in Catalog
              │
              ▼
  ┌───────────────────────────────┐
  │ STAGE 1: CANDIDATE GENERATION │  (~5ms)
  │ Two-Tower ANN Retrieval       │
  │ Output: ~500 Candidates       │
  └───────────────────────────────┘
              │
              ▼
  ┌───────────────────────────────┐
  │ STAGE 2: RANKING              │  (~50ms)
  │ Deep Neural Ranker (DCNv2)    │
  │ Multi-Objective Scoring       │
  │ Output: ~50 Scored Items      │
  └───────────────────────────────┘
              │
              ▼
  ┌───────────────────────────────┐
  │ STAGE 3: RE-RANKING           │  (~10ms)
  │ Business Rules & Diversity    │
  │ MMR, Freshness, Dedup, Ads    │
  │ Output: Final Feed (20 items) │
  └───────────────────────────────┘

Stage 1: Candidate Generation

The goal is to narrow 10 million items down to a few hundred quickly. You run multiple retrieval sources in parallel:

1. Two-Tower Embedding Retrieval: Encode the user into a vector, search the precomputed item vector index using HNSW. Returns items semantically similar to user preferences.
2. Collaborative Filtering: Items similar to what similar users engaged with.
3. Popular/Trending: Globally trending items as a cold-start fallback.
4. Same-Author/Category: Items from creators or categories the user recently interacted with.

Each source returns 100 to 200 candidates. After deduplication you have roughly 500 unique candidates.

Stage 2: Ranking

A deep neural model (like DCNv2 or a Multi-Gate Mixture-of-Experts network) scores each of the 500 candidates. The model predicts multiple objectives:

$$\text{Score} = w_1 \cdot P(\text{Click}) + w_2 \cdot E[\text{WatchTime}] + w_3 \cdot P(\text{Share}) - w_4 \cdot P(\text{Hide})$$

Features include user profile, item metadata, real-time session features (last 5 clicks from the Feature Store), and cross features (user-item interaction history).

Stage 3: Re-Ranking

The top 50 ranked items pass through business logic:

• Diversity: Apply Maximal Marginal Relevance (MMR) so the feed is not all the same category.
• Freshness: Boost recently published content so new creators get visibility.
• Deduplication: Remove near-duplicate items.
• Ad Insertion: Place sponsored items at designated feed positions.

The final 20 items are returned to the client.

Key Design Decisions to Discuss

1. Cold-Start Users: Fall back to popular items and demographic-based recommendations until enough interaction data accumulates.
2. Real-Time Features: Use a streaming pipeline (Kafka and Flink) to update the Feature Store with sub-second freshness so that clicks made 10 seconds ago influence the next feed refresh.
3. Feedback Loops: Reserve 5% of feed slots for exploration (Thompson Sampling) to prevent popularity bias amplification.

Say this out loud

A recommendation feed uses a multi-stage funnel. Candidate Generation retrieves hundreds of items from millions using Two-Tower embeddings and ANN search. A deep Ranking model scores candidates across multiple objectives. Re-ranking applies diversity, freshness, and business rules before serving the final feed.

Followups to expect

1. How do you handle position bias in training data? Use the Position Feature trick: include rank position as a training feature but set it to a constant at inference time.
2. How do you evaluate the full pipeline offline? Use replay-based offline evaluation with historical interaction logs, measuring NDCG@K and Hit Rate@K against held-out positive interactions.