Sequential & Session-Based RecSys

Beyond Static Matrix Factorization

Traditional recommendation models (Matrix Factorization, Collaborative Filtering) treat user history as an un-ordered static bag of items.

However, real-world user behavior is inherently sequential:

• A user who buys a Phone $\to$ is likely to next buy a Phone Case $\to$ then a Screen Protector.
• An un-authenticated guest user browsing a retail website has no user ID profile; recommendations must depend entirely on the sequence of items clicked during the active 5-minute session.

Sequential & Session-Based Recommendation models item interactions as an ordered sequence:

$$S_u = \left( i_1, i_2, i_3, \dots, i_t \right) \implies \text{Predict Next Item } i_{t+1}$$

┌──────────────────────────┬──────────────────────────┐
│ 1. SEQUENTIAL RECSYS     │ 2. SESSION-BASED RECSYS │
├──────────────────────────┼──────────────────────────┤
│ Long-term user history   │ Short-term guest session │
│ across months/years.     │ (no user ID!). Focuses   │
│ Balances past profile +  │ strictly on real-time    │
│ recent item shifts.      │ session click intent.    │
└──────────────────────────┴──────────────────────────┘

Architectural Evolution

┌──────────────────────────┬──────────────────────────┬──────────────────────────┐
│ GRU4REC (Hidasi 2015)    │ CASER (Tang 2018)        │ SASREC (Kang 2018)       │
├──────────────────────────┼──────────────────────────┼──────────────────────────┤
│ Uses Gated Recurrent     │ Uses 2D & 1D Convolution │ Uses Causal Masked       │
│ Units (GRU) to process   │ filters to capture local │ Transformer Self-Attention│
│ item click sequences.    │ sequential n-grams.      │ SOTA Sequential RecSys!  │
└──────────────────────────┴──────────────────────────┴──────────────────────────┘

1. GRU4Rec (Hidasi et al., 2015)

The pioneer of deep session-based recommendation. Uses Gated Recurrent Units (GRU) to process item sequences, trained using Pairwise Ranking Losses (BPR / TOP1 Loss).

2. SASRec (Self-Attentive Sequential Recommendation - Kang & McAuley, 2018)

SASRec frames sequential recommendation identically to a Causal Language Model (GPT)!

It represents historical item IDs as token embeddings, adds positional encodings, and processes sequences through Causal Masked Multi-Head Self-Attention layers:

  Item Sequence:  [ Phone ] ──► [ Phone Case ] ──► [ Screen Protector ]
                      │               │                   │
                      ▼               ▼                   ▼
            [ CAUSAL MASKED TRANSFORMER SELF-ATTENTION STACK ]
                                      │
                                      ▼
                    Predict Next Item Vector: [ Wireless Charger ]

3. BERT4Rec (Sun et al., 2019)

Replaces causal unidirectional attention with Bidirectional Self-Attention using a Masked Item Prediction objective (Cloze Task):

• Randomly mask out $20%$ of items in the user interaction sequence: [Item A, [MASK], Item C, Item D].
• Train the model to reconstruct the missing [MASK] item IDs using bidirectional context.

Key Industrial Techniques

1. Time-Aware Positional Encodings: Injecting continuous time delta embeddings ($t_{i+1} - t_i$) into self-attention to distinguish clicks made 30 seconds ago from clicks made 3 weeks ago.
2. Item Content Fusion: Combining item ID embeddings with rich multimodal features (item title text embeddings, image vectors, and brand categories).

Say this out loud

Sequential and Session Based Recommendation systems predict a user's next action by modeling temporal item sequences. GRU4Rec introduced RNNs for session click streams, while SASRec applied causal masked self attention to model long term user preferences and short term intent. BERT4Rec uses bidirectional masked item pretraining to capture sequence context.

Followups to expect

1. How does Session-Based RecSys handle anonymous guest users? Session-based models initialize a fresh hidden state vector for the active session, updating session intent dynamically with every new item click without requiring a persistent user ID.
2. What is SR-GNN (Session-based Recommendation with Graph Neural Networks)? Modeling a session click sequence as a directed sub-graph, applying Graph Neural Networks to capture complex item transition transitions across different sessions.