Query Understanding & Expansion

What is Query Understanding?

In web search and e-commerce, user search queries are notoriously difficult to process:

• Short Length: Average query length is $2\text{--}3$ words.
• Ambiguity & Typos: Typos ("nke shos"), acronyms ("TV 4k"), and ambiguous terms ("apple").
• Vocabulary Mismatch: Users search for "cheap laptop", while product titles contain "discount notebook".

Query Understanding is the pre-retrieval pipeline that converts noisy raw text into structured search intent:

  Raw Noisy User Query: "red nke running shos size 10"
                            │
                            ▼
  [ QUERY UNDERSTANDING PIPELINE ]
  1. Spelling Correction:  "red nike running shoes size 10"
  2. Intent Classification: Category = Footwear / Shopping
  3. Slot Tagging (NER):   Brand=Nike, Color=Red, Style=Running, Size=10
  4. Query Expansion:      Synonyms += ["sneakers", "athletic footwear"]
                            │
                            ▼
  Structured Filter Query ──► [ SEARCH RETRIEVAL ENGINE ]

The 4 Stages of Query Understanding

┌──────────────────────────┬──────────────────────────┬──────────────────────────┬──────────────────────────┐
│ 1. SPELLING CORRECTION   │ 2. INTENT CLASSIFICATION │ 3. SLOT FILLING (NER)    │ 4. QUERY EXPANSION       │
├──────────────────────────┼──────────────────────────┼──────────────────────────┼──────────────────────────┤
│ Corrects typos using     │ Predicts global user goal│ Extracts structured      │ Adds synonyms and LLM    │
│ Levenshtein distance or  │ (Transactional,          │ attributes (Brand,       │ rewrites to eliminate    │
│ masked Transformer models│ Informational, Nav).     │ Product, Color, Size).   │ vocabulary mismatch.     │
└──────────────────────────┴──────────────────────────┴──────────────────────────┘

Stage 1: Spelling Correction & Normalization

Uses character-level Levenshtein edit distance, SymSpell, or BERT Masked LM ("nke" $\to$ "Nike").

Stage 2: Intent Classification

Classifies the global search goal using a fast text classifier:

• Navigational: "Facebook login" $\to$ Direct user to login URL.
• Transactional / Commercial: "Buy iPhone 15" $\to$ Route to product catalog search.
• Informational: "How to change car oil" $\to$ Route to blog articles and video content.

Stage 3: Named Entity Recognition (NER) / Slot Filling

Parses raw query tokens into structured key-value filters:

Input Query:  "blue Samsung 4k television under $500"
Named Entities:
- Brand:     Samsung
- Color:     Blue
- Resolution: 4k
- Category:  Television
- Max_Price: 500

Structured slots are passed directly to SQL or ElasticSearch structured filters (WHERE brand='Samsung' AND price <= 500).

Stage 4: Query Expansion & LLM Rewriting

To fix Vocabulary Mismatch:

• Synonym Expansion: Add related terms from a domain dictionary ("shoes" $\to$ "shoes OR sneakers OR footwear").
• LLM Conversational Rewriting (RAG): In multi-turn chat systems, the user says "What about its battery life?". The LLM rewrites this into a self-contained query: "What is the battery life of iPhone 15 Pro?".

Say this out loud

Query Understanding transforms noisy raw user queries into structured search intent. It uses spelling correction for typos, intent classification for user goals, Named Entity Recognition for structured slot filling, and query expansion or LLM rewriting to bridge vocabulary mismatch gaps before database retrieval.

Followups to expect

1. What is PRF (Pseudo Relevance Feedback) in Query Expansion? Running an initial search query, extracting top 5 retrieved documents, selecting high TF-IDF terms from those documents, and appending them to expand the original search query.
2. What is Query Relaxation? If a structured query with 5 slot filters (Brand=Nike AND Color=Red AND Size=14 AND Style=Tennis) returns 0 search results, Query Relaxation systematically drops strict filters to return close matches.