Extractive vs Abstractive QA

Two Question Answering Paradigms

Question Answering (QA) models answer natural language questions based on a context document.

  CONTEXT DOCUMENT: "The Eiffel Tower was completed in 1889 in Paris, France."
  QUESTION:         "When was the Eiffel Tower built?"

  EXTRACTIVE QA (BERT):    Predicts exact text span -> "1889" (High Precision, Rigid)
  ABSTRACTIVE QA (GPT-4):  Generates new sentence  -> "The Eiffel Tower was constructed in 1889." (Fluent, Generative)

1. Extractive Question Answering (BERT / SQuAD)

Extractive QA treats question answering as a Span Selection Task.

The model receives a concatenated sequence: [CLS] Question [SEP] Context Passage.

It does NOT generate new words. It predicts two token index numbers:

1. Start Token Index ($i$): Where the answer starts in the passage.
2. End Token Index ($j$): Where the answer ends in the passage.

  Passage Tokens:   The   Eiffel   Tower   was   completed   in   [1889]   in   Paris
  Start Score:     0.01    0.01    0.01   0.01     0.01     0.01   0.92   0.01   0.01  (Index 7!)
  End Score:       0.01    0.01    0.01   0.01     0.01     0.01   0.94   0.01   0.01  (Index 7!)

• Pros: Zero risk of generating false words (zero hallucination). Extremely fast sub-10ms inference.
• Cons: Rigid. Cannot answer questions requiring multi-sentence reasoning or synthesis.

2. Abstractive Question Answering (Generative LLMs)

Abstractive QA reads context documents and generates fluent, full sentence answers in its own words using autoregressive decoder LLMs (GPT-4, LLaMA 3).

  Prompt: "Answer the question based ONLY on context: <Context> Question: <Question>"
  LLM Generation: "According to the provided document, construction finished in 1889."

• Pros: Fluent, handles multi-document synthesis, rephrasing, and complex reasoning.
• Cons: Slower inference, requires GPU VRAM, and risks Hallucination if un-grounded.

Production Benchmark Datasets

1. SQuAD (Stanford Question Answering Dataset): Benchmark dataset of 100,000+ Wikipedia passage question pairs for Extractive QA.
2. MS MARCO (Microsoft): Real Bing search queries and web passages for information retrieval and QA evaluation.
3. TriviaQA / Natural Questions: Open domain trivia questions testing general knowledge retrieval.

Say this out loud

Extractive QA predicts exact start and end token indices within a source passage to highlight answers directly without hallucination. Abstractive QA uses generative LLMs to synthesize fluent natural language answers in new words. Production systems use RAG to retrieve ground context passages for abstractive LLM generation.

Followups to expect

1. What is RAG (Retrieval-Augmented Generation)? Combining dense vector retrieval (finding top-K relevant documents from a database) with an abstractive LLM generator to ground answer generation on real business data.
2. What is Exact Match (EM) vs F1 Score in SQuAD evaluation? Exact Match requires the predicted span string to match human ground truth character for character. F1 measures word level precision and recall overlap between predicted and target spans.