NER & Sequence Labelling

What is Named Entity Recognition?

Named Entity Recognition (NER) extracts key structured information from unstructured text documents.

It identifies and classifies text spans into predefined real world categories:

  Raw Input Text:
  "Steve Jobs founded Apple in Cupertino on April 1, 1976."

  NER Extraction:
  - [Steve Jobs]   ──► PERSON
  - [Apple]        ──► ORGANIZATION
  - [Cupertino]    ──► LOCATION
  - [April 1, 1976]──► DATE

The BIO Tagging Scheme

To detect multi word entities (like "New York City"), NER frames the problem as Token Level Sequence Labelling using the BIO Tagging Scheme:

1. B- (Begin): First token of a named entity span.
2. I- (Inside): Subsequent tokens of a multi word entity span.
3. O (Outside): Non entity token.

  Token:   Steve     Jobs     founded    Apple      in     Cupertino   .
  Tag:     B-PER     I-PER    O          B-ORG      O      B-LOC       O

Architectures: From BiLSTM-CRF to BERT

┌──────────────────────────┬──────────────────────────┐
│ 1. BILSTM-CRF            │ 2. BERT TOKEN CLASSIFIER │
├──────────────────────────┼──────────────────────────┤
│ Uses Bidirectional LSTM  │ Uses pretrained BERT     │
│ + Conditional Random     │ contextual embeddings +  │
│ Field (CRF) layer to     │ Linear Token Head.       │
│ enforce valid tag rules. │ State of the art accuracy│
└──────────────────────────┴──────────────────────────┘

Why Add a CRF Layer?

Independent token classifiers can output illegal tag transitions (for example, outputting O followed directly by I-PER without a preceding B-PER).

A Conditional Random Field (CRF) layer models transition probability matrix $A_{i,j} = P(\text{Tag}_j \mid \text{Tag}_i)$ between adjacent labels, enforcing valid global tag sequences across the entire sentence.

Subword Alignment Challenge in BERT

BERT uses subword tokenization (BPE / WordPiece). A word like "Cupertino" might be split into subwords ["Cuper", "##tino"].

Standard Practice:

• Assign the ground truth BIO tag (B-LOC) to the first subword ("Cuper").
• Assign a special Ignored Index (-100) to subsequent subwords ("##tino"), masking their loss during training.

Say this out loud

Named Entity Recognition extracts structured entities like Person, Location, and Date from unstructured text. It uses the BIO tagging format (Begin, Inside, Outside) to represent multi word entity boundaries. Architectures evolved from BiLSTM-CRF models that enforce valid tag sequence transitions to fine tuned BERT models with linear token classification heads.

Followups to expect

1. What is Nested NER? When entities exist inside other entities (for example, "Bank of America" is an Organization, but "America" inside it is a Location). Handled using span based classifiers or hypergraph models instead of simple BIO tagging.
2. What is Zero Shot NER with LLMs? Using prompt instructions with structured JSON output formatting ("Extract entities as a JSON list") to perform NER without training custom token classification heads.