Chain-of-Thought & Reasoning Models

What is Chain of Thought (CoT)?

When humans solve complex math or logic puzzles, we do not blurt out the answer instantly. We grab a pencil and work through intermediate sub steps.

Standard LLM generation attempts to output the final answer in a single step:

  Standard Direct Prompting:
  Question: "Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many balls does he have?"
  Model Answer: "11."  <-- (Wrong! Guessing without intermediate computation!)

Chain of Thought (CoT - Wei et al., 2022 / Google) prompts the model to output step-by-step intermediate reasoning tokens:

  Chain of Thought Prompting:
  Question: "Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many balls does he have?"
  Model CoT Answer: 
  "Roger started with 5 balls. 
   2 cans of 3 balls each is 6 tennis balls (2 x 3 = 6). 
   5 + 6 = 11. 
   The answer is 11."  <-- (Correct reasoning path!)

Why CoT Works: Test-Time Compute Allocation

In a Transformer, every generated token passes through a fixed number of layers ($L$).

This means the model executes a fixed amount of compute FLOPs per token.

If you force an LLM to answer a complex 5-step math problem in 1 single output token, it has only 1 forward pass of compute to solve the entire problem!

Generating 100 intermediate reasoning tokens gives the model 100 forward passes of compute, allowing self-attention to process information step by step across sequential time.

┌──────────────────────────┬──────────────────────────┐
│ 1. ZERO-SHOT CoT         │ 2. FEW-SHOT CoT          │
├──────────────────────────┼──────────────────────────┤
│ Append trigger phrase:   │ Provide prompt exemplars │
│ "Let's think step by     │ demonstrating explicit   │
│ step." (Kojima et al.)   │ step-by-step reasoning.  │
└──────────────────────────┴──────────────────────────┘

The Evolution to Reasoning Models (o1, DeepSeek-R1)

In 2024, AI labs shifted from prompting CoT to training dedicated Reasoning Models:

  TRADITIONAL LLM GENERATION:     Fixed Prompt ──► [ Instant Output Generation ]
  REASONING MODEL GENERATION:     Fixed Prompt ──► [ Extended Internal CoT Reasoning ] ──► Final Response
                                                    (Generates 5,000 hidden reasoning tokens!)

Key Innovations in Reasoning Models (OpenAI o1, DeepSeek-R1)

1. Reinforcement Learning on Reasoning Paths: Fine-tuned via RL (PPO / GRPO) using reward signals for correct final answers, forcing the model to discover self-correction, backtracking, and verification strategies.
2. Dynamic Test-Time Compute Scaling: Spends seconds or minutes generating internal hidden reasoning tokens before outputting a concise final answer.

Say this out loud

Chain of Thought encourages LLMs to generate step by step intermediate reasoning tokens before outputting a final answer. Because Transformers compute fixed FLOPs per token, generating intermediate CoT tokens allocates extra test-time compute, allowing multi step problem solving. Modern reasoning models internalize long CoT paths via reinforcement learning.

Followups to expect

1. What is Tree of Thoughts (ToT - Yao et al., 2023)? Extending CoT by exploring multiple reasoning branches in parallel, using search algorithms (BFS / DFS) and LLM self-evaluations to backtrack when a reasoning path hits a dead end.
2. What is Self-Consistency in CoT (Wang et al., 2022)? Sampling multiple independent CoT reasoning paths at high temperature (e.g. 50 paths) and taking a majority vote on the final answer to improve accuracy.