Chain-of-Thought (CoT) Prompting: Make AI Think Step by Step

Chain-of-Thought prompting is the technique of asking an AI to show its reasoning step by step before giving a final answer. It sounds almost too simple — but it is one of the highest-impact improvements you can make to any prompt that involves logic, calculation, or multi-step decisions.

1. Introduction

If you have ever asked an AI a maths question and gotten a confident wrong answer, you have experienced what happens when a model leaps straight to a conclusion. Large language models are extremely good at pattern-matching — but that same strength becomes a weakness when a problem actually requires careful step-by-step reasoning. The shortcut instinct fires and produces a plausible-sounding but incorrect result.

Chain-of-Thought (CoT) prompting was introduced in a landmark 2022 paper by Wei et al. at Google Brain. The insight was simple: if you ask a model to write down intermediate reasoning steps, accuracy on complex tasks jumps dramatically. In some benchmarks, adding "Let's think step by step." to a prompt improved accuracy from 18% to 79% on grade-school math problems.

We touched on CoT in Foundations Topic 4 (Zero-Shot, Few-Shot, Chain-of-Thought). This tutorial goes deeper — covering zero-shot CoT, few-shot CoT, self-ask variants, and when not to use it.

2. The Concept Explained

There are two main flavours of Chain-of-Thought prompting:

• Zero-Shot CoT: You simply append a reasoning instruction to your prompt — phrases like "Think step by step", "Let's reason through this carefully", or "Walk me through your reasoning before giving the final answer." No examples needed.
• Few-Shot CoT: You provide two to four worked examples where both the reasoning and the answer are shown. The model learns your preferred reasoning style and applies it to the new problem.

The mechanism is subtle but powerful. By generating intermediate tokens that represent reasoning steps, the model builds up context that influences later tokens. The reasoning acts as a scratchpad — it forces the model to slow down and commit to intermediate conclusions before reaching the final one.

3. The Problem Without CoT

Consider this prompt without any chain-of-thought instruction:

A shop sells apples for £0.40 each and oranges
for £0.60 each. Sara buys 7 apples and 4 oranges.
She pays with a £10 note. How much change does
she get?

A shop sells apples for £0.40 each and oranges
for £0.60 each. Sara buys 7 apples and 4 oranges.
She pays with a £10 note. How much change does
she get?

Think step by step before giving the final answer.

4. Zero-Shot CoT in Action

The magic phrase is simple. Any of these trigger chain-of-thought reasoning:

• Think step by step.
• Let's reason through this carefully.
• Walk me through your reasoning before giving the final answer.
• Work this out step by step, then state the answer clearly at the end.

Here is a richer real-world example — a business decision prompt:

You are a senior product manager. Our SaaS product has two pricing options:

Option A: £29/month, no contract, 3,000 monthly active users, 8% monthly churn.
Option B: £49/month, annual contract, 1,200 monthly active users, 2% monthly churn.

Which option generates more revenue after 12 months, assuming no new customers
are acquired in either scenario?

Think step by step. Show all calculations. Then give a clear recommendation.

A model given this prompt will compute monthly retention for both options, project user counts month by month, multiply by price, and sum the revenue — producing a transparent, auditable analysis rather than a gut-feel recommendation.

5. Few-Shot CoT in Action

When you need consistent reasoning in a particular domain or format, provide worked examples where the reasoning is shown explicitly:

Answer each word problem by reasoning step by step,
then stating the answer on its own line starting with "Answer:".

---
Problem: A train travels at 80 km/h for 2.5 hours. How far does it go?
Reasoning: Distance = speed × time = 80 × 2.5 = 200 km.
Answer: 200 km

---
Problem: A recipe uses 3 eggs per batch and makes 24 cookies. How many eggs
are needed to make 84 cookies?
Reasoning: First find the ratio: 84 ÷ 24 = 3.5 batches.
Eggs needed = 3 × 3.5 = 10.5 eggs, so round up to 11 eggs.
Answer: 11 eggs

---
Problem: A cinema has 320 seats. It was 75% full on Friday and 90% full on
Saturday. How many more people attended on Saturday than Friday?
Reasoning:

The model follows the pattern: reason through each step, then output Answer: on its own line. This makes the output easy to parse programmatically.

6. When to Use (and When Not to Use) CoT

Note: Modern reasoning models (like OpenAI o1, o3, and Claude's extended thinking mode) perform CoT automatically and internally. Even so, explicitly requesting step-by-step reasoning in your prompt still helps — it signals to the model what type of task you expect, and it ensures the reasoning is surfaced in the response where you can read and verify it.

7. Practice Exercises

8. Key Takeaways