Chain-of-Thought Prompting

The Reasoning Problem

In the previous lesson, we explored few-shot learning. Now let’s build on that foundation. Here’s a puzzle:

A farmer has 15 apples. He gives 1/3 to his neighbor, then eats 2. His son takes half of what’s left. How many apples remain?

Ask an AI without guidance, and it might just say “4 apples” or “5 apples.” Sometimes right, sometimes wrong. You can’t tell if it actually reasoned through it or just guessed.

Now ask it to show its work:

Walk through this step by step before giving your final answer.

Suddenly you get:

• Started with 15 apples
• 1/3 given to neighbor: 15 × (1/3) = 5 given away, leaving 10
• Eats 2: 10 - 2 = 8 remaining
• Son takes half: 8 ÷ 2 = 4 taken, leaving 4

Final answer: 4 apples

Same model. Same question. Different reliability.

This is chain-of-thought prompting—and it’s one of the most significant discoveries in prompt engineering.

Why Showing Work Helps

It seems almost too simple. Just ask the AI to think step by step, and it gets smarter?

Yes. And there’s a reason.

AI models generate text one token at a time. Each token depends on what came before. When you ask for just the answer, the model has to make a huge leap—from question directly to conclusion.

When you ask for reasoning first, each step builds on the previous. The model can “think” through intermediate states. Each step is a smaller, more accurate leap.

Think of it like this: Jumping from one side of a river to the other is hard. Jumping stone to stone is easy.

The Magic Phrase

Researchers at Google found that a simple phrase dramatically improves reasoning:

“Let’s think step by step”

That’s it. Adding just those words increased accuracy on math and logic problems from ~18% to ~79% in some tests.

You can also use:

• “Walk through this step by step”
• “Think through this carefully before answering”
• “Show your reasoning process”
• “Break this down step by step”

The exact wording matters less than explicitly requesting stepwise reasoning.

When to Use Chain-of-Thought

Chain-of-thought shines on specific task types:

Great for:

Math and calculations

Calculate the total cost including 8.25% tax for: 3 items at $24.99, 2 items at $15.50, and 1 item at $89.99. Think through each step.

Logic problems

If all Bloops are Razzles and all Razzles are Lazzles, are all Bloops Lazzles? Reason through this carefully.

Multi-step analysis

Analyze whether this startup should pursue market A or market B. Consider market size, competition, team strengths, and timing. Think through each factor before concluding.

Decision-making with trade-offs

Should we build or buy this feature? Walk through the considerations: cost, time, expertise, maintenance, strategic fit.

Complex comparisons

Compare React vs Vue for this project. Think through: learning curve, performance needs, team experience, ecosystem requirements.

Not necessary for:

• Simple factual questions (“What’s the capital of France?”)
• Creative generation without logic (“Write a poem about autumn”)
• Straightforward text transformation (“Translate this to Spanish”)
• Basic formatting tasks (“Convert this to bullet points”)

Using chain-of-thought on simple tasks just adds unnecessary length.

Chain-of-Thought Patterns

Pattern 1: Suffix Request

Add reasoning request at the end.

[Your question or task]

Think through this step by step before giving your final answer.

Pattern 2: Structured Reasoning

Specify the exact steps to reason through.

Decide whether to approve this loan application.

Think through:
1. Credit score assessment
2. Income-to-debt ratio
3. Employment stability
4. Collateral value
5. Overall risk rating

Then give your approval decision with reasoning.

Pattern 3: Reasoning Template

Provide a format for the reasoning.

Quick check: Before moving on, can you recall the key concept we just covered? Try to explain it in your own words before continuing.

Analyze this business decision.

Format your response as:
CONSIDERATIONS:
- [List key factors]

ANALYSIS:
- [Think through each factor]

TRADE-OFFS:
- [What you gain vs lose with each option]

RECOMMENDATION:
- [Your conclusion with reasoning]

Pattern 4: Few-Shot Chain-of-Thought

Combine with examples showing reasoning.

Solve word problems by reasoning step by step.

Example:
Problem: A train leaves Station A at 9am traveling 60mph. Another train leaves Station B (300 miles away) at 10am traveling 90mph toward Station A. When do they meet?

Reasoning:
- At 10am, first train has traveled 1 hour × 60mph = 60 miles
- Distance remaining: 300 - 60 = 240 miles
- Combined speed: 60 + 90 = 150mph
- Time to meet: 240 ÷ 150 = 1.6 hours after 10am
- Answer: 11:36am

---

Now solve:
Problem: [Your problem here]

Reasoning:

Self-Consistency: Multiple Reasoning Paths

For important decisions, you can go further. Ask the AI to reason through the problem multiple times, then pick the answer that appears most often.

I need to decide whether to launch product A or product B first.

Reason through this decision three separate times, considering different angles:

Pass 1: Focus on market timing
Pass 2: Focus on resource constraints
Pass 3: Focus on competitive response

Then identify which conclusion appears in most passes.

If 2 out of 3 reasoning paths reach the same conclusion, you have higher confidence. If all three disagree, the decision might need more information.

Common Mistakes

Mistake 1: Requesting reasoning for trivial tasks Chain-of-thought adds tokens and time. Use it when reasoning actually helps.

Mistake 2: Not reading the reasoning The whole point is to verify the logic. If you skip to the answer, you lose the benefit.

Mistake 3: Vague reasoning requests “Think about it” is weaker than “Think through step by step.”

Mistake 4: Not providing structure for complex reasoning For multi-factor decisions, specify what factors to consider. Don’t leave it open-ended.

Practical Exercise

Take this problem:

A company has $500,000 budget. They can hire 5 engineers at $80,000 each or 3 engineers plus $200,000 in contracted services. The engineers will take 2 months to ramp up. The contracted work can start immediately but requires 10 hours/week of internal management. Which option should they choose if they need to ship a product in 6 months?

Write a prompt that asks for chain-of-thought reasoning, specifying the factors to consider.

A company has $500,000 budget for a 6-month project to ship a new product.

Option A: Hire 5 engineers at $80,000 each ($400K total)
- Engineers need 2 months to ramp up
- 5 engineers × 4 productive months = 20 engineer-months of work

Option B: Hire 3 engineers at $80,000 each ($240K) + $200K contracted services ($440K total)
- Contractors can start immediately
- Engineers need 2 months ramp up
- Requires 10 hrs/week internal management of contractors

Think through this decision step by step:

1. EFFECTIVE CAPACITY
   - Calculate productive work-months for each option

2. COST ANALYSIS
   - Total spend and remaining budget for each

3. RISK FACTORS
   - What could go wrong with each approach?

4. TIMELINE FIT
   - Which better fits the 6-month deadline?

5. RECOMMENDATION
   - Which option should they choose and why?

Key Takeaways

• Chain-of-thought prompting asks AI to show reasoning before conclusions
• Adding “Let’s think step by step” can dramatically improve accuracy
• Best for: math, logic, multi-step analysis, complex decisions
• Not necessary for: simple questions, basic transformations, creative tasks
• For important decisions, try multiple reasoning passes and look for consensus

Next up: proven prompt patterns. We’ll look at battle-tested templates for common tasks—so you don’t have to reinvent the wheel every time.

Up next: In the next lesson, we’ll dive into Prompt Patterns.