Here’s something that surprised me when I first learned it.
You can make AI significantly smarter at reasoning—not by using a better model, not by paying more money, but by adding five words to your prompt.
Those five words? “Let’s think step by step.”
This technique is called Chain of Thought prompting, and it’s one of the most useful things I’ve learned about working with AI.
The Problem: AI Rushes to Answers
By default, AI tries to jump straight from question to answer. Ask it a math problem, it spits out a number. Ask for analysis, it gives you a conclusion.
Sometimes that’s fine. But for anything requiring actual reasoning—logic puzzles, multi-step math, complex analysis, debugging—this shortcut-taking leads to mistakes.
The AI knows it should show its work. It just… doesn’t. Unless you ask.
What Chain of Thought Does
Chain of Thought (CoT) prompting asks the AI to show its reasoning process before giving an answer.
Instead of:
Question → Answer
You get:
Question → Step 1 → Step 2 → Step 3 → Answer
This simple change has profound effects.
When AI has to articulate each step, it catches its own mistakes. It considers factors it might otherwise skip. It builds toward answers rather than guessing at them.
Google Brain researchers published a landmark paper on this in 2022. They found that CoT prompting dramatically improved performance on arithmetic, commonsense reasoning, and symbolic logic tasks—without any additional training.
Same AI. Same knowledge. Much better results. All from asking it to think out loud.
The Simplest Version: Zero-Shot CoT
The easiest way to use Chain of Thought is embarrassingly simple.
Just add “Let’s think step by step” to your prompt.
Without CoT:
A bat and ball cost $1.10 together. The bat costs $1 more than the ball. How much does the ball cost?
AI often blurts out “$0.10”—which is wrong.
With CoT:
A bat and ball cost $1.10 together. The bat costs $1 more than the ball. How much does the ball cost? Let’s think step by step.
Now AI will reason through it:
- Let’s call the ball’s price “x”
- The bat costs $1 more, so the bat is “x + $1”
- Together they cost $1.10, so: x + (x + 1) = 1.10
- That means 2x + 1 = 1.10
- So 2x = 0.10
- Therefore x = $0.05
The ball costs $0.05 (and the bat costs $1.05).
Same question. Correct answer this time. All because we asked AI to show its work.
Few-Shot CoT: Show Examples First
For even better results, show AI an example of the reasoning you want before asking your question.
This is called “few-shot” CoT—you provide one or more worked examples, then ask your actual question.
Example prompt:
Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls.
Each can has 3 tennis balls. How many tennis balls does he have now?
A: Let's work through this step by step.
- Roger starts with 5 tennis balls
- He buys 2 cans, each with 3 balls
- That's 2 × 3 = 6 new balls
- Total: 5 + 6 = 11 tennis balls
Q: The cafeteria had 23 apples. They used 20 to make lunch and bought
6 more. How many apples do they have?
A:
The AI will now follow the same step-by-step format for the new question.
Few-shot CoT takes more effort to set up, but it’s more reliable—especially for complex or unusual problems where “let’s think step by step” might not trigger the right approach.
When Chain of Thought Works Best
CoT isn’t a magic wand for everything. Here’s where it really shines:
Math and Logic
Anything with multiple steps. Word problems, calculations, logical deductions.
Try: “Calculate the compound interest on $10,000 at 5% over 3 years. Think through each year step by step.”
Code Debugging
Understanding what code does wrong requires tracing through execution.
Try: “Here’s my code [paste code]. It’s giving the wrong output. Walk through what happens step by step when I call this function.”
Analysis and Decision-Making
Complex decisions have multiple factors. CoT helps weigh them systematically.
Try: “I’m deciding between these three job offers [details]. Think through the pros and cons of each step by step before recommending one.”
Reading Comprehension
Long documents with implicit information need careful reasoning.
Try: “Based on this contract [paste text], determine who is liable if the delivery is late. Work through the relevant clauses step by step.”
When to Skip CoT
Not everything benefits from step-by-step thinking:
- Simple factual questions: “What’s the capital of France?” doesn’t need reasoning.
- Creative writing: Stories and content benefit more from examples and tone guidance than logical steps.
- Quick tasks: If you just need a list or a simple rewrite, CoT adds unnecessary length.
- Smaller models: CoT works best with capable models. Smaller AI models may produce incoherent reasoning chains.
Use CoT for problems that involve reasoning. Skip it for everything else.
Five Ready-to-Use CoT Prompts
Copy these directly:
1. Math Word Problem
[Your problem here]
Solve this step by step. Show your work for each calculation
before giving the final answer.
2. Debugging Help
Here's my code:
[paste code]
Here's the error:
[paste error]
Walk through what the code does step by step, identify where
it goes wrong, then explain the fix.
3. Decision Analysis
I need to decide between: [option A] and [option B]
Context: [your situation]
Analyze this step by step:
1. List the pros and cons of each option
2. Consider my specific situation
3. Identify the key tradeoffs
4. Make a recommendation with reasoning
4. Document Analysis
Based on this document:
[paste document]
Answer this question: [your question]
Work through the relevant sections step by step before
giving your final answer. Quote specific passages that
support your conclusion.
5. Complex Explanation
Explain [complex topic] to someone who understands [their background].
Break this down step by step:
1. Start with what they already know
2. Build up to the new concept
3. Use analogies where helpful
4. End with how it all connects
Advanced: Self-Consistency
For high-stakes problems, there’s a more sophisticated technique: self-consistency.
Instead of generating one reasoning chain, you ask AI to solve the problem multiple times (with some randomness) and then take the most common answer.
It’s like asking five experts to independently work through a problem. If four of them reach the same conclusion, that’s probably right.
This is harder to do manually but worth knowing about for critical decisions.
The Takeaway
Chain of Thought prompting is one of those rare techniques that’s both simple and powerful.
For reasoning tasks—math, logic, debugging, analysis—it often makes the difference between wrong answers and right ones. And all it costs is a few extra words in your prompt.
Next time you’re frustrated with AI’s reasoning, try these magic words:
“Let’s think step by step.”
You might be surprised what happens.