How AI Actually Works

Recall: The Communication Gap

In Lesson 1, we saw that the difference between effective and ineffective AI use comes down to communication. But why does communication matter so much?

To answer that, we need to understand what’s actually happening when you chat with AI.

By the end of this lesson, you’ll understand:

• How AI generates responses (the simple version)
• Why AI can be confident but wrong
• What the “context window” is and why it matters

How AI “Thinks” (It Doesn’t)

Let’s clear up a common misconception: AI doesn’t think, understand, or know things the way humans do.

Here’s what actually happens when you send a message to ChatGPT, Claude, or any large language model:

1. Your text gets broken into tokens (roughly, pieces of words)
2. The AI predicts the most likely next token based on patterns it learned during training
3. It generates that token, then predicts the next one
4. Repeat until the response is complete

That’s it. The AI is playing a very sophisticated game of “what word probably comes next?”

A Simple Analogy

Imagine you’re playing a word prediction game. I give you: “The cat sat on the…”

You’d probably guess “mat” or “floor” or “couch”—common things cats sit on.

AI does the same thing, but with trillions of examples from books, websites, and documents. It’s seen so many patterns that it can generate remarkably coherent text. But it’s still just predicting likely sequences—not reasoning about truth.

Why This Matters: The Confidence Problem

Here’s the crucial insight: AI generates what’s likely, not what’s true.

If you ask “Who invented the telephone?”, the AI will generate “Alexander Graham Bell” because that’s the statistically likely answer based on its training data.

But if you ask about something obscure, or something that requires current information, the AI might generate a confident-sounding response that’s completely wrong. This is called hallucination—the AI isn’t lying; it’s just generating plausible-sounding text.

Key Insight: AI confidence ≠ AI accuracy. The same confident tone that delivers correct answers also delivers incorrect ones.

Quick Check

Before reading on, answer this: If an AI gives you a confident, detailed response, should you assume it’s accurate?

(Think about it before scrolling…)

Answer: No. You should verify important information, especially facts, statistics, quotes, or anything with real-world consequences. The AI’s confidence level tells you nothing about accuracy.

The Context Window: Your AI’s Working Memory

Every AI has a context window—the maximum amount of text it can consider at once. Think of it as the AI’s working memory.

What does this mean for you?

1. Everything in your conversation counts. Your prompt, the AI’s response, follow-up questions—it all uses context space.

2. Longer isn’t always better. A focused 500-word prompt often beats a rambling 2,000-word one.

3. Include what matters. Since space is limited, be intentional about what context you provide.

Practical Example

Say you want help editing a document. You could:

❌ Paste your entire 50-page report and say “make it better”

✅ Paste the specific section that needs work and explain exactly what kind of editing you want

The second approach uses context more efficiently and gets better results.

What AI Can and Can’t Do

Based on how AI works, here’s a realistic picture of its capabilities:

AI is Great At:

• Generating text in various styles and formats
• Summarizing long documents
• Brainstorming ideas and alternatives
• Explaining concepts in different ways
• Transforming content (rewriting, translating, formatting)
• Following patterns you demonstrate

AI Struggles With:

• Current events (training data has a cutoff date)
• Math (it predicts likely digits, not calculates)
• Facts about obscure topics (hallucination risk)
• Counting (seriously, try asking it to count words)
• Knowing what you mean (it only sees what you write)

The Implications for Prompting

Understanding how AI works changes how you should use it:

1. Be explicit. AI can’t read your mind—it only sees your text.

2. Provide context. Give AI the information it needs to generate relevant responses.

3. Verify important facts. Don’t trust AI for accuracy without checking.

4. Use AI for transformation, not creation. AI excels at reshaping content you provide, not inventing accurate new information.

5. Iterate. If the first response isn’t right, refine your prompt—you’re tuning a prediction engine.

Key Takeaways

• AI generates responses by predicting likely next words, not by reasoning or understanding
• AI can be confidently wrong (hallucination)—always verify important information
• The context window is the AI’s working memory; use it intentionally
• Knowing AI’s strengths and limits helps you choose the right tasks for it

Up Next

Now that you understand how AI works, you’re ready to learn the fundamentals of effective prompting. Lesson 3 covers the core techniques that separate good prompts from great ones.