1.2
Why It Sometimes Sounds Smart and Is Still Wrong
The thing that explains almost everything else.
What you'll leave with
By the end of this lesson, you'll understand the autocomplete model well enough to use AI more intelligently, and to start anticipating when it might fail.
Why this matters
There is one idea about how AI works that will change how you use it. You do not need to understand the engineering. You just need this.
The idea
Generative AI was trained on an enormous amount of text: books, articles, websites, conversations, code. During that training, it learned patterns: what words tend to follow what other words, what responses tend to follow what kinds of questions.
When you ask it something, it does not look up the answer. It generates a response that fits the pattern of what a good answer to your question would look like.
A useful, if slightly oversimplified, way to think about it: it is a very sophisticated autocomplete.
What to know
This explains several things at once.
- Why it sounds natural and fluent: because it learned from human writing.
- Why it is good at drafting, summarizing, and explaining: these are all language tasks that follow patterns.
- Why it can be confidently wrong: it is generating what a correct answer would look like, not verifying that it actually is one.
That last point is the most important thing in this course. We will come back to it.
Example
Ask it something well-known:
What is the capital of Australia?
It will say Canberra, correctly and confidently.
Now ask it something obscure: a precise detail about your town, a statistic from your field, a fact about someone who is not famous:
What year was the [your local landmark] built, and who designed it?
It may be right. It may also be wrong in a way that sounds exactly as confident as Canberra did. Both answers arrive with the same tone of certainty. That is the thing to notice: the tone tells you nothing about the truth.
Try this now
Ask AI something you already know the answer to.
Choose something specific: a local fact, a detail from your own life, or something from your field of work.
Notice how it responds. Does it get it right? Does it hedge, or does it sound completely sure?
You are not testing the tool yet. You are building intuition.
Save this
It is generating what a correct answer would look like, not verifying that it actually is one.
Quiet takeaway
Knowing how it works helps you use it better. The pattern-matching model is not a flaw. It is how the tool works, and it has a specific failure mode. Now you know what to watch for.
Next
Knowing how it works helps you use it better. In the next lesson, we look at what it is actually good at, and these are real, practical things that will save you time.