Quick question before you read on: how AI-literate are you, one to ten?
Hold that number. There’s a good chance it’s wrong, and wrong in one specific direction.
Researchers at Aalto University ran a proper version of this test. Across two studies (around 246 people, then 452), they had people work through LSAT-style logic problems with ChatGPT, then guess how well they’d done. The AI bumped their scores by roughly 3 points. But they overestimated their own score by about 4. They walked away thinking the machine’s win was theirs.
Here’s the part that should stop you. Normally the least skilled people over-rate themselves the most. That’s the famous Dunning-Kruger effect. In this study it flipped. The people who called themselves the most AI-literate had the worst read on their own performance. More confident, less accurate. Welsch and his co-authors gave the paper a blunt title: “AI makes you smarter but none the wiser.”
Why does it flip? One-shot offloading. People asked once, took the answer, and never checked. No checking, no feedback. No feedback, no way to calibrate.
So if heavy use quietly inflates your self-rating, a “rate yourself 1 to 5” quiz is worse than useless. It measures your confidence, which is the exact thing that’s broken. A real check has to ask something confidence can’t fake: what did you actually do last time?
That’s what this is. Eight questions, about five minutes, built around the four skills that make up AI fluency. Not a vibe check. A behavior check.
How this self-check works
Two questions per skill. Eight total. Each one points at a behavior, not a feeling, so the honest answer is either “yeah, I did that” or “no, I didn’t.” No comfortable middle.
And you won’t get one score at the end. You’ll get a profile. Most people are strong on one or two of the four skills and quietly skipping the rest. Knowing which one you skip is the whole prize.
This isn’t just our opinion. The researchers who build AI-literacy tests for a living reached the same place. The strongest academic scales, like SAILS (twelve real scenarios) and the Meta AI Literacy Scale, deliberately avoid “rate your skill” and drop you into situations instead, because self-ratings just don’t track with real ability. The four skills themselves trace back to Ng and colleagues in 2021, whose four pillars (know, use, evaluate, and the ethics around it) mirror Anthropic’s four D’s. Want the deep version? Here’s the full guide to the 4 skills. The short version’s below, one line each.
The self-check
Read each pair. Answer out loud if you can. A vague answer counts as a no.
Delegation: did you choose, or just reach?
Think back to the last task you handed AI. Did you actually decide it was worth handing over? Or did you ask because the box was sitting right there?
Then this one: name a single thing this week you could have given AI and deliberately kept for yourself. If nothing comes to mind, that’s your answer. Reaching for it on everything isn’t fluency. It’s a reflex.
Weak spot here? The skill you want is knowing when to use AI and, just as much, when not to. Start with AI Fundamentals.
Description: do you steer, or just regenerate?
Last time the first answer missed, what did you change on the retry?
Be honest about the answer. If it was “nothing, I just hit regenerate and hoped,” there’s the gap. Steering means changing the input, adding the context and the example of good, not rerolling the dice on the same prompt.
Weak spot here? This is the prompting-and-context skill, and it’s the one people mistake for the whole game. Prompt Engineering goes deep on it.
Discernment: when did you last catch it lying?
When did you last catch AI being confidently, specifically wrong? A fake citation, a number that didn’t add up, a name it invented on the spot.
And how did you catch it? If you can’t remember catching one lately, be honest with yourself: the likely reason isn’t that the AI stopped making things up. It’s that you stopped looking.
Weak spot here? Learn to spot hallucinations and pressure-test output before you trust it. AI Fundamentals covers it.
Diligence: did you own it, or just send it?
Before your last AI-assisted thing left your hands, an email, a report, a slide, did you verify the facts you couldn’t personally vouch for?
And did you handle the rest of it? Say AI helped, where that mattered. Make sure you weren’t about to paste in something private, a password, a client’s data, an unreleased number.
Weak spot here? This is the review-and-accountability habit. AI Fundamentals again.
Let the tool measure your habits (with one caveat)
Your memory of last week is its own kind of self-report, and you now know how much to trust those. So here’s a way to check yourself against actual data: let the tool show you what you really did.
On July 9, Anthropic shipped a feature called Reflect. People started calling it “Claude Wrapped,” or Spotify Wrapped for AI, and that’s about right. It’s a dashboard in Settings on Claude web and desktop, free tier included, that reads your usage back to you: your topics, your patterns (most active day, peak hour), and what you spent time on by category, across a one, three, six, or twelve-month window. It needs Memory switched on to work.
The part that matters for us: Anthropic maps it straight onto the same four D’s. Reflect will literally tell you how your Delegation and Description showed up this month.
It even hands you a reflective prompt now and then. One of them: What’s one thing you want to keep doing yourself, even if Claude could do it faster? That’s a genuinely good question to sit with.
Now the caveat, because it’s a real one. TechCrunch called Reflect a way of “quietly selling you on AI,” and they have a point. Look at the screenshot again. Those buttons, “Create a focused project” and “Copy a prompt,” nudge you deeper into Anthropic’s own paid features. Never toward a competitor. The whole thing runs on Memory, which you have to leave on. It’s a mirror, sure. But it’s a mirror the store installed, and it wants you to buy more.
(One small thing so you’re not caught out: a cumulative “time spent” view is announced as coming soon, not live yet. Don’t go hunting for it.)
So use Reflect. It’s genuinely handy for seeing your own patterns. Just don’t let the vendor be the one who grades your fluency. That’s the whole reason for a neutral, behavior-anchored check like this one.
What to do with your result
Depends who you are.
- If you manage people: your soft spots are probably Delegation and Diligence, because your real job is deciding what work AI should touch and setting the norm for checking it. Run the four questions in your next team meeting, out loud. Watch how few people can answer the Discernment one.
- If you’re job-hunting: employers are done being impressed by “I use ChatGPT.” They’re screening for Discernment and a result they can see. Pick your weakest skill, fix it this month, and walk in with a story. (The money’s real too, if you need the push: AI-fluent workers earn more.)
- If you’re a student: bet on Discernment. The tools will change five times before you graduate. Knowing when the answer is wrong won’t go out of date.
What a self-check can’t do
Let me be straight about the limits, because I’d rather you trust this than oversell it.
It’s not a certificate. Nobody’s putting “passed my own quiz” on a resume. This is a diagnostic, not a credential.
One good week isn’t fluency. You can ace all eight questions because you happened to be careful on one project. Fluency is the habit held across months, when you’re tired and rushing and the deadline is now.
It measures habits, not outcomes. Doing the right things is the best predictor we’ve got, but the check can only see your process, not your actual results.
And it goes stale. New tools, new failure modes, new things worth checking. Re-take it every few months, and watch how your honest answers drift. That drift tells you more than any single score.
The bottom line
Using AI a lot doesn’t make you good at judging your own AI skill. If anything, the Aalto data says it makes you a little worse: more sure, less right. So set aside the number you gave yourself at the top of this page. The better question was never how good you feel. It’s what you did last time. Did you choose the task, steer the answer, catch the mistake, own the result?
If a couple of those landed as an honest “no,” good. That’s what a check is for. Now you know where to aim.
Want the structured version instead of a gut check? Our AI Fundamentals course walks all four skills from zero. Start with your weakest one. That’s where the fastest gains are hiding.
