Tool Detective
Every great detective does the same thing: they do not just accept the first story they hear. They investigate. They test. They ask questions. They look for clues. Today, you are becoming a Tool Detective. Your job is to investigate an AI tool — not just use it, but figure out how it actually works by running your own experiments and asking sharp questions. Are you ready? Let us open the case file.
A good detective never assumes. They gather evidence, test ideas, and follow the clues wherever they lead. That is exactly how you become an expert on any tool.
Your Detective Toolkit
Before you start investigating, every great detective needs tools. Here is your Tool Detective toolkit. Clue 1 — Ask about limits: Does the AI know about things that happened very recently? Ask it about something from last week. Its answer tells you about its knowledge cutoff — the point in time when it stopped learning new things. Clue 2 — Test for confidence vs. accuracy: Ask the AI something you already know the answer to. Does it get it right? Does it sound confident even if it gets it wrong? This tells you how much to trust its tone. Clue 3 — Push with How and Why: After any answer, ask How do you know? This can reveal whether the AI has solid reasoning or whether it is guessing. Clue 4 — Ask about itself: Ask the AI what it is good at and what it is not so good at. Surprisingly, many AI tools will give you an honest answer — and that honesty is very useful. Clue 5 — Look for patterns: Ask three similar questions in different ways. Do the answers match? Inconsistencies are clues that something deserves more investigation.
The Full Tool Detective Investigation
- You are going to run a real investigation on an AI tool. Follow all five steps and write your findings in a Detective Report.
- STEP 1 — Introduction: Write the name of the AI tool you are investigating at the top of your report.
- STEP 2 — Test the limits: Ask the AI about something very recent — something that happened in the last few weeks. Write down what it says. Does it know? Does it admit it does not know? Write your finding.
- STEP 3 — Test confidence vs. accuracy: Think of a fact you are certain about — maybe from your favorite subject at school. Ask the AI about it. Is the answer correct? How confident did the AI sound? Write your finding.
- STEP 4 — Ask how it knows: Take one of the AI's answers and ask it: How do you know that? Write down whether the explanation made sense or whether something seemed off.
- STEP 5 — Ask about itself: Ask the AI: What are you good at? What are you not so good at? Write down what it says. Does it seem honest?
- STEP 6 — Your conclusion: Based on your investigation, write two things this AI is great for and one thing you should always double-check when using it.
- Share your Detective Report with a partner. Did you discover the same things, or different things? Why might two detectives investigating the same tool reach different conclusions?
What Your Investigation Reveals
When you run these experiments, you are doing something scientists and engineers do too — they call it evaluation, which means testing a tool carefully to understand its strengths and weaknesses. What you will probably find is that the AI is impressively good at some things: explaining ideas, helping you brainstorm, writing in different styles. And you will find that it is shaky in other areas: very recent events, precise calculations, remembering specific details from long ago. Both findings are valuable. Knowing strengths tells you when to lean on the tool. Knowing weaknesses tells you when to verify or look elsewhere. That is exactly the knowledge a smart, sovereign user has.
Terms
Definitions
Drag terms onto their definitions, or click a term then click a definition to match.
What does a 'knowledge cutoff' mean for an AI?
Why is it useful to ask the AI what it is NOT good at?