Module Check
You have covered the full arc of large language models: from the probabilistic foundation of what a language model is, through tokenization and the Transformer architecture, to pretraining and alignment, and finally to the genuine capabilities and limits of these systems and how to work with them critically. This module check is not a formality. It asks you to retrieve and apply everything — to demonstrate not just that you saw the material but that you can reason with it.
Flashcards — click each card to reveal the answer
Recall and Reasoning
A language model assigns probability 0.42 to the word 'rain' and 0.003 to the word 'telescope' as the next token after 'The forecast calls for.' What does this tell us about how the model works?
The word 'antidisestablishmentarianism' is unlikely to appear as a single token in an LLM's vocabulary. What will the tokenizer most likely do with it?
Two researchers debate whether LLM capabilities are due to 'real understanding' or 'sophisticated pattern matching.' Which response best represents the current state of knowledge?
A model was aligned using RLHF. Human raters consistently preferred longer, more detailed answers. A user now notices the model gives unnecessarily verbose responses to simple questions. This is best explained by:
A student claims: 'The reason chain-of-thought prompting helps is that it gives the model more time to think.' A classmate counters: 'No, it is because generating intermediate reasoning steps produces token context that makes the final correct answer statistically more likely.' Which student is correct?
A journalist uses an LLM to draft a story and includes a statistic the LLM provided. The statistic turns out to be hallucinated. Who is responsible?
Every lesson in this module connects to one central insight: a large language model is a statistical system trained to predict plausible text. Its capabilities flow from this — it has learned the patterns of human language, knowledge, and reasoning across an enormous training corpus. Its limitations flow from the same source — it predicts what is likely, not what is true; it has no access to the external world; it cannot verify its own outputs. The most sophisticated LLM user is one who holds both of these truths simultaneously: genuine capability and genuine limitation, neither dismissed nor overestimated.
Capstone: Brief an AI Policy Committee
- You have been asked to brief a school's AI Policy Committee — three administrators, two teachers, and two parents — on what they need to understand about large language models before deciding on an AI use policy for students.
- Your brief must address all of the following in plain, accurate language (no jargon without definition):
- 1. What a language model fundamentally is and how it produces text (one paragraph, accurate and accessible).
- 2. Three specific, concrete capabilities that make LLMs educationally valuable — with a concrete example of each.
- 3. Three specific, concrete limitations that require policy attention — with a concrete example of each and the potential harm if the limitation is not accounted for.
- 4. Two specific policy recommendations. Each recommendation must be actionable (a specific rule or procedure) and grounded in the technical reality you described above.
- 5. One open question — something about LLMs that experts genuinely disagree about or do not yet know — that the committee should be aware of as the technology continues to evolve.
- Write the brief in full sentences. Aim for approximately 500-600 words. You will be evaluated not only on technical accuracy but on how clearly and honestly you communicate genuine complexity to a non-expert audience.