What LLMs Can Do
The public conversation about large language models oscillates between two extremes: breathless hype claiming LLMs can do everything, and dismissive skepticism claiming they do nothing genuinely useful. Neither is accurate. LLMs have specific, real capabilities that are genuinely impressive and practically valuable. They also have specific, real limitations that make uncritical reliance on them dangerous. This lesson covers the capabilities honestly — the next covers the limits with equal rigor.
Fluent Text Generation and Transformation
The most immediate and obvious capability of an LLM is producing fluent, grammatically correct, contextually appropriate text. This encompasses several distinct tasks: Drafting: Given a topic, tone, audience, and purpose, an LLM can produce a draft of an email, essay, report, speech, or creative work. The draft may need revision, but it can provide a starting point that is structurally sound and stylistically consistent. Summarization: Given a long document, an LLM can produce a condensed version that preserves the main points. This is one of the strongest LLM capabilities — research literature, legal documents, and meeting transcripts can be reduced to actionable summaries in seconds. Translation: LLMs trained on multilingual text can translate between languages at quality levels that rival or exceed earlier specialized translation systems for major language pairs. For less common languages, quality degrades but is often still useful. Style transfer: An LLM can rewrite a document in a different register — converting technical jargon to plain language, rewriting formal prose as conversational text, or adapting content for a different reading level. Editing: LLMs can identify grammatical errors, awkward phrasing, and structural weaknesses in text, and suggest or implement revisions. They can also maintain consistency (checking that character names are spelled consistently throughout a story, for example).
LLMs are trained almost entirely on text, optimizing to produce text that is statistically likely given context. Text generation, transformation, and analysis tasks play directly to this strength. The model has seen enormous quantities of good writing, translation pairs, summaries, and edited documents — and has learned the statistical patterns that distinguish them.
Explanation and instruction: LLMs can explain concepts across an enormous range of domains — science, history, mathematics, law, medicine, programming — at varying levels of depth and technicality. They can adapt an explanation to a specified audience, give analogies, anticipate common misconceptions, and answer follow-up questions. This makes them useful as on-demand tutors and reference assistants. Code generation: Among LLM capabilities, code generation has seen some of the most dramatic real-world impact. LLMs trained on billions of lines of code can write, explain, debug, translate between programming languages, and document code. For common tasks in well-represented languages (Python, JavaScript, SQL), LLM-generated code is often correct on the first try. For unusual tasks or niche languages, it is often a useful starting point that requires review. Reasoning assistance: LLMs can assist with structured reasoning — decomposing a problem, identifying relevant considerations, generating counterarguments, or checking the consistency of an argument. They do not always reason correctly, and they can be confidently wrong, but they can also catch errors a human missed, suggest angles that were not considered, and help organize complex problems. Brainstorming and ideation: Because LLMs have broad exposure to human knowledge and creative output, they are unusually good at generating diverse ideas, analogies, framings, and examples. The ideas vary in quality, but the breadth and speed with which they can produce candidates is often valuable in early-stage creative or analytical work.
Information Synthesis and Pattern Recognition
LLMs can synthesize information across a context window — drawing connections between concepts mentioned at different points in a long document, summarizing multiple perspectives on a question, or identifying thematic patterns across many examples provided at once. For structured data provided as text (tables, lists, JSON), LLMs can perform extraction, classification, comparison, and simple analysis. Given a list of customer reviews, an LLM can identify recurring themes, classify sentiment, and extract product features mentioned — without being explicitly programmed for each task. In all of this, one principle applies: LLMs do these things well when the task is well-represented in their training distribution. They have seen enormous amounts of good summarization, explanation, code, and translation. They are correspondingly less reliable when the task requires knowledge they do not have, reasoning steps they have not been trained on, or information beyond their training cutoff. The capabilities and the limits are two sides of the same coin.
Complete the statement about LLM capabilities.
The best mental model for working with an LLM is not 'query an oracle that knows the answer' but 'collaborate with a fast, broadly-read assistant who needs supervision.' For drafting, explaining, and brainstorming, lean on the LLM's speed and breadth. For facts, calculations, and verification, always check independently.
Which of the following is a genuine LLM capability well-supported by how these models are trained?
An LLM is asked to explain a complex concept in physics to a 12-year-old. It produces a clear, engaging explanation using appropriate analogies. This works well primarily because:
Evaluate an LLM Response
- This activity develops your critical evaluation skills for LLM output.
- Scenario: Imagine you asked an LLM: 'Explain how HTTPS keeps web communications secure, for someone who understands basic programming but has never studied cryptography.'
- The LLM produced this response:
- 'HTTPS uses TLS (Transport Layer Security) to protect your data. When your browser connects to a secure site, it first does a handshake where the server proves its identity using a certificate signed by a trusted authority. Then both sides agree on an encryption key using something called a key exchange — a clever math trick where two parties can agree on a secret number even if someone is listening to their conversation. After that, all data is encrypted with that shared key, so an eavesdropper sees only scrambled bytes. HTTPS also checks that data has not been altered in transit using cryptographic hashes.'
- Step 1: List what this response does well. Be specific.
- Step 2: List what is missing, oversimplified, or potentially misleading. Research any terms you do not know.
- Step 3: Write one follow-up question you would ask the LLM to improve or extend the response.
- Step 4: Identify one claim in the response that you would want to verify independently before using this explanation in a presentation. How would you verify it?