1.3 in The AI Operating System
How LLMs Actually Work
Understanding the engine makes you a dramatically better driver
Concept
Tokens, attention mechanisms, transformers — explained entirely through analogy. No math required.
Application
Predict AI behavior before you prompt. Stop being surprised by failures.
Exercise
Token-count 5 prompts. Optimize each for both quality and cost simultaneously.
Deep Lesson Notes
Large language models process text as tokens. A token can be a word, part of a word, punctuation, or symbol. The model reads tokens, calculates relationships between them, and predicts what should come next.
The transformer architecture is powerful because attention lets the model weigh which parts of the context matter most. In plain English: the model scans the conversation and decides what to pay attention to when forming the next response.
This explains many common behaviors. Models are sensitive to wording because wording changes the token pattern. They can lose the plot in long conversations because important details compete for attention. They can imitate expertise because they learned expert patterns. They can fail at exact arithmetic because text prediction is not the same as calculation.
When you understand tokens, attention, and context, prompting stops feeling mystical. You start writing instructions that make the important details easier for the model to notice and harder for it to ignore.
Video Tutorial Blueprint
Videos coming soon
Original AI Academy video lessons are in production.
The written lesson, applied lab, worksheet, working prompt, quizzes, rubrics, and approved third-party references are available now. The original video version of this lesson will be added soon.
Use this structure when we produce the original AI Academy video lesson.
- Animate text turning into tokens.
- Show attention as highlights moving across a prompt.
- Demo messy prompt versus structured prompt.
- Show why output format changes model behavior.
Working Prompt Example
How LLMs Actually Work working prompt
Use this when applying how llms actually work to a real portfolio, business, career, or product workflow.
You are an expert AI Academy mentor helping me complete the lesson "How LLMs Actually Work" from the module "The AI Operating System".
My context:
- Goal: [describe the business, career, creative, research, or technical outcome]
- Audience or user: [who will rely on the output]
- Current inputs: [paste notes, data, draft, link summaries, requirements, or constraints]
- Quality bar: [what a strong result must include]
- Risk: [accuracy, privacy, compliance, brand, safety, cost, or user trust concern]
Task:
1. Explain how this lesson applies to my context using this expert frame: AI literacy starts with understanding capabilities, limits, tool routing, verification, and responsible use.
2. Build a practical workflow inspired by this real-world case: A learner can use AI more safely by matching each task to the right tool and adding a verification step before trusting output.
3. Produce the first version of the artifact: Token-count 5 prompts. Optimize each for both quality and cost simultaneously.
4. Critique the artifact against accuracy, usefulness, originality, risk, and whether a real person would trust it.
5. Improve the artifact using this lab frame: Test a real workflow with and without AI, compare the output, then document the verification method.
6. Identify this likely failure mode and how to prevent it: The failure mode is trusting fluent AI output without checking whether the task needs sources, calculation, current facts, or human approval.
7. Give me a final version, a short checklist, and the next experiment I should run.
Output format:
- Situation summary
- Recommended workflow
- Draft artifact
- Critique
- Improved artifact
- Risk controls
- Portfolio-ready checklist
- Next experimentWhy this works
- It forces the model to work from the learner's real context instead of generic advice.
- It separates drafting, critique, improvement, and risk review into distinct steps.
- It produces a reusable artifact and checklist instead of a one-off answer.
Applied Training Lab
AI Tool Reliability Drill
Choose one everyday task, run it through two AI tools, and compare quality, speed, hallucination risk, and ease of revision.
Source inspiration
Inspired by the training session's hands-on approach to testing AI tools before using them in real workflows.
- Pick a small task you already understand well.
- Run the task in two AI tools with the same inputs.
- Score each output for accuracy, usefulness, clarity, and risk.
- Write a rule for which tool you would use next time and why.
Lab prompt starter
Help me complete the AI Academy applied lab "AI Tool Reliability Drill" for the lesson "How LLMs Actually Work".
My real context:
- Project or workflow: [describe it]
- Audience or user: [describe who benefits]
- Current materials: [paste notes, data, links, rough ideas, or constraints]
- Tool stack: [tools available]
- Definition of done: [what finished looks like]
Use this structure:
1. Translate the lesson into my context using this frame: AI literacy starts with understanding capabilities, limits, tool routing, verification, and responsible use.
2. Apply this scenario: Choose one everyday task, run it through two AI tools, and compare quality, speed, hallucination risk, and ease of revision.
3. Walk me through the lab steps one by one.
4. Use this risk lens: The failure mode is trusting fluent AI output without checking whether the task needs sources, calculation, current facts, or human approval.
5. Produce the final artifact, a review checklist, and the next improvement.Approved YouTube Teachings
But what is a GPT? Visual intro to transformers3Blue1Brown
Best for visual intuition about transformers, token prediction, and why context changes output.
Attention in transformers, visually explained3Blue1BrownBest for seeing how attention lets models weigh context instead of treating every word equally.
Deep Dive into LLMs like ChatGPTAndrej KarpathyBest for learners who want a deeper mental model of tokens, training, inference, and system behavior.
Third-Party Shout Outs
YouTube creators and education channels
External videos are used as learner references only. AI Academy is not affiliated with these creators unless explicitly stated.
OpenAI, Anthropic, Google, IBM, NVIDIA, Hugging Face, DeepLearning.AI, 3Blue1Brown, and Andrej KarpathyShout out to the public educators, labs, and companies whose free materials help learners build a stronger foundation.
Step-by-Step Workflow
- Write one messy prompt with too much background and no clear output format.
- Highlight the tokens that matter most: role, task, context, constraints, examples, output format.
- Rewrite the prompt so the important information appears in a clean structure.
- Ask the model to produce a short answer, then a structured answer, then a JSON-style answer.
- Compare how structure changes quality and consistency.
- Estimate cost risk by marking which prompts are short, medium, or long context tasks.
Practice Lab
- Take five prompts you actually use.
- Cut unnecessary context while preserving the decision-critical details.
- Add a clear output format and test whether the result improves.
Portfolio Deliverable
A Prompt Optimization Sheet showing before/after prompts, what changed, and how the output improved.
Prompt Optimization Sheet
- Original prompt
- Important context
- Removed noise
- New structure
- Output comparison
- Cost/length note
Knowledge Check
1. What is the main professional outcome of "How LLMs Actually Work"?
- Produce a reusable artifact or decision improvement
- Memorize every tool name in the module
- Replace human review entirely
- Use the longest possible prompt
Answer: Produce a reusable artifact or decision improvement
The AI Operating System is designed around practical operating skill: learners should leave with an artifact, workflow, or decision they can reuse.
2. Which review step should happen before using AI-assisted work with real users or business data?
- Check accuracy, usefulness, risk, and human trust
- Publish the first draft immediately
- Remove all constraints from the prompt
- Ignore source quality if the writing sounds confident
Answer: Check accuracy, usefulness, risk, and human trust
AI output becomes professional only after verification, risk review, and context-aware judgment.
3. What is the likely failure mode to watch for in this lesson?
- Skipping verification turns polished AI output into operational risk
- Too much verified evidence
- Too many human approval points
- Over-documenting the final artifact
Answer: Skipping verification turns polished AI output into operational risk