Build Your First LLM from ScratchPart 1 · Section 2 of 9

What We're Building: A Text Calculator

A child robot learning math with number blocks, showing how an LLM learns patterns — A young robot learning the basics — just like an LLM

In this course, you'll build a simple but complete LLM from scratch—a text-based calculator that understands questions like "two plus three" and responds with "five". It's small enough to train on your laptop, yet contains all the core components of GPT and other large language models.

Why a calculator? Because it's the perfect learning project:

Simple to verify — you can easily check if the answers are correct
Clear patterns — the model must learn mathematical relationships, not just memorize
Complete pipeline — covers tokenization, embeddings, attention, and generation
Trainable locally — no expensive GPUs or cloud services needed

Let's trace the complete journey of a single question through our calculator model:

Input	Output
"two plus three"	"five"

At a high level, "five" is the best and closest word to "two plus three" based on the model's training. The model learned that when it sees "two plus three", the most probable next word is "five".

Let's understand how the model finds its path to "five":

Helpful?

What is an LLM?Step 1: Tokenization