Environment Setup

Before we start building, let's get your environment ready. We'll use standard Python with PyTorch — no notebooks required.

Why These Tools?

Python 3.9+ — Readable syntax, rich ecosystem, industry standard (GPT, LLaMA, Claude all use Python)
PyTorch — Pythonic design, dynamic computation, research favorite with great documentation

Clone the Repository

Get the complete source code:

bash

1git clone https://github.com/slahiri/small_calculator_model.git
2cd small_calculator_model

Well-commented code: The source files in src/ contain detailed comments explaining every component. This tutorial shows simplified snippets — refer to the GitHub repo for the full, annotated code.

Install Dependencies

Create a virtual environment and install PyTorch:

bash

1python -m venv venv
2source venv/bin/activate  # On Windows: venv\Scripts\activate
3pip install torch

Verify Setup

Run this to verify PyTorch is working:

bash

1python -c "import torch; print(f'PyTorch {torch.__version__} ✓')"

Project Structure

small_calculator_model/
├── config/
│   ├── config.json      # Model hyperparameters
│   └── vocab.json       # Vocabulary mapping
├── src/
│   ├── tokenizer.py     # Text → token IDs
│   ├── model.py         # Transformer architecture
│   ├── data.py          # Training data generation
│   ├── train.py         # Training loop
│   └── generate.py      # Inference & evaluation
├── tests/               # 78 pytest tests
│   ├── test_tokenizer.py
│   ├── test_model.py
│   ├── test_data.py
│   ├── test_train.py
│   └── test_generate.py
└── app.py               # Gradio web demo

Run Tests

The project includes 78 tests. Run them to verify everything works:

bash

1pip install pytest
2pytest tests/ -v

No GPU required. This model is tiny (~100K parameters) and trains in minutes on CPU.

Helpful?

Overview The Vocabulary