Plans / Plan 02 · Six weeks

Understand the model your agent runs on

You end up withYou've written backprop, a small GPT and a byte-pair tokenizer, and you can read GPT-2's forward pass line by line.

The steps

628 points·ML course, step 1: receipts and HN's criticisms

A two-layer network with the gradients done by hand, so the next step has something to automate.

171 points·ML course, step 2: receipts and HN's criticisms

Backpropagation from first principles, in about 150 lines. Everything later is this, scaled up.

50 points·ML course, step 3: receipts and HN's criticisms

Do: the first two videos (bigram counts and the MLP).

The language-model setup the GPT video builds on: tokens in, next-token probabilities out.

1,112 points·ML course, step 4: receipts and HN's criticisms

Self-attention and the transformer block, written out. This is the architecture your agent runs.

722 points·ML course, step 5: receipts and HN's criticisms

Tokenization explains a surprising share of odd model behavior, from spelling to arithmetic.

1,563 points·ML course, step 6: receipts and HN's criticisms

The check: GPT-2 with real weights in 60 lines of NumPy. If you can read every line, you've got it.

Stages 4 and 5 of the ML course: training a real model, Llama's architecture and fast inference. They need more time and a GPU.

Steps 7 and 8 of the ML course train a real one.

Plan written 2026-11-09. Points are each source's HN total, from its course's receipts.