Tinker!
Course
Sign in

Learn by playing with the math.

Every lesson is something you grab, drag, and break until it clicks. Start at f(x) = x². Finish by training a real transformer in your browser.

Start learning for free Explore the course

92 interactive lessons · free in your browser

Tinker the apple, the mascot of Tinker

Every dot is a letter's learned position. Drag any of them and the loss climbs as you pull a letter away from where training put it. Red dots are vowels, blue are common consonants.

What you'll actually build

Three builds. Start with the one you came for.

  1. Backprop, from scratch

    Build micrograd one node at a time. Watch the gradient flow backward through a tanh, see how loss.backward() actually works. The keystone module of the course.

    Module 12 · Backpropagation →

  2. Attention, as a soft dictionary

    The thing you played with up top is an embedding, the foundation of attention. Drag a query vector, watch it pivot toward the right keys. Build the math behind the T in GPT.

    Module 15 · Attention →

  3. A real transformer, in your browser

    Capstone: 4 layers, 4 heads, ~200k params. Trains in ~5 minutes on WebGPU. Generates Shakespeare-flavored nonsense. Yours to keep.

    Module 18 · Capstone →

The whole course

Every lesson. One continuous climb.

From f(x) = x² to a transformer you train yourself. Open any module to see its lessons. Sign in and the map lights up with what you've finished.

5arcs
19modules
92lessons
~28hof interactive math
  1. Arc 0

    Foundations

    Safety floor. Skippable via diagnostic; anyone who can already factor a quadratic starts in calculus.

    3 modules · 11 lessons · ~5h

    1. M0 Diagnostic & Placement A 20-question adaptive assessment that routes you to the right entry point, so you don't sit through fraction arithmetic if you already factor quadratics in your sleep. Placement · ~5 min Take it →
      1. 01 Quantities live on a line 18m
      2. 02 There are only two operations 22m
      3. 03 A fraction is one number 25m
      4. 04 Expressions are trees 20m
      5. 05 Variables, expressions, equations 28m
      1. 01 Same value, different shape 25m
      2. 02 The only shape with a slope 30m
      3. 03 Two lines, three outcomes 25m
      4. 04 Wiring machines together 35m
      5. 05 Quadratics and the parabola 30m
      6. 06 The one identity we live by 40m
  2. Arc 1

    Prerequisite Math

    Trig, calculus, linear algebra, probability, information theory. Every module ends with how it plugs into a transformer.

    7 modules · 34 lessons · ~10h

      1. 01 What a radian actually is 20m
      2. 02 Sine and cosine are coordinates 24m
      3. 03 The wave they make 20m
      4. 04 What rotation actually is 26m
      5. 05 Polar coordinates and the position fingerprint 24m
      1. 01 The zoo of functions, and one grammar 30m
      2. 02 Sequences, series, and the first infinite process 30m
      3. 03 What approaching actually means 30m
      4. 04 The number that makes calculus clean 30m
      1. 01 What is a derivative? 12m
      2. 02 The derivative as a function 10m
      3. 03 Differentiation rules 14m
      4. 04 Product and quotient rules 8m
      5. 05 The Chain Rule 16m
      6. 06 Second derivatives and finding optima 12m
      1. 01 Partial Derivatives: one knob at a time 15m
      2. 02 The Gradient: every partial, bundled into one arrow 16m
      3. 03 Local Linear Models and Saddle Points 16m
      4. 04 Jacobians and the Multivariable Chain Rule 20m
      5. 05 loss.backward() is a Vector–Jacobian Product 20m
      1. 01 What is a vector, really? 14m
      2. 02 Dot product, the alignment number 14m
      3. 03 A Matrix Is a Transformation 18m
      4. 04 Composition, Determinants, and Inverses 22m
      5. 05 Eigenvectors, Change of Basis, and a Glimpse of SVD 22m
      1. 01 What does probability measure? 12m
      2. 02 Joint, marginal, conditional 16m
      3. 03 What you can say about a random variable in one number 18m
      4. 04 Maximum likelihood: fit the data, with a formula 14m
      5. 05 Drawing samples (and what could possibly go wrong) 14m
      1. 01 How surprised should you be? 12m
      2. 02 Two distributions in the same room 15m
      3. 03 Why classifiers have a softmax head 12m
      4. 04 Perplexity and the floor of English 14m
  3. Arc 2

    Machine Learning Foundations

    Optimization, neural networks, backpropagation from scratch, training dynamics. The keystone arc, where micrograd gets built live.

    4 modules · 20 lessons · ~6h

      1. 01 What does it mean to minimize a function? 16m
      2. 02 Batches, Stochastic, and the Noise Ball 16m
      3. 03 Momentum: Gradient Descent with a Memory 18m
      4. 04 Per-parameter Steps: RMSProp, and then Adam 20m
      5. 05 Schedules, Pathologies, and What nanoGPT Actually Does 18m
      1. 01 What's a perceptron, really? 15m
      2. 02 The XOR moment 18m
      3. 03 Why a stack of linear layers is just one linear layer 12m
      4. 04 ReLU, and the activation zoo 18m
      5. 05 Forward pass, end to end 22m
      1. 01 Draw the graph 14m
      2. 02 Walk it backward 16m
      3. 03 Build the Value class 30m
      4. 04 From scalars to tensors 18m
      5. 05 Check it, break it, fix it 18m
      1. 01 When your model memorizes the training set 12m
      2. 02 Two ways to keep weights small (and what they really mean) 14m
      3. 03 Why your gradients explode 15m
      4. 04 Normalize what, exactly? 15m
      5. 05 The tricks that make depth possible 16m
  4. Arc 3

    Sequence Models & Transformers

    Bigrams, RNNs, attention, the transformer block. By the end of this arc you've built a transformer architecture in-browser.

    3 modules · 17 lessons · ~4h

      1. 01 Predicting the next character 14m
      2. 02 Two paths to the same model 18m
      3. 03 Loss, perplexity, and how to sample 16m
      4. 04 Why bigrams aren't enough 16m
      5. 05 Carrying memory forward, the RNN 16m
      6. 06 Training the RNN, and where it breaks 16m
      1. 01 Soft dictionary lookup 13m
      2. 02 Why divide by √dₖ 12m
      3. 03 Three projections of the same X 18m
      4. 04 Position, three ways 14m
      5. 05 Multi-head: parallel subspaces, same budget 10m
      6. 06 The cost and the cache 10m
      1. 01 One block, top to bottom 15m
      2. 02 Why pre-LN 14m
      3. 03 The residual stream as the object 18m
      4. 04 Stacking N and the full forward pass 16m
      5. 05 Counting the cost 12m
  5. Arc 4

    Capstone Build & Train

    Tokenization, sampling, and a real training run that produces a model you keep.

    2 modules · 10 lessons · ~3h

      1. 01 Text becomes integers 14m
      2. 02 How a tokenizer is built 18m
      3. 03 Wrapping the training loop around the transformer 18m
      4. 04 What the loss curve is telling you 12m
      5. 05 How distributions become text 22m
      1. 01 Press start 15m
      2. 02 Watch it learn 25m
      3. 03 Your checkpoint 15m
      4. 04 Now make it talk 30m
      5. 05 The credits roll 18m
Open the full course →

The first lesson takes fifteen minutes. The capstone is a GPT you trained.

All 92 lessons run free in your browser, on the GPU you already have. No install, no setup, no video lectures.

Start learning for free Find your level · ~5 min