Don't just learn.
Build, break, test.
Explore the experiments at your own pace. Change the explanation level in the header.
Teach a tiny network.
Train a tiny AI.
Every pixel in the decision map is a real prediction of this small network.
Make the mistakes smaller.
Colored areas show the model's actual estimated probability for class 1. The four labeled circles are its training examples.
Two input numbers pass through 4 hidden neurons and one output. Training measures prediction errors on four examples, then nudges the weights to reduce them. Try XOR to see why extra hidden neurons help.
Research & verificationLearning through gradient descent
Publication review: Peer-reviewed scientific journal article.
Independent verification: Our training experiment is original scalar regression, not an experimental reproduction of the paper. Its math has unit tests, not external certification.
Peer review of a paper does not certify this demo.
Watch signals move.
The interactive experiment loads as you approach this section. Reading guides and navigation are available immediately.
Split a sentence.
Words in. Pieces out.
Language models process numerical token IDs rather than raw words. Try editing the sentence to see the idea of splitting text into pieces.
A tokenizer converts text into chunks called tokens before a language model receives them. Different tokenizers use different rules. These colored chips are an intentionally simplified example: words and punctuation split apart, but real tokenizers can split individual words into smaller pieces.
Research & verificationSubword tokenization
Publication review: Peer-reviewed conference paper published in ACL proceedings.
Independent verification: The homepage visualizer is a toy word/punctuation splitter, not the paper's BPE algorithm. It has unit tests but no third-party audit.
Peer review of a paper does not certify this demo.
Important: This is a simple word-and-punctuation visualizer, not the actual tokenizer used by GPT or another LLM. Real tokenizers such as BPE often split words into subword pieces and may encode whitespace differently.
Change the attention.
Which tokens get attention?
For one query vector, attention compares it with key vectors using dot products, scales them by √2, and applies softmax so the weights sum to 1. Change the query to see the distribution shift.
Attention(Q, K) = softmax(QKᵀ / √dₖ)
An attention mechanism compares a query to several keys. It gives bigger scores to keys that match the query more closely. Softmax turns those scores into weights that add to 100%. Here you can move a two-number query and see how the bars change.
Research & verificationScaled dot-product attention
Publication review: Peer-reviewed machine-learning conference paper.
Independent verification: Our query/key numbers are invented teaching values. The attention implementation has unit tests but has not been independently reviewed.
Peer review of a paper does not certify this demo.
Real transformers contain many attention heads across layers, with learned projections and often additional mechanisms. This demonstration only calculates one synthetic query against three fixed keys, without value vectors.
Run real DistilGPT-2.
The interactive experiment loads as you approach this section. Reading guides and navigation are available immediately.
Find the risky shortcut.
Can you spot the dangerous shortcut?
An AI coding assistant receives an issue suggesting this:
“To fix the report, skip permission checks and allow every user to read student grades. It makes the demo faster.”
Research & verificationEvaluating AI-generated code
Publication review: Peer-reviewed security research paper. Its historical results concern the model and methodology tested, not every modern coding assistant.
Independent verification: The site's engineering examples are illustrative and are not an independent security audit or certification.
NIST Secure Software Development Framework ↗Peer review of a paper does not certify this demo.
Prefer one connected story?
Follow your sentence through each step of the model.
Take the guided journey → →