โ† HOME/ PLAIN-ENGLISH DEFINITIONS

The AI glossary. No buzzwords required.

38 definitions explaining the math, terminology and safety issues behind modern AI. Each term links to a fuller lesson or hands-on experiment.

A

Artificial intelligence (AI)

Broad category of systems designed to perform tasks associated with intelligence, such as language generation, perception, prediction and planning. Not all AI systems are language models.

Explore this topic โ†—

Activation function

A mathematical function applied to a neuron's pre-activation value. Nonlinear activations help multi-layer networks represent more complex relationships.

Explore this topic โ†—

Attention

A computation that scores relationships between positions and combines value vectors using normalized weights. It does not by itself prove understanding.

Explore this topic โ†—

AI hallucination

When a generative model produces false or unsupported content, sometimes expressed confidently. A high next-token probability is not a factual guarantee.

Explore this topic โ†—

Authorization

Deciding whether a particular user or system is allowed to perform an action on a specific resource. In AI products, this must be enforced by trusted application code.

Explore this topic โ†—

B

Bias (neural network)

A trainable numerical offset added to a weighted sum before an activation function. This is different from social or statistical bias in a dataset.

Explore this topic โ†—

Byte pair encoding (BPE)

A family of subword tokenization techniques that builds larger units from frequently occurring adjacent sequences. GPT-2 uses byte-level BPE.

Explore this topic โ†—

Backpropagation

Applying the chain rule through a computational graph to calculate gradients of the loss with respect to network parameters.

Explore this topic โ†—

C

Cosine similarity

A mathematical measure comparing the angle between two nonzero vectors. It describes geometric alignment, not guaranteed sameness of meaning.

Explore this topic โ†—

Causal mask

A restriction that blocks attention to future sequence positions during autoregressive language-model prediction.

Explore this topic โ†—

Context window

The token sequence made available to a model for prediction. Length limits and truncation behavior depend on the architecture and inference setup.

Explore this topic โ†—

E

Embedding

A learned numerical vector representing a discrete input such as a token. For DistilGPT-2, the original token embedding matrix has 768 coordinates per ID.

Explore this topic โ†—

Embedding matrix

A table of learned vectors indexed by token IDs. The original DistilGPT-2 token embedding matrix has 50,257 rows and 768 columns.

Explore this topic โ†—

G

I

Inference

Using the model's current learned parameters to calculate a prediction or output. Ordinary inference does not automatically update those weights.

Explore this topic โ†—

L

Large language model (LLM)

A neural language model trained on large collections of token sequences. Many generate text one token at a time; products may add tools, memory or other software around the model.

Explore this topic โ†—

Logit

An unnormalized model score for a possible output. The model's final token logits can be converted into a probability distribution using softmax.

Explore this topic โ†—

Learning rate

A step-size parameter used by an optimizer when changing weights in response to gradients. Too large a value can destabilize training.

Explore this topic โ†—

Loss function

A mathematical objective measuring prediction error or other undesirable behavior. Training generally tries to reduce the loss.

Explore this topic โ†—

M

Machine learning

An approach to building systems that identify statistical patterns from data and use them to make predictions or decisions, rather than relying solely on hand-written rules.

Explore this topic โ†—

Matrix

A rectangular arrangement of numbers. Neural networks use matrix multiplication to compute many weighted combinations efficiently.

Explore this topic โ†—

Model quantization

Representing model parameters using reduced numerical precision to lower storage or compute costs. It can introduce differences from original full-precision values.

Explore this topic โ†—

N

Neural network

A collection of connected numerical operations organized into layers. Networks can learn adjustable weights that transform inputs into outputs.

Explore this topic โ†—

O

Overfitting

When a model adapts strongly to its training data but performs poorly on fresh inputs. Training performance alone does not show generalization.

Explore this topic โ†—

P

Prompt injection

An attempt to use untrusted text or other content to redirect an AI system's behavior or actions. Trusted software controls, not instructions alone, must enforce permissions.

Explore this topic โ†—

Q

Query, key and value (Q/K/V)

Learned projections used in transformer attention. Queries are compared with keys; their normalized scores weight the combination of value vectors.

Explore this topic โ†—

R

ReLU

Rectified Linear Unit: the function max(0, x). It returns zero for negative values and leaves positive values unchanged.

Explore this topic โ†—

S

Sigmoid

A function that maps a real-valued input into a number between zero and one. It often appears in binary classifiers.

Explore this topic โ†—

Softmax

A normalization operation that converts a set of scores into nonnegative values summing to one. Commonly used for attention weights and token probabilities.

Explore this topic โ†—

Sampling temperature

A positive value used to scale model logits before softmax. Lower values make the distribution sharper; higher values make it flatter.

Explore this topic โ†—

T

Tanh

Hyperbolic tangent: a smooth activation mapping values between minus one and one. Used in the hidden layer of our tiny trainable classifier.

Explore this topic โ†—

Token

A piece of text represented by a vocabulary ID. Depending on the tokenizer, it may be a word, word fragment, punctuation, space or byte-related piece.

Explore this topic โ†—

Tokenizer

Software that converts text into a sequence of vocabulary IDs and can reconstruct text from those IDs. Different models use different tokenizers.

Explore this topic โ†—

Transformer

A neural-network architecture that uses attention mechanisms and feed-forward layers to process sequences. GPT-2 belongs to the decoder-only transformer family.

Explore this topic โ†—

Top-k sampling

A decoding method that limits sampling to the k highest-scoring candidate tokens after model prediction.

Explore this topic โ†—

Training

An optimization process that adjusts model parameters using data and a defined objective or loss.

Explore this topic โ†—

V

Vector

An ordered list of numbers. A model can calculate with vectors to represent tokens, features and intermediate states.

Explore this topic โ†—

W

Weight

A trainable numerical parameter that controls how strongly an input contributes to a calculation. Weights acquire useful values through optimization.

Explore this topic โ†—

Prefer a guided explanation?

Read the AI learning guides, start the interactive prediction journey, or check the public research papers and verification disclosures.