Glossary

Every term with its one-sentence definition. These are the same sentences that appear in the hover cards and in search results — written once, reused everywhere.

A

A/B testing: Randomly assigning subjects to two variants and comparing an outcome, so that the difference you measure can be attributed to the change rather than to who happened to see it.
Accuracy: The fraction of predictions a classifier got right — correct predictions divided by total predictions.
Adam: An optimiser that keeps a running average of both the gradient and its square, giving every parameter its own effective step size — the default choice for training neural networks.
Attention: Letting every position compute a weighted average of every other, with the weights produced by comparing a query against keys — a learned, content-based routing of information.
Autocorrelation: The correlation of a series with a shifted copy of itself. It is how you find a cycle you were not told about, and how you discover that your sample size is smaller than you thought.

B

Backpropagation: The chain rule applied backwards through a computational graph. One traversal produces the derivative with respect to every parameter, at roughly the cost of one forward pass.
Bag of words: Text as a vector of word counts with the order thrown away — crude enough that two sentences meaning opposite things collide, useful enough that it was the default representation for thirty years.
Bagging: Fitting the same unstable model to many bootstrap resamples of one dataset and averaging the results — variance falls, bias does not, and nothing about the model itself has to change.
Batch normalization: Re-centring and re-scaling each layer's inputs using the statistics of the current mini-batch — a layer that made much deeper networks trainable and much less sensitive to how they were initialised.
Bayes' theorem: A rule for turning the probability of the evidence given a hypothesis into the probability of the hypothesis given the evidence. The two are not the same, and the gap between them is the prior.
Bayesian linear regression: Linear regression that returns a distribution over coefficients rather than single values — so every prediction carries an honest uncertainty, and the regularization strength is learned, not tuned.
BERT: A transformer encoder pretrained by hiding 15% of the tokens and predicting them from both sides at once, then fine-tuned on a task with one extra output layer and no architectural surgery.
BFGS: An optimiser that builds an approximation of the curvature from the gradients it has already seen, using it to take far better-aimed steps than gradient descent.
Bias-variance tradeoff: The classical account of why a model can be too simple or too complex, and the modern result that breaks it — past the point of fitting the training data exactly, test error can start falling again.
BLEU: A precision score over n-gram overlap with a reference translation, times a penalty for being too short. A parameterised family, not a single number.
Boosting: Fitting weak learners in sequence, each one on a version of the data reweighted to emphasise what the previous ones got wrong, and combining them into a weighted vote.
Bootstrap: Resample your own data with replacement, recompute the statistic, and read the spread. It gives you an interval for statistics that have no formula — and quietly fails for some that do.

C

Central limit theorem: Add up enough independent values from almost any distribution and the sum is approximately normal. Almost is doing real work — the theorem has conditions, and heavy-tailed data breaks them.
Class imbalance: One class is rare, and every default in the toolchain assumes it is not. The accuracy looks excellent, the model predicts the rare class never, and most of the standard fixes only move the threshold.
CLIP: Train an image encoder and a text encoder together so a picture and its caption land in the same place. The label set becomes something you write, not something you train for.
Confidence interval: A range built so that, across repeated experiments, a stated fraction of such ranges contain the true value. The fraction is a property of the method, not of the interval in front of you.
Confounding: A third variable that causes both the thing you changed and the thing you measured. Adjusting for the right one rescues an estimate; adjusting for the wrong one invents an effect that is not there.
Confusion matrix: A table counting how a classifier's predictions line up against the truth — correct and incorrect, broken down by which class was predicted and which was actual.
Conjugate gradient: An optimiser that chooses each search direction so it does not undo the progress of the previous ones — the fix for gradient descent zig-zagging down a narrow valley.
Convolution: Sliding a small learned kernel across an input and taking a weighted sum at each position — the same weights everywhere, which is how a layer can look at an image without a parameter per pixel.
Convolutional neural network: A stack that alternates convolution and downsampling so that features grow from edges to objects while the grid shrinks — the architecture that made image models tractable.
Correlation: One number for how tightly two variables sit on a straight line. It says nothing about the slope of that line, and near zero it does not mean the two are unrelated.
Cross-entropy: The loss every classifier on this site is trained with. It charges for being wrong and for being confident about it, and its floor is set by the base rate rather than by zero.
Cross-validation: Splitting the data k ways and letting each part be the test set once, so every sample is predicted by a model that never saw it — and the estimate uses all the data instead of a single lucky split.

D

Data augmentation: Applying random label-preserving transformations to training examples, so the model sees each one many ways and cannot rely on the accidents of how it happened to be captured.
Data leakage: Any way information reaches the model that would not exist at prediction time. It does not make the model worse — it makes the score better, which is why it survives review and fails in production.
Decision tree: A model that predicts by asking a sequence of yes/no questions about the features, splitting the data at each answer until it reaches a leaf holding a prediction.
Decoding: The rule that turns a language model's next-token distribution into actual text. Choosing the most probable continuation at every step reliably produces the worst output.
Depthwise separable convolution: Split a convolution into filtering each channel and then mixing channels. Roughly nine times cheaper for a 3×3 kernel, and what is left is mostly matrix multiplication.
Diffusion model: Destroy an image with noise in a thousand fixed steps, then train a network to undo one step at a time. Sampling runs the chain backwards from pure noise.
Distance-weighted KNN: KNN where each neighbour's vote counts in inverse proportion to how far away it is — so a point sitting right next to the query outweighs one at the edge of the neighbourhood.
Dropout: Zero a random fraction of activations during training and divide the rest by what survived. The model that ships is a cheap stand-in for an ensemble it never builds.

E

Early stopping: Stop training when held-out loss stops improving, and hand back the best weights rather than the last ones. It is regularization by limiting epochs instead of weights.
Eigenvectors and eigenvalues: The directions a matrix does not turn, and how much it stretches each of them. Almost every method that reduces, ranks or propagates through a linear map is a question about these.
Elastic Net: Linear regression penalised by a weighted blend of the L1 and L2 terms — it selects features like Lasso while keeping correlated groups together like Ridge.
Entropy: The average surprise in a distribution, measured in bits. It is the floor on how short a code can be, and every loss on this site that says “log” is paying it.
Expectation and variance: The two numbers that summarise a random quantity — where it sits and how far it moves. Each carries a condition people forget, and one of them does not always exist.

F

F1-score: The harmonic mean of precision and recall — one number that is only high when a model is both trustworthy when it fires and thorough in what it finds.
Feature engineering: Deciding what the columns are — rescaling, encoding, combining and deriving inputs so that the structure a model can represent is the structure that is actually there.
Feature importance: Ranking inputs by how much a fitted model relies on them — where the default measure for tree ensembles is biased toward high-cardinality features and the honest one is measured on held-out data.
Fréchet Inception Distance: Fit a Gaussian to Inception features for real and generated images, then measure the distance between them. The number you compute is not the number you want.

G

Gaussian discriminant analysis: A classifier that models what each class's data looks like — fitting a Gaussian per class — and then uses Bayes' rule to turn that into a prediction.
Generalized linear models: One framework that covers linear, logistic and Poisson regression at once — a linear predictor, a link function connecting it to the response, and a distribution for the noise.
Generative adversarial network: Two networks in a game — one invents samples, the other tries to spot them. The generator never sees the data, only gradients that come back through its opponent.
Gradient: The vector of a function's partial derivatives — it points in the direction of steepest increase, and its length says how steep that increase is.
Gradient boosting: Boosting generalised to any differentiable loss — each new tree is fitted to the negative gradient of the loss at the current predictions, and added with a shrinkage factor.
Gradient descent: Repeatedly step in the direction the loss decreases fastest, by an amount set by the learning rate — the algorithm underneath almost all model training.

H

Hallucination: Generated text that the source does not support — either contradicting it or going beyond it. The second kind is not automatically wrong, and no overlap metric can detect the first.
Huber loss: A loss that behaves like MSE for small errors and like MAE for large ones — smooth enough to optimise easily, robust enough that outliers stop dragging the fit.
Hyperparameter tuning: Searching the settings a model cannot learn for itself. Which search you use matters less than noticing that most settings do not matter, and that a grid spends its budget as if they all did.
Hypothesis testing: Assume there is no effect, work out how often chance alone would produce a result as striking as yours, and call it real if that is rare enough. The method is a proof by contradiction.

I

Image classification: One image in, one label out. The simplest shape in vision, and the benchmark whose single-label assumption shaped everything built on top of it.
Instance segmentation: A mask per object, not a class per pixel. It is detection and segmentation at once, and the thing that made it work was removing two roundings.
Instruction tuning: Fine-tuning a pretrained model on tasks phrased as instructions, and then on human preferences between its own outputs. What turns a text completer into something that answers the question you asked.
Intersection over union: The area two regions share divided by the area they cover between them. It is read as "how correct is this box", and it is stricter and scale-blind in ways that number hides.

J

Jupyter Notebook: A document that interleaves executable code, prose and its own outputs, run cell by cell against a live kernel — the standard tool for exploratory analysis, and a known hazard for reproducibility.

K

K-means: Pick k centres, assign every point to the nearest, move each centre to the mean of what it caught, repeat. It always converges and it can converge somewhere arbitrarily bad.
KNN: Classify a new point by looking at the k closest training examples and taking their majority vote — a model that stores the data instead of learning from it.

L

Language model: A probability distribution over sequences of tokens, factorised so that each token is predicted from the ones before it. Counting works up to about two words of context and then runs out of data.
Lasso regression: Linear regression with a penalty on the sum of absolute coefficients — it drives weak coefficients to exactly zero, so fitting the model also selects the features.
Leaky ReLU: ReLU with a small non-zero slope for negative inputs instead of a flat zero — so a unit that goes negative still receives a gradient and can come back.
Learning rate: How far each gradient step moves. On a quadratic the curvature sets a hard ceiling and the best value is exactly half of it; perhaps the most important hyperparameter to tune.
Linear regression: A model that predicts a numeric outcome as a weighted sum of the input features, fitted by choosing the weights that make the squared prediction errors as small as possible.
Logistic regression: A linear model for classification — it feeds a weighted sum through a sigmoid to produce a calibrated probability rather than a raw score, despite the "regression" in its name.

M

Machine learning workflow: The end-to-end loop a model lives in — framing, data, features, training, evaluation, serving and the feedback from production that changes all of them.
MAE: The average size of the errors, ignoring their direction — the plain, honest answer to "how far off is this model typically?"
Manifold learning: Non-linear projections that keep neighbours together at the cost of everything else. They make wonderful pictures, and almost nothing you can measure off those pictures is real.
MAPE: The average error expressed as a percentage of the true value — scale-free and intuitive, and unusable when the target can approach zero.
Maximum likelihood: Pick the parameters that make the observed data as probable as possible. Nearly every loss function in machine learning is this rule with a minus sign and a logarithm in front of it.
Mean average precision: The area under a precision-recall curve, averaged over classes — and in detection, also over IoU thresholds. The threshold you report it at decides whether it can see localisation quality at all.
Mini-batch gradient descent: Estimating the gradient from a small group of examples rather than one or all of them — the compromise that everything is actually trained with.
Missing data: Whether dropping or filling missing values biases your answer depends on why the values are missing — and on which answer you wanted. One of those two questions is usually unanswerable.
Model calibration: Making a model's stated confidence mean what it says — of the predictions it makes at 90% confidence, 90% should be right. Accuracy does not measure this, and modern networks fail it badly.
MSE: The average of the squared differences between predicted and actual values — the standard way to measure how wrong a regression model's predictions are.
MSLE: MSE computed on the logarithms of the values — it measures the ratio between prediction and truth rather than the difference, so being wrong by a factor counts the same at any scale.
Multiple comparisons: Run twenty tests at the 5% level and you will usually find something. The corrections differ in what they promise, and the honest choice is which kind of mistake you would rather make.

N

Named entity recognition: A label for every token, read back as spans — people, places, organisations. The output is structured, so one wrong token invalidates a whole entity and accuracy stops being the right metric.
Natural language processing: The study of how computers handle human language — turning text into representations a model can work with, then using those for classification, inference, translation and question answering.

O

Object detection: Finding every object in an image and drawing a box around each. The output is a variable-length set, which is what makes it harder than classification and why the machinery looks the way it does.
Outliers: Values far from the rest, which may be mistakes or may be the finding. Detecting them with a mean and a standard deviation fails in a specific way — the more of them there are, the fewer you find.
Overfitting: Fitting the training sample more closely than the process that generated it — so the model reproduces noise that will not recur, and does worse on new data the better it looks on old.

P

PCA: A rotation of the axes onto the directions of greatest spread. It makes the data uncorrelated and rank-orders the new axes, so you can drop the last ones and know exactly what that costs.
Perceptron: The simplest neural unit — it multiplies each input by a weight, adds them up with a bias, and fires 1 if the total clears a threshold and 0 if it doesn't.
Perplexity: The exponential of a model's average negative log-likelihood on held-out text — read as the effective number of choices it faces per token. It is per token, so it is not comparable across tokenizers.
PointNet: A neural network that consumes a raw 3D point cloud directly, respecting the fact that the points are a set — reorder the file and the answer must not change.
Polynomial regression: Fitting a curve by adding powers and products of the original features, then running ordinary linear regression on the expanded set — still a linear model, now with a curved shape.
Pooling: Replacing each window of a feature map with a single summary — usually its maximum — to shrink the grid and make the representation less sensitive to exactly where a feature sat.
Precision: Of everything the model flagged as positive, the fraction that really was positive — the measure of how much you can trust a positive prediction.
Prompting: Specifying a task in the model's own input, with an instruction and perhaps a few examples and no weight updates. Good at saying what the task is; poor at supplying an ability.

R

R²: How much better a model predicts than simply always guessing the mean — the one regression score that is comparable across different datasets.
Random forest: Bagged decision trees with a second source of randomness — each split may only consider a random subset of the features, which decorrelates the trees and lets averaging do far more work.
Recall: Of everything that really was positive, the fraction the model actually found — the measure of how much it misses.
Receptive field: The region of the input a single unit can possibly depend on — which grows only linearly with depth, and whose influence is concentrated in a Gaussian centre far smaller than the region itself.
Recurrent neural network: A network that carries a hidden state from one step of a sequence to the next, so one set of weights handles any length. The same recurrence destroys the gradient.
Regularization: Adding a penalty on coefficient size to a model's objective, so it trades a little fit on the training data for far better behaviour on data it has not seen.
ReLU: Pass positive values through unchanged and clamp everything negative to zero — an activation so simple it is barely a function, and the default choice in modern networks because of it.
Residual connection: Adding a block's input to its output so the block only has to learn the difference — which made networks of a hundred layers and more trainable, and gave gradients a path that skips the weights.
Retrieval-augmented generation: Retrieve passages from a document store, put them in the model's input, and generate from those. Moves facts out of the weights and into an index you can inspect, correct and replace.
Ridge regression: Linear regression with a penalty on the sum of squared coefficients — it shrinks every coefficient toward zero without ever removing one, which stabilises models with correlated features.
RMSE: The square root of the MSE — the same error measure, brought back into the units of the thing being predicted so a human can read it.
ROC AUC: The area under the curve traced by a classifier's true and false positive rates as the decision threshold sweeps — one number summarising performance at every threshold at once.

S

Self-supervised learning: Build the labels out of the data itself — two augmented views of one image are a positive pair — and the representation transfers as well as a supervised one.
Semantic segmentation: A class label for every pixel. The output is the same size as the input, which forces an encoder-decoder shape and makes pixel accuracy a useless way to score it.
Sentence embeddings: One fixed-length vector per sentence, arranged so that cosine similarity means semantic similarity. It makes search, clustering and deduplication linear in the collection instead of quadratic.
Sequence-to-sequence: An encoder reads a variable-length input and a decoder writes a variable-length output. Squeezing that into one fixed vector is the bottleneck attention was invented to remove.
SGD: Gradient descent that estimates the gradient from one example instead of the whole dataset — a much noisier direction, computed thousands of times more cheaply.
Sigmoid: An S-shaped activation that squashes any real number into (0, 1) — the historical default for neural networks, now largely displaced from hidden layers because it strangles the gradient.
Softmax: Turns a vector of scores into a probability distribution by exponentiating and normalising — the standard last step of a multi-class classifier, and the standard place for a numerical overflow.
Specificity: Of everything that really was negative, the fraction the model correctly left alone — recall measured on the negative class.
Stationarity: The assumption that a series' mean, variance and correlation structure do not change over time. Nearly every time-series method needs it, and ordinary regression fails loudly without it.
SVM: A classifier that places its boundary to leave the widest possible gap between the classes, with only the points on the edge of that gap deciding where it goes.

T

Tanh: A rescaled sigmoid that outputs between −1 and 1 instead of 0 and 1 — zero-centred, which makes it better behaved than sigmoid, though it saturates just the same.
TF-IDF: A weight that multiplies how often a term appears in one document by how rare it is elsewhere, so a word appearing everywhere contributes nothing and a word appearing in few documents dominates.
Time series decomposition: Splitting a series into trend, a repeating seasonal pattern, and what is left. Three readable things in place of one unreadable one, and the basis of most forecasting methods.
Tokenization: Cutting text into the units a model actually consumes — where splitting on words leaves every unseen word as a hole, and subword methods earn their vocabulary from frequency instead.
Train-test split: Withhold rows from training so the score you report was measured on data the model never saw. The split has to come first, be large enough, and respect how the rows arrive.
Transfer learning: Taking a network trained on a large dataset, replacing only its output layer and training on your much smaller one — the features are inherited, the classifier is all you learn from scratch.
Transformer: A stack of identical layers, each self-attention followed by a position-wise feed-forward network, both wrapped in a residual connection and layer normalisation, with word order supplied separately.
Transposed convolution: A convolution run backwards — it scatters the kernel instead of gathering it, turning a small feature map into a larger one. The name is literal, not a metaphor.

V

Vision transformer: A transformer applied to an image cut into fixed-size patches, with no convolution anywhere. It has far less built-in knowledge about images than a CNN, and it has to buy the difference with data.

W

Word embeddings: Dense vectors for words, learned from the company they keep — so that words used in similar contexts end up near each other, without anyone defining what similar means.