Linear model family

We email you a link and a six-digit code. Either one signs you in; there is no password.Enter your email. We send a link and a six-digit code to confirm it — no password to remember. The account is free.

You come back to this page, right where you were.

Already have an account? The same email simply signs you in.

Comparison

Name What it changes	Penalty	Zeroes coefficients?	Hyperparameters
Linear regression No penalty. Minimises squared error and nothing else, so it overfits freely when features are many or correlated. Full page →	none	No	none
Ridge L2 penalty. Shrinks every coefficient smoothly, splitting effect across correlated features instead of playing them against each other. Full page →	L2 — ‖w‖²	No	alpha
Lasso L1 penalty. Drives weak coefficients to exactly zero, so fitting the model also selects the features. Full page →	L1 — ‖w‖₁	Yes	alpha
Elastic Net Both penalties, weighted. Selects like Lasso while keeping correlated groups together like Ridge. Full page →	L1 + L2	Yes	alpha, l1_ratio
Bayesian ridge L2 as a prior rather than a setting. Learns the penalty strength from the data and returns uncertainty with every prediction. Full page →	L2, learned	No	none tuned
Polynomial Not a penalty but an expansion — adds powers and cross-terms so the same linear machinery fits a curve. Full page →	none by itself	No	degree
Logistic Same weighted sum through a sigmoid, fitted by likelihood. Outputs a probability, so it classifies rather than predicts a quantity. Full page →	L2 by default	with l1_ratio	C, l1_ratio

One model, several objectives

Every row above predicts with the same expression — a weighted sum of features plus an intercept. None of them differ in what they compute at prediction time. They differ in what they minimise at fitting time, and that is entirely a matter of which penalty is bolted onto the squared-error loss.

Elastic Net makes the relationship explicit, because it contains the other two: its objective carries both penalties weighted by l1_ratio, and at l1_ratio = 0 the penalty is purely L2 — Ridge — while at l1_ratio = 1 it is purely L1 — Lasso [1, l1_ratio]. Ridge and Lasso are not siblings of Elastic Net so much as its endpoints.

Choosing is a claim about your features

The decision is not "which performs best" but "what do I believe is true":

Most features carry a little signal → Ridge. Nothing is discarded, everything is quietened.
Most features are irrelevant → Lasso. L1 "results in a solution that is more sparse" [2, §7.1.2]. Sparsity is an assumption, and the model will act on it.
Relevant features come in correlated groups → Elastic Net. Lasso would keep one of each group arbitrarily; the L2 component stops that.
I do not want to tune anything and I need uncertainty → Bayesian ridge.

If you are unsure, Elastic Net can reach any of the three by search — at the cost of a second hyperparameter, l1_ratio [1, l1_ratio].

The two that are not penalties

Two rows break the pattern and are worth separating out.

Polynomial regression changes the features, not the objective. It is a preprocessing step that hands ordinary least squares a wider matrix, and it adds capacity rather than removing it — which is why it is almost always paired with a penalty rather than used alone.

Logistic regression changes the task. It is the only row that does not predict a quantity: the weighted sum goes through a sigmoid and comes out as a probability. Its penalty parameter also runs backwards — C is the inverse of regularization strength, where Ridge and Lasso use alpha directly.

Practical advice

Scale your features before any of the penalised rows. All of these penalties act on raw coefficient magnitude, and coefficient magnitude depends on units — so without a scaler you are penalising features for being measured in millimetres.

Tune with the built-in cross-validated variants (RidgeCV, LassoCV, ElasticNetCV) rather than a hand-rolled grid. They exploit the regularization path and are substantially faster than refitting from scratch at every value.