What is ridge regression?
A ridge regression is an OLS regression that uses L2-regularization.
The regularized loss function on some dataset is thus:
Let be the training-set and note and the corresponding design matrix and output vector.
We can compute the value of the parameter vector that minimizes the regularized loss using differentiation.
Setting all directional derivatives to , we get: