OLS regressions in simple terms
This article is a sequel to our previous article: Linear regression in simple terms. We will use the same notations and start where we left.
The loss function is the MSE loss, defined by:
The best vector of parameters can be found by minimizing this loss function using an optimization algorithm, or using the analytic solution that we will introduce now.
Finding the parameters
With these notations, the predictions for the set are in the predicted vector :
and the loss function is writen:
To shorten notations, let and be the design matrix and output vector for our training-set. Provided is invertible, the vector that minimizes this loss is: