Machine-learning 5 minutes

# Linear regressions in simple terms

A linear regression is a model used to predict the value of a (continuous) variable.

We are given a dataset $\sets$ made of $\sn$ records $\sr_1, \dotsc, \sr_\sn$. Each record contains data about an object:

Where $\vx_\si$ can be a single variable or a vector of several variables.

The aim is to predict the output value $\sy_{\si}$ based on the input vector $\vx_\si$ using a linear function $\ff_{\vw}$:

Where the function $\ff_{\vw}$ is defined by:

Before we can make predictions, we need to learn the parameter vector $\vw$. This is called fitting the model.

## Model fitting

We will fit the model using a subset $\trainset$ of the dataset:

We are looking for the value of $\vw$ that minimizes the error between the output values $\sy_si$ and the predictions $\ff_{\vw}(\vx_{\si})$.

The total error made on a dataset $\sets_x$ using the parameter $\vw$ is measured by the loss function:

Using this (yet undefined) loss function, we can compute the best parameters:

These parameters depend on the trainset used. Different trainsets might yield (slightly) different parameters.

## Making predictions

Using the best parameters $\hat{\vw}$, we can make predictions on the whole dataset $\sets$ and compute an estimate $\hat{\sy}_\si$ for the output value $\sy_\si$:

## Assessment

To assess our model’s performance on new data, we can compute the loss on another subset $\testset \subset \sets$ which is disjoint from our trainset: