Description

1. [6 points] Linear Regression Basics
Consider a linear model of the form ˆy(i) = w|x(i) + b, where w,x ∈ RK and b ∈ R. Next, we are given a training dataset, D = {(x(i),y(i))} denoting the corresponding input-target example pairs.
(a) What is the loss function, L, for training a linear regression model? (Don’t forget the
)
(f) For convenience, we group w and b together into u, then we denote z = [x 1]. (i.e. yˆ = u|[x,1] = w|x + b). What are the optimal parameters u∗ = [w∗,b∗]? Use the notation Z ∈ R|D|×(K+1) and y ∈ R|D| in the answer. Where, each row of Z,y denotes
an example input-target pair in the dataset.
Your answer:
u∗ = (Z|Z)−1Z|y
2. [2 points] Linear Regression Probabilistic Interpretation
Consider that the input x(i) ∈ R and target variable y(i) ∈ R to have to following relationship.

where, is independently and identically distributed according to a Gaussian distribution with zero mean and unit variance.
(a) What is the conditional probability p(y(i)|x(i),w).

(b) Given a dataset D = {(x(i),y(i))}, what is the negative log likelihood of the dataset according to our model? (Simplify.)

2