Description

MA202 Numerical Techniques LAB#9 Regression
As a reasonable means, we consider the least-squares (LS) approach to minimizing the sum of squared errors, where the error is described by the vertical distance to the curve from the data points. We will look over various types of fitting functions in this section.
Linear least-squares regression using straight line fit (polynomial approximation of first degree)
Given a set of M input/output data points’ pairs (x1,y1),(x2,y2),…,(xM,yM), we would like to establish following linear relationship using linear least-squares regression,
θ1x + θ0 = y (1)
Where θ1 is the slope and θ0 is the intercept of the resultant straight line equation. Using the available data points pairs, we can write a set of linear simultaneous equations as follows:
θ1×1 + θ0 = y1 θ1×2 + θ0 = y2 θ1×3 + θ0 = y3
……..
θ1xM + θ0 = yM
The corresponding matrix form can be written as,
(2)
Since it yields an overdetermined system of expression, i.e., more number of equations than unknowns (θ0 and θ1), we can solve the same in norm-2 sense, i.e., Euclidean norm, by writing the normal equations,
(3)
This is the normal equation which minimizes the objective function J = kek2 = J = kAθ − yk2 = [Aθ − y]T[Aθ − y]
Polynomial Curve Fit: A Polynomial Function of Higher Degree
(4)
Procedure for Linear Regression (Fitting y = θ0+θ1x) using Least Square Method
1. Form normal equations:
Py = nθ0 + θ1Px
Pxy = θ0Px + θ1Px2
2. Solve normal equations as simulataneous equations for θ0 and θ1
3. Substitute the value of θ0 and θ1 in y = θ0 + θ1x which is required line of best fit.
Linear Regression Algorithm (Fitting y = θ0 + θ1x)
1. Start
2. Read Number of Data (n)
3. For i = 1 to n:
Read Xi and Yi Next i
4. Initialize:
sumX = 0 sumX2 = 0 sumY = 0 sumXY = 0
5. Calculate Required SumFor i = 1 to n: sumX = sumX + Xi sumX2 = sumX2 + Xi ∗ Xi sumY = sumY + Yi sumXY = sumXY + Xi ∗ Y i
Next i
6. Calculate Required Constant θ0 and b of y = θ0 + θ1x: b = (n ∗ sumXY − sumX ∗ sumY )/(n ∗ sumX2 − sumX ∗ sumX) θ0 = (sumY − θ1 ∗ sumX)/n
7. Display value of θ0 and θ1 8. Stop
Polynomial Curve Fit by LS (Least Squares).
Q. 1: Given these data points:
x: [-3 -2 -1 0 1 2 3]
y: [0.2774 0.8958 1.5651 3.4565 3.0601 4.8568 3.8982]
Fit these data into polynomials of degree 1, 3, 5, and 7. Plot the polynomials of different degree in independent subgraphs using MATLAB command subplot. Observe oscillation of the fitting curve between the data points with higher degree.
Q. 2: Given the following data:
x: [0.8 1.4 2.7 3.8 4.8 4.9] y: [0.69 1.00 2.00 2.39 2.34 2.83]
Fit these data into polynomials of degree 1, 3, 5, and 7. Observe oscillation of the fitting curve between the data points with higher degree.
Page 2 of 2