Scribd
Upload
106 views3 pages

Linear Regression and Gradient Descent Assignment

This document contains instructions for an assignment on linear regression and overfitting. Students are asked to: 1. Fit linear and polynomial regression models to dataset assigned based on roll number and calculate errors. 2. Use Lasso regression to obtain sparse polynomial models and compare accuracies. 3. Implement gradient descent to minimize an objective function involving roll number, iterating to convergence with different learning rates. Values are to be recorded in a table.

Uploaded by

Velamuru sai Yashwitha Reddy
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
106 views3 pages

Linear Regression and Gradient Descent Assignment

This document contains instructions for an assignment on linear regression and overfitting. Students are asked to: 1. Fit linear and polynomial regression models to dataset assigned based on roll number and calculate errors. 2. Use Lasso regression to obtain sparse polynomial models and compare accuracies. 3. Implement gradient descent to minimize an objective function involving roll number, iterating to convergence with different learning rates. Values are to be recorded in a table.

Uploaded by

Velamuru sai Yashwitha Reddy
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Name: Roll Number:

Amrita Vishwa Vidyapeetham, Amrita School of Artificial Intelligence, Coimbatore


Assignment-1 – February 2024, Sixth Semester
21MAT311 - Mathematics for Intelligent Systems - 6
Duration = 6 hours Total Marks = 50 marks

Linear Regression and Over-fitting (3+3+5+5+2+5+2+5(forCode)= 30 marks)

1. Read data from the data file assigned for your roll number. The ‘.csv’ data contains data
arranged in the order of [y x]. Fit a linear model of the form y = f(x,β). To begin with, plot
a graph between y and x. (0 marks)

Split the data into train and test data with 80:20 proportion. Objective is to find a polynomial
model using the training data. But we proceed a step-by-step process.

(a) First of all, tabulate the values of errors (least square errors) obtained for a polynomial
fit of different degrees (p) as shown below (remember to use p upto 50). Also plot a
graph with error on the y-axis and degree of the polynomial (p) on the x-axis. (Make
sure you use the error values you obtained for test data and not the training
data)

Degree of the polynomial (p) Error


1 -
2 -
3 -
.. ..
. .
50 -

Table 1: Fill up the table using Least square regression (try to fit the table on a single page)

(b) Now, recreate the the same table and the plot using LASSO model.
(c) From Table-1, what you think is an apt model? Write P
down the polynomial model you
obtained using Table-1. (An equation of the form y = p+1
j=1 βj x
p+1−j )

(d) From Table-2, what you think is an apt model? Write down the
Psparse polynomial model
you obtained using Table - 2. (An equation of the form y = p+1 β
j=1 j x p+1−j )

(e) Make a comparison of the accuracies of the two models you obtained using Table-1 and
Table-2.

1
Degree of the polynomial (p) Error
1 -
2 -
3 -
.. ..
. .
50 -

Table 2: Fill up the table using LASSO regression (try to fit the table on a single page)

(f) Employ leave-p-out cross validation and obtain a sparse polynomial model. Write down
Pp+1
the model you obtained for the given data. (An equation of the form y = j=1 βj xp+1−j )
(g) Compare the accuracy of the model you obtained using leave-p-out cross validation and
the one you obtained from Table-2.

Gradient Descent (4+4+4+12(forCode)=20 marks)

2. Consider the following:

f (x) = (x − (r ∗ 0.1))T A(x − (r ∗ 0.1)) (1)

where matrix A is given by:


 
r + 0.1 0
A= (2)
0 r − 0.1

where x is a vector and x ∈ R2 space; also r = log(roll number)/2. (roll number = the last
two-digits of your roll number).
   
x0 r−5
Choose initial condition x0 = = .
y0 r+5
Fill the table given below using the values obtained using the first 30 iterations of gradient
descent method. (Note: No need to fill up values for all 30 iterations, if your algorithm
converges before 30 iterations. Fill up the values till convergence.).

2
Table 3: Fill up the table using gradient descent algorithm. (try to fit the table on a single page)
iteration δ = 0.1 δ = 0.4 δ=
(i) xi f(xi ) xi f(xi ) xi f(xi )
1
(initial guess)
2
3
..
.
30

You might also like