Description

Assignment 1
Instructions
• Please read each question carefully
• Note that the last question is only for groups with four members
• After completing your assignment compress all your files into a single zip file and name the file with your group’s name (e.g., OverAchievers.zip)
Points for The Questions
3-member groups 4-member groups
Question 1 10 pts. 7.5 pts.
Question 2 10 pts. 10 pts.
Question 3 – 2.5 pts.
Question 1
Recall that the coordinate descent algorithm can also be applied with weighted samples (see slide 31 of the first lecture notes). Show the steps to obtain the following analytical solution of the one-dimensional subproblem for j ̸= 0:
0 and λα < βˆj;
0 and λα < −βˆj;
.
Here,
.
What will you turn in?
A PDF file showing the steps that you followed to obtain the solution. The name of the file should be your group’s name (e.g., OverAchievers.pdf). Add the renamed file into the final compressed file that you will submit.
Evaluation. This question will be evaluated in terms of correctness and clarity of your steps.
Question 2
Implement the coordinate descent algorithm (page 30 of the first lecture notes) in Python to solve the Elastic Net problem. The input and output of your Python function are as follows:
• Input
– Training data, {(xi,yi) : i = 1,…,n}, xi ∈Rp (standardized), yi ∈R (centered)
– Hyperparameters, λ > 0 and α ∈ [0,1]
– Maximum number of iterations (default value of 1000), i.e., maxiter=1000
• Output
– Weight vector, β¯
– Duality gap, η(β,¯ v¯) where ¯v is the dual feasible vector
Here are some remarks and requirements:
If that turns out to be the case, your creativity, persistence, and skills are required.
• Do not forget to fully comment your Python function.
What will you turn in?
Add your function to the designated place in the testEN.py template script and rename the file after appending your group’s name as a single word (e.g., testENOverAchievers.py). Add the renamed file into the final compressed file that you will submit.
Evaluation. The results of your implementation will be tested against the sklearn Elastic Net package. The template script testEN.py also shows our testing setup as well as the datasets. The clarity and correctness of your implementation along with the comments will determine your score.
Question 3 – Only for groups with four members
Write down the dual of the linear programming model on page 23 of the second lecture notes.
What will you turn in?
A PDF file showing the steps that you followed to obtain the dual. The name of the file should be your group’s name (single word) appended with “Extra” (e.g. OverAchieversExtra.pdf). Add the renamed file into the final compressed file that you will submit.
Evaluation. This question will be evaluated in terms of correctness and clarity of your steps.