Description

General Instructions

The following document contains the solutions to the theory-based questions for
Question 1

Given x ∈ RN Let x = [x1 x2 ⋯ xN]T
N
a) Average, μ = N1 ∑xn Since, the daily “price” of a stock can never be negative:
n=1
|xn| = xn
N
|xn| n=1
⟹ μ = N1 ∥x∥1
b) We calculate the variance as follows:
N
σ2 = N1 ∑(xn − μ)2
n=1
N
N
n=1
= 1
N
As we know, |x2n| = x2n
⟹ σ2 = 1
N
n=1 n=1
c) We now find a recursive relation for μ as
N+1
μN+1 = N 1+ 1 ∑ xn
n=1
= N

d) We now find a recursive relation for σ2 as
N
∑ (xn − μN+1)
n=1
N+1
= 1 ∑ (x2n − 2μN+1xn + μ2N+1) N + 1
n=1

Question 2

Let two independent vectors a and b. The scalar projection of vector a onto vector b is given by:
⟨a,b⟩ Scalar Projection
Now, we know that the Cauchy-Schwarz inequality states that:
⟨a,b⟩ ≤ ∥a∥2 ∥b∥2
∴ ⟨a,b⟩ ≤ ∥a∥
Hence, proved.
Question 3

Given a ∈ R+ and ∥a∥1 = 1. Let a = [a1 a2 ⋯ aN]T such that
n
∑ai = 1
i=1
Let us define a new vector x = 1 where 1 is a vector with all its elements equal to 1.
Using Cauchy-Schwarz Inequality for a and x,
|⟨a,x⟩| ≤ ∥a∥2∥x∥2
⟹ |⟨a,x⟩| ≤ ∥a∥2√n
n
⟹ ∑aixi ≤ ∥a∥2√n
i=1 n

⟹ ∑ai ≤ ∥a∥2√n
i=1
⟹
√n
Therefore, the minimum value of ||a||2 is 1
√n
Question 4

a. Considering the function f : R → R as follows
f(α1) = ∥a + α1b∥22
= ⟨a + α1b,a + α1b⟩
⟨a,b⟩
The given expression can be minimized by minimizing f(α1)
⟨a,b⟩

The given expression will have minima when .
Therefore, for the required minima,
⟨a,b⟩ + 2α1|b|22 = 0
⟨a,b⟩
⟹
b. Similarly considering the function g : R → R as follows
g(α2) = ∥a − α2b∥22
= ⟨a − α2b,a − α2b⟩
= ∥a∥22 − 2α2 ⟨a,b⟩ + α22∥b∥22 The given expression can be minimized by minimizing g(α1)
⟨a,b⟩

The given expression will have maxima when .
Therefore for the required minima,
⟨a,b⟩
⟨a,b⟩
⟹
Question 5

Given x ∈ RN. Let x = [x1 x2 x3 ⋯xN]T. The LP norm of a vector x:
|xi|P)
Consider ∥x∥21 i.e., ∥x∥21 = ∥x∥1∥x∥1

⟹ |xi||xj|
i=1 i,j
i≠j
N
Analyse the first summation of the RHS with respect to the L2 norm i.e., |xi|2
i=1
The first summation is simply the L2 norm of x
⟹ ∥x∥21 = ∥x∥22 + ∑|xi||xj|
i,j
i≠j
Now analyse the second summation. It is a summation of positive definites(absolute values). Hence, the summation is positive definite i.e.,
∑|xi||xj| ≥ 0
i,j
i≠j
⟹ ∥x∥21 ≥ ∥x∥22
Note that one of the criteria to be classified as a norm is the positive definite property i.e.,
|xi|P)
⟹ ∥x∥1 ≥ ∥x∥2