Description

Department of Electrical Engineering and Computer Sciences
EECS 126: Probability and Random Processes
Discussion 13
1. Orthogonal LLSE
(a) Consider zero-mean random variables X,Y,Z such that Y,Z are orthogonal. Show that L[X | Y,Z] = L[X | Y ] + L[X | Z].
(b) Explain why for any zero-mean random variables X,Y,Z it holds that:

2. MMSE for Jointly Gaussian Random Variables
Provide justification for each of the following steps (1 – 5) to prove that the LLSE is equal to the MMSE estimator for jointly Gaussian random variables X and Y . Let g(X) = L[Y | X].
E[(Y − g(X))X] = 0 (1)
=⇒ cov(Y − g(X),X) = 0 (2)
=⇒ Y − g(X) is independent of X (3) =⇒ E[(Y − g(X))f(X)] = 0 ∀f(·) (4)
=⇒ g(X) = E[Y | X] (5)
3. Stochastic Linear System MMSE
Let (Vn, n ∈ N) be i.i.d. N(0,σ2) and independent of X0 = N(0,u2). Let |a| < 1. Define
Xn+1 = aXn + Vn, n ∈ N.
(a) What is the distribution of Xn, where n is a positive integer?
(b) Find E[Xn+m | Xn] for m,n ∈ N, m ≥ 1.
(c) Find u so that the distribution of Xn is the same for all n ∈ N.
4. (Optional, included for practice) Random Walk with Unknown Drift Consider a random walk with unknown drift. The dynamics are given, for n ∈ N, as
X1(n + 1) = X1(n) + X2(n) + V (n),
X2(n + 1) = X2(n),
Y (n) = X1(n) + W(n).
Here, X1 represents the position of the particle and X2 represents the velocity of the particle
(which is unknown but constant throughout time). Y is the observation. V and W are independent Gaussian noise variables with mean zero and variance σV2 and σW2 respectively.
1
(a) Write down the dynamics of the system in matrix-vector form and write down the Kalman filter recursive equations for this system.
(b) Let k be a positive integer. Compute the prediction E(X(n+k) | Y (n)), where Y (n) is the history of the observations Y0,…,Yn, in terms of the estimate Xˆ(n) := E(X(n) | Y (n)).
(c) Now let k = 1 and compute the smoothing estimate E(X(n) | Y (n+1)) in terms of the quantities that appear in the Kalman filter equation.
Hint: Use the law of total expectation
,
as well as the innovation
X˜(n + 1) := X(n + 1) − L[X(n + 1) | Y (n)].
2