Description

1 Homework 1 – Berkeley STAT 157
1. Write all code in the notebook.
2. Write all text in the notebook. You can use MathJax to insert math or generic Markdown to insert figures (it’s unlikely you’ll need the latter).
3. Execute the notebook and save the results.
4. To be safe, print the notebook as PDF and add it to the repository, too. Your repository should contain two files: homework1.ipynb and homework1.pdf.
In [1]: from mxnet import ndarray as nd
1.1 1. Speedtest for vectorization
Your goal is to measure the speed of linear algebra operations for different levels of vectorization. You need to use wait_to_read() on the output to ensure that the result is computed completely, since NDArray uses asynchronous computation. Please see http://beta.mxnet.io/api/ndarray/_autogen/mxnet.ndarray.NDArray.wait_to_read.html for details.
1. Construct two matrices A and B with Gaussian random entries of size 4096 × 4096.
2. Compute C = AB using matrix-matrix operations and report the time.
3. Compute C = AB, treating A as a matrix but computing the result for each column of B one at a time. Report the time.
4. Compute C = AB, treating A and B as collections of vectors. Report the time.
5. Bonus question – what changes if you execute this on a GPU?
In [ ]:
1.2 2. Semidefinite Matrices
Assume that A ∈ Rm×n is an arbitrary matrix and that D ∈ Rn×n is a diagonal matrix with nonnegative entries.
1. Prove that B = ADA⊤ is a positive semidefinite matrix.
2. When would it be useful to work with B and when is it better to use A and D?
1
1.3 3. MXNet on GPUs
1. Install GPU drivers (if needed)
2. Install MXNet on a GPU instance
3. Display !nvidia-smi
4. Create a 2 × 2 matrix on the GPU and print it. See http://d2l.ai/chapter_deep-learningcomputation/use-gpu.html for details.
In [ ]:
1.4 4. NDArray and NumPy
Your goal is to measure the speed penalty between MXNet Gluon and Python when converting data between both. We are going to do this as follows:
1. Create two Gaussian random matrices A, B of size 4096 × 4096 in NDArray. 2. Compute a vector c ∈ R4096 where ci = ∥ABi·∥2 where c is a NumPy vector.
1. Compute ∥ABi·∥2 one at a time and assign its outcome to ci directly.
2. Use an intermediate storage vector d in NDArray for assignments and copy to NumPy at the end.
In [ ]:
1.5 5. Memory efficient computation
We want to compute C ← A· B+C, where A, B and C are all matrices. Implement this in the most memory efficient manner. Pay attention to the following two things:
1. Do not allocate new memory for the new value of C.
2. Do not allocate new memory for intermediate results if possible.
In [ ]:
1.6 6. Broadcast Operations
In order to perform polynomial fitting we want to compute a design matrix A with
Aij = xij
Our goal is to implement this withoutasingleforloop entirely using vectorization and broadcast. Here 1 ≤ j ≤ 20 and x = {−10, −9.9, . . . 10}. Implement code that generates such a matrix.
In [ ]:
2