Description

Machine Learning

Assignment 4

General information
Report all results in a single, *.pdf- le. Other formats, such as Word, Rmd, Jupyter Notebook, or similar, will automatically be failed. Although, you are allowed rst to knit your document to Word or HTML and then print the assignment as a PDF from Word or HTML if you nd it di cult to get TeX to work.
The report should be written in English.
If a question is unclear in the assignment. Write down how you interpret the question and formulate your answer.
To pass the assignments, you should answer all questions not marked with *, and get at least 75% correct.
A report that does not contain the general information (see the template), will be automatically rejected.
When working with R, we recommend writing the reports using R markdown and the provided R markdown template. The template includes the formatting instructions and how to include code and gures.
Instead of R markdown, you can use other software to make the pdf report, but you should use the same instructions for formatting. These instructions are also available in the PDF produced from the R markdown template.
The course has its own R package uuml with data and functionality to simplify coding. To install the package just run the following:
1. install.packages(“remotes”)
2. remotes::install_github(“MansMeg/IntroML”, subdir = “rpackage”)
We collect common questions regarding installation and technical problems in a course Frequently Asked Questions (FAQ). This can be found here.
You are not allowed to show your assignments (text or code) to anyone. Only discuss the assignments with your fellow students. The student that show their assignment to anyone else could also be considered to cheat. Similarly, on zoom labs, only screen share when you are in a separate zoom room with teaching assistants.
If you have any suggestions or improvements to the course material, please post in the course chat feedback channel, create an issue here, or submit a pull request to the public repository.

Contents
1 Convolutional neural networks using Keras 4
2 * Implementing a convolutional layer 7
3 * Transfer learning using VGG16 9

Keras, Tensor ow, and Google Colab
This assignment can be computationally heavy. If your own an old computer, you might want to run this task in the computer room in the Department or on Google Colab. Google Colab also enable GPU support if you want to try that.
We have setup a Jupyter Notebook with R and Tensor ow at MansMeg/IntroML/additional_material/colab_nb. We suggest not to run the code in markdown. Instead, run the code in R/Colab and copy the results as output (see the assignment template for an example).
As a rst step, you must install tensorflow and keras on your local computer. You can nd detailed information on how to install tensorflow and keras [here]. You can nd details on di erent architecture components in the keras reference found [here].
1 Convolutional neural networks using Keras
If you have installed keras, you can load the MNIST dataset. This dataset contains data on handwritten digits that we want to classify. To access the data, we use keras as follows.
library(keras) mnist <- dataset_mnist() mnist$train$x <- mnist$train$x/255 mnist$test$x <- mnist$test$x/255
1. (1p) As a rst step, we visualize a couple of digits. Include the rst three you can nd in the training dataset in the report as an image.
idx <- 1
im <- mnist$train$x[idx,,] # Transpose the image im <- t(apply(im, 2, rev))
image(1:28, 1:28, im, col=gray((0:255)/255), xlab = “”, ylab = “”,
xaxt=’n’, yaxt=’n’, main=paste(mnist$train$y[idx]))
2. (1p) Implement a Convolutional Neural Network for the MNIST dataset. The network should have two convolutional layers as follows.
## _______________________________________________________________________
## Layer (type) Output Shape Param #
## =======================================================================
## conv2d (Conv2D) (None, 26, 26, 32) 320
## _______________________________________________________________________
## max_pooling2d (MaxPooling2D) (None, 13, 13, 32) 0
## _______________________________________________________________________
## conv2d (Conv2D) (None, 11, 11, 32) 9248
## _______________________________________________________________________
## flatten (Flatten) (None, 3872) 0
## _______________________________________________________________________
## dense (Dense) (None, 64) 247872
## _______________________________________________________________________
## dense (Dense) (None, 10) 650
## =======================================================================
## Total params: 258,090
## Trainable params: 258,090
## Non-trainable params: 0
3. (1p) Explain why there are 320 parameters in the rst layer. How many are kernel weights (and why), and how many biases?
4. (1p) Train the network using Keras. What is your loss and accuracy on the MNIST dataset?
6. (1p) As the next step, we will implement a convolutional neural network for the CIFAR-10 dataset using dataset_cifar10(). See the tutorial for details on how to load data. Implement a similar CNN as in the tutorial. That is:
## Model: “sequential”
## ___________________________________________________________________________
## Layer (type) Output Shape Param #
## ===========================================================================
## conv2d (Conv2D) (None, 30, 30, 32) 896
## ___________________________________________________________________________
## max_pooling2d (MaxPooling2D) (None, 15, 15, 32) 0
## ___________________________________________________________________________
## conv2d_1 (Conv2D) (None, 13, 13, 64) 18496
## ___________________________________________________________________________
## max_pooling2d_1 (MaxPooling2D) (None, 6, 6, 64) 0
## ___________________________________________________________________________
## conv2d_2 (Conv2D) (None, 4, 4, 64) 36928
## ___________________________________________________________________________
## flatten (Flatten) (None, 1024) 0
## ___________________________________________________________________________
## dense (Dense) (None, 64) 65600
## ___________________________________________________________________________
## dense_1 (Dense) (None, 10) 650
## ===========================================================================
## Total params: 122,570
## Trainable params: 122,570
## Non-trainable params: 0
## ___________________________________________________________________________
Note! This problem is more complex than the MNIST problem, so don’t be surprised if you get lower accuracy than you got on the MNIST dataset.
7. (1p) Visualize one image of your choice from the CIFAR dataset together with its class/category.
8. (1p) Why do we now have 896 parameters in the rst convolutional layer?
2 * Implementing a convolutional layer
To get VG on this assignment, you can choose to make this task OR the task on transfer learning below. You are also free to do both, if you want. As long as you pass one of the two VG assignments you will get a VG point. Although, please only write down the time it took to do the VG assignment for one of the tasks.
This task is only necessary to get one point toward a pass with distinction (VG) grade. Hence, if you do not want to get a pass with distinction (VG) point, you can ignore this part of the assignment.
As a rst step, we will implement a layer of a convolutional neural network with one lter. We will here not train the layer, only implement it to understand the inner workings. You are not allowed to use convolve() in R in the nal solution, but you can use it to debug your code if you want to.
Start by importing examples from the MNIST dataset as follows.
library(uuml) data(“mnist_example”)
To visualize an image use:
im <- mnist_example[[“5”]] image(1:ncol(im), 1:nrow(im), im, xlab = “”, ylab = “”,
xaxt=’n’, yaxt=’n’, main=”4″)
1. (1p) Visualize the MNIST example digit 4.
2. (3p) Implement a convolution function called convolution(X, K) that takes an input MNIST image (X), an arbitrary large square kernel (K) and returns a valid feature map. Below is an example of how it should work.
X <- mnist_example[[“4”]][12:15,12:15] X
## [,1] [,2] [,3] [,4] ## [1,] 56 250 116 0 ## [2,] 0 240 144 0
## [3,] 0 198 150 0
## [4,] 0 143 241 0
K <- matrix(c(-0.5, -0.5, -0.5, 1, 1, 1,
-0.5, -0.5, -0.5), nrow = 3, byrow = TRUE) K
## [,1] [,2] [,3] ## [1,] -0.5 -0.5 -0.5 ## [2,] 1.0 1.0 1.0 ## [3,] -0.5 -0.5 -0.5
convolution(X, K)
## [,1] [,2]
## [1,] -1 27 ## [2,] -36 -36
3. (1p) Visualize the feature map of MNIST example digit 4 using the above two by two kernel K.
4. (3p) Now implement all steps in a convolutional_layer(X, K, b, activation) function that takes the kernel, bias and activation function. It should work as follows.
relu <- function(x) max(0, x) convolutional_layer(X, K, 100, relu)
## [,1] [,2] ## [1,] 99 127 ## [2,] 64 64
5. (2p) Run your convolutional layer on MNIST example digit 4 with bias -150. Visualize the feature map as you visualize the original image. What does the lter seem to capture?
6. (1p) Now transpose your lter and run your convolutional layer on MNIST example digit 4 with bias -150. Visualize the feature map. What does that transposed lter seem to capture?
7. (3p) As the last step in our convolutional layer, implement a two by two, two stride max-pooling layer. It should work as follows.
X <- mnist_example[[“4”]][12:15,12:15] maxpool_layer(X)
## [,1] [,2] ## [1,] 250 144
## [2,] 198 241
8. (1p) Now put it all together and visualize the nal output of your own convolutional layer. Visualize the feature map.
X <- mnist_example[[“4”]] relu <- function(x) max(0, x)
output <- maxpool_layer(convolutional_layer(X, K, -370, relu))
3 * Transfer learning using VGG16
To get VG on this assignment, you can choose to make this task OR the task on implementing a convolutional layer above. You are also free to do both, if you want. As long as you pass one of the two VG assignments you will get a VG point. Although, please only write down the time it took to do the VG assignment for one of the tasks.
This task is only necessary to get one point toward a pass with distinction (VG) grade. Hence, if you do not want to get a pass with distinction (VG) point, you can ignore this part of the assignment.
Note! This task can be a computationally heavy.
You can nd the current state-of-the-art neural networks for the cifar-10 dataset here. Feel free to read some of the papers and be inspired on how to improve your neural network.
1. (2p) Using Keras, download the VGG16 convolutional neural network convolutional base. Just download the convolutional base. We are going to use the network on the cifar-10 dataset so change the input_size to c(32, 32, 3).
2. (2p) Now add a dense layer with 64 hidden nodes (as in the previous exercise). Include the Keras model output in the assignment report. How many parameters do you have in the dense top layer (compared to the CNN in the previous part)?
3. (2p) Now, freeze the convolutional base and train the dense top layer of your network on the cifar-10 dataset. Run it for ve epochs. Note! This will take time. Don’t be surprised if you need roughly 200s per epoch or more. Also, remember to freeze the convolutional base!
4. (2p) Report your nal accuracy. How does this compare to the previous model for the cifar data?
References
Andrew Ng. C4w1l07 one layer of a convolutional net, 2017.
URL https://www.youtube.com/watch?v=jPOAS7uCODQ&list=
PLkDaE6sCZn6Gl29AoE31iwdVwSG-KnDzF&index=8.