Description

Special Topics: Deep Reinforcement Learning Homework – 0
Xu Cao
xc2057@nyu.edu
Note
Conceptual questions
1. What is K-Means clustering? How is it different from Meanshift?
K-Means clustering is an unsupervised learning algorithm. Given a set of samples (x1,x2,…,xn), where each sample is a d-dimensional vector, k-means clustering aims to partition the n observations into k sets (k <=n). It use recurrence update method to to minimize the within-cluster sum of squares (WCSS) or we can say within-cluster variance.
Different from K-Means, Mean-Shift algorithm has a parameter called bandwidth, which is using to control the within-cluster variance. Since Mean-Shift algorithm has bandwidth, it do not need to define k (the number of output class). For K-Means, we have to fix the k.
2. What is softmax? Under what cases does softmax computation become unstable?
The softmax function takes as input a vector z of K real numbers, and normalizes it into a probability distribution consisting of K probabilities proportional to the exponential of the input numbers. Thus, after applying softmax, each component will be in the interval [0,1].
3. What is regularization in CNNs? Explain one common technique used forregularization.
Regularization is the process of adding extra-information and computation in order to prevent over-fitting (eg. L1 regularization, L2 regularization, dropout).
One common technique used for regularization is dropout. Dropout means randomly omitting some neurons during the training process of a neural network. For testing and validation process, the dropout will close.
4. What is a GAN? What does it mean for a GAN to ‘mode collapse’?
Generative Adversarial Network (GAN) is an unsupervised learning method. It uses two neural networks (generative network and discriminative network) contesting with each other in a game process. The generative network learns to map from a latent space to a data distribution of interest, while the discriminative network distinguishes candidates produced by the generator from the true data distribution. The generative network’s training objective is to increase the error rate of the discriminative network.
Mode collapse means that the GAN only produce one class of sample and ignore other samples. It considers that the result that satisfies a certain distribution is true, and the others are False.
5. What are some common initialization methods for Convolution layers andFully Connected layers in a CNN?
Initialization methods include Xavier initialization, He initialization, and initialize the network based on a pre-training step. All of these method is to solve the vanishing gradients problem in deep networks. Xavier initialization works pretty well for trivial and tanh activation while He initialization works well for relu activation. Pre-training is a concept based on transfer learning.
Coding Challenges
6. Optimizing a Function
Use Monte Carlo method, we can solve this problem easily.
Here is the code link:
Answer Link
7. Detecting Breast Cancer
Here is the code link:
Answer Link
I used a two layers shallow fully connected neural network to solve this problem. For each layer, I also apply dropout with probability 0.25. The learning rate for training is 0.001, and the batch size is 10. You can check my code to see what I used in this model.
The final test loss is 0.09724803268909454.
The final test accuracy is 96.42857142857143%
Visualizations of training loss and validation loss.

Figur 1: Visualizations of training loss and validation loss.
Visualizations of validation accuracy.

Figur 2: Visualizations of validation accuracy.
Reference
Most of the concept is refer to Wikipedia.
https://intoli.com/blog/neural-network-initialization/