Description

The project is intended to give you experience to working on neural networks in a research application. It is also intended to give you experience working with some of the topics covered in the last three weeks of class, for which this project is the evaluation. This means, in this project, you should evaluate at least one of the algorithms we discuss post-CNNs, or maybe even evaluate neural network topics we don’t explicitly get to cover in class.
• Recurrent neural networks
• Variational autoencoders
• Generative adversarial networks
• Policy gradient / deep reinforcement learning
Project dataset
In class, we have worked with CIFAR-10 extensively. But what about data with temporal components and structure? This project will explore this, using datasets collected from electroencephalography (EEG). A more complete description of the data is available here: http://www.bbci.de/ competition/iv/desc_2a.pdf
EEG reflects the coordinated activity of millions of neurons near a non-invasive scalp electrode. Because these are scalp potentials, necessarily, they have relatively poor spatiotemporal resolution compared to other neural recording techniques. EEG is believed to be recording dipoles that are transmitted through the scalp.
This dataset is made publicly available through: http://www.bbci.de/competition/iv/. However, typically people process these datasets with MATLAB. We have formatted the data in a way that it will be easily loadable for you, as opposed to working with the raw .gdf files. These datasets are internal for this class and should not be distributed. If you would like to publish based off of this data, you should visit the BCI competition site and download the raw gdf files after filling out the following agreement:
Each participant has to agree to give reference to the group(s) which recorded the data and to cite (one of) the paper listed in the respective description in each of her/his publications where one of those data sets is analyzed. Furthermore, we request each author to report any publication involving BCI Competiton data sets to us for including it in our list.
After filling out the form and pushing the ”I Agree” button an automatic e-mail will be generated containing location and access information for the data set download area.
Again, we provide a processed version for teaching purposes, but if you want to use this dataset for any research purposes, you should download the data from their website after filling out the appropriate agreement.
For each subject, they record from 25 EEG electrodes while the user imagines performing one of four actions. Therefore, this is a classification task (with four outcome classes), where the EEG is used to determine what action the subject was imagining. Follow the instructions below to load the data.
import numpy as np import h5py
Load the data, storing the EEG data as X and the corresponding labels as y
A01T = h5py.File(’A01T_slice.mat’, ’r’) X = np.copy(A01T[’image’]) y = np.copy(A01T[’type’]) y = y[0,0:image.shape[0]:1] y = np.asarray(y, dtype=np.int32) You should see the following outputs:

This indicates that there are 288 trials; each trial has corresponding EEG data from 25 electrodes over 313 time bins. Please look at the dataset documentation to know more about the data. E.g., Table 2 lists what class labels (769, 770, 771, 772) correspond to.
EEG questions that we suggest you answer
There are many projects you can imagine doing from this data, but if you’d like a “default” project, we suggest you do the following:
1. Optimize the classification accuracy for subject 1. Does it help to train across all subjects?
2. Optimize the classification accuracy across all subjects. How does the classifier do? Do you notice any interesting trends?
3. Evaluate the classification accuracy as a function of time (e.g., does it increase as you have data over longer periods of time? how much time is required to get a reasonable classification accuracy?)
If you are working with the EEG dataset, randomly sample 50 trials (of the 288) in each dataset for testing, and evaluate performance on these 50 trials. Train and validate on the remaining data.
Project submittables
The writeup must adhere to the following template: http://www.pamitc.org/cvpr15/files/ cvpr2015AuthorKit.zip – so that we can judge all writeups in the same manner without having to worry about different font sizes, etc.
Project writeup
In the writeup, there should be the following sections:
Abstract
A brief description of what you did in the project and the results observed.
Introduction
Do not use the introduction to formulate the general problem of EEG decoding (unless you are doing a project from your own research, in which case it should be brief), as we are all familiar with the EEG problem. Instead, use the introduction to set up and motivate the architectures you pursued and why.
Results
State the results of your experiments.
Discussion
Discuss insights gained from your project, e.g., what resulted in good performance, and any hypotheses for why this might be the case.
References
List references used in your writeup.
Project grading
1. Creativity (7 points).
Does the project reveal some insight about the choice of various parameters on the performance of the algorithm? How about hyperparameters and architectures? Is there reasonable insight into their results? (i.e., you should not just blindly apply different algorithms to a problem and compare them.)
3. Performance (6 points).
4. Write-up (4 points).
Are the approach, insight, and results clearly presented and explained? Dissemination of results is an important component to any project.
Last note
CNNs with the same dataset: https://arxiv.org/pdf/1703.05051.pdf