Description

IANNwTF
Submit your homework via https://forms.gle/ApAZ5ubY8ewgNmJA9
Remember that you now have to review the homework submission of two other groups.
Contents
1 Recap on Reviews 2
2 Assignment: CIFAR-10 Classification 2
2.1 Prepare the Dataset 3
2.2 The CNN Model 3
2.3 Training the network 3
2.4 Visualization 3
3 Adjust hyperparameters and architecture 3
3.1 Sharing your results 4

1 Recap on Reviews
Welcome back to the fifth homework for IANNwTF. Here’s just a short refresher on how to do reviews.
In addition to handing in your homework, you will have to review last week’s homework of two groups and note down which group you reviewed on the homework submission form. This requires you to find two other groups and have a short (10 min) meeting where you go over each others homework submission, discuss it and then write a short summary of what you discussed on each others forum pages. We recommend using the Q&A timeslots for this purpose, but you can meet however and whenever you like. The main review part of this should be the discussion you have together. The written review in the forum should merely be a recap of the main points you discussed so that we can see that you did something and the reviewed group has access to a reminder of the feedback they received. If there are any open questions regarding your or any other groups code afterwards, please feel invited to discuss this with us in the QnA sessions, so we can help you sort out how a perfect solution would have looked like! Just to make this obvious: We will not penalize any submission (by you or any of the groups you reviewed) based on such questions. The purpose is helping you understand and code better only.
As to how a discussion could look like, you could for example have the group being reviewed walking the other two groups through their code. The other two groups should try to give feedback, e.g. ”I like how you designed your data pipeline, looks very efficient.”, ”For this function you implemented there actually exists a method in e.g. NumPy or TensorFlow you could’ve used instead.” or ”Here you could’ve used a slightly different network architecture and higher learning rate to probably achieve better results.”, and note down their main points in the forum.
Important! Not every member of every group has to be present for this. You could for example implement a rotation where every group member of yours only has to participate in a review every third week.
2 Assignment: CIFAR-10 Classification
During the last two weeks you learned how to make use of TensorFlow functionalities to build neural networks. This week you will do the same implementing a Convolutional Neural Network. So far, the instructions in the previous homework assignments have been rather specific about how things should be done. This week’s however is going to be a little bit more free, which should give you the opportunity to play around with your model setup and hyperparameters. In the end you can share your best results or interesting results for comparison with your fellow students!
2.1 Prepare the Dataset
First familiarise yourself with the dataset. It is included as a TensorFlow Dataset but can also be obtained from other sources. Try to understand how it works (information in the data, dataset size, image size, are there color channels, type the data is stored as, etc.). Understanding the data you are working with and applying respective preprocessing steps is a major part of the process for real life applications.
Now that you familiarised yourself with it, load the data and apply respective preprocessing steps using a data pipeline (In case of problems refer to your old homework tasks or the Courseware). Visualize a sample of the dataset.
2.2 The CNN Model
Build a Convolutional Neural Network inspired by the information gained in this weeks Courseware. You are going to need more than the dense-layers used so far. Start with a basic structure and don’t necessarily think about optimizing the architecture too much, as this will be part of the last step.
2.3 Training the network
Define a training loop function. Define your hyperparameters, think of an appropriate loss function and optimizer and initialize everything. Store loss and accuracy for training and test data. Training your network for a maximum of 15 epochs should be enough.
2.4 Visualization
After training visualize the performance of your model and the values that you collected during training and testing.
3 Adjust hyperparameters and architecture
Now to the fun part. You should have a working model at this point. There are tons of different architectures online that you can use but what we would like, is for you to try out different combinations of layers to achieve different results. You can also play around with the hyperparameters, optimizer or whatever ideas you can come up with. Whatever you do, remember to keep your network reasonably small. Also try to find the middle ground between the depth of the network and the number of filters used in each layer.
3.1 Sharing your results
A strength of the AI-community is the open exchange of knowledge. This opensource spirit is important. We have prepared a google form in which you we encourage you to post your model-architecture as well as a list of your hyperparameters and a plot of your results (loss & accuracy). This way you can see what your peers did and how different implementations perform.
Link to: IANNwTF homework 05 CNNS – Collaboratory results.
Each group should try to at least post one architecture of their best results but feel free to post more. Coding is all about the community. And most importantly: Enjoy the process!