Resources
Prerequisites
- Linear Algebra
- Vector, matrix, norm, rank, linearly independent
- Vector Calculus
- Chain Rule, Calculate Gradients for Multiple Variables
- Probability
- Bayes Rule, Maximum Likelihood
- Coding
- Python
- Basic Concepts of Machine Learning
- Classification vs. Regression, Supervised learning vs. Unsupervised learning
Having taken CS 165 A/B before not required
Tools
- Google Colab
- Ipython notebook
- Can access GPU run-time
- Support both coding and text editing
- LaTex
- Very useful for scientific wwriting
- LaTex + Colab
Grading
- Homework (60%)
- Including the self-assessment
- No handwritten homework: Colab notebook
- No late homework
- Midterm (20%)
- November
- In-class / evening exam
- Open book but no internet
- No makeup exam
- Final (20%)
- Format undecided
- Exam? Large Coding Project? Coding test within 3-hour test time?
Topics
- Linear models
- Regression
- Classification, both binary and multi-class settings
- Multi-layer perceptron / Fully connected neural networks
- Gradient descent, back-propagation, optimizers
- Training recipe for deep models
- Overfitting, data augmentation, activation function, normalization, etc.
- CNN, ResNet, RNN, Transformer Architectures
Class Format
- In-class lecture (by instructor)
- Concepts behind the topic
- Mathematical derivation
- Examples
- In-class implementation (by instructor + TAs)
- Implement together in class
- Make sure you understand the concept and avoid common mistakes
- Laptops are needed
- Homework
- Extension based on the already-implemented code in class
- Discussion session (by TAs)
- Extended Topics
- e.g you learned GD / SGD in class, you will learn Adam in these sessions
Textbook
No required textbook
Optional resources
- Deep Learning by Ian Goodfellow, Yoshua Bengio, Aaron Courville
- Dive into Deep Learning by Aston Zhang, Mu Li, Alexander J. Smola, Zachary Lipton