IE643 - Deep Learning
The course is of moderate difficulty. Unlike a lot of other deep learning courses at Insti, this course initially focuses a lot on the mathematics behind deep learning, and thus builds a good conceptual base to understanding the working of more complex architectures later on.
Time Commitment Required
6-8 hours per week
Grading Policy and Statistics
The course instructor asked sophomores to de-register when I took this course.
Basic knowledge of Linear Algebra is expected (MA106 is more than enough)
Mid-Term Exam: 20%
Assignments (Theoretical / Programming): 10%
Scribing, class participation, other activities: 10%
Challenge Programming contests: 10%
Course Project: 40%
Topics Covered in the Course
The Perceptron, Feed-forward networks and Multi-layer perceptron, ConvNets, Recurrent architectures like RNNs / LSTMs / GRUs, transfer learning, Attention based networks, Auto encoders, Generative Adversarial Networks, Deep Neural Recommenders. Non-convex Optimization tools for Deep Networks. Learning theory for Deep Neural Networks. Several Applications covering operations research, computer vision, natural language processing, multi-media analytics, proof checking.
Concepts are covered in live lectures, with the professor sharing some short reading material / pre-recorded lectures to go through before the lecture. The teaching is clear, and easy to follow. Initial part of the course focuses a lot on the mathematical concepts of Deep Learning so the pace is slower, but the course really picks up speed after the midsem.
The Assignments are of good quality. The programming assignment requires you to completely code an entire neural network from scratch, for different kind of tasks, and implement the backpropagation yourself (derived in class in detail). The second assignment had 3-4 open ended problem statements that required the students to do their own research, check out the latest literature, and then suggest the approach towards solving these problem statement. This assignment brings focus on the problem solving approach to a deep learning task.
Feedback on Exams
The exams are of moderate difficulty, and most questions just focus on the conceptual understanding and problem solving approach rather than mathematical ability.
The high-weightage course project is a good opportunity to explore and do significant work on an awesome problem statement of you choice (from the pool of topics shared by the professor).
How strongly would I recommend this course?
This course is recommended for anyone who is looking for a more mathematical understanding of Deep Learning, that is lacking in a lot of MOOCs on Deep Learning. The course project is an added benefit, and your dedicated performance there can contribute to a very good grade due to the high weightage. This course is also in the Mechanical Department Electives basket, so that’s an added motivation.
Look out for IE 663 - Advanced Topics in Deep Learning taken by the same professor in the following semester. Also consider EE 782 - Advanced Topics in Machine Learning by Prof. Amit Sethi (odd semester), CS726 - Advanced Machine Learning by Prof. Sunita Sarawagi (even semester)
IE 643 Review By: Shubham Lohiya