Instructor

Balamurugan Palaniappan

Semester

Autumn ‘20

Course Difficulty

The course is of moderate difficulty. Unlike a lot of other deep learning courses at Insti, this course initially focuses a lot on the mathematics behind deep learning, and thus builds a good conceptual base to understanding the working of more complex architectures later on.

Time Commitment Required

6-8 hours per week

Grading Policy and Statistics

AA 5
AB 14
AU 5
BB 15
BC 13
CC 19
CD 4
DD 9
FR 3
Total 87

Pre-requisites

The course instructor asked sophomores to de-register when I took this course.
Basic knowledge of Linear Algebra is expected (MA106 is more than enough)

Evaluation Scheme

Mid-Term Exam: 20%
Quiz: 10%
Assignments (Theoretical / Programming): 10%
Scribing, class participation, other activities: 10%
Challenge Programming contests: 10%
Course Project: 40%

Topics Covered in the Course

The Perceptron, Feed-forward networks and Multi-layer perceptron, ConvNets, Recurrent architectures like RNNs / LSTMs / GRUs, transfer learning, Attention based networks, Auto encoders, Generative Adversarial Networks, Deep Neural Recommenders. Non-convex Optimization tools for Deep Networks. Learning theory for Deep Neural Networks. Several Applications covering operations research, computer vision, natural language processing, multi-media analytics, proof checking.

Teaching Style

Concepts are covered in live lectures, with the professor sharing some short reading material / pre-recorded lectures to go through before the lecture. The teaching is clear, and easy to follow. Initial part of the course focuses a lot on the mathematical concepts of Deep Learning so the pace is slower, but the course really picks up speed after the midsem.

Tutorials/Assignments/Projects

The Assignments are of good quality. The programming assignment requires you to completely code an entire neural network from scratch, for different kind of tasks, and implement the backpropagation yourself (derived in class in detail). The second assignment had 3-4 open ended problem statements that required the students to do their own research, check out the latest literature, and then suggest the approach towards solving these problem statement. This assignment brings focus on the problem solving approach to a deep learning task.

Feedback on Exams

The exams are of moderate difficulty, and most questions just focus on the conceptual understanding and problem solving approach rather than mathematical ability.

Course Highlights

The high-weightage course project is a good opportunity to explore and do significant work on an awesome problem statement of you choice (from the pool of topics shared by the professor).

How strongly would I recommend this course?

This course is recommended for anyone who is looking for a more mathematical understanding of Deep Learning, that is lacking in a lot of MOOCs on Deep Learning. The course project is an added benefit, and your dedicated performance there can contribute to a very good grade due to the high weightage. This course is also in the Mechanical Department Electives basket, so that’s an added motivation.

Going Forward

Look out for IE 663 - Advanced Topics in Deep Learning taken by the same professor in the following semester. Also consider EE 782 - Advanced Topics in Machine Learning by Prof. Amit Sethi (odd semester), CS726 - Advanced Machine Learning by Prof. Sunita Sarawagi (even semester)

IE 643 Review By: Shubham Lohiya