Instructor

Shivaram Kalyanakrishnan

Semester

Autumn ‘20

Course Difficulty

The course is not very difficult, and the lectures are very good and enough to score well in the course. The recommended textbook and readings are also relevant, and give a deep understanding of the topics.

Time Commitment Required

5-7 hours per week

Grading Policy and Statistics

AA 103
AB 33
AU 1
BB 25
BC 24
CC 13
CD 6
DD 2
FR 6
Total 213

Pre-requisites

The course is open to all Ph.D. students, all masters students, and undergraduate/dual-degree students in their third (or higher) year of study.
Understanding of basic probability theory is expected. Can also be picked up towards the beginning of the course.
Basic familiarity with coding in Python is required, as the assignments require you to code in Python.

Evaluation Scheme

Weekly Quizzes of 2 marks each (capped to 20 marks)
4 coding assignments (10 marks each)
Endsem (40 marks)

Topics Covered in the Course

(1) Agency, intelligence, and learning (2) Exploration and multi-armed bandits (3) Markov Decision Problems and planning (4) Reinforcement learning (5) Search (6) Multi-agent systems and multi-agent learning (7) Case studies

Teaching Style

The lectures are very clear, and well paced, with the right amount of concept depth explored in lectures. The supplementary reading material is nicely curated, and extremely relevant in exploring the topics further to get a good understanding. The professor also holds regular discussion / doubt sessions. The weekly quizzes and assignment are very nicely designed, and greatly help the learning process.

Tutorials/Assignments/Projects

Programming Assignments are very intellectually stimulating, and give a deep understanding of the theory. They are time consuming though, required as much as 5-10 hours of effort, but they are definitely worth it.

Feedback on Exams

The weekly quizzes are conceptual, and based on only the lecture content of the current week. They are easy to score in, and help keep up with the course content.
The Endsem was a take-home exam for us, with very stimulating questions, and was on an overall level moderately difficult. Though attempting this paper was a lot of fun.

Course Importance

This course is important if you want to explore the field of reinforcement learning, as it is the perfect introduction to the field. RL is a very up and coming field of AI, with a lot of research going on, and applications in robotics, trading, banking, health care, manufacturing, agriculture, etc.

How strongly would I recommend this course?

If you are even moderately intrigued / amazed by the field of RL (a must for taking the course), you SHOULD do this course.

When to take this course?

I took this course in my 5th semester. This would be the ideal semester for taking this course, as you can’t take this course before this, and taking it as early as you can allows to enough time to explore the field further before you graduate.

Going Forward

After finishing this course, you can take up the project-based advanced CS748 course by the same professor in the following semester, explore RL in fields like Robotics, Finance, Health Care, Manufacturing, etc., explore research under a professor in the form of a BTP or RnD, take up self projects in RL, or apply for university interns.

References Used

Reinforcement Learning - An Introduction (Richard S. Sutton and Andrew G. Barto) (https://web.stanford.edu/class/psych209/Readings/SuttonBartoIPRLBook2ndEd.pdf)

All content from the Autumn ‘20 iteration from CS747 (https://www.cse.iitb.ac.in/~shivaram/teaching/old/cs747-a2020/index.html)

Other Remarks

Take the ethical code of conduct very very seriously in this course, as the professor is intolerant of any breaches, whatever may be the cause.

CS 747 Review By: Shubham Lohiya