Instructor

Shivaram Kalyanakrishnan

Semester

Spring ‘20

Course Difficulty

The course is primarily project based, which is chosen by your team (of a size of up to 3) after discussing with the professor, so difficulty is variable. Other than the semester long project, the course lectures cover additional advanced RL topics, with weekly quizzes which are not very difficult.

Time Commitment Required

5-7 hours per week

Grading Policy and Statistics

AA 10
AB 14
AU 1
BB 5
Total 30

• Students auditing the course must score 50 or more marks in the course to be awarded an “AU” grade
• An outstanding research project will bypass the regular evaluation criteria and automatically result in an “AA” grade for the concerned team

Pre-requisites

CS747 - Foundations of Intelligent and Learning Agents is a hard prerequisite for this course.

Evaluation Scheme

There are 10-12 weekly quizzes, each worth 4–6 marks, and together totaling at least 50 marks. The marks contributed by the quizzes to the grade will be the maximum of the total marks earned in the quizzes and 40.

The research project carries 60 marks, divided as: 5 marks for an introductory presentation, 5 marks for a proposal, 10 marks for a mid-stage presentation, 10 marks for the final presentation, and 30 marks for the final report.

Topics Covered in the Course

Advanced topics and research projects building on topics covered in “Foundations of Intelligent and Learning Agents”. Specific topics covered generally include: (1) Contextual bandits (2) Partially Observable Markov Decision Problems (3) Function approximation for reinforcement learning (4) Sample complexity of reinforcement learning (5) Monte Carlo tree search (6) Evolutionary algorithms

Tutorials/Assignments/Projects

Project: After the problem statement is decided, the team is expected to work regularly throughout the semester, and not just a few days before the mid-term and end-term submission, where the latter case leads to low quality projects with lower learning output. There is no significant push from the Professor’s side, so the team doing the project has to be motivated enough to maintain regularity. The professor does hold weekly meets where you can join to discuss your project progress or ask doubts.

Motivation for taking this course

This course is a wonderful opportunity to explore advanced topics in Reinforcement Learning, while doing a semester-long guided research project.

How strongly would I recommend this course?

If you enjoyed CS747 a lot, and would like to dive deeper into RL, you can consider this course. If you also want to do a research-level semester long project with guidance, this course is highly recommended. But, if you get an RnD or SLP in a topic that interests you, I would recommend you choose that over this.

When to take this course?

I took this course in my 6th semester. The ideal semester for taking this course is the one after the semester you took CS747 in (for which Sem 5 is recommended)

References Used

Reinforcement Learning - An Introduction (Richard S. Sutton and Andrew G. Barto) (https://web.stanford.edu/class/psych209/Readings/SuttonBartoIPRLBook2ndEd.pdf)

All content and references from the Spring ‘20 iteration of CS748 (https://www.cse.iitb.ac.in/~shivaram/teaching/cs748-s2021/index.html)

CS 748 Review By: Shubham Lohiya