Contact the Faculty of Science

Rupam Ashique

Rupam Mahmood

Assistant Professor


Computing Science


My research objective is developing a computational and scientific understanding of general-purpose mind-like systems for robots. Currently, I am working on developing policy and representation learning methods for real-time systems.


CMPUT 652: Reinforcement Learning with Robots (Fall 2019)

In this course, we will study the foundations of RL to be able to develop policy learning methods and learn about systematic ways of studying a real-time system to reveal the uncertainties involved in real-world tasks. This investigation will allow us to understand the differences between real-world and standard simulated tasks so that we can adapt task setups and algorithmic implementations to the real world as well as enhance the simulated tasks to incorporate the additional challenges in real-time systems. En route, we will learn about other promising approaches to learning in robotics that are not performed in real-time, such as learning from demonstration and simulation-to-reality transfer.

CMPUT 397: Reinforcement Learning (Winter 2020)

In this course, we study the design, analysis, and applications of reinforcement learning agents that interact with a complex, uncertain world to achieve a goal. We will emphasize agents that can make near-optimal decisions in a timely manner with incomplete information and limited computational resources. The course will cover Markov decision processes, reinforcement learning, planning, function approximation (online supervised learning) and real-world applications. The course will take an information-processing approach to the concept of mind and briefly touch on perspectives from artificial intelligence, psychology, neuroscience, philosophy, and robotics.

The course will use a recently created MOOC on Reinforcement Learning, created by the Instructors of this course. Much of the lecture material and assignments will come from the MOOC. In-class time will be largely spent on discussion and thinking about the material, with some supplementary lectures.

Pre-requisites: The course will use Python 3. You should either know Python 3 or be sufficiently experienced with programming in other languages that you can learn Python 3 quickly. We will use elementary ideas of probability, calculus, and linear algebra, such as expectations of random variables, conditional expectations, partial derivatives, vectors, and matrices. Students should either be familiar with these topics or be ready to pick them up quickly as needed by consulting outside resources or the teaching assistants.