Associate Professor Department of Computing Science University of Alberta Edmonton, Alberta Canada T6G 2E8

Office: 311 Athabasca Hall Email: szepesva AT cs DOT ualberta DOT ca Phone: (780) 492-8581 Fax: (780) 492-6393

[en/hu dict] [RLFA RG] [CMPUT 412] [calendar] [math genealogy]

Who am I?

Faculty at the Department of Computing Science, one of the 10 PIs at AICML, member of RLAI. On leave from SZTAKI. However, more importantly, I am the head of a fun family. If I am the head, Beáta is the neck. Our kids are Dávid, Réka, Eszter and Csongor. If nothing but they make sure that my brain is kept immersed in a "persistently exciting" environment!

News

I gave a talk titled Towards robust reinforcement learning algorithms at the European Workshop on Reinforcement Learning. The talk investigates how we can relax assumptions that we usually make in reinforcement learning while still arriving at meaningful results. The pdf slides are here. Talk on the full characterization of the complexity of stochastic partial monitoring (slides) at the Workshop on On-line Trading of Exploration and Exploitation 2 , July 2, 2011 (50 mins) at ICML, video! Future plans for the summer: Visiting COLT in Budapest in July, Mini-Workshop on "Mathematics of Machine Learning" at Oberwolfach August 21-27. In addition, we have a student-run course on Monte-Carlo and Simulation, tea-time talks twice a week, LLL talks at noon on Wednesdays. (13/09/2010) Talk at the workshop on Stochastic Approximation about off-policy learning (slides). (01/08/2010) Gabor has won the best student paper award at ALT with this paper on partial monitoring. I am very proud of you Gabor (and David)!! Tea-time talk (29/07/2010): Policy iteration is strongly polynomial. The slides are here. Book! (10/07/2010) Algorithms for Reinforcement Learning. The draft of the book is here. The book is intended for researchers and students who want to learn about the state of the art in RL. The focus is on the algorithms (basic and advanced), what we know about them and what we don't but would be good to know. And I tried to keep it short (~100 pages). As usual, I am interested in your for feedback:) Rich and I gave a tutorial at AAAI (10/07/2010). The tutorial webpage is here. The tutorial was based on my just published book. The slides are on the webpage. They should give you a good idea about what is in the book (but they go beyond the book).

Toward the understanding of partial-monitoring games

A video made by my daughters, Eszter and Reka (then in grade 9 and 10) about the final project of my 2009 CMPUT 412 class. The video was submitted to the IJCAI-09 where it won the "community appreciation award". Well done Eszter and Reka! And thanks for my students who made this possible by their excellent work in the class.

• Prospective grad students who are interested in joining the Statistical Machine Learning degree specialization program, which is a joint program between our department and the MathStat department should look here. Further info here.
• Co-organizing an ICML-2010 workshop with Peter Auer and Samuel Kaski. The topic is Reinforcement Learning and Search in Very Large Spaces
• Recently, I have been working on an RL overview paper, which I plan to keep updating for a while and then eventually I will send it to some journal. The target audience is students and researchers interested in algorithms of reinforcement learning. The survey is fairly comprehensive, yet I tried to keep it succinct (50 pages without references, 65 pages with references). Comments, suggestions are welcome.
• Old news: I am teaching CMPUT 412, Undergraduate robotics. I have decided to write up lecture notes for the course, which will be posted to the website. Stay tuned.
• Gradient Descent Methods for Reinforcement Learning , talk at WUSTL (13/11/2009)
• Cryptography and beyond: my "Lunch and Learn" seminar (29/07/2009)
• Responding to an emergency situation, I have spent a few hours by searching on the IEEE website to collect recent references on applications of RL. Here are the results which are now linked to the page on Successes of RL. See also Satinder's similarly titled page here.
• In Fall 2009 I was teaching On-line learning.
• Participating in the "Subspace methods for system identification" reading group discussing this book.
• Participating in the "Beyond Lasso" reading group
• Organizing the On-line Learning with Limited Feedback workshop to be held at COLT-09, June 18, Montreal, Canada.
  
• I enjoyed participating in Max Planck Symposium on Autonomous Systems in Tübingen.
• I have graduated my first students at UofA!
• The Compressive sensing reading group was functional during 2008 Fall.
• I gave some talks at MLSS'08, Ile de Re:
  • Online learning in stochastic environments
  • Batch learning in Markovian decision processes
  • Online learning in adversarial environments
• I talked at MLSS'08 (Kioloa) on "Reinforcement Learning". Topics covered: MDPs, dynamic programming - the operator-theoretic approach, (approximate) planning algorithms and the curse of dimensionality, efficient on-line learning, classical and new algorithms (TD and least-squares algorithms).
• My slides on batch RL presented at the RL Workshop in Barbados. I gave a summary of what is known (theoretically) and listed ca. 10 open questions. If you are interested in any of these problems, drop me an e-mail!

Research interests