In a historic move for the global artificial intelligence community, one of the world's leading AI research companies, DeepMind, opened its first satellite research lab outside the United Kingdom this summer in Edmonton. Headlines reflected a universal optimism that DeepMind Alberta will do for the Canadian startup AI community what DeepMind did for the London AI community, as well as create places for University of Alberta grads to stay rather than being forced to find work beyond Canadian borders.
What is DeepMind?
DeepMind is on a scientific mission to push the boundaries of artificial intelligence, developing programs that can learn to solve complex problems without being taught how, all with the goal of ultimate application to the most pressing real-world issues. DeepMind researchers typically use games as test beds for their research to develop and benchmark their algorithms. Acquired by Google in 2014, DeepMind is now part of the Alphabet group. DeepMind Alberta signals DeepMind's commitment to accelerating Alberta's and Canada's AI research community.
The connections between DeepMind and the university run deep, with roughly a dozen UAlberta alumni- some of whom played important roles in some of Deep- Mind's signature advances with reinforcement learning in AlphaGo and Atari-working at the company in their headquarters in London, England.
David Silver ('09 PhD), one of the most well known DeepMind-ers, caught the world's attention in 2015, when Alpha- Go, the program he started during his PhD studying with computing science professors Rich Sutton and Martin Mueller, bested the world's Go Champion, Lee Sedol. "It was an amazing feeling to see that dream realized in Seoul in such dramatic fashion," says Silver. "During my PhD, I dreamed of one day challenging a world champion. AlphaGo was the culmination of 12 years of research that I began back in Alberta."
Go was long viewed as the most challenging of classic games for artificial intelligence. Resulting in a landmark paper in Nature, AlphaGo further demonstrated just how intelligent AI can be, using games as a platform to demonstrate applicability, an area for which the University of Alberta is globally renowned.
The advice not taken
Silver fondly reflects on memories of his UAlberta student days, including a departmental ski trip to Jasper-when freezing temperatures kept everyone in the basement playing board games-as well as taking part in the Pi Day beard-growing competition. However, one of the most important things he learned during his time at UAlberta was following his gut rather than well-meaning yet perhaps overly cautious advice.
From left, University of Alberta professors Rich Sutton, Michael Bowling, and Patrick Pilarski are heading up the new DeepMind Alberta research office in Edmonton.
Meet the DeepMind Alberta Team
Based in Edmonton to work closely with the Department of Computing Science, the lab is being led by computing science professors Rich Sutton and Michael Bowling as well as adjunct professor Patrick Pilarski. DeepMind Alberta is rounded out with seven more researchers, all with roots at UAlberta.
Bowling is best known for his research in poker, most notably with two milestone discoveries, both published in Science. Cepheus, in 2015, solved heads-up limit Texas hold'em, and DeepStack, in late 2016, achieves professional-level play in heads-up no-limit Texas hold'em.
Pilarski, the Canada Research Chair in Machine Intelligence for Rehabilitation, is an assistant professor in the Department of Medicine, looking at human-machine interaction, rehabilitation technology, and assistive robotics.
Sutton is world-renowned for his foundational research in reinforcement learning-he literally wrote the textbook-in which machines learn based on their environment, much like humans, trying to replicate good outcomes and avoid bad outcomes based on learned experiences. He is the most cited researcher worldwide in reinforcement learning and the 11th most influential researcher in all of computing science, and his textbook has been called "the single most influential publication in computing science" by the Allen Institute for AI.
"The head of the department told me to switch my topic away from computer Go as it was too difficult. But what was 'impossible' before is now realizable with current technology. We all need to adjust our ambitions and expectations accordingly. Do not underestimate the power of reinforcement learning. Even if initial results don't go your way, never give up, don't compromise your goals, keep digging to understand the reasons for failure and searching for new and better approaches. Sometimes you have to have enough belief in your own ideas to ignore well-intentioned advice."
In addition to his inner voice, Silver says he often feels like he has an "inner Rich Sutton" sitting on his shoulder, giving advice on his research.
AI angel on their shoulders
Speaking of Sutton's advice, the AI pioneer's work with DeepMind can be traced back to the inception of the company. In fact, DeepMind CEO and co-founder Demis Hassabis says Sutton has been instrumental to the development of DeepMind. "I first met with Rich-our first ever adviser-seven years ago when DeepMind was just a handful of people with a big idea. He saw our potential and encouraged us from day one. So when we chose to set up our first international AI research office, the obvious choice was his base in Edmonton, in close collaboration with the University of Alberta, a leader in reinforcement learning research thanks to his pioneering work."
According to Sutton's former post-doctoral fellow Hado van Hasselt, reinforcement learning is the most exciting thing to work on. "Even after having worked on reinforcement learning algorithms for more than a decade now, the idea that we can create algorithms and build agents that can solve problems themselves without us needing to specify a solution in advance is magical and fascinating to me," says van Hasselt from London.
Teamwork makes the dream work
Van Hasselt says he learned from his supervisor Sutton how to pay attention to what is important and more importantly, to communicate it accordingly. Van Hasselt says Sutton always reinforced the need for clear communication in the lab and beyond. "Without collaboration and communication, there is little hope for progress of research as a whole," says van Hasselt. "If you discover something but do not communicate this clearly, this can greatly diminish the potential impact and thereby limit progress. Perhaps the single most important lesson I learned from Rich Sutton was following these three rules for communication: Speak clearly; say important things first; and explain why you say what you say, or in other words, show your intent."
Communication and collaboration just got a whole lot more interesting and important for the DeepMind researchers, who, thanks to the new Alberta-based lab, will be communicating and collaborating across the globe on a scale previously unexplored.
Continuing in the DeepMind tradition of a unique hybrid of academia and startup culture, the new lab represents a pipeline to keep talented people in the country. Sutton, Bowling, and Pilarski, all three of whom will remain with the Alberta Machine Intelligence Institute, will also continue teaching and supervising graduate students at the university to further foster the Canadian AI talent pipeline and grow the country's technology ecosystem. "Edmonton is sticky," says Sutton with a smile. "A lot of people who get trained here want to stay here, and now there's an opportunity to retain good people and attract people back."
Home(town)grown
One of Sutton's other former students working at DeepMind in London now has an opportunity to ponder a return to Edmonton. Joseph Modayil ('97 BSc, '99 MSc), a math grad who eventually post-docced with Sutton, speaks with optimism about the new initiative in his hometown. "The new DeepMind research lab in Edmonton will cause the AI community to grow and advance the field faster." Modayil credits his education in mathematics and computing science for providing a foundation for careful reasoning in formal domains where conventional intuition isn't available.