VR technology could reveal how the brain forms memories in the real world

Psychology researcher’s innovative work recognized with prestigious Sloan Research Fellowship.


Peggy St. Jacques will use her Sloan Research Fellowship to advance her groundbreaking research into how we form memories, which could shed new light on disorders including Alzheimer’s and PTSD. (Photo: Dawn Graves)

When you think of a vivid personal memory, you don’t merely recall the bare facts of what happened — you remember additional details like the emotions you felt, perhaps where you were standing when things unfolded, scents and sounds in the air. 

“Memories play such a key component in shaping who we are,” said Peggy St. Jacques, a psychology researcher in the University of Alberta’s Faculty of Science.

Autobiographical memories, also known as real-world memories from one’s personal past, are formed in a complex fashion, which is what makes them so challenging to study. In addition to the knowledge of an event itself, these memories include components like visual perspective and spatial context. To be fully understood, they need to be examined in an immersive environment. 

Memories are studied through functional neuroimaging such as fMRI scans, but this technology has offered limited insight into real-world memories because it doesn’t allow for a person’s brain to be placed in that same immersive environment. St. Jacques’s research is changing that.

Using extended reality technologies like 360-degree 3D video cameras and MRI-compatible binocular headsets allows individuals whose brains are being scanned to essentially be plunged into the real world, said St. Jacques, one of 118 exceptional early-career researchers across the U.S. and Canada who were named Sloan Research Fellows today.

“Although we know a lot about how the brain remembers real-world memories, understanding how the brain supports how we initially form real-world memories has been elusive,” said St. Jacques, who also holds the Canada Research Chair in Cognitive Neuroscience of Memory and is a member of the Neuroscience and Mental Health Institute at the U of A.

As she explained, participants in St. Jacques’s Memory for Events Lab can don virtual reality headsets while within an MRI scanner so that when the scan is being taken, they’re experiencing an immersive, 360-degree 3D video of the event.

“While participants are doing that, they’re forming memories for those experiences,” she said.

Once they exit the MRI scanner, participants can be asked questions about what they recall from the experience, providing valuable insight into what’s going on in the brain during the complex process of real-world memory formation.

“We are understanding for the first time how the brain forms real-world memories. We’ve never been able to do that before.” 

A thorough understanding of how these real-world memories are formed and how the brain supports them is a critical tool in understanding disorders that affect our memories.

“Tackling this big question of how the brain creates real-world memories has the potential to not only radically change how we understand memory, but it could lead to new insights in impairments and treatments of Alzheimer’s disease and PTSD.”

The Sloan Research Fellowships are two-year, $75,000 awards given out each year by the Alfred P. Sloan Foundation in recognition of distinguished performance by researchers considered to be the next generation of leaders in their fields. Fifty-one previous fellows have gone on to win the Nobel Prize.