Gold medal winner explores the relationship between body movement and speech

A conversation with Jennifer Hinnell, '20 PhD (Linguistics)

FGSR Staff - 10 June 2020

Every year during Spring convocation, the University of Alberta awards the Governor General's Gold Medal to the doctoral graduate who achieves the highest academic standing. This year's recipient, Jennifer Hinnell, is an extraordinary scholar with intriguing interests: she studies how the words we say and the ways we move are interconnected. We spoke with Jennifer this week.

Tell us about your thesis research.
My dissertation is titled Language in the body: Multimodality in grammar and discourse. It's a look at the relationship between what we say in our speech and how we move our bodies when we're speaking. The "multimodality" part refers to this speech/body connection and usually includes the gestures, shoulder movements, head movements, gaze, and sometimes even eyebrow movements that accompany speech.

For example, when someone says, "I dunno", it's easy to imagine them shrugging their shoulders and raising their forearms on each side at the same time. People can't help but gesture when they're talking, but linguistic analysis has tended to focus almost exclusively on text.

How do you study the speech/body connection?
One of the key contributions of my research is that I look at conversational data. As soon as you're looking at conversation, you have to include the body in the linguistic analysis. I use technologies like motion capture and really large databases of broadcast media to look at hundreds of spontaneous conversations. Then I analyze these conversations for particular speech patterns and coordinated patterns in the body.

What have you found, and what are the implications?
My research suggests that for studying and teaching language we should be starting with these speech/body patterns that are so prevalent in conversation. I have also started to explore how my research could make language production more human-like in technologies such as virtual agents, and in dialogues in animations, films, and video games.

Whom would you like to acknowledge?
Research never happens in a vacuum. I'm sincerely grateful for the support of my supervisor Dr. Sally Rice, who invested so much time and energy in me over the last number of years, and to collaborators in Germany and the US who mentored me and really helped me push the edge of what I'm doing. I'm also grateful to my department, the university, and SSHRC and Killam Trusts for financially supporting me at various times throughout my PhD. Working on a PhD was a commitment for everyone around me as much as it was for me. I really appreciate all the people who were in my corner - especially my husband, my family, and a fantastic group of moms with young kids in my neighbourhood.

You completed your previous degrees in British Columbia. What kinds of hobbies and activities have you pursued here in Edmonton?
I'm an avid cross-country skier, so Edmonton is a good place to be from that perspective. I did a lot of thinking about my research while skiing in the city's parks and on trails east of Edmonton. Mountains are a soulful place for me, so that's where I go to feel restored. Getting into an excellent fiction book, going to the opera, and biking and skiing with my kids are also things that bring me joy. I also sing in a choir (the Richard Eaton Singers - we rehearse on campus and I toured to Germany with them in 2013) - I love it, it's a close community, the music is gorgeous, and rehearsals take such a different physical and mental energy than analyzing data or synthesizing research findings!

What's up next for you, Dr. Hinnell?
A Killam Postdoctoral Research Fellowship at the University of British Columbia to continue my research with Dr. Barbara Dancygier. We'll be exploring multimodal linguistic patterns across a variety of media genres, focussing on how people assert, negate, and reject beliefs. This is something that is front and centre of public discourse right now, where the phenomenon of post-truth is really dominating airwaves and the internet. I'm also really interested in pursuing the applications of my research to AI and gaming in particular, so I'll be seeking out people in those areas to talk to.

Learn more about Jennifer's research in this profile by the Faculty of Arts.