Artificial intelligence could make studying birds easier, thanks to the work of a University of Alberta student.
Priscilla Adebanji, a computing science student at Augustana Campus, spent her summer experimenting with AI to improve the analysis of U of A research video showing red-winged blackbirds and their nests.
The tweaks, once fully developed, could save hours of time needed to manually review the video to identify specific birds — a task vital to her work, says Augustana professor Ivana Schoepf, who is exploring the effects of parasitism on bird behaviour.
Currently, the only way to detect the identity and actions of each bird is by listening for their distinct calls, which means reviewing hours of video. It also requires expertise, Schoepf notes.
“There has to be a level of knowledge about bird biology, and it also takes time to learn the birds and their calls.”
Though existing software can track the movements of animals like mice in the lab, “for the field it’s more complicated,” she says. “There’s a lot of background vegetation and noise, like wind, so recording conditions are not optimal for automatic detection.”
Nor is the quality of the video high, “so it can be impossible to tell the male from the female birds,” Schoepf adds. The nests are also well hidden in marsh cattails, making it difficult to see whether the birds are feeding their chicks, cleaning the nest or leaving it using an off-camera route, she notes.
With two field seasons’ worth of video — totalling about 30 hours of film on 30 nests — Schoepf approached Adebanji’s computing science professor Thibaud Lutellier, looking for a way to automatically detect the birds without having to review the tape.
“We felt there were a lot of applications in AI that could help, though we had no idea what to expect,” says Lutellier. “We had to figure out what kind of machine learning to use. There was a lot of groundwork and research involved.”
Adebanji worked through various challenges, including false readings generated by existing AI models. “They seemed to sometimes detect the bird as an airplane, and things like shadows were falsely detected as bears in the background,” she says.
Using computer vision tools and motion detection algorithms to help analyze the video, she reduced the tracking sensitivity enough to count the birds, single them out from other objects and determine whether the same bird had already been recorded.
By the end of the summer, Adebanji was able to develop software that refined Schoepf’s raw footage enough to pinpoint the exact times the birds visit their nests to feed their chicks, along with whether they are entering or leaving.
“It’s a big step up for saving time,” says Adebanji. “We have gone from eight hours to a couple of minutes to automatically detect everything.”
Though the software still needs more work, Schoepf hopes she’ll eventually be using it to reduce the labour-intensive process involved in collecting data for her research.
“It will save time in reviewing the videos, and effort in terms of having to train people to analyze them — and you need to have at least two people involved, to compare results for consistency.”
She’s also excited about the software’s potential for the larger field of animal biology.
“This could provide broad applicability to a multitude of species and projects in different settings, beyond birds. It could improve and expand ecological research.”
Further work on the program will include exploring AI applications for determining how long the birds visit their nests, and analyzing their calls to determine gender. Future students will also be tasked with better “cleaning” the videos of background sounds like wind, and seeing whether the software can apply to other animals like rodents, Lutellier says.
For the AI field, small, innovative projects like this one “highlight some of the issues we have with computer vision” in motion-detecting technologies such as self-driving vehicles and drones, he notes.
“This is useful for developing better AI, because there was a lot of trial and error during the process of developing this software. Rather than testing on very precise, high-quality benchmarks, we were working with a lot of unknown factors — and that’s how real life is. AI has to deal with it safely and successfully.”
Now in her fourth year of undergraduate studies, Adebanji says the project has sparked her interest in exploring AI as a career and taken her beyond the classroom.
“It feels good to work on a real-world application, to be able to help people with the knowledge I’ve learned.”
Adebanji’s work was funded by the U of A’s Undergraduate Researcher Stipend.