Folio Letters University of Alberta Edmonton, Canada June 13, 1997

Internal KPIs: Street drug of the university

Central Administration has recently presented to GFC a draft of its own "performance measures" (Folio, 30 May), because apparently they have some reservations about the KPIs developed by the Minister of AECD. I read the draft of these internal KPIs at GFC with considerable disappointment. None of them actually measures the quality of what we do. They have the stamp of over-simplicity and irrelevance that one has come to expect of all such attempts to reduce complex activities into simple numbers. What is this set of measures that Central Administration wishes to adopt?:

1. To assess student quality, they will monitor two indices:

   (a) the proportion of Alberta high school graduates achieving greater than 80 per cent and greater than 90 per cent matriculation average who attend the U of A, and

   (b) the geographic source of our incoming students. They wish to increase the proportion of students from afar. Are we to infer from this that student quality somehow improves with the distance from Edmonton and Alberta? Or perhaps there is a perception that attracting more students from outside Alberta is one reflection of our growing national and international reputation. But our president has stated before that the quality of Alberta high school students is better than that of Ontario and Quebec students (see Folio, 29 November 1996). Why then would we want to increase student intake from these provinces? The whole field of KPI-manipulation is contaminated with this sort of fuzzy logic.

2. To assess learning experience, there are two measures of "program quality," and both involve student satisfaction with their educational experience. I agree that student satisfaction is an important indicator in its own right, but surely it does not substitute for more objective measures of program quality.

3. To assess career outcomes, they will monitor overall employment success and relevance of employment to the graduate's educational experience. Unfortunately, there are many variables which influence overall employment, most of which are unrelated to anything that the university can do. "Relevance to employment" is whatever you wish to make of it. Is flipping hamburgers relevant employment for a business grad or a food science and nutrition grad, but irrelevant for all others?

   Who defines 'relevance'? Isn't it more important to have employment which is challenging, interesting and relatively long-term irrespective of any link to one's degree program?

4. Academic quality (really "quality of academic staff"). This indicator will have several measures associated with teaching and research. Research quality will be measured largely in terms of how much external research funding is attracted to the university. In principle this is not too bad as an intra-institutional measure, where the proportion of high-expense and low-expense research programs might remain relatively constant over time. But during a period of faculty renewal, many of our senior faculty (with presumably larger research grants) are being replaced by junior faculty, who may be attracting smaller grants in the short to medium term. How will our administration massage these data to our advantage?

   There is a single measure used for teaching quality: the number of 3M Teaching Awards. We are all rightfully proud of the 16 Award winners from the U of A since 1986, but how does such an indicator relate even tangentially to overall teaching quality? Perhaps Central Administration has developed a plan to ensure that henceforth our students will be taught only by our 3M Teaching Fellows?!

   The University already has a mechanism in place for collecting an incredibly rich data base on student opinion of teaching. Perhaps there are ways of revising the universal teaching questionnaire so that it can provide us also with objective data useful for gauging course quality?

5. Alumni financial support is the final performance measure proposed by the Administration. Considering that we've just launched the most aggressive fund-raising campaign in our history, surely it's a foregone conclusion that we'll look very good on this score over the next few years.

Why am I so suspicious of these activities; surely they are no worse than harmless? Wrong! It costs a surprising amount of money to collect, tabulate and analyze these data. Much of it is totally buried and perhaps fragmented in the system, and thus unlikely to be easily visible in any budget or financial statement. The British government imposed its own very complex system of KPIs on post-secondary institutions in the early eighties. It is estimated to approximately ten million pounds ^§ a year to administer. To waste valuable resources for which nobody is held accountable in order to collect meaningless data for the purpose of showing simplistic (though perhaps colorful) PowerPoint® presentations to the Board of Governors and GFC simply does not make any sense!

I urge all who care about our university to resist this wasteful exercise. None of us would want these irrelevant numbers to determine how we teach or do research. Education emerges from thoughtful interaction among people who study and do research together, and none of this can be measured by Maclean's, by AECD nor by our administration with current KPIs. To my colleagues at GFC and everywhere, just say NO!

Reuben Kaufman, president, AAS:UA

Reuben Kaufman

Student thanks Dr. Elizabeth Crown for outstanding support

It's Thursday evening, 9:30 pm. It's time to tidy up the desk, time to pick out an outfit to wear tomorrow, time to double check the number of copies, time to rehearse one last time, time to sleep, time to stop worrying! It's the evening before an oral presentation and the butterflies are having a festival in my stomach.

The telephone screams and I jerk, startled and panicky. "Can I answer, do I have time to talk to anyone? Who doesn't know it is my big day tomorrow, who would phone to chat tonight?" The rings are like sirens, deafening and demanding. I submit.

"Hello Kay, this is Betty. I've just finished re-reading your final chapter and called to congratulate you. It flows nicely, you've done a good job. Tomorrow will go very well for you."

I whispered my appreciation and sank down onto the chair, relief flooding over me. A voice of support, a vote of confidence when I need it most. A watchful star is shining over me tonight, tomorrow will be a day to celebrate.

This short story is one of teaching with a heart. It is the story of a nervous and excited graduate student preparing for her master's oral defence and a thesis supervisor who is also a professor and department chair with many, many responsibilities that regularly extend her workday late into the evening. It is a story of a supervisor who, despite administrative pressures, understands and takes time for students. It is a true story and one which happened. It is with regret that we will no longer call on you, Dr. Crown, as chair of the Department of Human Ecology. You have served your colleagues and the university well throughout difficult faculty transitions. You have persevered with optimism and creative problem solving during devastating budget cuts. You have inspired students with technical expertise and gentle coaching when minutes were mythical and seconds were sacred. For me, you have been a role model in teaching professionalism and caring. And I thank you - humbly and most sincerely.

Kay McFadyen
PhD student,
Department of Educational Policy Studies

Letters published in the June 13, 1997 edition of Folio.

Folio Information

Office of Public Affairs

University of Alberta