With the economic and labour disruption wrought by COVID-19, for a time it seemed Ontario and Alberta had realized the folly of judging their universities’ performance against metrics over whichthe universities themselves had little or no control. Refreshingly, both provinces made an about-face and rightly pressed pause on their plans. However, this apparent change of heart was short-lived. In November 2020, Ontario announced its intention to push ahead and fully implement its performance-based funding plan as previously detailed in April 2019, joining Alberta, who had already vowed for the need to press on with its plans back in June 2020.
Both province’s proposed indicators for determining university performance are linked to labour-market and economic outcomes. For example, among the 10 indicators Ontario plans to use are “Graduate employment earnings,” “Graduate employment rate in a related field,” and “Research funding from industry sources.” Alberta previously suggested it intends to use similar indicators, but has not officially announced the final set of metrics it will employ.
At stake is nothing less than the hearts and souls of our universities
Let’s not bury the lede here. It is difficult to see these exercises as anything more than heavy-handed ideological attempts to redesign the fundamental mission of our universities.
It is difficult to see these exercises as anything more than heavy-handed ideological attempts to redesign the fundamental mission of our universities.
Under such schemes, universities will be coerced away from their traditional aspirations of fostering critical, creative, and well-rounded citizens—while performing research in the public interest—toward drastically retooled, narrowly conceived “outcomes” focused on trying to serve the current labour market and performing corporate-styled research and development.
Performance-based funding simply does not work
There is a compelling body of evidence for permanently shelving the implementation of performance-based funding for universities, with the most significant finding being that it simply does not work. For example, in a recent study published in December 2020 in the journal Educational Evaluation and Policy Analysis, Justin Ortagus and his team undertook a systematic and comprehensive review of 52 of the best peer-reviewed studies published between 1998 and 2019 that examined outcomes of performance-based funding in the 41 U.S. states that adopted the funding model. After meticulous review, they concluded that performance-based funding (PBF) “is generally associated with null or modest positive effects on the intended outcomes of retention and graduation, but there is also compelling evidence that PBF policies lead to unintended outcomes related to restricting access, gaming of the PBF system, and disadvantages for under-served student groups and under-resourced institution types.”
Labour force discrimination and university admittance
Indeed, Ortagus and his team’s research confirms what many of us feared. Tying student enrolments to specific outcomes, such as “Graduate earnings” and “Graduate employment rate” will skew funding towards institutions that enroll students with the best chances of being employed at the highest pay immediately after graduating. Such an approach, as is planned under Ontario and Alberta’s performance-based funding models, will happen at the expense of prospective students from Canada’s most marginalized groups, since equally qualified, but racialized Canadians are hired with less frequency and for less pay than their non-racialized counterparts. Thus, these plans will sabotage any hopes for equity, diversity, and inclusion gains amongst students pursuing a postsecondary education.
Today’s labour market
There are also persistent questions about the metrics used by these schemes. For example, the rationale for using current labour-market realities to direct future postsecondary education funding is questionable at best. The labour market can change swiftly, making a shortsighted degree program ineffective by the time students graduate. A case in point is Alberta’s optimistic investment in petroleum engineers ten years ago and the stark reality that that job market has dried up.
Moreover, as highlighted by the federal government’s 2017 Expert Panel on Youth Employment, the nature of work is changing. We are shifting away from manufacturing and towards service and knowledge economies with a greater emphasis on problem-solving, communication, interpersonal, and critical thinking skills. The report concludes, perhaps obviously, that “the world of work is transforming rapidly” and that the key to navigating such a future is to remain flexible and fluid; it goes on to state, “Some of the next job opportunities may not even exist today.”
Universities excel in teaching flexible thinking and problem-solving skills, with a main benefit of these skills being that they are portable and may be applied in many different, ever-changing, and unanticipated contexts. Why the metric “Graduate employment in a related field” would even be considered seems regressive, harkening back to an age when freshly minted graduates might have had the luxury of remaining in their newly acquired jobs until retirement. Sadly, this does not correspond with the current nature of work and the reality of Canada’s growing gig economy. It also conveniently overlooks the reality that industry must bear some responsibility for worker development and training, should they require specialized skills. This critique extends to ill-considered micro-credential plans now being floated in both provinces.
Governments should not be judging or punishing universities for graduating students who might choose lower paying careers that they find more meaningful and fulfilling—especially when many of these professions are vitally important to society. Given that students are increasingly asked to shoulder a greater percentage of the cost of their degree programs, tackling the growing cost of tuition would seem a better government priority.
Standardized testing for universities
The ironies continue as one considers the “Skills and competencies” metric. Here the government is actually planning to expand standardized computer-based testing beyond K–12 and into postsecondary education. Do we need to look any further than the high-stakes testing craze that has all but strangled sound pedagogy in so many other countries for clues to what could go wrong? Externally developed standardized tests undermine traditional collegial authority, autonomy, and education quality.
It seems foolhardy to privilege a few computerized tests over the many authentic assessments professors have developed to best evaluate performance in their courses—assessments that are reviewed by expert professionals and subject matter specialists. How any standardized test could be more heavily weighted or even compared to a university degree’s existing course and program requirements betrays a cynical de-professionalization agenda. A standard four-year undergraduate experience includes approximately 20–40 “second-opinion” expert evaluations of student achievement and subject matter coverage as students progress through their programs. No one-size-fits-all, quantified, out-of-context test administered via computer could ever compare.
Then there is the internal inconsistency of rewarding universities for higher graduation rates while incentivizing the removal of challenging courses and subject matter. Instead, it would be much better to look at year-to-year retention, identifying areas of concern, discerning potentially under-served groups, and directing resources to better support these areas and groups. Indicators showing what percentage of resources are allocated to student supports would also be more effective, as would examining whether or not there is a year-over-year increase in the percentage of students enrolled from non-traditional groups. The extent to which faculty teaching is supported by teaching and learning centres is also an important measure of an institution’s overall excellence.
Universities are already accountable
To imply that universities are currently unaccountable is not just misleading, it is untrue. External and internal program reviews, scholarly peer review, student teaching evaluations, professional accreditation bodies, and strict financial audits all act as existing accountability measures.
In Ontario’s case, it appears that the government was so busy applying audit culture via performance-based funding everywhere it doesn’t belong that it lost sight of the one place auditing does belong: accounting. As George Fallis points out in the Financial Post “In its fascination with goals and metrics, the province lost sight of the basics.”
To imply that universities are currently unaccountable is not just misleading, it is untrue.
Universities already commit a great deal of resources toward gathering and responding to data and publishing their performance to better serve, attract, and retain students—and ultimately to better serve society. Such work is organized through dedicated research offices. Universities Canada also tracks data, for example, in the area of Indigenous outreach, leadership and academic programs.
In Ontario, one of the new metrics will be “Experiential learning,” and similarly proposed as “Proportion of students who participate in work-integrated learning” in Alberta. Here, again, is an area where universities do not need incentive to engage. As universities look for ways to attract students, they are constantly seeking to improve programs, courses, and learning formats. One area where universities have been focusing efforts in recent years is work-integrated learning, where students can enroll in co-op internship programs as part of their education. Popular with students, postsecondary institutions have been investing in these programs, with enrolments increasing from 53,000 in 2006 to 65,000 in 2013 to 75,000 in more recent estimates from Co-operative Education and Work-Integrated Learning Canada—all without the punitive stick of performance-based funding.
Corporate research and development
Judging and funding universities on their success attracting research funds (in Alberta under “sponsored research funding”) could well lead to a narrowing of scholarship, both in teaching and research. As the pressure placed on institutions invariably trickles down to the individual level, academics will be encouraged to focus on what is rewarded instead of what matters.
In Ontario, a similar metric rewards private research funding from industry. Somewhat perversely, this approach will tie public funding to private funding and clearly incentivizes the further commercialization of university research.
Making matters worse is the perverse incentivization of competition between universities rather than collaboration.
Such an emphasis impacts society by devaluing less costly but no less important scholarship, including risky, yet innovative research; community-engaged research; and other valuable research endeavours that cannot easily be measured or reflected by a simple financial calculus. Rather than uncovering ground-breaking new ideas, following uncertain but innovative paths that become potential game-changers, or working in the service of the communities in which they reside, scholars will be incentivized to get in and out of research initiatives and funding cycles with something (anything?) that ticks the suitable boxes. Making matters worse is the perverse incentivization of competition between universities rather than collaboration.
Growing administrative costs
Imposing performance-based funding systems will invariably lead to the addition of another layer of costly bureaucracy at both institution and ministry levels. Universities will need to create new or re-classified management positions whose sole purpose will be to assess, report, target, and, ultimately, game the new metrics. On the government side, bureaucrats will be needed to gather, evaluate, monitor and, in the longer term, respond to the manipulated metrics as well as to their unintended consequences.
Unintended outcomes: The U.K.’s Teaching and Research Excellence Frameworks
The United Kingdom offers an example of the distortions and ever-increasing costs that occur when coercive metrics are imposed. Various iterations of the Research Excellence Framework (REF) have led to unintended consequences, such as ballooning costs, increasing publish-or-perish pressures, and a de-emphasis of university teaching. The government followed this by creating a Teaching Excellence Framework (TEF) to add to an ever-expanding set of auditing frameworks.
Although the TEF is, for now, still voluntary, it was linked to how much universities would be permitted to raise tuition—something revisited following criticism. It is still too early to determine what effects the TEF’s gold, silver, and bronze ratings will have on teaching. What we do know is that the TEF was suspended in January 2020 so that it could be redesigned and redeveloped.
Now, the REF is going through its own re-forming. Tellingly, in a speech published on October 20th, 2020, Amanda Solloway, U.K Parliamentary Under-Secretary of State for Science, Research and Innovation stated: “the processes researchers use to communicate with each other have now become so ingrained into the recognition and reward system that publication and citation seem to have become ends in themselves. This gives rise to related issues—we know people feel pressured to show significant results from their work, to get it published, just to justify the effort and investment involved. This could be having a profound effect on the very integrity of science itself—leading to questionable research practices and evidence of a growing crisis in the reproducibility of research. A crisis which over half of surveyed academics recognize as significant. We have created this situation, in part because of the way we evaluate success.”
It is no surprise that these frameworks have led to drastic deformations and growing bureaucratic bloat, while diverting larger and larger pieces of the pie away from teaching, research, and service—the very budget line items that best serve students and society.
Campbell’s Law
It is not difficult to predict what will happen under performance-based funding exercises. These are classic examples of “Campbell’s Law.” Put simply, the more an indicator becomes a target the more it distorts that which it set out to measure. In discussing examples taken from
the former U.S.S.R., Donald Campbell relays accounts of nail factories overproducing their biggest nails when targets were set by weight and overproducing small nails when targets were set by quantity. As performance indicators were used as quantifiable and enforced production targets, they would inevitably lead to the underproduction of needed items and the overproduction of redundant items.
It’s time to just say no
Let’s face it, whenever governments start floating performance-based funding schemes, one can be certain they will be quickly followed by budget cuts. It is so predictable. The fact is that these are tired, costly ideas adopted by unoriginal and cynical governments copying other governments and jurisdictions where these ideas have already failed. Let’s not be drawn into a government-run version of The Postsecondary Hunger Games. The public deserves and should demand better.
Universities are much more than entrepreneurial training centres to be rewarded for performing short-sighted corporate-styled research and worker development. With that mandate, they cease to be universities in any sense of the word. To create a future where we can all thrive, our citizens need to not only have the skills to prosper today, but be capable of imagining and implementing a better tomorrow.
