The case of the Toronto School of Theology raises serious questions about the misuse of quality assurance processes in the province.
In the Fall of 2011, the Toronto School of Theology (TST) within the University of Toronto (UofT) underwent a rigorous quality assurance review of its academic programs by its theological accrediting agency in North America, the Association of Theological Schools (ATS). In all, twenty-one scholars from university-related or free-standing accredited theological institutions with an intimate knowledge of North American theological education—in teams of three, one for each of the six member-institutions of the Toronto School of Theology, and one for the TST consortium itself—strongly affirmed the quality of the programs, including the Doctor of Theology (ThD) research degree program.
In January of 2012, the Toronto School of Theology was put through a second quality assurance review process by the University of Toronto—referred to as the University of Toronto Quality Assurance Process (UTQAP). The UTQAP is governed by the Quality Assurance Framework of the Ontario Universities Council on Quality Assurance (the Quality Council) of the Council of Ontario Universities (COU). Three scholars, one each from the University of Chicago, University of Cambridge, and McGill University, were invited to undertake this cyclical degree program review, which included a two-day site visit (January 10 and 11, 2012).
The UTQAP external assessors were aware of the results of the earlier ATS assessment of TST’s theological programs, but, nevertheless, declared the research ThD degree program to be “below standard” and advised that it be closed. However, they also admitted (in their April 10, 2012 report) both that their “visit was not focused on the quality of specific programs but rather on the institution that is TST, and its relationship to the University of Toronto,” and that they had “lacked both the data and, during [their] visit, the time to do a full academic assessment …[of TST’s programs]” (p7). Despite this major discrepancy in the results of the two quality assurance reviews of TST’s academic programs, the University not only unhesitatingly accepted the results of the UTQAP review process, but immediately informed TST to suspend admissions to its doctoral programs, including the ThD degree program.
The Toronto School of Theology, and individual faculty within it, brought to the University’s attention that the UTQAP external assessors had admitted that they had not properly carried out a full review of TST’s programs. Those concerns were not only ignored but evidence justifying those concerns was removed from the report. The university, that is, suggested that the external assessors delete the compromising comments about the limitations of the review. This was an extraordinary intervention by the university.
A different version of the report, with the problematic elements removed (dated April 26, 2012) was presented to the Committee on Academic Policy and Programs and to the Academic Board as the basis for its actions against TST. Under pressure from the university, TST reluctantly agreed to suspend admission to its ThD program for the 2013-2014 academic year. Within weeks, however, the university withdrew its demand for such a suspension of admissions, even though it still claimed that the ThD was not up to standard. To this day, the university has not informed the Academic Board as to whether or not the ThD degree is up to standard, despite its obligations under the cyclical program review protocol to do so.
It is difficult to fully understand why the senior administration of a world-class university behaved in this manner, especially when the welfare of TST’s students was at stake. Some sense can be made of this as a political maneuver when one recognizes that the UTQAP cyclical degree program review was used inappropriately—by both UofT and the leadership of the TST—to resolve a long-standing tension regarding the use of the PhD designation for the TST Doctor of Theology degree.
The Toronto School of Theology has over many years (since the early 1980s) requested the university to consider approving a change in nomenclature of its research doctoral degree program from ThD to PhD. As the Director of TST puts it, the PhD nomenclature is the recognized international standard for doctoral research degrees and is, therefore, a better credential in the academic job market. Frustrated by the lack of progress in this effort, TST permitted its ThD students to graduate with a PhD degree in theology by way of an existing PhD degree program it was administering for one of its member colleges. This in itself was an irritation to the university.
Moreover, the manner in which TST transferred ThD students to what the university seems to have perceived as a “rogue” PhD program further exacerbated the tension between the two institutions. Because the ThD is conjointly granted by TST and the UofT, it is a government-funded program, whereas the PhD degree program is not, making the tuition costs for the PhD considerably higher than they were for the ThD program. In order to keep student debt to a minimum for ThD students who wished to graduate with the PhD degree, TST permitted students to transfer into the PhD program in the final year of their ThD degree program, thus reducing the individual student’s costs significantly. Although this was a long-standing practice of which a succession of deans of the School of Graduate Studies (SGS) had full knowledge, current senior university administrators were not pleased with the practice. The claim was made that the university would be greatly embarrassed should knowledge of this practice reach the office of the Minister of Training, Colleges, and Universities. As the dean of SGS at that time explained the matter to me, not only did the university want this practice stopped, it believed that TST needed to be “taught a lesson” about proper academic practice.
As already noted, TST had not only passed a rigorous external review of its degree programs but had also committed itself to further improvements of its ThD degree program recommended by the ATS assessors. One of these recommendations was that TST bring to an end the practice of transferring ThD students to the rogue PhD degree program. This gave increased urgency to TST’s campaign for a conjoint UofT/TST PhD degree program in theological studies. Consequently, TST requested to use the government mandated UTQAP cyclical degree program review process as an occasion to consider that proposal, and the University accepted the offer. Here is how the director of TST described this decision in a letter to the teaching staff in the member colleges of TST:
“Unlike most quality assurance reviews under the UTQAP, which focus their attention on program objectives, admissions, curriculum, assessment of learning, resources, and quality indicators, the report which we received for TST was overwhelmingly concerned with TST’s relation to the University. This was an unusual approach, but it came about in large part because we ourselves had invited it. With the University’s agreement, we were using UTQAP to test the possibility of a conjoint PhD and MA, and this purpose really did require the reviewers to deal with larger issues, not just the character of our existing programming” (October 3, 2012; emphasis added).
It was, of course, inappropriate for the university to have agreed to use the cyclical degree program review process for any other objective than the assessment of academic programs as set out by the COU’s Quality Council. The admission that TST and the UofT diverted the attention of the UTQAP external assessors from program assessment to resolution of the PhD problem accounts for the fact that they had neither the time nor the data to do a proper review of TST’s degree programs. Removing that admission from the external assessors’ report amounts to deliberately hiding the fact that the UTQAP assessors were not in a position to provide academic justification for their claims from the University’s Committee on Academic Policy and Programs and from the university’s Academic Board. The agreement between TST and the UofT, therefore, led to an unjustified negative judgment about the ThD which, to this day, has not been corrected.
The university not only undermined the credibility of TST’s ThD degree program, it has also refused to consider TST’s original request for a change of nomenclature of its ThD degree program to a PhD in theological studies—presumably on the grounds that the ThD was not up to standard as a research doctoral degree program. But the university was willing to approve a “brand new” PhD degree program in theological studies and it created a joint TST/UofT committee to create the new program. That committee, however, did not in reality create a new program, despite claims to the contrary. A comparative analysis of the current ThD degree program and the new PhD degree program proposed by the university shows that the latter is merely a slightly modified version of the former with no substantially different program requirements and no substantially different program outcomes. The PhD degree program makes use of all the current ThD program courses and teaching staff.
All of this was brought to the university’s attention—and to the attention of the Quality Council of COU and the Ministry of Training, Colleges, and Universities—with the request that the University recognize both that the ThD was unjustifiably claimed to be below standard, and that it agree to a change of nomenclature of the ThD degree to the PhD. The university ignored the evidence showing that their proposed PhD degree program was not a new program, and it proceeded to have the proposal considered by external assessors under the New Degree Program Approval Protocol of its UTQAP framework. Because the PhD is not actually a new program, this action was again a misuse of the UTQAP process that further harmed TST’s ThD students. Because the university claimed the PhD was a brand-new degree, current ThD students were not given the opportunity to graduate with the PhD. They will now be at a significant disadvantage in the academic job market against new, incoming students who will be graduating from the same program, but who now have the PhD degree.
One might have expected the TST administrators (and the chief executive officers of its member colleges) to have balked at this process and to have protected its own ThD students by insisting that they be allowed to graduate with the new designation of PhD (particularly because it is essentially the ThD students’ current program, slightly modified).
In fact, they did the opposite. They filed an official petition with the ATS for a change of nomenclature for the existing ThD degree program to PhD, highlighting the false distinction between the existing ThD and the supposedly “new” PhD program. But correspondence between the ATS and TST shows that TST took the same line as the university was imposing on them, namely, that the proposed new PhD was meaningfully distinct from the ThD.
The ATS, however, did not accept the argument that the proposed new PhD proposal was a new degree program, and it informed TST that:
“While the province and the University of Toronto consider a change in nomenclature such as this to comprise a new degree program, the Commission staff understand this change to be a nomenclature change with only minor adjustments and the addition of two required courses” (Letter from Dr. Tisa Lewis, Senior Director, Accreditation and Institutional Evaluation; January 6, 2015).
The letter from Lewis also indicated that ATS expects TST to allow ThD degree students to be able to choose either designation upon graduation. However, the university refuses to allow TST to proceed in this manner. In contrast, when the university changed the nomenclature of its law degree program from the LL.B. to the JD, it permitted all current students, and former graduates, to choose use of the JD nomenclature. The fact that TST is not objecting to the university’s stance on this issue is likely not based on agreement with the university’s position, but rather on fear that objecting to the university’s position may jeopardize the now agreed upon UofT/TST conjoint PhD degree program in theological studies.
Despite the evidence given here of the university’s mishandling of the TST program reviews, it has been impossible to get an independent investigation of the university’s actions. Without public scrutiny of university administrative practice, such flawed processes are, unfortunately, likely to continue. It might be reasonable to expect the possibility of such examination from the Quality Council of the Council of Ontario Universities, to whom Ontario universities are responsible for proper execution of their quality assurance reviews and adherence to approved program review protocols. I was surprised, however, to learn from the director of the Quality Council that it “does not intervene in the cyclical program reviews that are undertaken by universities” (Letter, August 16, 2012) no matter the evidence of failures to comply with the cyclical program review protocol.
In reality, once an Ontario university’s Institutional Quality Assurance Process (IQAP) has been approved, it can be administered as the university sees fit (at least until it undergoes an audit process by the COU once every eight years; Toronto’s next audit is slated for 2016-17). From my perspective, this is at odds with the COU’s stated goals of being “publicly accountable” in its task of ensuring “rigorous quality assurance” of the academic programs of Ontario’s public universities. The hands-off stance of the director of the Quality Council was further confirmed in letters to me from the President and CEO of the Council of Ontario Universities (March 25, 2013), the Chair of its Executive Committee (April 18, 2013), and the Chair of its Appraisal and Audit Committee (July 2, 2013). This position may well be due to the nature of the COU as an organization that comprises senior administrators from universities across the province. Regardless of the reasons for their unwillingness to investigate or intervene, it is clear that modifications need to be made to the quality assurance processes for Ontario university programs. As the University of Toronto Faculty Association put the matter to the provost of the University, it is important to see that measures are taken “to ensure that the authority of the Provost’s Office is exercised in a sufficiently collegial and accountable fashion.”
The acknowledged unwillingness of the Council of Ontario Universities to investigate possible misconduct by universities in carrying out quality assurance reviews, and the fact that there are at present few, if any, credible measures in place to ensure accountability in the exercise of the authority of the Provost’s Office in overseeing quality assurance reviews, raises serious questions about the efficacy of the COU’s entire quality assurance framework. AM
Donald Wiebe is a Professor in the Faculty of Divinity, Trinity College in the University of Toronto.
