ORCID iDThis post is part of an ongoing series where I intend to develop my full personal (not institutional) response to the HE Green Paper. Comments are welcome to refine this.
The Green Paper asks (as question 2):
“How can information from the TEF be used to better inform student and employer decision making? Please quantify these benefits as far as you can.”
The challenge in answering this question is twofold, pertaining to information currently available and also to consumer paradigms of HE.
Firstly, the Green Paper does not specify what TEF will measure. Instead the Green Paper says that: “A technical consultation will be run in 2016 which will cover the operational detail of metrics and of the assessment criteria, process and outcomes, as well as looking at the evidence to be submitted alongside applications and how it will be used for provider level assessment”. Since the modes in which this information can be disseminated will be dependent upon what information it is, it is hard to recommend how this information should be used to inform student and employer decision making. This question should be re-posed after the technical consultation, should it go ahead.
Secondly, existing efforts to alter student and employer decision making have utterly failed because the brand power of some institutions is so strong that it does not matter what they actually offer. Instead, students know that employers will use the institutional brand name as a proxy for quality. League tables, run by external entities on an international stage, also provide strong brand measures that TEF will struggle to influence. In order to better inform student and employer decision making, TEF information would have to be viewed, by those parties, as more significant and a better indication of return than either international league tables or individual institutional brand power. This will be difficult to achieve.
That said, the other aspect here that should be considered is how a multi-dimensional dataset will be used. The Paper acknowledges that “excellence is the sum of many factors”. Again, it is hard to say what this will look like because the Green Paper treats the actual metrics as something yet to be decided. But, for instance, it is clear that multi-dimensional data can be used as marketing spin by institutions across different axes, such as in REF. Comparability (of which I am sceptical in any case) though, depends on any system of metrics provided like-for-like data.
The problem of which the Green Paper currently complains is that existing mechanisms do not “allow reliable comparisons to be made on teaching quality”. But this is the role of inter-institutional validation, which ensures that grading and classifications are set at a comparable standard. This appears to be contradictory, then. With respect to new providers, Jo Johnson recently said in a speech that “the requirement for new providers to seek out a suitable validating body from amongst the pool of incumbents is quite frankly anti-competitive”. Yet it will be very hard to achieve the comparability that is required without some form of inter-institutional validation. Furthermore, the very epistemology of HE requires such validation. If HE is to be “higher” then it must be about the teaching of new knowledge (which requires a research link). Because this is not a standardised curriculum (and cannot be), the function of peer review in validation is vital. The system of which the Minister complains is peer review in teaching, the recognised international standard. Would the same accusations be levelled against peer review in research? To better inform student and employer decision making, TEF information would have to produce comparable data across institutions. This will require a system of inter-institutional validation and audit to be in place. This should not be dismantled as it forms the touchstone of the inter-institutional comparability that is sought.
This aside, though, my final point is that UK HE exists within a global context. It is clear that other countries are not following the same recommendations here. By measuring UK institutions by measures that are not globally adopted, such as those in TEF, the output will produce a non-comparable dataset to international institutions. This will damage the competitiveness of UK HE by adding to student confusion over different measures, particularly for foreign students. If TEF information is to meaningfully inform student and employer decision making then it must be viewed as an internationally accepted and valid measure. The timescales on which TEF is being introduced make it unlikely that this will be the case as no international consultation has taken place.
In summary: while it may be possible to present TEF information in a clear way to students and employers, it is likely to be a highly reductive quantification of complex social processes that are not fundamentally reducible to such numbers. Spreading this across a multi-dimensional dataset will make the narratives that emerge from such quantification subject to gaming and spin. Insufficient information has been given at this stage about the measures that will be used, and so this question is hard to answer. The arguments about the need for comparability sit in contrast to the Minister’s rejection of recognised best practices of peer review. Only inter-institutional validation will provide this comparability in an HE context. International HE league tables, sometimes predicated on research, will continue to act as more powerful brand signals than any TEF result, meaning that TEF will be less likely to influence student and employer behaviour. TEF is not sufficiently aware of the global challenge for the competitiveness of UK HE and seeks to impose a framework that will not be internationally recognised, thereby making UK insitutions both behave and report in ways that will be unfamiliar to those outside of a UK context.