Source: https://www.universityworldnews.com/post.php?story=20230519155707244

GLOBAL

You have been warned: A breakthrough in rankings is coming

Waldemar Siwinski  20 May 2023

In a tweet sent from Tashkent, Uzbekistan, after the closing of the IREG Observatory on Academic Ranking and Excellence conference I wrote: “IREG 2023 clearly articulated that in the coming years nothing in the world of rankings will be the same. The main word of the conference was ‘breakthrough’. We discussed where and when it will happen, and we looked for its first signs…”

Since university rankings have been a hot issue in higher education debates for a long time and several people have asked me about this future breakthrough, I will try to explain what I mean.

A view from Central Asia

Far away from the traditional ranking conferences, Tashkent turned out to be an excellent site for the IREG 2023 Conference. Once, its cities of Samarkand and Bukhara were a key link on the Silk Road and now Uzbekistan is a fast-developing country that has made education and science the basis of its modernisation efforts.

The annual IREG conferences are the world’s only neutral place where rankers, higher education experts and analysts as well as universities – often represented by rectors – meet.

The issues discussed there often have a direct impact on rankings and their standards.

Judging by the feedback, IREG 2023 was a creative and refreshing event. Here are three characteristic comments:

• Laylo Shokhabiddinova, head specialist of the international rankings department at Tashkent State Transport University, said: “From around the world, we have come to share our experiences, ideas and insights on ranking in higher education. It has been an eye-opening experience for me, and I am grateful for the chance to connect and collaborate with such an esteemed group of professionals. As universities continue to navigate a rapidly changing landscape, it is more important than ever to stay connected and learn from each other.”

• Alex Usher, president of Higher Education Strategy Associates, Canada, stated: “One of the most interesting ranking discussions I’ve heard in years. The difference in discourse around rankings after leaving a rich country is huge – here there is much less about marketing and much more about system control or benchmarking.”

• Komiljon Karimov, first deputy minister of higher education, science and innovation in Uzbekistan, commented: “I often participate in various conferences and seminars, but I have not met such a substantive and engaged discussion for many years.”

The conference in Tashkent showed how pragmatic and hopeful expectations about rankings are among universities and governments of countries outside Europe and North America.

They need the rankings as a tool to monitor implementation of reforms in higher education, to improve the quality of education and not for the sake of prestige. But this aspect has already been analysed by Usher in his highly recommended blog under the title “Rankings Discourses: West, East and South”.

Historical background

So, what new trends and ideas emerged in Tashkent about the global rankings landscape? To properly interpret new trends, we need to go back to the turn of the century and the beginning of the era of massification and globalisation of higher education. The UNESCO World Conference of 1998 was not able to properly describe this phenomenon. There was simply no comparable data available. It’s hard to believe, but the situation has not changed much since.

Concern about this state of affairs was sounded by Philip Altbach 10 years ago in a University World News article, “Long-term thinking needed in higher education”, in which he regretted that none of the global or regional organisations with the necessary potential and prestige (the UN, UNESCO and the OECD) had taken the necessary action on data collection.

He even went so far as to say that they had “abdicated” responsibility from the roles they should perform. He wrote: “There is a desperate need for ongoing international debate, discussion and regular data collection on higher education. At present, we have only a fragmented picture at best.”

Similarly unsuccessful in setting up an updated database was the European Union. Projects financed by the EU, such as the European Tertiary Education Register (ETER) had been undertaken but never fully implemented. This is why rankings, first national, then international, have been a discovery, introducing ‘countability’ to higher education.

The first conference of an international group of ranking experts (from which IREG was born) was in Warsaw in June 2002. There were no international rankings yet, but national rankings (including US News Best Colleges in the USA, Maclean’s in Canada, CHE in Germany and Perspektywy in Poland) were such a fascinating phenomenon that Jan Sadlak from UNESCO-CEPES and Andrzej Kozminski, rector of Kozminski University, invited a group of ranking creators to a debate in order to exchange information. I spoke there about Perspektywy’s first 10 years in ranking.

The next meeting, in Washington DC, in 2004, included Nian Cai Liu with his brand new Shanghai Ranking. At the 2006 meeting in Berlin, the IREG group adopted the famous “Berlin Principles on Ranking of Higher Education Institutions” which introduced quality standards for rapidly emerging new rankings.

Had UNESCO or any other international organisation been able to collect data globally and annually update reliable information on higher education, international rankings would probably never have gained such momentum and importance. But politicians, or rather bureaucrats, have failed, and not for the first time.

As a result, the short history of modern rankings (40 years of national rankings and 20 of international ones) has seen them gain an inflated role which they did not initially aspire to. It is a fascinating history of exploration, confusion and rivalry.

Don’t break the thermometer

We all see and feel that the world is changing and that it is changing in many ways. Geopolitical forces are shifting and technological advancements and artificial intelligence are both promising and scary. All this has a strong impact on higher education and, consequently, on the academic rankings. The rankings landscape is changing quickly too.

A new approach to rankings has emerged in the form of the impact rankings, reflecting university contributions to social goals. The range of regional and ‘by subject’ rankings have also grown.

At the same time, criticism of rankings has intensified, particularly from the European higher education and research community. There are new initiatives (the Coalition for Advancing Research Assessment to name one) which are searching for new conceptual approaches and tools for university assessment.

IREG Observatory welcomes the search for ever better tools to measure and evaluate excellence in research and higher education. However, we see no need to breed a ranking phobia in the process. Assessment and rankings serve different purposes. IREG expressed this very clearly in its position paper, “Assessment and rankings are different tools”, published last December.

Higher education around the world faces many difficult problems, but rankings are not the cause of them. Rankings are like a thermometer that signals various academic illnesses. But neither sickness nor fever disappears when we break the thermometer. The same applies to rankings.

At the countless ‘summits’ organised almost weekly by the major ranking players, the audience is told that the rankings and methodologies are approaching the pinnacle of human achievement, and that the answer to every university rector’s dreams is to purchase a ‘marketing package’, a kind of miracle medicine.

It is not for nothing that some now consider the big ranking organisations to be little more than ‘prestige-selling companies’ rather than just a rankings provider, with rankings serving merely as the ‘cherry on the cake’.

Big data

There can be no meaningful analysis without good data, data that meets the agreed standard, that is properly collected, externally validated, updated at least once a year, if not more frequently, and is widely available though not necessarily free, because quality costs. The fees, however, should be reasonable.

Such data, however, cannot be obtained without the cooperation of national education authorities. In turn, they must collect such data for the efficient realisation of their public policy and economic strategy. The data collected directly from universities by ranking organisations and adjusted to their requirements have many imperfections that have been well identified.

This applies to QS, Times Higher Education and other organisations that use surveys sent to and collected from universities. Of course, when no other options are available, flawed data serves better than none at all. However, let me point out that better databases, anchored in national systems, have already appeared and, year by year, are increasingly available.

M’hamed el Aisati, vice-president at Elsevier, and an analyst endowed with an impressive “ranking intuition”, emphasised in his speech in Tashkent the growing need for big data platforms to evaluate the impact of research, its visibility, academic collaboration and innovation.

He then presented the first already tested and operational national research evaluation platforms which are being used in Japan and Egypt.

What is a national research evaluation platform? As Aisati explained: “It is a big data platform that provides a knowledge network to support high-value decisions. It is a knowledge network that provides an integrated view of relevant research data, ensuring that high-value client decisions are based on an inclusive, truthful and unbiased view of their country’s research ecosystem.”

Such a platform is modular. It collects dispersed data, including digitisation. Non-English language data is being translated into English. Linking and profiling of references as well as automatic classification and metrics are provided.

In less specialised language: the big data national research evaluation platforms cover the entire research output of Japan and Egypt, including their academic characteristics. Hearing this at the IREG 2023 Conference, Professor Piotr Stepnowski, rector of the University of Gdansk in Poland, said in an emotional outburst: “This will radically change evaluation of individual countries in world science!”

Yes, it will! In fact, it already signals the beginning of a ‘breakthrough’, the word that best describes the essence of the discussion in Tashkent.

The role of AI

The subject of big data has appeared in various ranking-related conversations for over a dozen years. It is obvious that the global ‘data ocean’ on science and higher education can create new analytical possibilities and, consequently, new solutions in the ranking area. By the way, ‘data ocean’ was one of the key phrases of the IREG 2019 Conference in Bologna.

But the excitement about this potentially revolutionary technology has been effectively dampened by the so-called ‘artificial intelligence winter’. It was impossible to meet the huge expectations presented by the first AI algorithms because two things were still missing: data that could be sent in bulk, and computing power of sufficient capacity.

Only in the last 10 years has computing power, as well as very fast, high-bandwidth communication, like the 5G systems, allowed artificial intelligence to be used effectively. The emergence of ChatGPT and the rapid spread of this application has brought millions of people closer to the practical implementation of AI.

At the same time, in many countries solid database systems covering higher education and science have been created (the author of this text was the leader of the team that prepared the design of such a system in Poland, currently functioning under the name POL-on).

So we now have both the data and the desired computational power. It would be naive not to expect that a platform based on giant higher education databases and AI algorithms would soon make an appearance in the university rankings field. The question is: ‘Who will be the first to come up with such a ranking platform or application?’.

Waldemar Siwinski is president of the IREG Observatory on Academic Ranking and Excellence, and founder of the Perspektywy Education Foundation, Poland.