Science is better off without rankings

The international position of Dutch science is under pressure. That is the concern that Hans Clevers expressed in the podcast Dr Kelder and Co about possible adverse effects of the ‘Recognize and Value’ programme. Clevers, a much-publicized stem cell biologist who recently switched to pharmaceutical company Roche, is not alone in his concerns. And that was reason for VVD MP Hatte van der Woude to discuss this parliamentary questions to the Minister of Education: is this program a threat to our international position?

Recognition and Value is a proposal by the Dutch universities and the subsidy providers NWO and ZonMW to recognize and value scientists more based on the nature and content of all their work. By not only looking at the research they do, but also at educational tasks, social impact and leadership. And by no longer staring blindly at the amount of work that is being done, but by focusing on quality, content and creativity.

Abolish

The concern of Clevers and others seems to be mainly about the proposal to discard ‘objective’ performance indicators when assessing grant applications. The researcher’s resume is shortened to a narrative description of his or her research qualifications and the top ten contributions to science. The rest of the oeuvre is not taken into account. How many articles you have published as a researcher and how many citations you have collected in your career are no longer allowed to be mentioned.

It is not clear to me why the abolition of such indicators will lead to research of lesser quality and thus, in the long run, to a deterioration of the international position. The reasoning assumes that the quality of the researcher can be inferred from the performance indicators and not from the top ten publications. But that is not obvious.

The performance indicators appear to be an objective measure of the impact of publications and the quality of the scientists. But they are not. They depend too much on factors that have nothing to do with impact and quality. Those who publish many review articles are often quoted, but they are not innovative. Anyone who collaborates a lot easily has more publications and citations, but has not always made a substantial contribution to the creation of the work. And someone who is approaching retirement easily has a higher citation score than someone who has just started. Performance indicators are too erratic and therefore unsuitable for assessing and comparing individual scientists.

Focus on the top

Moreover, the international position of Dutch science does not exist. There are international rankings of universities, but even those are not very objective. Each ranking chooses on the basis of which indicators it ranks universities. The World University Rankings, a list of more than 1,600 universities, calculates a rank score based on 13 performance indicators, including the number of publications and citations and the results of subjective reputation surveys among students and academics. The ShanghaiRankinga ranking of 1,800 universities, opts for a focus on the top of the top and calculates the score from, among other things, the number of Nobel prizes among the university’s alumni and the number of articles in Nature and science among its employees. In the World University Rankings, seven Dutch universities are in the top 100, in the ShanghaiRanking only three. How we do internationally depends on which list you look at.

Scientific performance indicators and rankings are notoriously ambiguous. They pretend to measure something that can hardly be measured and which they therefore do not measure. And they compare researchers and institutes that cannot be compared. We don’t want them and we don’t need them. Not even to motivate researchers.

International position

Van der Woude asks whether the minister shares the opinion “that it no longer makes sense for academics to build up a good CV, including international criteria, if what is on that CV may not or much less be taken into account when applying for a job?” I expect the minister to be happy to let these scientists go. He will want to invest in researchers who are driven by curiosity, not their resume. Researchers who want to get to the core of something, who want to develop and renew themselves, who want to grow and let others share in this. Researchers who get the motivation from within know that half work is not enough. Researchers who go for quality over quantity.

I expect him to argue that that quality is recognizable and visible, even on an abbreviated resume that lists only the top ten contributions to science. And that that quality will translate into better publications and more citations, even if we no longer count them in the assessment in the Netherlands. The international position of Dutch science does not depend on what we measure. It depends on the quality in which we invest. And Recognize and Appreciate gives space for that.

Cecile Janssens is professor of translational epidemiology at Emory University in Atlanta.

Leave a Reply