Effective high-quality research funding is backed up by responsibly implemented evaluation of applications. The goal of application review is to enable the funding and operation of promising and scientifically ambitious projects. A crucial tool that the Academy of Finland has in the review of applications is high-quality international peer review, and Academy staff are committed to following and promoting the principles of responsible science in organising it.

The Academy of Finland takes questions of research ethics, equality, open science, sustainable development and responsible researcher evaluation into account in connection with responsible science. The goals of responsible science also form the preconditions for the most important tool of the Academy of Finland – reviewing applications based on scientific peer review. Responsible reviews of applications guarantee a foundation and preconditions for the high quality and impact of the research to be funded.

At the strategic level, the Academy promotes responsible science by committing to national and international declarations. These include, among other things, the Declaration on Research Assessment DORA.

The goals of responsible science are not detached from other policies that guide the activities of the Academy, and matters related to responsible evaluation of researchers have been part of the Academy's principles for funding for nearly ten years. At the Academy we are constantly developing our review practices to promote the goals of responsible science. The principles of responsible researcher evaluation, among other things, have been expanded as a part of our entire funding process.

International peer review panels are compiled mainly based on fields of research, or thematically

International peer reviews of applications are carried out at the Academy of Finland by panels of experts. The panels are mainly compiled based on the fields and the themes of the research. In the panels based on fields of research, Academy staff invite specialists who have been successful in their own fields to join the review work. For example, in thematic programme calls for strategic research, it is possible to invite both international experts and Finnish or foreign experts in science and societal impact to join the panels.

In compiling the panels, consideration is also given to the experts’ prior review experience. Officials at the Academy use different kinds of tools in the search of suitable experts. These also include Scopus and the Web of Science. In addition, the Academy makes use of the databases of its Administration Office to contact experts who participated in previous reviews.

When forming panels based on fields of research, we aim at making sure that the expertise of the panellists corresponds to the content of the applications as well as possible. Applications can also be moved to the panel of another research council for evaluation if the panel in question has expertise that is better suited for the content of the application. With applications for which no scientific panel of experts can be found, separate statements by two individual experts may be requested. In addition, a statement of support by an individual expert may be requested later, if it is revealed at a meeting of the panel that the panel's expertise is insufficient to review the application. Thematic programme calls, meanwhile, deviate from panels based on fields of research in that the panels are compiled based on the goals of the programme call, for example.

Panels set up to be all-inclusive

Both the research-based and the thematic panels of the Academy of Finland are always multidisciplinary. According to a study by the Administration Office’s internal TIMO core team, based on subcategories of the research fields, about 80 per cent of the applications submitted to the Academy are of a multidisciplinary character. Consequently, the precise knowledge of an individual expert concerning a specific area of research is not at the core of panel activities. Instead, the focus of the activity is the comprehensive peer review conducted by the panel of experts. There is also turnover in the review panels and their composition, which means that the panels often have different members in the different application rounds. Changes in the composition of the panels also reflect changes in the content of the topics of research and in what branches of science are emphasised.

International review panels are built on the expertise of the members of the panels. However, expertise is not the only principle that guides the Academy in the formation of a panel. The Academy of Finland is committed to the principles of responsible science in the funding of science, and these themes are also reflected in the how the panel work is prepared.

Expert panels with extensive compositions enable the review of different kinds of applications. Both applications based on fields of research and thematic applications are extensive in content and include actors of many backgrounds. This diversity requirement is also addressed by the principles of responsible science.

One consideration in compiling the panels is to have experts of different genders and nationalities, for example. In addition, factors affecting the selection of experts include different points of emphasis for different types of funding. For example, in the review process for the September call, the selection is also affected by what phases of the research careers of the reviewers are in. Recruiting members for a panel often begins from women, as young female experts are internationally in very high demand among funders from different countries. That is why recruiting them for review panels can be challenging. In the September 2019 application round, 31 per cent of the experts in the review panels were women and 69 per cent were men. The gender balance among the experts was slightly better in the 2020 round: 36 per cent of the experts were women, while 64 per cent were men. The panels for the 2020 application round worked remotely.

Remote panels offer new opportunities for convening panels

Our goal is to get an extensive representation in the review panels from different countries. In the September 2019 application round, 44 per cent of the background organisations were from Western Europe, 18 per cent were from the Nordic countries, 17 per cent were from Central Europe and 10 per cent were from Southern Europe. About 6 per cent of the experts were from organisations with North American backgrounds. The statistics show that Western scientific traditions have a powerful presence in the Academy’s review panels.

Practices related to travel and time differences set constraints on the recruitment of experts. However, it is worth noting that the statistics reflect the time before the Covid-19 pandemic and the switch to remote panels. The solutions offered using remote panels open new possibilities for the recruitment of experts from outside the regular areas such as Europe.

The review of the applications will be discussed in greater detail in the next blog article on the Academy's review and decision-making process.