As the summer came to an end, so did the first five years of our strategic research and our first strategic research programmes. The first evaluation of our strategic research programmes is scheduled for next year. Our aim is to understand the social impact of our strategic research programmes and to improve the funding instrument.

A new approach to impact assessment

When it was first founded, the Academy of Finland’s Strategic Research Council (SRC) was legally mandated to both monitor and evaluate the research projects that it finances (Finlex, in Finnish). This was a completely new kind of mandate and therefore called for a new approach to financing research. There were no previous models that could be easily copied either, as the SRC’s mission to find solutions to the major challenges facing society and promote interaction between the scientific community and decision-makers is quite unique in the context of competitive research funding.

Both Finland and other countries have naturally been investing in the social impact of research, its promotion and the evaluation of results for a long time. The SRC and its secretariat at the Academy of Finland have always been lucky, as the funding instrument has attracted researchers who are not only interested in influencing society with their research but also in studying the social impact of their work. Some project teams have even formulated clear and versatile procedures for monitoring how well they are meeting their targets. Decision-makers and government officials therefore have a pool of enthusiastic partners whom to engage in constructive discussions.

Nevertheless, evaluating strategic research projects and programmes requires a unique approach. The social impact of research has been explored and pursued previously as well, but it has rarely been a key criterion for granting research funding. Even fewer are the examples of impact assessments focusing on specific pre-determined social challenges. The overarching objective of the funding instrument – to create new interdisciplinary systems and social networks – also has an unusually high degree of influence over research goals.

And not just in terms of researchers’ work. In fact, the first question to answer should be how well the SRC has met the objective set in the Finnish Government’s decision on strategic research themes in terms of devising submission schemes and granting funding to projects that fit in with the funding instrument and securing enough funding for the programme. The ultimate question, on the other hand, comes down to the opportunities of researchers and research teams to work together to actually convince decision-makers, operators and, where necessary, the general public of the results of their work – and to whether the projects have created solutions to social challenges as stipulated in the programme criteria. The evaluation process itself also has its idiosyncrasies, as this is not an academic study but an assessment carried out by a public research funding agency. It has been highly encouraging to see so many researchers getting excited about studying the potential impact and implementation of not only strategic research but science in general.

What constitutes social impact in the context of research?

Studying the social impact of research is about unravelling a complex tangle, as it is not always possible to demonstrate or prove results in this respect. Research can have an impact on society years after a specific research project comes to an end. Research into a specific issue and its impact can easily be lost among a multitude of different projects and initiatives.

Earlier approaches to impact assessment have focused on achievements that appear to be easy to verify and effects that materialise quickly. However, strategic research as a concept is extremely complex, and so are the actions and achievements that are needed to enable and bring about tangible impacts on society. It is impossible to see the big picture by simply looking at performance indicators or case studies. Instead, the focus should be on the effectiveness of interaction in practice, understanding its impacts, reflecting on potential future consequences and also on the ability to analyse failures and goals that have proven impossible.

The SRC has set parameters for impact assessment in its funding principles. The SRC’s focus is on the performance of each programme as a whole. The aim is to evaluate the social impact or future social impact of the research and interactive initiatives financed by the SRC and to make the strategic research funding instrument more effective.

Focusing on each programme as a whole forces the SRC to remember that each strategic research programme is different in terms of both its theme and the methods of funding and implementation. The operation and outcomes of each programme must be approached with the framework of that specific programme in mind rather than comparing the successes of different programmes against each other. The probabilities of finding answers to different challenges vary, as do the roles that different areas of science play in society.

The SRC is committed to evaluating the impact of its work openly and responsibly. The Academy of Finland’s promise last summer to take the San Francisco Declaration on Research Assessment into account in its evaluations is a good starting point for assessing the impact of strategic research programmes. For example, the interdisciplinary nature of strategic research and its focus on finding new combinations of expertise mean that any assessments of research quality cannot be based on the views of individual branches of science on the best ways to publish results.

Three elements of impact assesment

The programme evaluation (pdf, in Finnish) will consist of three phases: the first phase is based on self-assessment and requires the project teams that have participated in the programme to reflect on the success of their projects and the programme as a whole, any challenges encountered in respect of programming and how well the description of the programme matched the reality. Being able to learn about the operators’ perspective in their own words will be crucial for external evaluators to understand the motives and actions of the project teams.

The external evaluators can then focus on analysing and assessing the programme’s social impact and potential. This phase of the evaluation looks into the role of partners in research projects and the ways in which the projects’ findings help them to understand and interact with the world: how the partners perceived the interactive elements, how involved they were in the projects in practice, and what the research findings ultimately allowed them to achieve – or could potentially allow them to achieve in the future – in terms of solving the social challenges that they are facing. The evaluation should not focus just on individual decision-makers and institutions but also on the promotion of public debate as a means to influence society.

The final phase of the evaluation consists of an external scientific review of the outcomes of the research projects included in the programme. This phase focuses, above all, on links between the researchers’ work and results and their interaction with society and the social impact of the projects. The characteristics, successes and challenges of interdisciplinary research are also examined at this point. We need high-quality research to solve the challenges facing society, and we also need information about how efficient this funding instrument is in promoting such research.

The evaluation is set to begin once the final reports of the projects funded (pdf, in Finnish) through the programme have been published. The findings will accumulate in the aforementioned order. Evaluation reports on the four strategic research programmes that ended in August of 2019 are expected to be ready towards the end of 2020.

The project teams have begun to publish synopses and summaries of their findings and achievements in recent months. The first results were featured in the SRC’s August newsletter, and there are plans to pool and showcase the teams’ achievements more extensively in the near future. From the perspective of the social impact of the funding instrument and research in general, more comprehensive and longer-term evaluation provides the best possible way to understand and interpret the role and achievements of strategic research.

Original text in Finnish by Jyrki Hakapää