The funding periods of four SRC projects ended in August 2019, and the first ever strategic research programme evaluation commenced at the beginning of 2020. The goal of the evaluation is to understand the multifaceted social impact of strategic research and to develop the funding instrument itself.

This blog post continues the theme of Jyrki Hakapää’s 18 October 2019 post Understanding the impact of strategic research.

The Strategic Research Council (SRC) funds programme-based research that has societal impact and is of high scientific quality. SRC programmes consist of two or more multidisciplinary projects that each have their own impact goals based on research and interaction activities. Unlike many other funding bodies, the SRC has a statutory responsibility to monitor the projects it funds and assess their impact. Additionally, the SRC has decided that impact assessments shall also be carried out on the programme level.

In the context of strategic research, the term ‘impact’ often denotes social impact specifically. When assessing social impact, it is accepted that it may take a long period of time before said impact becomes apparent. As such, assessing social impact is not a simple task. The complex nature of impact requires the utilisation of many different data forms in assessment, which puts a particular emphasis on a qualitative understanding.

Due to the time lag in the manifestation of social impact, the assessment of impact is approached in the IOOI (Input-Output-Outcome-Impact) impact chain framework in programme evaluations. Social impact is therefore not only examined from the perspective of the ultimate impact, but attention is also paid to the entire impact chain. The inputs (e.g. time, money, ideas) of research and interaction activities produce outputs (e.g. publications, events) which, in turn, produce short-term outcomes (e.g. new operating methods). When outcomes become permanent, they become social impacts. When examining impact chains, it is also important to understand their network-like nature. The pursuit of social impact in strategic research is based, first and foremost, on interaction and co-creation with actors in stakeholder networks.

Strategic research programmes are broad entities, and it is important to take the special characteristic of the programmes’ constituent projects into account during evaluation. Though evaluation takes place at the programme level, it is still mostly based on the research and interaction activities of individual projects. Programmes can therefore be understood as multi-level operations. In programme evaluation, attention is paid to the cooperation between projects within the programme as well as project and programme-level goals. Additionally, at the programme level, the uniting and coordinating role of the programme coordinator is significant. The interactive cooperation taking place in stakeholder networks is vital at every level, and therefore, in programme evaluation, particular attention is paid to the structure and activities of the network. From the perspective of a programme as a whole, it is difficult or even impossible to ultimately separate what is internal activity and what is, for example, project-specific interaction. As such, a comprehensive understanding of programme activities that takes into account the special characteristics and operating methods of different projects is emphasised in the evaluation of SRC programmes.

Concluded programmes in evaluation

The funding periods of four strategic research projects ended in August 2019: Urbanising Society (URBAN); Health, Welfare and Lifestyles (HEALTH); Skilled Employees – Successful Labour Market (WORK) and Security in a Networked World (SECURITY). The programmes contained a comprehensive and multidisciplinary set of 13 projects that pursued social impact.

The projects and programme coordinator of each programme each produced a final report concerning their work during the funding period, and their reports were delivered to the Academy of Finland in late 2019. The established progress reporting instruments, impact reports and output indicators for strategic research also form the framework of the final reports. Through the impact reports, both projects and programme coordinators examine their own activities from the perspective of the research and interaction work done to achieve influence, as well as from the perspective of the impact of methods and achieved, intended and unintended impacts. The output indicators, in turn, contain quantitative information about subjects such as concluded events, publications, policy briefs and, for example, international visits. Additionally, information about the most important interaction partners was requested in the final reports from projects and programme coordinators for the purposes of mapping partner networks. The approval of final reports from concluded projects and programme coordinators is on the SRC agenda for May 2020.

Programme evaluation consists of three stages: self-assessment, assessment of social impact and evaluation of scientific activities. The final reports described above act as supporting material in all stages of programme evaluation. The self-assessment stage has already been completed. In this stage, project representatives and programme coordinators discussed their views on programme activities during the funding period on a programme-specific basis in a focus group interview facilitated by the Academy of Finland. The purpose of the events was to offer an opportunity to reflect on experiences relevant to the projects’ activities together with other parties who took part in the programme. Attention in the self-assessments is specifically on the programme’s research and interaction activities and their social impact according to the perspectives and interpretations of those taking part in the discussions. Records of the self-assessment events will be published as part of the report on the evaluation of social impact.

The next stage of programme evaluation is the assessment of social impact, which will be carried out by a party that is independent of the funding body and not involved in the programme activities. The selection of the assessing party was done by a competitive selection process, which ended on 15 April 2020. The last stage of evaluation will focus on the evaluation of the scientific work and results of the SRC programmes that ended in 2019. The overall programme evaluation will be completed in 2021. For more information about programme evaluation and SRC funding, see the SRC funding principles.