One of the special sessions at the 48th FEBS Congress (Milano, Italy, 29 June– 3 July 2024) was titled “Reforming research assessment to support the improvement of research quality”. The session was premised on the growing consensus that current research assessment systems need to change to enhance the quality and impact of research. It introduced for discussion initiatives, actions, and points of view of different stakeholders to explore how to bring about this reform.

The session was chaired by Menico Rizzi, from University of Piemonte Orientale, Italy, who also gave an overview of the early steps of the Coalition for Advancing Research Assessment (CoARA), a coalition of organisations that was established on the basis of the Agreement on Reforming Research Assessment (ARRA).

In addition, the session had two speakers: Alessandra Ferrari, from the European Research Council (ERC) Executive Agency, who talked about the rationale of recent changes to the evaluation of research proposals at the ERC, and Daniela Ruffell, the FEBS Press Publisher, who spoke about responsible publishing and how researchers’ choice of journal has an impact on both science and on their reputation. The session ended with a lively discussion that reflected the high level of interest that these topics generated in the audience.

On this post we share background information on CoARA and highlight several of its activities, to help continue the movement towards responsible reform of the research assessment system.

CoARA: a global initiative for a systemic change of research assessment

There is a broad consensus among research communities worldwide that the existing tools of academic rewards and recognition criteria, such as h-indexes or the weight of publisher prestige, in particular if determined on the basic of indicators such as the journal impact factor, have ceased to accurately reflect what we most value in, and need from, research. A wide range of innovative, born-digital scholarship such as databases, visualisations, software development, or contributions to research infrastructures, are still invisible from formal research administration and assessment. Besides, beyond focussing solely on the end products of research, in the Open Science paradigm it is also clear that it is the integrity and transparency of research processes that, at the end, lead to truly innovative, open and high-quality research; therefore it is essential that a qualitative-centered approach and not a quantitative one, must characterize research assessment activities.

Building on progress made so far (DORA, Leiden Manifesto, Hong Kong Principles), over 700 research organisations, funders, assessment authorities, professional societies, and their associations have agreed on a common direction and principles for reforming the assessment of research, researchers and research organisations, outlined in the Agreement on Reforming Research Assessment (ARRA). They commit to a common vision, which is that the assessment of research, researchers and research organisations recognises the diverse outputs, practices and activities that maximise the quality and impact of research. This requires basing assessment primarily on qualitative judgement, for which peer-review is central, supported by responsible use of quantitative indicators. They also pledge to disclose their progression in evaluating or constructing criteria, tools and procedures, aligned with the core commitments and following an action plan with milestones defined by the community.

Additionally, they can also join the Coalition of Reforming Research Assessment (COARA), a global coalition that offers a platform for member organisations for collaboration and mutual learning. It was founded in December 2022 and currently supports 13 Working Groups and 16 National Chapters to facilitate exchange and develop resources that member organisations can rely on for their reform journeys. Working groups, jointly formed by several Institutions who joined CoARA, cover a variety of topics of relevance for research assessment, e.g., "Ethics and Research Integrity Policy in Responsible Research Assessment for Data and Artificial Intelligence", "Towards an Inclusive Evaluation of Research", "Supporting the alignment of research assessment systems with CoARA in biomedical disciplines through administrative reforms and governance", "Early-and-mid-Career Researchers (EMCRs) – Assessment and Research Culture", "Experiments in Assessment – Idea generation, co-creation, and piloting".

The COARA initiative started in Europe and is growing as it must for an equitable global system of research. Keeping the essentially global, transnational nature of research in mind, the mobility of researchers and ideas, enabling a systemic change is impossible without the involvement of research and research-related institutions in the broadest possible scope. Therefore, widening the Coalition’s membership in Europe and beyond, developing equitable policies and practices that benefit all is a strategic priority for CoARA.

CoARA’s activities

The invitation to sign ARRA and join the coalition is open to research funding organisations, research performing organisations, national/regional assessment authorities and agencies, as well as associations of the above organisations, learned societies and other relevant organisations.

Instituions who are CoARA members can join the Working Groups, as described above, and get involved in current and new activities that way. As an example, the Experiments in Assessment Working Group is currently collecting from the research community examples of experiments in research assessment – new ways to measure and recognize quality and excellence, no matter how crazy or unconventional. The results will be compiled in a catalogue and shared.

In addition, the European Commission launched a call for research proposals supporting the reform of research assessment as defined by the CoARA principles and commitments. The selected proposal, CoARA-Boost, offers funding through its cascade funding program – whose first call closed in June 2024 – to enhance institutional change and target pilot and exchange-of-knowledge initiatives (information on future cascade funding calls will be shared on CoARA’s website and  social media channels).

CoARA organises a range of workshops, webinars, and exchange forums to help organisations and researchers move towards a reformed research assessment system. Several of them have been recorded and shared on CoARA’s YouTube channel, and can provide additional background to those interested in joining the coalition. Furthermore, institutions joining CoARA are committed, within one year, to elaborate their own specific action plan toward reforming research assessment, which are publicly accessible in the Zenodo platform. You can also sign up to the CoARA Membership Newsletter, as well as follow the individual Working Groups, or contact the organization directly. CoARA is also present on social media:

• X (Twitter): @CoARAssessment
• LinkedIn: Coalition for Advancing Research Assessment
• Mastodon: @CoARAssessment
• YouTube: @CoARAssessment

Top image of the special sesison at the 48th FEBS Congress "Reforming research assessment to support the improvement of research quality", with Daniela Ruffell (speaking), Menico Rizzi, and Alessandra Ferrari.