During my career the number of scientists in biomedical research has grown dramatically, as has the number of journals publishing scientific papers. In parallel, the size, complexity and interdisciplinarity of each paper has increased. This has made it more challenging to be sure if the research is of high quality (even for experts in the field). When postdocs and students tell me that they see the top next-career-stage academic jobs going to the candidates with papers in Nature, Science and Cell, they are right. These journals are the markers of quality that search panels can readily understand because the work has been peer-reviewed and bench-marked against the best work in the field.
The Declaration on Research Assessment (DORA) and internet-based open access publishing has not changed the perception that the title and impact factor of the journal that we publish in is a surrogate for research quality and for the quality of the researcher. This is a human response to the sheer volume of information and the need to assimilate facts and concepts into a coherent view. Inevitably, we specialise our academic interests with the unintended consequence of limiting our exposure to new ideas/concepts and narrowing our world view. This makes it difficult to make sound judgements of individuals, papers or groups of scientists. Whether we find something interesting or exciting is a matter of personal taste and may bear little relation to the novelty and importance of the science (which may yet not be widely appreciated). Although journal impact factors, publication metrics and altmetrics are a guide to what might be the best science, they can be inflated in a way that has little relation to the underlying quality. And, just because something is popular doesn’t make it true. But there are some additional objective criteria that are relevant to evaluation to which we should pay more attention. They relate to honesty, competency and reliability. These characteristics of individuals are as important as impact factor and the title of the journal they just published in.
Reproducibility should be an unambiguous criterion for the quality of research and it has been argued – https://www.nature.com/news/faculty-promotion-must-assess-reproducibility-1.22596 – that it be made an important consideration for appointment or promotion at faculty level.
Of course, the timescale for assessing reproducibility is necessarily a slow one and honest mistakes do happen. Yet, there may be some actions that can help ensure reproducibility. It has been the practice in my lab for some time for experiments to be repeated, where sensible and feasible, by different individuals in the lab prior to publication. Far from undermining trust between individuals, this has brought people closer together and sometimes improves experimental design and interpretation. The sharing of know-how and research tools with other labs, even if they are feared as competitors, is another means to ensure reproducibility. If properly managed this should be a win–win situation.
A further objective criterion I want to emphasise is creativity. This is another highly valued and arguably less subjective scientific skill which can be developed throughout a career. It is the foundation of originality. At the Babraham Institute mechanisms are being developed to allow non-group-leader scientists to develop projects that they own and, if they move, can take away with them. This requires consistent communication and competent management. It is challenging, but its importance is that it emphasises the culture of creativity that we want to encourage and nurture. Indeed, time out from core projects should be encouraged for scientists at every level to enable diversification and head-space for creativity.
A third criterion is the courage to do things differently (and better) than the ways of our elders. Change of this kind is a sure sign of progress. Where we submit our manuscripts, how we evaluate them and how we judge and celebrate success of individuals and teams can change and is changing. So too is the research environment that we recognise as being most nurturing for the best science. A recent survey by Wellcome, a major science funder – https://wellcome.ac.uk/reports/what-researchers-think-about-research-culture –makes clear what the concerns of the next generation of scientists are. They are asking for a kinder and more supportive environment. Who wouldn’t want that? A brave response to this is required at all levels of the community. Inappropriate or malign behaviour is often endured for far too long by those on the receiving end, who may feel great reluctance to come forward, leading to the inevitable destruction of relationships. Senior colleagues are often too busy, blind or even scared to get involved and there is a general feeling that there is inadequate protection for whistle-blowers. Active mentoring, training and ongoing support is necessary to reduce these problems. Scientists at all levels must be communicating and properly engaged with each other if the culture to which they belong is to remain healthy and produce the best science.
Top image of post: by Gordon Johnson from Pixabay