Take a look at any journal, conference, internal seminar, or poster session: results! Beautiful shiny results: new players identified, mechanisms elucidated, interactions uncovered, “to the best of our knowledge, this phenomenon has not been observed before”. Then spend a day on the bench, sit at a lab meeting, chat to PhD students at the poster session, and the picture painted is different: the talk will usually be of failed experiments. But what do they mean by 'failed'? The technique was sound, the controls proper, the execution careful, the p-value acceptable. And yet, the results did not point where they 'should' point to. Eventually these results end up deep in some drawer, never to be seen again.

Many of us have a sense that the positive results adorning our journals represent only a fraction of the community’s work, but what is really the magnitude of the phenomenon? A 2012 study examined 4000 papers published between 1990 and 2007, and found that the literature has indeed taken a turn towards positivity. In 2007 more than 85% of published studies that have tested a hypothesis found positive support for it, a 22% increase from 1990 (Fanelli). 

The reasons behind this trend span research culture, competition, limited funding, short-term contracts and more, but the consequences are more straightforward: a shift towards preferably publishing positive results can (and probably already does) exacerbate the questions around reproducibility in science – with scientists often feeling pressured to present their results under the best possible light. On a different, but very practical note, reporting a hypothesis that the results did not support, or an otherwise successful method that was found to be inapplicable in a certain system, will save much time, funds and frustration. Speaking of frustration, recognising the value of negative results can alleviate some of the pressures on early-career researchers, who are often struggling with imposter syndrome, exacerbated by the constant stream of publications highlighting success after success. Finally, we should be cautious that the focus on publishing positive results might discourage innovation and risk-taking, and favor straightforward, more predictable projects instead. Tackling publication bias can signal a positive culture shift in academia.

A channel for negative or unclear results

So what can you do with your failed hypotheses? Start by rephrasing that as “hypothesis unsupported after my best and most earnest effort”. Then look around for options. While some journals consider negative results, publishing in a journal can still require substantial effort to prepare the paper in a way that fits the journal expectations, or carry a financial burden.

There’s clearly still an unmet need for a channel for those negative results, and preprint servers might well be the perfect solution. Preprints are flexible in format and absolutely free to publish and access, making them a relatively quick, easy and painless way to ensure that your negative results are out there for the community to reap those benefits. The preprint server bioRxiv indeed already categorizes its preprints into “New results,” “Confirmatory results,” and “Contradictory results”, highlighting how different types of results contribute to scientific progress. Furthermore, the nature of preprints invites discussion, which can provide further context for that finding and how to potentially use a negative result as a stepping stone for a future discovery.

At ASAPbio and within the ASAPbio Fellows program, we advocate for open and innovative avenues to communicate all research outputs. At one of the working groups of the 2022 ASAPbio Fellows cohort, we got thinking about negative results and their importance. The members of our group came from diverse research cultures, scientific backgrounds and career stages, and yet we all agreed on the value of negative results and that preprints could be the way forward to showcasing it.

Helpful competition

With the goal of highlighting how preprints offer a solution to share different types of results, we have organized a competition that calls for the submission of preprints that report a negative/null or inconclusive result as the main finding. The competition leverages the flexibility in format that preprints afford: it is open to results that did not validate a study hypothesis, results that did not replicate a previous published paper, or inconclusive results that highlight uncertainty about a particular method or line of research. Preprints posted any time from 1 January 2022 in any field of experimental or computational biology – including studies involving cell, plant, or animal models, or bioinformatics or translational approaches – are eligible for the competition. Manuscripts that have been subsequently published at a journal can also be submitted, provided that the paper was first shared as a preprint. The winners will receive a $1,000 award and will be highlighted via the ASAPbio blog.

A panel of judges that includes ASAPbio Fellows, ASAPbio community members and authors of preprints reporting negative results will assess the entries to the competition to select up to three winners. The judges will evaluate submissions according to the perceived value of the negative result to the field, the clarity in presentation and the alignment of the preprint to open science practices (e.g., regarding availability of the underlying data, code and/or other resources). 

The reason it is important to publish negative experiments is because if you don't, we are all doomed to repeat them.

Nandakumar Narayanan
Associate Professor at the University of Iowa and one of the judges of the competition.

Submit your paper to the ASAPbio competition

Do you have any negative results that you have been meaning to write up? Any inconclusive data that you wondered whether you should share? Then write up your results, post them as a preprint and submit to our competition by 14 January 2023.

Negative/null results are invaluable to build rigorous scientific evidence. We must support greater recognition for the importance of this type of research results. Be the change you want to see: share your negative results via preprints and be part of the shift in research culture around what we view as failure and success in science.

Full details about the competition at asapbio.org/competition2022. 

This post was jointly written by Anastasia Moraiti (ASAPbio Fellow and PhD student, Francis Crick Institute) & Iratxe Puebla (Director of Strategic Initiatives & Community for ASAPbio).

Top image by Matt Ridley on Unsplash