like many PubPeer is like an anonymous place on the Internet. There, under a randomly assigned taxonomic name, Actinopolyspora visculensis (bacteria) and Hoya camphorifolia (flowering plant), “Detectives” meticulously record errors in the scientific literature. They write about all sorts of errors, from clumsy statistics to pointless methodology, but their collective expertise lies in manipulated images. Protein clouds exhibiting suspiciously sharp edges, or identical arrangements of cells in two presumably different experiments. In some cases, these irregularities are simply an attempt by researchers to glorify a person before submitting to a journal. But they still raise red flags.
PubPeer’s tenuous forensic community has produced an unlikely celebrity. Elisabeth Bik uses her extraordinary acuity to spot image overlaps that are virtually invisible to other observers. Such duplication allows scientists to conjure results out of nowhere by frankensteining parts of many images together. It could also be argued that one image represents two separate experiments of hers that yielded similar results. But even Bik’s supernatural eyes have their limits. It’s possible to fake an experiment without actually using the same image of her twice. “If the two pictures overlap a little, I can nail you,” she says. Fighting fraud, or even studying it, can seem impossible when the most high-profile experts in fraud can’t always identify it.
Nevertheless, good scientific practice can effectively reduce the impact of fraud, or outright falsification, on science, whether it is discovered or not. Fraud cannot be eliminated from science any more than we can eliminate fraud,” says Marcel Van Assen, principal investigator at the Meta-Research Center at the Tilburg School of Social and Behavioral Sciences. But as researchers and advocates continue to push for science to be more open and impartial, he says, “there will be less fraud in the future.”
Alongside detectives like Bick, “meta-scientists” like Van Assen are the world’s fraud experts. These researchers systematically track the scientific literature to ensure it is as accurate and robust as possible. Metascience was founded in 2005 by John Ioannidis (a once-acclaimed Stanford University professor who has recently been tarnished for his views on the Covid-19 pandemic, including his vehement opposition to lockdowns), as a provocative Title: Why Most Published Research Findings Are False. Ioannidis noted that erroneous conclusions often persist in the literature due to small sample sizes and biases, and that scientists tend to advance their own research agendas rather than try to replicate the work of their colleagues. , which meant that errors in Since that his paper, metascientists have honed their techniques to study bias, a term that covers everything from so-called “questionable research practices” to blatant data fabrication or falsification.
They capture the pulse of this bias by looking at overall patterns in the literature rather than individual studies. For example, if smaller studies on a particular topic tended to show more dramatic results than larger studies, it could be an indicator of bias. Smaller studies have more variability, so chance can lead to dramatic results. In a world that favors dramatic results, such studies will be published more frequently. Other approaches look at the p-value, a number that indicates whether a particular result is statistically significant. If there are too many significant p-values and too few non-significant p-values across the literature for a particular research question, scientists may be using questionable approaches to make their results more meaningful. there is.
However, these patterns do not indicate the degree of bias due to fraud rather than dishonest data analysis or innocent errors. There is a sense that cheating is inherently immeasurable, says Jennifer Byrne, a professor of molecular oncology at the University of Sydney. “Deception is about intent. It’s a psychological state of mind,” she says. “How do you infer mental states and intentions from published papers?”
