Questionable research practices and biases
Questionable research practices: the grey zone
From one end of the research integrity spectrum of clear misconduct (e.g. falsifying results or plagiarizing someones work) to acceptable and responsible research practices (i.e. what Open Science movement strives to create), where do the selective reporting or misrepresentations of data fit in? This grey zone of Questionable Research Practices (QRPs) of less overt and subtle behaviors (compared to misconduct) nonetheless damages the credibility of science (Entradas et al. 2026). The prevalence and impact of QRPs has received less attention than the clear cut examples of serious research misconduct (although see case study below).
HARKing
Hypothesizing after the results are known (HARKing) is a QRP where a researcher changes their hypothesis based on or informed by the results, but present their hypothesis as if it were a priori (Kerr 1998).
p-Hacking
p-Hacking is a QRP which describes the many ways researchers can artificially increase the likelihood of obtaining a statistically significant result (Pennington 2023). p-Hacking increases the amount of false positive results and inflates effect sizes published in the literature. There is even an app that let’s you play around with different p-hacking strategies! Warning: the app is designed to understand how p-Hacking works, please do not intentionally p-Hack in your own work!
Cherry picking
Cherry picking is where researchers select the most favorable looking results that support their hypotheses to report and omit the non-significant ones. This practice is an issue for readers interpreting findings as there is not a complete account of the findings (Andrade 2021).
Fishing and data dredging
When researchers look for associations between different combinations of variables (by mix and matching or omitting variables) with the intention of finding a statistically significant outcome. Fishing can occur with HARKing. This practice leads to increased probability of false positive (Andrade 2021). Data dredging is a more extreme case of fishing, where researcher carryout an extensive search for significant findings without a specific hypothesis to test.
Researcher biases
There are wide range of cognitive biases (e.g. hindsight bias, confirmatory bias, bias blind spot) that lead to systematic differences in how we perceive information and make decisions.
Hindsight bias
The “I knew it all along” is the phrase associated with hindsight bias, which is the tendency to frame previous decisions or events as more predictable after the outcome is known (Hardwicke and Wagenmakers 2023).
Confirmatory bias
The tendency to seek out, interpret, favor and recall information in a way that supports one’s prior values, beliefs, expectations, or hypothesis (Parsons et al. 2022).
Bias blind spot
The lack of awareness of how one’s decisions are shaped by bias (Hardwicke and Wagenmakers 2023).
Summary
Questionable research practices can overlap with each other and biases. Engaging in QRPs may not be intentional. The important thing is to be aware and course correct when you do know. The range of biases highlights that we are all human and it is important to be aware of how cognitive biases influence how we do our research and take steps to mitigate.
Interested in learning more about questionable research practices? Take a look at the recent study by Entradas et al. (2026) where they examined how ~1500 researchers understand and engage in questionable research practices.
Quick Quiz on QRPs!
Test your knowledge on questionable research practices by taking this short quiz.
- HARKing - Hypothesizing after the results are known is a:
The correct answer is B, HARKing is a questionable research practice.
Refer back to Section 1.1 for more information
- The tendency to frame previous decisions or events as more predictable after the outcome is known, is an example of which type of cognitive bias?
The answer is C. Hindsight bias can be summarized as “I knew it all along” type of thinking.
Refer back to Section 2.1 for more information
Revisiting Max’s Dilemma
Remember back to Max’s dilemma in the introduction, can you identify which of the questionable research practices option A (“remove that variable from the analysis”) would correspond to?
Option A would fall under fishing, as Max would be trying to find an association between the variables by omitting a variable to produce a significant result.