Although both types of research aim for findings that are reliable and valid, there are important differences between them.
In confirmatory (also called hypothesis-testing) research, the researcher has a pretty specific idea about the relationship between the variables under investigation. In this approach, the researcher is trying to see if a theory, specified as hypotheses, is supported by data.
Depending on the type of research you are doing, your approach and research design will be different. Imagine you are doing a confirmatory study to test the hypothesis that playing table tennis at work increases the employees’ creativity. In this case, you will have precise ideas about which measurements to use (e.g., measure the time played table tennis and count the ideas generated in a subsequent brainstorming session or the scores in a creativity test). You will (hopefully) also have a good theoretical foundation to justify why there should be a connection between playing games at work and creativity in the first place, and (hopefully) also a good rationale for measuring “playing games” and “creativity” in a certain way.
If your research was exploratory, however, you would not have any hypotheses in advance, but you would still be interested in finding out what may increase employees’ creativity. You might collect extensive qualitative data through interviews with employees. When analysing these interviews, you may notice that the topics of leisure activities and games come up quite frequently. Based on this insight, you could then develop the post-hoc hypothesis that playing games in the office (and, more specifically, table tennis) might have a positive impact on creativity.
Why does it matter?
It’s dangerous to confuse these two types of research. All too often researchers treat exploratory results as confirmatory, and this hindsight bias (also known as the ‘I-knew-it-all-along effect’) can make us feel as though we had a prediction all along, even though we didn’t. For example, you might unexpectedly discover that playing table tennis at work decreases creativity, and then find post-hoc reasons for why this is plausible after all. This is called HARKing (Hypothesizing After the Results are Known) and it’s a poor research practice, for several reasons:
- HARKing can make exploratory findings more publishable by falsely giving the impression that an unanticipated result was expected. This may lead fellow researchers to believe that your finding has been empirically tested more times than it actually has (and that it is more robust than it actually is), thus creating unwarranted confidence in a result and ultimately reducing reproducibility.
- Valuable information about your original hypothesis might be lost
- HARKing may promote (conscious or inadvertent) fudging of statistical analyses
- It presents a distorted, inaccurate model of science
- It violates a fundamental ethical principle of science: to communicate one’s work honestly and completely
- HARKing promotes narrow (i.e. context and paradigm bound) new theory (rather than powerful, general theory)
One way to tackle HARKing is to preregister your research. Preregistration helps to distinguish analyses and outcomes that result from prediction (i.e., confirmatory, hypothesis-testing research) from those that result from postdiction (i.e., exploratory, hypothesis-generating research). At the end of the day, we need both exploratory and confirmatory research to do good science. For more information on how to preregister, check out the preregistration article in this guide.