Motivation
Misinformation has serious societal consequences, with COVID-19 misinformation resulting in health-endangering behaviors, vaccine conspiracy theories resulting in vaccine hesitancy, and climate misinformation delaying urgently needed mitigation policies. Evidence-based interventions are needed to effectively mitigate damaging misinformation, especially on social media where it often spreads (Vosoughi, Roy, & Aral, 2018).
Past research has explored two main approaches to countering misinformation: fact-based or logic-based corrections (Banas & Miller, 2013). Fact-based corrections show how misinformation is false through factual explanations while logic-based corrections reveal the rhetorical techniques used to mislead. While both methods are effective against climate and health misinformation (Schmid & Betsch, 2019), the logic-based approach has unique benefits. Logic-based corrections work across topics, with one study finding that correction of a rhetorical technique used in tobacco misinformation neutralized the same technique in climate misinformation (Cook et al., 2017). This same study also found that the logic-based approach was depolarizing. The logic-based approach is effective regardless of whether it comes before or after the misinformation, while fact-based corrections can be rendered ineffective if the misinformation comes later (Vraga et al., 2020).
Combining the two approaches has been found to convey no additional benefit compared to each individual method alone (Schmid & Betsch, 2019). However, past research focused on scientific misconceptions. With my colleagues Emily Vraga and Sojung Kim, we recently conducted a pilot study finding that while a fact-based correction was more effective in reducing scientific misperceptions, the logic-based approach inspired greater information seeking. This implies that a combined approach may have a greater impact across a range of dependent variables than each approach alone. Further, no studies have explored the relative durability of either approach.
Our experiment will explore the following research questions:
- What is the relative effectiveness of fact-based and logic-based corrections in correcting misperceptions, engaging readers, provoking information seeking, and improving related behavioral intent?
- Does a combined fact + logic approach convey benefits across outcomes?
- Which type of correction conveys longer lasting effects across outcomes?
Methodology
We plan to conduct two experiments focused on (1) health or (2) climate change. Both experiments will consist of two waves. In wave 1, participants are randomly assigned to one of five conditions: control, misinformation-only, misinformation + logic-based correction, misinformation + fact-based correction, misinformation + logic-based + fact-based correction. The interventions are displayed in the form of mocked up tweets, simulating how people are exposed to misinformation and corrections on social media. Two weeks later in wave 2, the same dependent variables are measured in a follow-up survey.
In wave 1, participants fill out a pre-test capturing general demographics (3 mins). They then read the intervention tweets (2 mins). Finally, they fill out a post-test measuring post engagement, knowledge, trust in science, information seeking, policy support, and behavioral intent (10 mins). The total time for the wave 1 experiment is 15 minutes. In wave 2, occurring two weeks after wave 1, participants fill out the same post-test survey as in wave 1 (10 minutes).
In the climate change experiment, the misinformation is the argument that cold weather disproves global warming. The fact-based correction explains that the odds of hot records are increasing while cold records are decreasing. In the health experiment, the misinformation will cite a grandfather who smoked while living to 90 as evidence that smoking isn’t unhealthy. The fact-based correction will explain that while individuals show varied responses to smoking, on average, smokers experience more health impacts with shorter lifespans. In both experiments, the logic-based correction will explain that the myth commits the fallacy of anecdotal thinking, focusing on personal experience while ignoring the bigger picture.
Our analysis plan will be to conduct between-subjects ANOVA to explore the immediate effect of experimental condition on different dependent variables such as misperceptions, behavioral intent, engagement with the post, and information seeking intent. We will then conduct within-subjects ANOVA to explore which conditions resulted in long-term effects on the dependent variables. When significant effects are detected, pairwise comparisons will explore the most effective interventions.
Sample Size and Costs
Using power analysis with power 95% and alpha 0.05, and given the interventions are social media posts which in our previous studies have resulted in small effect sizes, we estimate requiring 165 participants per group in wave 2. Assuming a 25% dropout from wave 1 to wave 2, we will recruit 1100 participants in wave 1, offering £0.50 bonus to incentivize completion of wave 2. Total costs for each experiment will be £4400 with a total cost of £8800.
Preregistration
Our study design will be preregistered at OSF.
Impact
We will submit our research results in an open-access journal while making the anonymized data and code freely available via OSF. The coauthors of this research have published practical guides on debunking misinformation such as The Debunking Handbook 2020 and The Conspiracy Theory Handbook. These research findings will be incorporated in similar subsequent publications. The results will also be presented at skepticalscience.com and crankyuncle.com.
This research will yield valuable insights due to the dearth of research into logic-based corrections, despite the powerful benefits of this approach found in existing research. Deepening our theoretical understanding of the relative merits of both approaches will inform practical efforts of educators, scientists, and fact-checkers when countering misinformation.
References
Banas, J. A., & Miller, G. (2013). Inducing resistance to conspiracy theory propaganda: Testing inoculation and metainoculation strategies. Human Communication Research, 39(2), 184-207.
Cook, J., Lewandowsky, S., & Ecker, U. K. H. (2017). Neutralizing misinformation through inoculation: Exposing misleading argumentation techniques reduces their influence. PLOS ONE, 12, e0175799.
Schmid, P., & Betsch, C. (2019). Effective strategies for rebutting science denialism in public discussions. Nature Human Behaviour, 3, 931-939.
Vosoughi, S., Roy, D., & Aral, S. (2018). The spread of true and false news online. Science, 359(6380), 1146-1151.
Vraga, E. K., Kim, S. C., Cook, J., & Bode, L. (2020). Testing the effectiveness of correction placement and type on Instagram. The International Journal of Press/Politics, 25(4), 632-652.