[Proposal]
I’M NOT EMOTIONAL, I’M JUST MAKING A BAD DECISION OK?!
Importance of the Proposed Work
Decisions are nearly always influenced by mood and affect, yet we still know very little about the role of emotion in decision-making, and have struggled to effectively refute the idea that making a decision under high-emotion is highly problematic (Pfister & Böhm, 2008). There are many occupations, like the Fire Service, where decision-making under heightened emotional states is a regular occurance, yet there is relatively little known about how emotion influences decision-making, and thus how we can support decision-makers to make optimal decisions in such contexts (Evans, 2019).
One key contributing factor to such fragmentation in understanding is the lack of consistent empirical attention and central theories from which to base understanding. There are many studies within this field which have been deemed “cornerstone” in our understanding of decision-making yet which have received little-to-no replication attention. For example, Rottenstreich & Hsee (2001) proposed an s-shaped function whereby an affect-poor outcome is more popular and financially-justified when its likelihood is high, and vice-versa for when likelihood is low. This work has been highly-influential and has received over 1000 citations to-date. The Many Labs 2 project (Klein, 2018) replicated the first of three experiments and reported the opposite effect, questioning the underlying theory, but a comprehensive evaluation of the theory proposed has yet to be enacted.
We’re requesting funding to complete student-led replications via preregistrations and the Registered Report format to revisit and reassess studies in decision-making, such as this classic article by Rottenstreich and Hsee (2001), with extensions to further assess possible explanations or pathways for such effects.
Study Design
Rottenstreich and Hsee (2001) conducted three experiments where probability of a hypothetical lottery outcome was experimentally manipulated and participants either chose an outcome, or chose how much they would pay to participate, when the outcome was either affect-rich (e.g. opportunity to hug your favourite movie star) or affect-poor (tuition fee coupon). The high-powered replication and extension project planned attempts to combine all three experiments into one, testing both a UK and US sample, with extensions to assess the assumed differences in affect and all response options for all experiments. All aspects of the proposed study including materials, simulated dataset, power analyses, analysis code and more can be found on the OSF here and the work is currently under review at Psychological Science.
The Rottenstreich and Hsee (2001) project is just one of many influential works which have been chosen for independent replication and extension. The CORE mass replication project led by Dr Gilad Feldman (University of Hong Kong), and a similar educational training programme led by Dr Thomas Rhys Evans (University of Greenwich), uses preregistration and Registered Reports to support UG, MSc and PhD students to have vital practical experience with open science practices and lead contributions towards increasing the rigor of scholarship in the field.
Other study examples: Jones & Kavanagh (1996), Epstein et al. (1992), Hsee and Rottenstreich (2004), Ames, Flynn & Weber (2004), Hardisty, Johnson & Weber (2010), Gilovich & Medvec (1994), Monin & Miller (2001), Watkins et al., (2006), see https://mgto.org/pre-registered-replications/#planned.
Sample and Study Costs
Study costs vary across the studies completed but are all informed by pre-registered power analysis-informed estimates. For example, when replicating Rottenstreich and Hsee (2001), the original study featured three experiments with sample sizes of 40, 138 and 156 respectively. The effect sizes of the original results were calculated as φ = .3 for Experiment 1, d = 0.47 and 0.57 for Experiment 2, and d = 0.52 and 0.51 for Experiment 3. Using a power calculation in G*power with the smallest effect sizes of interest, as reported in the supplementary document on the OSF project page, the number of participants required to observe the smallest effect (d = 0.47) with 95% power was N = 416. To account for the expected ~10% of careless responses (Meade & Craig, 2012), the study aimed to collect data from a minimum of 450 participants from the UK. Should the total sample size post-exclusions not meet the 104 participants per group target, additional sets of 10 participants will be recruited. As such, a conservative estimate of 500 participants paid £1.50 for 10 minutes would require £1000. Adopting this study as representative of typical study designs chosen to be replicated, £10000 is requested to provide funding of 10 student-led open science replication and extension studies of influential decision-making research.
Preregistration
Funding will contribute to recruit participants for a number of preregistered or Registered Report publications. As such, all project designs will be fully preregistered alongside analysis code, study materials, and open data. For example, a Stage 1 Registered Report replication study of Rottenstreich and Hsee (2001) is currently under review at Psychological Science with the full protocol, materials and simulated data and data analysis plans available on the OSF here.
Openness
Every element of the studies funded, including preprints, materials, data, and analysis code, will be made permanently and publicly available on their respective OSF pages. As part of the Collaborative Open-REsearch (CORE) and Greenwich-based Registered Report groups, we have a clear public commitment to transparency in all aspects of the research cycle.
References
Evans, T. R. (2019). Emotions in the fire service: decision-making, risk, and coping. In Evans, T. R. and Steptoe-Warren, G. (Eds.), Applying Occupational Psychology to the Fire Service (pp. 13-57). Palgrave Macmillan.
Klein, R. A., Vianello, M., Hasselman, F., Adams, B. G., Adams Jr, R. B., Alper, S., … & Sowden, W. (2018). Many Labs 2: Investigating variation in replicability across samples and settings. Advances in Methods and Practices in Psychological Science, 1(4), 443-490.
Pfister, H. R., & Böhm, G. (2008). The multiplicity of emotions: A framework of emotional functions in decision making. Judgment and Decision Making, 3(1), 5–17.
Rottenstreich, Y., & Hsee, C. K. (2001). Money, kisses, and electric shocks: On the affective psychology of risk. Psychological Science, 12(3), 185-190.