Gamification describes the use of game-elements in cognitive tests and trainings, which is commonly believed to make arbitrary tasks more enjoyable and to increase user experience and engagement. Gamification is very promising for clinical trainings and applications, whereby patients or at-risk groups can enhance their cognitive system through many repetitions of otherwise mundane tasks. For example, gamification elements are employed in trainings that target inhibitory control functions to alleviate food selection and food intake in adults with binge eating disorder or obesity [1], but also already in children with loss-of-control eating [2,3]. Inhibitory control in a mundane stop task was associated with obese children’s weight, but also with weight change across one year [2]. To date, several studies are available that sometimes employ gamified tasks, which seems particularly promising as a prevention strategy in young populations who appear at risk for the development of an impulse control disorder [4–6].
However, not all gamification elements produce more enjoyment. Adding progress- and score-bars to track performance, for instance, has shown beneficial effects in some studies [7] whereas adding an interesting narrative and context to an otherwise mundane task had more controversial effects [8–10]. In this vein, we recently tested a full- blown gamified task on Prolific which reproduced the gap in inhibitory control between overweight and normal weight participants, but no changes in experience and enjoyment were observed despite considerable changes to the visual appearance, a feedback bar, a narrative and display of a companion character in the gamified version. The full publication of this study is openly available with the support of open access publishing from our institutions.
With this proposal, we want to go one step further and pursue two goals:
- We want to disentangle the effects of individual gamification components, in particular, the heightened distraction by context and the improved user experience by scores and progress bars in a 2-by-2 matrix. With this experiment, we can disentangle the individual and combined contribution of these single game components to user enjoyment
- Instead of gamify-ing an existing assessment, we want to inject the assessment from the previous study in an existing jump-and-run game. With this experiment, we test the validity of this stealth assessment and investigate if user enjoyment can be further improved
Methods
We implement all task variants within a web-based game engine (JavaScript) and record conventional behavioral and psychological measures (reaction times, questionnaires). This implementation is close to our original paper [8], but a full gamification will utilize an endless runner scenario. This is a cross-sectional experiment with random assignment of participants to the experimental groups.
Please see a live preview of the gamified version here.
For reliable online assessment of behavioral responses, the experiment will be implemented using the JavaScript library processing.js, along with additional code elements from jsPsych for questionnaires. On the server side, the experiment was controlled with the JATOS framework, a Java environment that allows recording data, starting experimental modules, and creating links for participants to run the experiment. Data will be stored on a dedicated server at the author’s institution.
Sample size determination. Even a small effect on user experience can motivate some individuals to continue with a behavioral task or training. Recent meta-analyses found medium to large effect sizes for user engagement [10,11]; in contrast to this, gamification had very little impact on subjective experience in our Prolific study. However, we were not able to detect more nuanced differences due to the small sample size. With an anticipated group size of 176 participants, we will be able to detect even small effect sizes (d = .3) in this project.
Description of the study costs. We apply for participant funding for N = 1056 participants (704 in experiment 1, 352 in experiment 2). With the funding, we wish to employ the current minimal wage rate in Germany (10€ / hour ~ GBP 8.62) + 33% service charge. Our previous experiment took participants on Prolific on average 35.6 minutes, we will thus offer GBP 5.12 (+VAT) for a comparable study length. Then, the total study costs amount to:
- Experiment_1: (5.12+0.33×5.12)×704 = 3,584 GBP
- Experiment_2: (5.12+0.33×5.12)×352 = 1,792 GBP
Programming and other study costs will be distributed among the study team and are considered acceptable due to our previous pilot study.
Preregistrations: Experiment1 / Experiment2
Open Science Statement: As with our previous research, we will share data and results openly in pre-prints and repositories.
Thank you for supporting our proposal!
References
[1] Lavagnino et al. Inhibitory control in obesity and binge eating disorder: A systematic review and meta-analysis of neurocognitive and neuroimaging studies. Neurosci Biobehav Rev. 2016;68:714–726.
[2] Nederkoorn et al. Impulsivity predicts treatment outcome in obese children. Behav Res Ther. 2007;45:1071–1075.
[3] Fogel et al. Associations between inhibitory control, eating behaviours and adiposity in 6-year-old children. Int J Obes. 2019;43:1344–1353.
[4] Verbeken et al. Executive function training with game elements for obese children: A novel treatment to enhance self-regulatory abilities for weight-control. Behav Res Ther. 2013;51:290–299.
[5] Dassen et al. Gamified working memory training in overweight individuals reduces food intake but not body weight. Appetite. 2018;124:89–98.
[6] Reinblatt et al. Pediatric loss of control eating syndrome: Association with attention-deficit/hyperactivity disorder and impulsivity. Int J Eat Disord. 2015;48:580–588.
[7] Wiley, Vedress, & Mandryk. How Points and Theme Affect Performance and Experience in a Gamified Cognitive Task. Conf Hum Factors Comput Syst 2020. Oahu, Hawai i, USA.; 2020.
[8] Schroeder, Lohmann, & Ninaus. Preserved Inhibitory Control Deficits of Overweight Participantsin a Gamified Stop-Signal Task:Experimental Study of Validity. JMIR Serious Games. 2021;9.
[9] Lau et al. Serious games for mental health: Are they accessible, feasible, and effective? A systematic review and meta-analysis. Front Psychiatry. 2017;7.
[10] Vermeir et al. The effects of gamification on computerized cognitive training: Systematic review and meta-analysis. JMIR Serious Games. 2020;8.
[11] Looyestyn et al. Does gamification increase engagement with online programs? A systematic review. PLoS One. 2017;12:1–19.