By the People, For the People: A Participatory Approach to Improving the Life in the UK Test
Wijnand A. P. van Tilburg & Nikhila Mahadevan, University of Essex
Who started the first curry house and what street was it on?
The answer is Sake Dean Mahomed, who opened the āHindoostane Coffee Houseā at George Street, London, in 1810. If the reader did not know, then, rest assured, they are hardly alone. Our recent study among a representative (Prolific) sample of UK residents, mostly UK citizens, showed that two-thirds failed their own citizenship test (Van Tilburg, Saadi, & Mahadevan, 2020). We propose an empirical vetting of questions such as these by a representative sample of UK residents to form the basis for policy change recommendations.
Our opening question is one of several that around 150,000 UK citizenship applicants per year may face as part of their obligatory Life in the UK Test (Home Office 2013; Brooks, 2013). Passing the test requires answering 18 out of 24 multiple choice questions correctly, based on content covered in a 160-page book. Performance is critical for living and working with equal rights to those already in possession of UK citizenship. The Life in the UK Test has been widely criticized for factual errors and trivia questions, excessive study material, absence of practical information, disagreement about its purpose, and the undue psychological distress it causes (Bassel et al., 2017; Brooks, 2013; Select Committee on Citizenship and Civic Engagement, 2018).
Despite this widespread criticism, despite the attention that the issue has received in Parliament, and despite the significance of this life changing test for those 150,000 yearly applicants, improvements have been marginal or dismissed (e.g., Home Office spokesperson in response to our 2020 findings; iNews, 2021). Yet, we found that the UK public overwhelmingly supported altering the Life in the UK Test (in particular, making it easier) after attempting the test themselves (Van Tilburg et al., 2020), regardless of political beliefs. Furthermore, support for its improvement seems a shared, rather than partisan, issue.
While top-down improvement of the test remains slow (e.g., initiated by the Home Office), a bottom-up approach where UK residents directly participate in suggested revisions can serve as powerful public tool in urging policy improvement to Parliament and Home Office. We initiate such a participatory approach to refining this citizenship test, by involving a representative UK sample in vetting existing questions. The envisioned outcome, aside from academic dissemination, will be a selection of Life in the UK Test questions vetted by the UK public that will be offered to the Home Office to spark the so desperately needed improvement of this life-altering test.
Design, Samples, and Method
Our project encompasses two studies, each relying on a representative sample of the UK population in age, gender, and ethnicity. Participants will be recruited via the online platform Prolific TM. Representative sampling is critical for these studies to serve as initial norms for policy change. Materials are in an OSF repository; data and code will be added. We pre-registered our studies. Analysis code and anonymized data are placed in a public Open Science Framework repository. Studies are administered online using Qualtrics TM.
The aim of Study 1 (September 2021) is to identity the suitability of Life in the UK Test questions. Participants give informed consent and report demographics (age, gender, ethnicity, education, household income) presented following Census format (Office for National Statistics, 2016), and citizenship status. Participants then evaluate 45 systematic-randomly drawn Life in the UK Test questions from 360 public-access questions (Van Tilburg et al., 2020). Participants simply categorize each as āsuitableā or āunsuitable.ā We will recruit 660 participants, which, after exclusion criteria (approx. 10%), yields approximately 75 participants per evaluated question. Individual questions are deemed āsuitableā when 54 of participants rate them such, as Bonferroni-corrected confidence intervals (Ī± = 0.5/360) then exceed 50% acceptance.
Study 2 (December 2021) serves to validate the suitability judgments from Study 1 and to compare perceived suitability of tests containing only the approved subset versus original tests. After reporting demographics (see Study 1), participants are randomly assigned to one of two conditions. In the āoriginal testā condition, participants answer 24 randomly selected questions from the 15 public-access Life in the UK Tests of Study 1. In the ārevised testā condition, these 24 questions will be drawn from the items deemed suitable in Study 1. For our dependent variable we ask participants to judge the suitability, on a scale, of their assigned test.
We will recruit 550 participants, which, after applying a priori exclusion criteria (approx. 10%), affords independent samples t -test sensitivity to differences between the conditions with effects sized, d = .29, with power of, (1 ā Ī²) = .90, and a Type-I error of, Ī± = .05 (two-tailed).
Study Costs
Total study costs are Ā£4,636.79.
Study 1 costs are participant remunerations at 660 Ć Ā£2 for 15 minutes (Ā£1,320). At the time of writing, Prolific adds to this a representative sample fee Ā£1,176.17, totaling Ā£2,496.17 for Study 1.
Study 2 costs are participant remunerations at 550 Ć Ā£2 for 15 minutes (Ā£1,100). Prolific adds to this a representative sample recruitment fee of Ā£1,040.62, totaling Ā£2,140.62. We appreciate that some potential overlap in samples is currently unavoidable. The preregistration outlines subsidiary analyses to address this.
Dissemination and Impact
We endeavor publishing results in a suitable Q1 ranked peer-reviewed journal, such as British Journal of Social Psychology . The submitted article will be publicly available on PsyArXiv. Precedent suggests likely media interest in the findings which we will nurture with a press release and support by the External Relations and Communication team at the University of Essex.
A succinct report of findings and recommendations will be offered to Home Secretary Rt. Hon Priti Patel, MP, and Shadow Home Secretary Hon Nick Thomas-Symonds, MP, in collaboration with the University Impact Team. Pending their responses, further distribution likely follows. We will furthermore monitor suitable Policy Engagement calls (e.g., Select Committees, Government Consultations, Parliamentary Enquiries) for opportunities to contribute.