🧪 How To Optimise Your Data Collection on Prolific

📌 Notice Board 📅 Community Events
1

@Andrew_Gordon, our Academic Lead, is going to give you all the tips and tricks to optimize your data collection on the platform!

Webinar Stream

If you have any questions, just pop them below! :slight_smile:

Webinar Contents with Timestamps

  1. Trends in online research - 1:20
  2. Demo of Prolific - 4:12
  3. How we ensure data quality on Prolific - 12:03
  4. Increasing participant engagement for your study - 18:11
  5. Increasing participant attentiveness for your study - 23:08
  6. Increasing the trustworthiness of your sample - 31:40
  7. Demo results - 35:25

Enjoy!

2 Likes
2

Reminder that this is happening tomorrow! Hopefully see you there :slight_smile:

@trust_level_0

3 Likes
3

I won’t be able to join the event at that time (kids home from pre-school). But I’d love to take part.
Will you record it?

1 Like
4

Hi @Jonas_Hjalmar_Blom, yes the event will be recorded and available here to watch on demand :slight_smile:

1 Like
5

Webinar is streaming now at the top of this page!

2 Likes
6

Can you say more about how participants might decide to participate in your study? For example,

How do they get notified for available studies?
Can / do they filter based on pay rate, estimated time?
Can they favorite you as a researcher?

7

Hi @cosanlab, thanks for the question. We actually ran a survey on ~1000 of our participants asking this exact question a few months ago (see image below for the results). It seems that amount of pay is the main reason, but interest in the topic and length of the study were popular also. For the responses that fell under ‘other’ see the data file here which factor

Regarding your specific questions:

1 - Participants are either notified directly through the website if they are logged in and on the study selection page (available studies will appear there), or via email (will be sent an email when a study they are eligible for becomes available). Some participants also use a Prolific chrome extension that notifies them.

2 - No they are not able to filter studies by these factors, but that information does appear on the study description so they can decide to participate or not based on those factors if they want to.

3 - No they are currently unable to do this.

I hope this helps? Do let me know if you have any other questions :slight_smile:

3 Likes
8

Hi there, any chance of getting the references with your presentations? I tried to google some of them, but many names are quite generic and don’t show up in a Google Scholar search.

2 Likes
9

Hi @Susanne_Flach sure thing:

1: Sassenberg, K., & Ditrich, L. (2019). Research in social psychology changed between 2011 and 2016: Larger sample sizes, more self-report measures, and more online studies. Advances in Methods and Practices in Psychological Science , 2 (2), 107-114.

2: Chandler, J., Paolacci, G., Peer, E., Mueller, P., & Ratliff, K. A. (2015). Using nonnaive participants can reduce effect sizes. Psychological science , 26 (7), 1131-1139.

3: DeVoe, S. E., & House, J. (2016). Replications with MTurkers who are naïve versus experienced with academic studies: A comment on Connors, Khamitov, Moroz, Campbell, and Henderson (2015). Journal of Experimental Social Psychology, 67, 65–67

4: Rand, D. G., Peysakhovich, A., Kraft-Todd, G. T., Newman, G. E., Wurzbacher, O., Nowak, M. A., & Greene, J. D. (2014). Social heuristics shape intuitive cooperation. Nature communications , 5 (1), 1-12.

5: Peer, E., Brandimarte, L., Samat, S., & Acquisti, A. (2017). Beyond the Turk: Alternative platforms for crowdsourcing behavioral research. Journal of Experimental Social Psychology , 70 , 153-163.

6: Spatharioti, S. E., & Cooper, S. (2017). On Variety, Complexity, and Engagement in Crowdsourced Disaster Response Tasks. In ISCRAM .

7: Online article: https://www.qualtrics.com/blog/using-attention-checks-in-your-surveys-may-harm-data-quality/

8: Hauser, D. J., Sunderrajan, A., Natarajan, M., & Schwarz, N. (2016). Prior exposure to instructional manipulation checks does not attenuate survey context effects driven by satisficing or gricean norms. Methods, data, analyses: a journal for quantitative methods and survey methodology (mda) , 10 (2), 195-220.

Thanks for raising this, I will make sure to add a full reference list to this presentation for the future!

3 Likes
10

Awesome, thanks! (Great talk, btw, forgot to mention that…!)

2 Likes