Tips for collecting data with very long surveys

[Dani_Levine]

I will have a survey likely containing ~500 items, which may take as much as 2 hours to complete. I am torn about the best approach, as I can’t imagine it makes sense to have participants complete this all at once. I could split it up into 2 or more chunks, but I’m concerned about the dropout rate after the first chunk is completed, or the low number of participants who would agree to a multi-part study if payment happened at the end and required completing all parts of the study. Has anyone successfully tested a very long survey on here and if so, have you learned any best practices that would be helpful for planning?

[paul]

hi @Dani_Levine , I don’t have any experience with this type of situation, but I strongly suspect that 500 items over 2 hours would result in unreliable data. Given that, I would probably err on the side of multiple studies to make sure that participants have a break. There’s probably a way to plan this with a sort of randomly intentional missing data approach so that the partial participants’ data counts for something, but missing data is just a bit outside my wheelhouse … might be able to find a reference or two lmk if that would be helpful

[Dani_Levine]

Thanks for your response! I have a plan for the missing data piece, but I was more curious about the distinction between having participants complete 2 studies where both are required for payment versus having participants complete 2 studies when the second is optional. In general, is it common for people to sign up for studies that require multiple sessions? Or, if completing the second/third follow-up studies was optional, is there some estimate of the percentage of participants who would typically complete those follow-ups?

[timtak]

Hi Dani

I really don’t know how many participants will come back
or the optimal number of studies to separate 2 hours of
experiment into, but regarding drop out rate…

The Prolific FAQ on (multi-part) Longitudinal Studies

How do I set up a longitudinal / multi-part study?

Suggests a drop out rate between 0%-50% but that is
for longer studies.

Please see this my recent post for suggestions
on how to make sure participants come back

Ask Anything Thread (Newbie Friendly)

Dear Madalina Welcome to the forum Yes, it is very possible to post a two part study, especially if the two parts are close together, on consecutive days. A FAQ is here In brief You need to make it very plain at the top of the study description that (perhaps all in capitals like this) THIS IS A TWO PART STUDY. THE NEXT PART WILL BE CARRIED OUT TOMORROW. COMPLETION OF BOTH PARTS IS A REQUIREMENT FOR PAYMENT Tell participants again that they will be required to participate …

but, bear in mind that it may not be possible to reject large numbers of participants from the first study if they don’t come back to the second.

Tim