Tips for collecting data with very long surveys

I will have a survey likely containing ~500 items, which may take as much as 2 hours to complete. I am torn about the best approach, as I can’t imagine it makes sense to have participants complete this all at once. I could split it up into 2 or more chunks, but I’m concerned about the dropout rate after the first chunk is completed, or the low number of participants who would agree to a multi-part study if payment happened at the end and required completing all parts of the study. Has anyone successfully tested a very long survey on here and if so, have you learned any best practices that would be helpful for planning?

hi @Dani_Levine , I don’t have any experience with this type of situation, but I strongly suspect that 500 items over 2 hours would result in unreliable data. Given that, I would probably err on the side of multiple studies to make sure that participants have a break. There’s probably a way to plan this with a sort of randomly intentional missing data approach so that the partial participants’ data counts for something, but missing data is just a bit outside my wheelhouse … might be able to find a reference or two lmk if that would be helpful

Thanks for your response! I have a plan for the missing data piece, but I was more curious about the distinction between having participants complete 2 studies where both are required for payment versus having participants complete 2 studies when the second is optional. In general, is it common for people to sign up for studies that require multiple sessions? Or, if completing the second/third follow-up studies was optional, is there some estimate of the percentage of participants who would typically complete those follow-ups?

Hi Dani

I really don’t know how many participants will come back
or the optimal number of studies to separate 2 hours of
experiment into, but regarding drop out rate…

The Prolific FAQ on (multi-part) Longitudinal Studies

Suggests a drop out rate between 0%-50% but that is
for longer studies.

Please see this my recent post for suggestions
on how to make sure participants come back

but, bear in mind that it may not be possible to reject large numbers of participants from the first study if they don’t come back to the second.


1 Like