I conducted a baseline survey in January 2022 with 1000 participants from the US. Now I am running a follow-up survey with the same participants, i.e., only those that took the first survey are eligible to do the second one. In general, the response rate is fine, we are currently at 40% and the survey just started yesterday.
However, a couple of users returned the invitation or timed out. Since I made good experience with prolific messaging app in the past to receive feedback from users, I was wondering, if I may also use it to actively contact those users were problems occurred and encourage them to reconsider their participation.
I simply intend to write a message of the kind “We saw that you returned/timed out our study, but please reconsider and try again to do the survey” to all IDs that are either described as “Returned” or “Timed Out” in the study overview screen.
Do you have any experience with such practice? Is it fine or do users consider it annoying to be contacted directly?
No, I haven’t experienced the exact same problem, so I didn’t need to ask participants to try the study again even if they returned in the first place. But in general, I’ve always had a good experience messaging participants so far. For instance, for understanding if there were technical issues that could explain long completion times, or if they had problems in the comprehension of the task. The only issue I could spot for you recontacting the participants that returned their submission is that they’ll do some task/reply to questions that potentially they have already seen. So think if it’s a problem for your design (e.g., do you need naive/impulsive replies? Could the fact that they were previously exposed to them bias your results?).
Dear Veronica,
thanks for your feedback. I had similar positive experience with messaging user. I guess since Josh also indicated that it is allowed to actively engage, I should give it a try.
Regarding the research design, the feature that the users already know about the survey and do it again is intended. We want to analyze how stable opinions on a particular issues are.
Hi @Felix_Bethke (& all),
I agree with everyone else on this - I’ve not run into this particular issue but I have had much more success than expected by messaging participants. Especially if there was some sort of genuine computer error or a participant did not realize that there was a problem, people have been quite responsive.
Good luck reaching out to everyone and feel free to update the forum here if you think others might learn from your experience!