I am planning to use Prolific to recruit participants to rate some videos. For that, I will have some check questions to make sure that they watched the whole video - if they do not, the data will be useless.
Therefore, I considering rejecting (kicking out) ones who fail the check questions mid-survey (not a hard rejection, they can still return). Is this legit on Prolific? I think in this way, less work for both sides: I do not need to go through their answers, they don’t need to complete the whole survey and get rejected.
Hey @Eric, apologies for the delay in getting back to you and welcome to the community!
Usually we would ask you not to remove participants from the study part-way through unless you are validating your prescreening questions and the participants provide a contradictory answer.
We’d rather that you allow the participants to complete the study and then review their answers holistically at the end in the context of how other participants have behaved / performed.
For example, there could be a tech issue that the you aren’t aware of and you could end up removing all participants due to this, when the participants should have been allowed to submit and be paid.
Specify that participants should be watching the whole video, and if participants don’t do this when their submission is reviewed at the end then this would be a valid reason for rejection or for asking them to return the submission (for the reason ‘The participant did not sufficiently engage in a task where the required level of engagement was clearly specified.’).
