Failing attention check question

Hi, if some respondents have failed the attention question, do they get paid?, how do I get that research money back? Thanks Ps: It is my first time running an styudy w/ Prolific

Dear Arancha

As far as I know you should contact the participants, explain that they failed the attention check , ask them to “Return their submission,” to ensure that they don’t have some valid excuse such as they failed to understand that it was an attention check.

For studies longer than 5 minutes, participants should fail more than one (two or more) attention checks before they are requested to return their submission.

If they do not respond, nor have a valid excuse, and have failed the one check in a less than 5 minute study or two or more in a longer study, then you can click on the Reject button on the survey results screen. Bear in mind that you can only reject a certain percentage of participants (5%? 8%? something like that) without special dispensation from Support which is unlikely to be forthcoming since the higher percentage of rejections may indicate that the survey was in some way specially difficult to understand.

All as I far as I know. I generally accept everyone and then throw some data away.

I hope that is not disheartening. I used to be more upset but these days I just accept it as part of doing surveys online. Not everyone is going to be paying attention all of the time.


Timothy Takemoto
Yamaguchi University