I always put several attention check questions in my surveys, that say:
“This is an attention check, please select strongly agree” or strongly disagree, or whatever is the middle value of a 5 or 7 point scale.
Today I ran a survey and got a pattern I have never seen before. There was only 1 wrong response to the attention check questions, but 20% of my respondents left all three blank. What’s going on? Has anyone seen that? Technically it’s a fail on the attention checks because they didn’t follow the instructions. But could it be a bot? i.e. it’s not discerning enough to parse the item and find the correct response so it just leaves those blank.
I have never seen that. Participants almost always get IMCs right, and never, in my experience, leave them out (that said, mine are compulsory for submission).
Could it be you only had numbers, 1 2 3 4 5, in your scale, not “strongly agree” (this would still not explain the omission of the “middle” ones).
If you PM me the survey link I’ll take a look. I could make my age 150 so you no not to use my data.
The odd thing is that except for one other incorrect response on one of the attention checks, everyone else got them correct. I double checked because I thought the fault might be on my side, as you suggested with some disconnect between the instructions and response scale, but everything is in order.
However, good point about the forced response. I have a set of diligence checks that I will make forced response, and also the ID checks, but typically I leave everything else as requested. So at least they would get a reminder.
TO me the weird thing is the pattern - every single one of them skipped the attention checks (which direct them to select a specific response) and the diligence checks (which ask them about the quality of their data, e.g. ‘I gave the survey my complete attention’). I also have an item asking them yes / no should I use their data, with the information that they will still get paid if they select no. And they all selected yes.
So here’s a strange update - I collected data for the remainder of the sample, an additional 250 respondents. Not a single instance of the blank attention or diligence checks. This just seems non-random