I just ran a study and I have 10 participants who show complete on Prolific and successfully provided a completion code, but they are nowhere to be found in my dataset. Mind you, I collected ~1200 participants and this has only happened with 10. I required all participants to fill out their PID # so they could not have advanced the study unless that question was filled out. These participants did not fill that question out (and thus did not fill out the rest of the survey). Does anyone know how this could have happened? What are possible scenarios here? Do participants communicate completion codes outside of the survey?
Thanks,
Logan
Hi Logan
Hmm. I think it is unlikely that the 10 participants got their completion code from another participant without doing the survey. In order to get to the page where they input the completion code they would have had to successfully reserve their place in the survey, input a Prolific ID, but ignored the survey, and waited for a friend or sock puppet, to inform them of the completion code. It is conceivable, but I would say unlikely because if any participant wanted to get the payment fraudulently it would be simpler, and less detectable, to answer pretty much at random except for the attention checks which they could likewise be informed of by their friend or sock-puppet.
I guess if they faced some sort of technical difficulty or timed out or something they might have asked a friend.
Are there duplicate Prolific IDs in your dataset? (That might indicate the use of a sock-puppet, which Prolific checks for vigilantly by the way. I know of a participant that happened to be using a VPN to watch Netflix or something and somehow Prolific detected their use of a VPN, and ejected them from the participant pool.).
Could it be that 10 of 12000 participants have input their Prolific ID incorrectly? Do the Prolific IDs match up or are there Prolific IDs in your dataset that are not shown on Prolific?
Can you contact the participants who have submitted but have not submitted data and ask them if they know why their data is not appearing.
Is there anyway for them to have obtained the completion code without pressing “submit data”? I have seen surveys were researchers have given the completion code on the penultimate, submit button page rather than on the page shown after submit (but if so then I think you would be missing more data).
I am sorry I can’t think of more scenarios.
Tim
There were no duplicates. They did not input it incorrectly because I have accounted for all PIDs that were inputted into the dataset. There also aren’t any in reverse (in the data but not in prolific). I contacted them and hopefully will see if there is a common theme. I’ll let you know if I find anything out. I did include the completion code on the last page, not in the thank you for participating message after advancing from the final page. From what you say, it seems that it is more common to put the completion code after the final page in the thank you message. I’ll adopt this practice next time.
Logan
I guess that alas the participants saw completion code, returned to prolific, and did not press submit, but shut down their browser 
From my experience as a participant it is more common to put the completion code in the message displayed after submit. I as a researcher this is what I do since it prevents participants from submitting completion codes at Prolific without having submitted their data.
However, I see that prolific is not giving this advice in the Google Forms integration guide
Instead they suggest just " making it very clear on your final page that participants need to click 'Submit ’ for their data to be recorded."
I have asked Prolific to update the integration guide.
And if you have already done this, Prolific suggests " you can ask the participant to return their submission or to repeat the study."
(Personally I think that it is a bit harsh since there is a way of making sure that the participants submit the study. But if you used Google forms, you are in luck.)
Tim