Missing data

⛑ Community Support 🕷️ Bug Reports
1

Hi all! I’ve been using Prolific in combination with Pavlovia to collect data for my online experiments (experiments written in JavaScript using jsPsych). One problem consistently occurs - the number of submissions as shown on the Prolific study page doesn’t match the number of data files appearing in my Pavlovia Gitlab repository. For example, Prolific tells me that 10 participants have completed my study, but I only get 7 data files on Pavlovia.

Of course, I still pay the participants for their time spent, but I don’t get any data from them. This missing data issue is costing me a lot of money. Have people experienced the same problem and any idea how to solve it? Thank you!

2

Hello Hellen,

Are you sure that this gap is not due to Returned or Timed-out submissions? If this is the case, you don’t have to pay these participants.

If instead it is a bug due to Pavlovia, I’m afraid I cannot be of more help - I have never used it for my studies!

Veronica

3

Thanks, Veronica! The missing data are from participants with successful submissions and the correct completion code (not Returned or Timed-out). I messaged some of them asking if anything went wrong in the experiment, but they all said everything went fine.
I’m not sure if it’s a problem with Pavlovia or Prolific or the way the two interact. On the Pavlovia user forum I’ve seen several people reporting missing data issues when using Pavlovia together with Prolific. Don’t think anyone on the Pavlovia forum has been able to identify or solve the problem, so I’m posting here to see if any luck from this side.

2 Likes
4

Hey @Hellen_Jing_Yuan,

I’m just checking with the team to see if there have been similar reports, and/or if there’s a potential fix.

In the meantime, we’ve listed a number of reasons why data might be missing. One of them could be relevant to your situation :slight_smile:

1 Like
5

Hey @Hellen_Jing_Yuan ,

We’ve investigated, and there’s not a known, widespread bug that we’re aware of. We have had a researcher report that, if participants reload the survey on Pavlovia, something goes wrong, so perhaps that’s the issue?

In general, we only ever expect you to pay for data you receive, so if you are recording participant IDs on Pavlovia, you should be able to see who you have a complete data set for and approve them on Prolific.

For any missing data, get in touch with the participant and ask them whether anything went wrong, or whether they reloaded the survey or opened it more than once.

It might also be worth getting in touch with Pavlovia customer support to see if their data is stored somewhere not obvious, perhaps in cases of incomplete submissions. Some survey sites will store this elsewhere. Some survey sites will also not record responses unless a final button is clicked, but hopefully that’s not the case here! :slight_smile:

1 Like
6

Thank you very much, @Josh!

1 Like