Request Return/Reject for Qualtrics Duplicate/Fraud Alerts

⛑ Community Support
1

I’ve noticed others are integrating Qualtrics with Prolific for their surveys. Qualtrics has built in fraud and duplicate response detection. Can researchers use these as grounds for rejection? They are detectable prior to completing the survey, should participants be asked to return the HIT?

Slightly related, can we build our survey to be inaccessible in incognito mode? And ask participants to return the HIT if they are uncomfortable completing it outside of incognito mode?

2

Dear Matthew

Welcome to the forum.

I am afraid that the short answer to your questions is I think that you would need to ask support.

Qualtric’s fraud and duplicate response detection sounds helpful but, first of all, I have heard that Prolific uses some sort of algorithm to detect duplicate response fraud. I am not sure how it works and how it could detect VPNs (that might be able to provide false IP addresses) but I believe Prolific does (partly because my mother was rejected from the Participant program due to the fact that she happened to be using a VPN for another purpose at the time), or if Prolific’s fraud protection system ever delivers false positives. What if there are two Prolific participants in the same household using the same IP address?

However, bearing in mind that Prolific has fraud protection of its own in place, I doubt that they would wish researchers to double down on this functionality and reject participants on this basis but, you would need to ask Prolific support to make sure.

With regard to the second question, there is first of all a policy that one should not screen participants in surveys, nor even in the descriptions of surveys. So if you asked participants to return their submission when they refuse to use incognito mode, this should be (according to Prolific policy) only a request, since we are not allowed to screen in studies. However, if you were to

  1. persuade Prolific to create a new screener asking whether participants would be willing to forgo incognito mode. Now that Prolific has “retreated” from these forums, I am not sure what happened to the regular request for new screeners that were added at a rate of one or two a month at the sub forum below
    Vote For New Screeners
  2. Create a preliminary screening survey of your own only asking the question “Are you willing to forgo the use of incognito mode”
    then you would I think be free to use a survey that is inaccessible in incognito mode.

By the way, once when I was doing an open ended survey I found that when I reordered responses by Prolific ID I found that there were a couple (of about 50?) responses that were very similar using adjacent Prolific IDs (Name a person you admire 1 My dog 2 My cat).

Again, all in my humble and non-Prolific affiliated opinion (IMNPAO) :slight_smile:

Tim

3

Thank you for the tips Tim! Very suspicious about the adjacent IDs… I will definitely reach out to support. Any idea how responsive they are?

4

Hi Matthew
I think that Prolific support are overworked :slight_smile:
Tim

5

Re. question 1, Prolific has a very narrow list of acceptable criteria for rejecting participants: Approvals, rejections & returns – Prolific

So no, you couldn’t reject participants because of their results on Qualtrics in-built fraud detection variables. Keep in mind that the participants know what the list of acceptable reasons is, so if you step outside it they will call you out on it.

1 Like