Hi, I have read through the guide (How do I recruit a custom sample? – Prolific) for recruiting a custom sample by using two surveys: one that is a set of screener questions and the second that is only available to those who fit the profile relevant to your study. I’m looking for any additional tips from researchers who have done this set-up before I start my own. For example:
Do you have an informed consent form for both surveys or only the second one (the actual research study)?
Do you have a rule of thumb for how many participants to open the screener questions to (e.g., x times the number you want to offer your main study to)?
I assume that the description of the first study should state that it is a screener for a larger study and that not everyone who completes it will be eligible for the second study (but will still get paid for completing the screener). Anything else that should be included here? Or, do you have a sample screener that you’d be willing to share?
Anything else that is important to keep in mind or that you wish you had known before doing your first study like this?
Hi @Jessica , I haven’t done this before. About the consent forms though, I would start by talking with your local IRB. Each one might have different policies, so it would be good to check with them first.
If you search through the newbie forum, you could find some users that have posted about these procedures and tag them here! It would be nice to have a thread with suggestions and best practices for everyone to find easily when they arrive at the forum.
I’ve done this and had success with it. I had a separate information sheet and consent form for both the screening survey and the main study.
You will have to over-recruit with your screening survey but by how much depends entirely on how many people pass your screening and how appealing/easy/etc your main study is. I recommend doing a test run one day and seeing what % of your screened people actually complete your main study and go from there.
One thing I would highly recommend is running the screening survey immediately before the real study. I have an R script that reads in my qualtrics screening survey data and outputs a list of eligible Prolific IDs straight away, and so I then invite people into the main study within an hour or so of them completing the screening survey. I find that success at filling your main study reduces quickly if you leave a gap between the screening and main study (even by a day or two) as many people will complete the screening survey and then go about their lives and miss the follow up invitation or no longer be available. My main study was quite heavy going though, so that may have been a factor!
Hi @Jade_Pickering,
Any chance that you would be willing to share the information sheet and consent form you used for your screening survey? I’m trying to get a feel for how much information one should include.
Hi! I agree with most of the advice I got to my original post - it’s probably best to check with your local IRB about when to do the informed consent / whether you need both of them. I’ve had different results in terms of how many people from the screener end up completing the main study, so I think that one you need to play by ear a little bit. I have had about twice as many people do the screener as I needed for the main study and sometimes that’s more than enough and sometimes it’s barely enough. I haven’t done the R script that @Jade_Pickering described but I do try to sort through the responses to the screener as quickly as possible (24-48 hours) to give eligible participants the main study while they may still be thinking about it.
Though in neither does Prolific mention Jade’s good suggestion about keeping the time between prescreening study and study proper as short as possible.
Some of the suggestions I made regarding prescreening for Zoom interviewees may be a little relevant.
I have been having a huge problem with the custom sample plan - Our Part 1 (screener question) got too large to download on my end some time in July. Prolific had to download it for me and email it to me. Since July 19, however, they have not been able to download it either. We ended up starting a new screener study in order to keep moving on data collection, but there are several weeks worth of data that no one has been able to access (from 7/19/23 through about 8/10/23) - all those participants have not yet been invited to Part 2 of the study, and I am seeing here that someone recommends sending Part 2 within an hour if possible!
Prolific had initially been responsive to my messages on this, though still not able to provide a solution, but lately the communication has fallen off. Does anyone have tips for reaching a high level person in the organization who can handle a complaint like this? I have started requesting a refund for that data as well, but it is so hard to reach an actual person to assist with issues.
