Introduction to integrations
If you haven't already checked out our basic study set-up guide, you should do so before exploring this advanced guide. Gorilla also has an extensive support centre that provides all of the information required to help you set up your study on their platform, including specific advice about how to recruit participants using Prolific. We encourage all of our researchers to review the information provided there before setting up their first study.
Getting consent
It is important to get consent to participate in your research from all of your participants (see here for what to include). Participants who do not consent to take part in your study should be redirected to a separate 'Reject Node', which asks them to return their submission so their study place becomes available again.
To setup your consent flow:
-
Review the Gorilla tutorial covering how to setup a consent form here: Gorilla consent form tutorial
-
Insert a 'branch node' after the consent form
-
This node should redirect any participant that does not give their consent to a 'reject node' via a 'questionnaire node'. This can be achieved by using 'embedded data' to store participants responses to the consent question (to see how to do this please go here)
-
The questionnaire node should use the 'rich text' option to say something like:
'As you do not wish to participate in this study, please click the button below to close this survey and then return your submission on Prolific by selecting the 'Stop without completing' button.'
-
All other participants (i.e., those who do consent) should continue along the other branch of your experiment.
Your survey design should look something like this upon completion of this step:
Recording participant IDs
You need to record participant IDs within your survey, so you can match your survey responses to individual Prolific submissions. Gorilla make it easy to record Prolific IDs in your experiment, as they can be recorded automatically via the URL.
To set up automatic ID recording:
-
Open your experiment on Gorilla and go to the 'Recruitment' tab
-
Click "Change Recruitment Policy" > "Recruitment Service" > choose "Prolific" > click "OK"
-
Read the information provided, then click 'Use This Policy' and Gorilla will provide you with your unique study URL that has been pre-filled with the required parameters:
-
Use this unique URL as your Study URL on Prolific, by copy & pasting this URL into the 'Study Link' section on Prolific. The option 'I'll use URL parameters' should be selected automatically and the parameter names will adjust automatically (the Prolific ID parameter must be named 'external_id', as below, in order for the recording of IDs to work correctly):
Validating your prescreeners
We recommend that you validate your prescreeners by asking the prescreening question(s) again in your survey. Any validation question(s) should be at the very beginning of your study and must be worded exactly as appears on Prolific.
To setup your validation flow:
-
Add a questionnaire node to your survey named ‘Screener Validation’, and add your prescreening questions to this node. Note this question should follow directly from the 'Did consent' branch.
-
Insert a 'branch node' after the Screener Validation questionnaire node
-
This node should redirect any participant that failed any one of the screener validation questions to a 'reject node' via a 'questionnaire node'. This can be achieved by using 'embedded data' to store participants responses to the screener questions (to see how to do this please go here)
-
The questionnaire node before the reject node should use the 'rich text' option to say something like:
'You are ineligible for this study as you have provided information which is inconsistent with your Prolific prescreening responses. Please click the 'next' button below to close the survey and then return your submission on Prolific by selecting the 'Stop without completing' button.'
-
All other participants (i.e., those who have provided consistent information with their Prolific prescreeners) should continue along the other branch of your experiment to your main experiment/survey.
Your survey design should look something like this upon completion of this step (note that this setup is for 2 screener validation questions):
Redirecting participants back to Prolific upon completion
Once a participant has completed your study they should be redirected back to Prolific to register their completion of your survey. There are two main methods of redirecting participants back to Prolific:
1) Automatic redirecting participants via a URL [Recommended]
Gorilla allows researchers to automatically redirect participants to a specific URL upon completion of the survey.
-
On Prolific, select 'I'll redirect them using a URL' in the STUDY COMPLETION' section of your study setup. Prolific then provides you with a redirect URL to insert into your Gorilla experiment
-
On Gorilla, create a new questionnaire node at the end of your experiment design
-
This node should use the 'rich text' option to say something like:
'Thank you for taking part in this survey, please click 'Next' below to submit your responses and be redirected back to Prolific'
-
Add a 'finish node' straight after this message, and enter your completion URL as the Onward URL (leave the rest of the option on their default settings)
Your survey design should look something like this upon completion of this step:
IMPORTANT: Participants should only be redirected to your completion URL if you wish to include them in your data and pay them the study reward. Any participants who you do not want to complete your study (e.g. those who do not give consent, or match the prescreening you’ve selected) should be redirected to a reject node using branches, and asked to return their submission as described above.
2) Giving them a completion code to copy and paste
-
On Prolific, select 'I'll give them the Completion Code to copy & paste' in the STUDY COMPLETION' section of your study setup. Prolific then gives you a code to provide to participants:
-
On Gorilla, create a new questionnaire node at the end of your survey
-
This node should use the 'rich text' option to say something like:
'Thank you for taking part in this survey, please enter the code below on Prolific to register your submission then click "Next" below to submit your responses on Gorilla'
-
Add a 'finish node' straight after this message - no customisation of this node is required.
Please note that providing the completion code in this manner may lead to participants not clicking 'Next' to submit their data on Gorilla, it is therefore critical to make it clear to participants that they must click this final button. Due to this possibility, we strongly recommend that you use automatic redirecting instead.
Previewing your study
To preview your study on Gorilla, and check that this flow is working exactly as you expect it to, check that all of the nodes are up to date, click 'Commit Version X' at the top-right of the screen and that, and then click 'Preview Experiment'.
You can also preview the study on Prolific by following the steps here: Previewing your study. This will allow you to check that your study link is correct and it is redirecting to the correct Completion URL at the end.
Please note that any changes you make to your survey on Gorilla will not be visible via your Prolific preview until you 'commit' them. You can read more about Survey Publishing on Gorilla by clicking here.
Gorilla Integration FAQs
- On Prolific, any participants who decide to leave your study early without completing (returned submissions), or who time-out after becoming inactive (timed-out submissions) are automatically excluded from your total number of submissions, and their spaces are opened up to new participants.
- However, Gorilla does not automatically reject these returned or timed-out submissions, so they will take up one of your participant tokens until you manually reject them. Unfortunately a loop can be created in these circumstances where all of the tokens are used up, but there are a few places left on Prolific. Participants continue to enter the study on Prolific and get returned due to the lack of tokens on Gorilla, then opening their place up on Prolific again. This will continue until either (a) a participant submits their study as completed on Prolific, (b) you manually stop the study, or (c) you manually reject the problem participant on Gorilla.
- One way to avoid this issue is to set a time limit for your experiment. This will mean that any participants who go over the time limit will be automatically rejected on Gorilla, and the space will be available again for a new Prolific participant. Similarly to the maximum allowed time on Prolific, this time limit should give participants plenty of time to complete all tasks in your study, and should be intended to reject inactive participants. As such, it should not be too close to your study’s estimated completion time, to avoid rejecting a diligent participant who has taken slightly longer to complete than on average.
- To minimise problems with returned submissions, you should follow Gorilla’s guidance to use Checkpoint Nodes and regularly check your data as it’s coming in to manually reject any incomplete submissions.
- Setting your recruitment target to Unlimited will avoid this problem altogether, but your license may not allow you to do so. Alternatively, setting the recruitment target to greater than your number of Maximum Submissions on Prolific will also help to account for a few returned/timed out participants taking up a space on Gorilla (e.g. 200 on Prolific/250 on Gorilla). This would require you to buy more participant tokens in advance, but you wouldn’t have to use up the additional tokens because you can reject the incomplete submissions at the end.
- Participants experiencing a server outage during their participation in your study on Gorilla is a rare occurrence, but it is still an important consideration. If a participant makes it 80% of the way through your study, but a server outage occurs meaning they cannot continue, you will not have access to their data on Gorilla until you manually include them by expending a participant token.
- Our policy when technical issues occur on the side of the survey software is that participants should be at least partially compensated to reflect the time they have invested into your study, or otherwise approved as normal. Therefore, to avoid having to compensate for potentially incomplete or missing data, here are some points to consider:
- Run your study in batches which are not too large. For example, if you were recruiting 200 participants for a long study (60 mins+), we would recommend running this in batches of ~20 to avoid too many participants being affected by a possible server outage. Gorilla recommend running studies 'in small enough batches that you can afford to lose every participant that is currently active'.
- Include checkpoint nodes at various stages in your experiment. This way, you’ll be able to see how far participants were able to get through your study before any server outage occurred, without having to expend a participant token and inspect the data manually. The ‘Current Node’ column on your participants tab will also be informative here.
- If a server outage does occur, and you wish to inspect the data of your participants without wanting to expend the participant tokens, consider contacting Gorilla’s support team with a list of the participant IDs that were affected. Should they find evidence of any technical issues that occurred, they’ll be happy to help you out.
-
If you’re running a study that requires participants to be assigned to one of multiple conditions (e.g. a treatment group vs. a control), you can program this into your Gorilla experiment by using multiple Start Nodes. This will generate N Unique URLs, where N is your number of conditions. In this example, I have two paths in my experiment - one for my treatment group, and one for controls:
-
You can then flexibly manage how many participants you wish to recruit for each condition by publishing separate studies on Prolific, one condition at a time (i.e., you should complete recruitment of a single condition before beginning recruitment of the next).
-
Once you’ve collected the data for one group, you can prevent these participants from taking part in the next study by adding a custom blocklist or previous studies filter to your prescreening.
- You can run longitudinal studies on Prolific by setting up multiple studies, linked together using a custom allowlist containing the Prolific IDs of the participants from the first stage. On Gorilla, you can build all stages of your study into one experiment, so you can use the same Study URL for each part.
- To do this, add a Redirect Node at the point in your experiment that you want each stage to end. Your redirect URL should be your Prolific study’s completion URL, so that participants are redirected back to Prolific at the end of each stage. If you have a specific time period set before you will invite participants to the next stage, you can change the Completion setting to ‘Delay’ and specify this time here. Otherwise, the default setting is fine.
- When a participant reaches the end of each stage in your experiment, the Redirect Node will record their active Prolific submissions as Awaiting Review. This will provide your list of IDs to include in the custom allowlist for the next Prolific study. The next time participants access your Gorilla experiment will be when you publish the subsequent stage on Prolific - here, they’ll pick up straight from where they left off in the next Node of your experiment tree.