Once a participant has completed your study, it is up to you as the researcher to review their data and decide if that participant should be approved and paid, rejected and penalised, or returned.
At Prolific we have very specific criteria for the acceptance and rejection of participant submissions.
This article will tell you how to make the decision on which participants to approve and which to reject, how to action these decisions within Prolific, and how to adjust previously made decisions on submissions.
How do I decide who to approve, reject, or ask to return?
You have a maximum of 21 days to review submissions. If you haven't reviewed submissions after 21 days, our system will automatically approve all submissions that are awaiting review.
After you have checked your participants’ submissions, you should pay them as soon as possible. Ideally, you would reward them within 24-48 hours after they have completed your study.
Although we ask that you action submissions as soon as possible, please do not rush this process. Ensure that you are happy with your decision and have read our guidelines before taking action.
Who should I approve?
Any participant who has completed your study and has provided you with data should be approved and paid unless they meet any of the rejection criteria listed below.
Who should I reject?
On Prolific, you have the option to reject participant submissions. However, please think carefully about whether the rejection is justified, and try to keep rejections to a minimum. Participants are penalised for rejections and if they receive too many they will be removed from the platform entirely. Please review all submissions holistically and remember that participants have spent time and effort taking your study.
|Valid reasons for rejection|
|Invalid reasons for rejection|
- If you are concerned that a rejected submission might have been made by a bot, then please let us know immediately.
- Prolific reserves the right to overturn invalid rejections in certain circumstances.
- If you have any questions or concerns about the above, you can contact our Support Team using the button at the bottom of this page.
Who should I ask to return their submission?
You may request that a participant returns their submission if:
- They only completed part of your study (and didn't finish it)
- They encountered technical problems (in which case, please consider awarding partial payments)
- They withdrew their consent (in which case please also delete any data that you have from that participant in your external software)
- You feel they should be given the opportunity to return their submission for another reason, rather than be penalised with a rejection. Remember that rejections should only be used in cases where a participant has been clearly negligent.
Participants are able to voluntarily return submissions and will often do so upon request if it's demonstrated that a mistake was made. Researchers cannot return submissions themselves.
You can ask a participant to return their awaiting review submission by clicking the blue and white arrow next to a their submission:
After you select a reason for requesting the return, a message will automatically be sent to the participant asking them to return their submission.
Please note that this button will only be activated if your study status is ‘active’ or ‘awaiting review’. It’s not possible to use this button on ‘paused’ studies. Instead, you can message the participants directly and ask them to return their submission.
The participant can return submissions from their 'Submissions page', by selecting the red circular arrow to 'Return + cancel reward':
Please allow the participant at least 7 days to respond to your request. If the participant does not respond to your request within 7 days then please contact the Researcher Support Team using the button at the bottom of this article to request the returns.
How do I make an approval or a rejection from the submissions page?
There are two methods of approving submissions on Prolific:
You can manually approve each submission in your list by clicking the green check mark on the right-hand side of the page:
Alternatively, you can select the check box to the left of the participant's ID and then click 'Approve selected' at the bottom of the list (note that this can be done for multiple participants at the same time):
Approve in bulk
You can approve submissions from specific participant IDs by using the 'Approve in bulk' feature. To do this select 'Bulk actions' > 'Approve in bulk':
On the resulting pop-up, simply enter the IDs of the participants you want to approve separated by a comma, and then click 'Approve':
To reject a participants submission simply click the red cross on the right-hand side of the page:
You will then be asked to explain why the rejection is being made:
We have provided a number of common rejection reasons in the dropdown menu for you to choose from, or you are welcome to submit your own reason by selecting 'Other' and providing detail in the message. Once you select a reason a preformatted message will appear that will be sent to the participant.
Please note that there is no option to bulk reject submissions, each rejection must be made one at a time.
All studies on Prolific have a limit on the number of submissions researchers can reject by default. This is because a rejection negatively impacts a participant's Prolific Score, which could prevent them from taking part in further studies on our platform.
We typically find that the automatic rejection limit is sufficient, but if you reach the maximum number of rejections for your study and require more, you will need to get in touch with the Support Team using the button at the bottom of this article to request a review of this limit.
It is important to remember that we are a platform that serves both researchers and participants, so we will need to make sure that your rejection reasons are fair and in line with the above guidelines. Therefore, when you submit your request, we will ask for the following information:
- Name of study (including internal name, if relevant)
- Study questions, the related concerning responses, and the relevant participant IDs (any format is fine)
- Any additional information that might be useful to assist our review (e.g. context about your study). If you are submitting a rejection limit increase request due to failed attention check items, please include a screenshot of the attention check(s) in question.
Please note, we won't be able to increase your rejection limit until we have this information. By providing this, you are helping us to make sure everything is fair for everyone on Prolific!
'Unreject' the submission
The submission will then show as 'awaiting review.'
Either approve the submission, or send the participant a message to let them know they can 'return' it (when a submission is returned, that spot opens back up in your study so that another participant can take part and you don't miss out on getting all the data you need)
If you don't hear back from participants after 7 days, our Support team can assist you with returning submissions.
- You will only be able to reverse a rejection if you have enough funds in your account to approve the submission. If necessary, you can top-up with additional funds, then request a refund.
- If you 'unreject' a submission after 21 days, the submission status would return to 'awaiting review'. It would then be automatically approved within 24 hours, or you can approve it manually before then.
- If you require the participant to return their submission after 21 days, please contact us using the button below for assistance. This is to avoid the submission automatically approved before the participant has had a chance to return it.