Skip to main content

Approvals, rejections & returns

Once a participant has completed your study, it is up to you as the researcher to review their data and decide if that participant should be approved and paid, rejected and penalised, or returned.

At Prolific we have very specific criteria for the acceptance and rejection of participant submissions.

This article will tell you how to make the decision on which participants to approve and which to reject, how to action these decisions within Prolific, and how to adjust previously made decisions on submissions.

How do I decide who to approve, reject, or ask to return?


After you have checked your participants’ submissions, you should pay them as soon as possible. Ideally, you would reward them within 24-48 hours after they have completed your study. You have a maximum of 21 days to review submissions. If you haven't reviewed submissions after 21 days, our system will automatically approve all unreviewed submissions.

Although we ask that you action submissions as soon as possible, please do not rush this process. Ensure that you are happy with your decision and have read our guidelines before taking action.

Who should I approve?

Our rule of thumb

Any participant who has completed your study and has provided you with data should be approved and paid unless they meet any of the rejection criteria listed below.

Who should I reject?


On Prolific, you have the option to reject participant submissions. However, please think carefully about whether the rejection is justified, and try to keep rejections to a minimum. Participants are penalised for rejections and if they receive too many they will be removed from the platform entirely. Please review all submissions holistically and remember that participants have spent time and effort taking your study.

 Valid reasons for rejection  Invalid reasons for rejection
 Valid reasons for rejection
  • The participant completed your study exceptionally fast

    • By 'exceptionally fast' we mean participants who are statistical outliers (3 standard deviations below the mean)

    • This is the only criteria under which speed of submission can be used to reject
  • The participant skipped crucial questions, i.e. questions that are critical to answering your specific research question.

  • These questions must not have been shown as optional for the participants 

    • Please note that demographic questions do not fall under 'crucial questions' because participants have the right to withhold such sensitive information (according to guidelines by the British Psychological Society)
  • The participant failed one (for studies under 5 minutes) or multiple (for studies 5 minutes or longer) fair attention checks

  • Please refer to our attention check policy for further guidance
  • The participant objectively demonstrated clear low-effort throughout the experiment, even when the required level of engagement was clearly specified

    • For example, the participant was asked to write a paragraph but only put a few words or gibberish
Please also remember that:
  • If you are concerned that a rejected submission might have been made by a bot, then please let us know immediately.
  • Prolific reserves the right to overturn invalid rejections in certain circumstances.

Who should I ask to return their submission?


You may request that a participant returns their submission if:

  • They only completed part of your study (and didn't finish it)
  • They encountered technical problems (in which case, please consider awarding partial payments)
  • They withdrew their consent (in which case please also delete any data that you have from that participant in your external software)
  • You feel they should be given the opportunity to return their submission for another reason, rather than be penalised with a rejection. Remember that rejections should only be used in cases where a participant has been clearly negligent.

Participants are able to voluntarily return submissions and will often do so upon request if it's demonstrated that a mistake was made. Researchers cannot return submissions themselves.

To ask a participant to return their submission, please send them a message explaining the reason for the request, and asking them to return the submission on their dashboard. The participant can return submissions from their 'Submissions page', by selecting the red circular arrow to 'Return + cancel reward':

returning.jpg

Please allow the participant at least 5 days to respond to your request. If the participant does not respond to your request within 5 days then please contact the Researcher Support Team using the button at the bottom of this article to request the returns.

How do I make an approval or a rejection from the submissions page?


Approvals

There are three methods of approving submissions on Prolific:

  1. Manual approval

    You can manually approve each submission in your list by clicking the green check mark on the right-hand side of the page:

    manual_approval.jpg

    Alternatively, you can select the check box to the left of the participant's ID and then click 'Approve' at the top of the list (note that this can be done for multiple participants at the same time):

    manual_approval_2.jpg

  2. Approve by upload

    You can approve submissions from specific participant IDs by using the 'Approve by upload feature. To do this select 'More' > 'Approve by Upload':

    by_upload.jpg

    On the resulting pop-up, simply enter the IDs of the participants you want to approve separated by a comma, and then click 'Approve':

    upload_list.JPG.jpg

  3. Approve all

    You can also approve all submissions that have the status 'Awaiting review' by clicking the 'Approve all' button at the top of the submissions list:

    approve_all.jpg

Rejections


To reject a participants submission simply click the red cross on the right-hand side of the page:

reject.jpg

You will then be asked to explain why the rejection is being made:

reject_reason.JPG.jpg

We have provided a number of common rejection reasons in the dropdown menu for you to choose from, or you are welcome to submit your own reason by selecting 'Other' and providing detail in the message. Once you select a reason a preformatted message will appear that will be sent to the participant.

Please note that there is no option to bulk reject submissions, each rejection must be made one at a time.

FAQs

I've reached my rejection limit - what should I do?

All studies on Prolific have a limit on the number of submissions researchers can reject by default. This is because a rejection negatively impacts a participant's Prolific Score, which could prevent them from taking part in further studies on our platform.

We typically find that the automatic rejection limit is sufficient, but if you reach the maximum number of rejections for your study and require more, you will need to get in touch with the Support Team using the button at the bottom of this article to request a review of this limit.

It is important to remember that we are a platform that serves both researchers and participants, so we will need to make sure that your rejection reasons are fair and in line with the above guidelines. Therefore, when you submit your request, we will ask for the following information:

  • Name of study (including internal name, if relevant)
  • Study questions, the related concerning responses, and the relevant participant IDs (any format is fine)
  • Any additional information that might be useful to assist our review (e.g. context about your study). If you are submitting a rejection limit increase request due to failed attention check items, please include a screenshot of the attention check(s) in question.

Please note, we won't be able to increase your rejection limit until we have this information. By providing this, you are helping us to make sure everything is fair for everyone on Prolific!

I've mistakenly rejected a submission instead of approving it or asking the participant to return it. What should I do?
  1. 'Unreject' the submission

    reject_-_unreject__2_.jpg

  2. The submission will then show as 'awaiting review.'

  3. Either approve the submission, or send the participant a message to let them know they can 'return' it (when a submission is returned, that spot opens back up in your study so that another participant can take part and you don't miss out on getting all the data you need)

    message.jpg

  4. If you don't hear back from participants after 5 days, our Support team can assist you with returning submissions.

Please note:

  • You will only be able to reverse a rejection if you have enough funds in your account to approve the submission. If necessary, you can top-up with additional funds, then request a refund.
  • If you 'unreject' a submission after 21 days, the submission status would return to 'awaiting review'. It would then be automatically approved within 24 hours, or you can approve it manually before then.
  • If you require the participant to return their submission after 21 days, please contact us using the button below for assistance. This is to avoid the submission automatically approved before the participant has had a chance to return it.

I need further help

 Click here to contact us

Was this article helpful?
powered by Typeform