Who is eligible to be in a representative sample of the UK / US?
In order for a participant to be eligible for a representative sample, there must be space remaining in their matching age, sex, or ethnicity subgroup. Additionally, they must be a current resident of the country being sampled and they must be fluent in the language of that country.
Where does Prolific get its census data from?
We use census data from the US Census Bureau and the UK Office of National Statistics.
- The age by sex by ethnic group proportions for the UK can be calculated from 2011 Census data found here.
- The age by sex by ethnic group proportions for the US can be calculated from US Census Bureau population group estimates from 2015, found here.
How does Prolific create the demographic subgroups used for representative samples?
Participant eligibility for representative samples is calculated on the basis of prescreening answers. We use three prescreeners: 'date of birth', 'sex’, and 'ethnicity (simplified)’.
- Starting with the youngest allowable participation age on Prolific, we stratify age using five 9-year brackets: 18-27, 28-37, 38-47, 48-57, and 58+.
- ‘Sex’ is stratified into male and female.
- ‘Ethnicity (simplified)’ is stratified into the five categories recommended by the UK Office of National Statistics: White, Mixed, Asian, Black and Other.
What type of allocation algorithm does Prolific use?
Cross stratifying on age (5 brackets), ethnicity (5 groups) and sex (2 groups) results in 50 subgroups: one for every combination of answers. Using census data, we can calculate the proportion of each subgroup in the national population. In order to ensure minimal representation, we first allocate 1 space per subgroup. We then allocate the remaining spaces in the sample proportionally, according to the national population.
Where we need to round subgroup sizes (i.e. proportionate allocation indicates a subgroup should have 2.6 participants) we try to round to the closest whole number, while balancing round-ups and round-downs to ensure the total sample size remains exactly what was asked for.
How are participants recruited to my representative sample?
Participants take part in a representative sample study in exactly the same way as a normal Prolific study. In other words, a representative sample is collected on a (mostly) first come, first serve basis: though we do have processes in place to ensure fair distribution of studies across the participant pool.
How long does it take to collect a representative sample?
Representative samples tend to take longer to recruit than Standard samples. Representative samples include demographics with smaller population numbers, so it can be hard to guarantee all demographics will be online on a daily basis to take your study. We're always striving to recruit more participants in niche demographics to improve this service for you. Typically, we expect a representative sample study to complete within 2 to 4 days.
What can I do if my sample is slow to fill?
If a representative sample study is still awaiting submissions 48 hours after you have launched it on Prolific, you will have the option to reallocate your sample by loosening the eligibility requirements of each remaining place a little in order to strike a balance between timely data collection and sample representativeness.
If you choose to reallocate your sample, you will not be able to undo this or later increase places on your study.
We find most representative samples are at least 90% full after 48 hours and fill completely after loosening. This normally results in a sample accuracy of 95% or more.
How does reallocating work?
Reallocating removes age bracket restrictions from all unfilled places in the sample.
As an example, if you have 2 unfilled places for 28-37-year-old Asian women in your study 48 hours after publication. These places will normally be opened up to all Asian women in the country, regardless of age.
Additionally, the algorithm will monitor the overall demographic distribution based on sex and ethnicity. This means that sometimes reallocated places may be reassigned to a different sex/ethnicity to give the most accurate representation of the population.
How can I calculate sample accuracy?
You can calculate final representative sample accuracy by summing the total number of subgroup requirements met by participants in the final sample, and dividing it by the total number of requirements that could have possibly been met.
For example: You have a final sample of 1000 participants.
- The total possible number of requirements your sample could meet is 3000. 3 (age, ethnicity, sex) x 1000 (the total sample size)
- 900 places have been filled by participants that met their subgroup’s requirements for age, ethnicity and sex: 900 x 3 = 2700 requirements met.
- 80 participants were collected after one round of reallocation: These participants met the requirements for ethnicity and sex: 80 x 2 = 160 requirements met.
- 20 participants were collected after two rounds of reallocation: These participants met the requirement for sex only: 20 x 1 = 20 requirements met.
- 2700 + 160 + 20 = 2880
- 2880 / 3000 = 96% sample accuracy.