Running experiments with adults

Important things to know

• IRB approval is required before running an experiment. Check with Katie to find out if your experiment can be added to one of our existing IRB protocols. If no, we have to create a new one (Katie and the lab manager will help you).

• All recruitment materials must be IRB approved. Consent forms, ads, flyers, forms, demographic surveys, etc. Make sure you use the exact right materials for your experiment (check with Katie to confirm).

• Always generate experiment links from our database. The database is expecting very specific URL parameters and (as a security feature) will not accept incoming data if they are missing or incorrect. Use the Run experiment links on your experiment's data page.

• Pay people fairly. Our lab pays a minimum of $12 per hour, even on Prolific. For reference, here is an estimate of the current living wage in Philadelphia.

Prolific

We have IRB approval to recruit and run adult participants on prolific and SONA. We prefer prolific; talk to Katie if you need to use SONA instead (e.g. your experiment is outside the scope of our existing grants, requires in-person participation, etc).

What is Prolific

Prolific is a website that connects researchers to participants. You post a link to your experiment, include a brief ad, and provide some eligibility criteria (e.g. native English speakers who are age 18-49). Prolific handles the participant logistics from there.

Posting an experiment

Use the lab account; details shared via Lab Passwords on lastpass

• Step 0: Make super extra sure your experiment code works and contains all lab-required elements. See Programming experiments for more details; briefly, these include (1) the exact right consent+demographics form, (2) the lab's prolific prescreener validation form, and (3) all data (including lab required data) writes to our database

• Step 1: Add a new study. Take care to choose the project that corresponds to the correct IRB protocol number.

• Step 2: Add study details. Some choices are specific to each experiment (e.g. tech requirements), but please use:

  • title "Language Learning Study" for all experiments. Participants see this name and the consistent title helps them avoid participating in our other studies (we also screen for that, but just in case).

  • internal name project/experiment/condition (from our database). For example, speaker-fluency/natnon-spoken-contrastive/native-distinct

  • the IRB approved study description, adding any tech requirements or browser restrictions the participant should know about (there are buttons for this, but also write in description)

• Step 3: Add data collection parameters. Choose external URL, add the link generated by our database, and select "I'll use URL parameters" to capture Prolific IDs via our link.

• Step 4: Select study completion options. Choose redirect URL (you'll need to add this to your experiment code) and submission will be manually reviewed

• Step 5: Select audience. Select (1) participants in the US and (2) a balanced sample. For the number of participants, choose something small (30 or less) just in case there is a bug and we have to do the experiment over again.

• Step 6: Select the lab-required prescreeners (IRB approved, first 7 correspond to our Prolific prescreener validation form):

  • Age: between 18 and 49

  • First Language: English

  • Fluent Languages : Select all

  • Were you raised monolingual? Select all

  • Language related disorders: None

  • Hearing difficulties: No

  • Vision: Yes

  • Exclude participants from previous studies: Select all that use your same paradigm or language

  • Approval Rate: between 85 and 100

• Step 7: Select study cost parameters. Pay people $12 per hour (eg $3 for 15 min) and be generous with time estimates. If the actual time it takes is 15 min, round up to 20. We want to pay people as fairly as possible (not the minimum we are permitted).

Rather than paying participants less, you can help the lab save money by being careful and double-checking (so we don't have to repeat your study). Please make sure your code works, your data writes to the database, and you are collecting the right data for your analysis plan.

Publish your study

• Step 1: Preview before you publish.

Approving submissions

• Step 0: take care to monitor prolific messages and th