Thousands of scientific studies had to toss out weeks of data because of a 56-second TikTok video by a teenager. From a report: The July 23rd video is short and simple. It opens with recent Florida high school graduate and self-described “teen author” Sarah Frank sitting in her bedroom and smiling at the camera. “Welcome to side hustles I recommend trying — part one,” she says in the video, pointing users to the website Prolific.co. “Basically, it’s a bunch of surveys for different amounts of money and different amounts of time.” That video got 4.1 million views in the month after it was posted and sent tens of thousands of new users flooding to the Prolific platform. Prolific, a tool for scientists conducting behavioral research, had no screening tools in place to make sure that it delivered representative population samples to each study. Suddenly, scientists used to getting a wide mix of subjects for their Prolific studies saw their surveys flooded with responses from young women around Frank’s age.
Though not particularly well known, Prolific is part of a small collection of online tools that have transformed the way corporations and scientists study the way people think and act. The first and largest of these research platforms is Amazon-owned Mechanical Turk, which was released in 2005 as a general-purpose platform for crowdsourcing work on repetitive tasks. Soon after it was released, behavioral scientists realized its potential value for their research, and it quickly revolutionized several research fields. […] The Behavioral Lab at Stanford mainly uses the newer, smaller Prolific platform for online studies these days, said Nicholas Hall, director of the Behavioral Lab at the Stanford School of Business. While many Mechanical Turk customers are big businesses conducting corporate research, Prolific gears its product to scientists.
The smaller platform offers more transparency, promises to treat survey participants more ethically, and promises higher-quality research subjects than alternative platforms like Mechanical Turk. Scientists doing this sort of research in the United States generally want a pool of subjects who speak English as a first language, are not too practiced at taking psychological surveys, and together make up a reasonably representative demographic sample of the American population. Prolific, most agreed, did a good job providing high-quality subjects. The sudden change in the platform’s demographics threatened to upend that reputation. In the days and weeks after Frank posted her video, researchers scrambled to figure out what was happening to their studies. A member of the Stanford Behavioral Laboratory posted on a Prolific forum, “we have noticed a huge leap in the number of participants on the platform in the US Pool, from 40k to 80k. Which is great, however, now a lot of our studies have a gender skew where maybe 85% of participants are women. Plus the age has been averaging around 21.”
Read more of this story at Slashdot.