Sapio has been a wonderful partner to work with. They listened closely to our business goals and objectives and created a research design that best fit our needs.
Success story
COVID-19 UK B2B Barometer
Tracking business confidence and helping give UK businesses clarity during this turbulent time.
For many of the research studies we conduct, we source our respondents from online community panels. People sign up to these websites in order to take surveys, and in exchange they receive cash incentives. The vast majority of panel members are genuine and honest, but there are always a few spoilsports that we need to look out for.
We’ve worked with online panels for a number of years and know the ones to trust and the ones to avoid (often learning the hard way!!). There are a number of checks we do to ensure the data our clients receive is accurate and reliable.
Sneaky Speeders – No, not a brand of swimming trunks, I’m talking about people who rush through the survey without reading the questions. We catch these out by using “trap” questions where if respondents contradict themselves or give a suspicious answer, they are removed from the survey. We also look at the amount of time people take to complete the survey, if it’s faster than expected they are also removed.
Straight-lining Sinners – Like their sneaky speeder cousins they look to complete surveys quickly without considering and answering questions properly by putting the same answer for each question (c, c, c etc.). We use software to spot unusual answer patterns and exclude these sinners.
International Ignoramuses – Each online community tends to be country specific. Some people believe they can get away with joining a panel even if they don’t live in the country it is dedicated to. Fortunately, the IP addresses of those that fill in our surveys are checked to ensure that they originate from the correct location.
Reckless Robots – Although many online panels take a number of security precautions, it would be naïve to think that they were impenetrable to hacking. There have been attempts in the past to create software programs that automatically fill in surveys, or trick the system into thinking they have completed the survey. It’s our job, and the panel owners job to be on the lookout for these technologies, and quash them before they become a problem. We check our data thoroughly for suspicious activity. Some of these checks include: looking for duplicate answers, checking respondent ID numbers match on our software and on our panel partners, and fluctuations in response rates, screen out rates, and interview lengths. We even use pictures in surveys as robots can’t tell elephants from mice.
Alter Ego Anonymous – Unsurprisingly it’s harder to get “C” level exec’s to complete surveys than general consumers and consequently we have to pay the harder to reach groups a higher incentive to persuade them to complete surveys. Sadly, there are a small group of unscrupulous people who are quite happy to pretend they’re somebody different to grab the extra cash. We use complex screening questions that make it difficult for respondents to guess who the target audience for the survey is so only genuine respondents can complete the survey.
These checks help us to be confident of the data that we provide to our clients, giving them the most accurate insight into their industry or topic of research. Always ask your research data provider:
- How do they ensure the respondents are who they say they are?
- What security measures do they have in place to pick up malicious software?
- How do they prevent people from going through a survey too quickly or not focusing on the questions asked?
> For more resources on this topic, read our full report Quality in Market Research Panels and our blog In data we trust.
