As a novel start-up in a poorly-researched space, we needed to validate the problems Whirli is solving. Sapio Research were fantastic partners from start to finish – creative in brainstorming angles, careful in designing the questions, and rigorous in analysing the results. The research brought a wealth of insights, backed by hard figures, for our business decision making and for us to talk about publicly in the press.
Success story
Iptor
Supply Chain KPI’s – Which really deliver?
The Market Research Society (MRS) has teamed up with CIPR and the Royal Statistical Society (RSS) to provide a comprehensive guide for those working in the PR industry on how to use statistics in communications. For the full guide, follow the link here.
We have provided a short summary of the guide as a quick go-to read to give you an idea of what it’s all about.
Statistics are notoriously easy to manipulate
Full fact is the UK’s independent fact checking agency. They spend their time checking out what people are publishing and saying, often with a focus on political speeches. Full Fact then publish their findings so that the public can easily find out whether the claims that have been made are fully substantiated, or if they have been taken out of context.
But, statistics are also incredibly powerful
Think about an argument or research without statistics, compared to one with. Being able to say “80% of UK adults prefer Cadbury’s to Galaxy” has much more impact than “Cadbury’s is generally preferred to Galaxy”. Statistics have the ability to substantiate (or refute) claims, influence thinking and help understanding.
How to make sure audiences know your claims are reliable
It’s important when using statistics that the audience is able to make an informed decision about how reliable they are. Thus, we need to include certain pieces of information for them to be able to do this:
- Client who commissioned the research
- Objectives of the research
- Sample size
- Audience being represented (e.g. UK adults, cat owners etc.)
- How data was collected (e.g. online, by phone, face-to-face)
- What data collection method was used (e.g. questionnaire, discussion guide etc.)
- When data was collected (fieldwork period)
- Sample demographics (e.g. age, region, gender etc.)
- Whether data is weighted
- Survey results, including margin of error and statistical reliability of the results
Common pitfalls to avoid
As mentioned above, statistics can easily be misunderstood or misrepresented, and not necessarily intentionally. Below are some of the pitfalls that the MRS suggest you look out for:
- Averages – make sure you know which average has been reported (mean, median or mode) and how these differ. Some are better suited for certain measurements than others.
- Sample size – if you have a sample of 1,000 UK adults then you have a robust sample to represent this group. However, if you then start breaking it down into age bands, regions, genders etc. you need to be careful – 100 is the suggested minimum for a subset, although for some samples 50 may be sufficient. Even so, it is important to make sure any results and comparisons made are statistically significant.
- Leading questions – make sure your questionnaire or discussion guide doesn’t include any leading questions. People can be more inclined to agree, or to be biased by the wording of the question. For example a leading question might be “do you agree that cats are better than dogs?”. A better way to ask this (if a closed question) is “Which animal do you think is better at XX? Cats, Dogs, Neither”.
- Calculating trend data – say last year 40% of people said that they preferred cats to dogs, and this year 48% of people said that they preferred cats to dogs. This is an increase of 8 percentage points, not an increase of 8% (the actual increase is 20%).
- Correlation vs. causation – just because more people now prefer cats to last year, and more people now drink Coke to last year does not mean that drinking Coke causes people to prefer cats.
At Sapio Research we have senior, experienced researchers. We can offer advice and expertise in survey design, statistical analysis and sample size. We also offer a press release check on all projects. For more information please contact us on team@sapioresearch.com
