Bad science in the headlines

I am sure you have all seen the ‘scary’ health and science headlines which pop up on our news feeds everyday- you know the ones- “processed meats increases your chance of getting cancer by 9%“, “people exposed to dirty air are 40% more likely to develop dementia“.

Eeeeek. 

But is it really as bad as the media makes out?

Probably not.

The media is very good for informing us about news, however it can also be really bad. Science is rarely reported well in the media. The problem is not the science per se but how the science in being reported.

Take the story behind the headline “Eating bacon, sausages and other processed meats increases breast cancer risk in older women” reported by The Independent among other newspapers, earlier in the year. The article tells us of how eating processed meats can increase the chance of women developing breast cancer by 9%, with just 9g of meat a week being enough to dramatically increasing cancer.

Holy smokes! 9%… that is a lot. Or is it?

Firstly, the study behind this story was based upon meta-analysis.

What is meta-analysis?

In meta-analysis data from several group studies are pooled together and then analysed. The individual group studies are normally made up of groups of individuals in which health changes and lifestyle factors are followed over a period of time. By themselves these studies can often not provide that much data, but by pooling the results from lots of different cohort studies together, trends between specific factors-  such as a link between processed meats and development of cancer can be investigated.  This type of data analysis is good for looking to see if there are any links between factors, however it is important to note that this does not prove that one factor is directly linked to another- there can be many underlying and unmeasured factors contributing.

Secondly, a 9% increase is not really a 9% increase…

What do I mean by this?

In this study it was found that women who ate processed meats had a increased risk of developing breast cancer by 9% compared to women who did not eat processed meats.

An average woman in the united states has a 12% lifetime risk of developing breast cancer. Eating processed meats means you have a 9% increase in that 12% increase of getting cancer. Overall this means your risk goes up from a 12% risk to 13% risk. Which really is not a lot at all (especially when you consider cigarettes increase cancer risk by a massive 500%!).

How do we keep ourselves from falling for those big headlines?

Here are a few of my tips for spotting bad science:

– Beware the percentages!! Percentages are often not reported well. As in the case above, the percentage increase reported does not really explain the full story.  Yes, eating processed meat does increase your risk by 9%- but it only increases your own risk by 9%.

–Ask “what type of study was it?” Was it based upon a questionnaire sent around a few people? Was it a large scale study? If it was a large study, was it diverse (ages, sexes, backgrounds..)

–Who published it? And what are the sources? Science stories coming direct from a person, or an organisation (especially a profit-based industrial organisation) are often less trustworthy than those based on research in peer-reviewed articles. It can sometimes be hard to access these original papers, but these are often the best sources of information.

–Watch out for hyped up words “Study proves”, “breakthrough research”… these are often red flags for over-hyped science and indicate that you should maybe go into reading a article with a healthy dose of pessimism.

If you are interested in finding out more about the science behind the headlines, the NHS has a great website which tells you everything you need to know about the new stories- from where the study came from, to what the research means, and interpretation of data : https://www.nhs.uk/news/

There are also a lot of great popular science books out there which look into the bad science around, and the underyling mis-representation in the media. I would highly recommend checking out https://www.badscience.net/ run by Dr Ben Goldacre, and his respective book.