Why Do Nutrition Studies Contradict Each Other?

In June 2014, Time magazine put a curl of butter on its cover under two words: Eat Butter. It still gets quoted more than ten years later.

It’s the best-known example of something that happens every few months. One headline says a food is fine after all. The next one says the opposite.

It’s tempting to decide that nutrition science has no idea what it’s doing. The usual explanation is more ordinary: a lot of the contradiction is built into how studies are designed and paid for.

Once you know where to look, you can often spot it in a couple of minutes.

Why Nutrition Studies Contradict Each Other

A lot of the contradiction is settled by the design, before anybody has eaten anything.

Pin

The comparison group: what did they eat instead?

A study that tells people to eat less saturated fat has to ask what they ate instead. In a typical Western diet, the easiest swap is refined starch and added sugar.

Pin

Here’s how much that matters. A Harvard analysis followed more than 127,000 nurses and health professionals for up to 30 years. Replacing 5% of calories from saturated fat with:

• carbohydrates from whole grains went with a 9% lower risk of coronary heart disease
• refined starch and added sugar made no significant difference
• polyunsaturated fat went with a 25% lower risk, though the study didn’t split those fats by where they came from

How we handle fat in a kitchen without oil is in our guide to cooking without oil.

So a study where people cut butter and ate more white bread will find that cutting butter did nothing. It will be right about the white bread.

No low end: you can’t measure what nobody is doing

In a large Swedish study in the city of Malmö, even the fifth of people eating the least saturated fat got about 13% of their calories from it. The top fifth got about 22 to 23%.

So the study can compare 13% with 23%.

Pin

It can’t tell you what happens far below 13%, because almost nobody in it ate that little.

The Oxford Vegetarian Study had a low end built in. It followed about 11,000 people in Britain, from vegans to meat eaters, for 13 years.

The third eating the most saturated animal fat had nearly three times the death rate from heart disease of the third eating the least. The margin of error is wide, from about 1.25 to 6 times. The direction is clear.

The starting point: the same egg, two different answers

The more cholesterol someone already eats, the less any extra shows up in their blood.

Pin

A 1992 review of 27 tightly controlled feeding studies found that curve.

Researchers had tested the low end eight years earlier, with 17 lactovegetarians eating about 97 mg of cholesterol a day.

For three weeks they ate one extra-large egg a day, and for three weeks matching foods without egg, without knowing which was which. On the egg, their LDL cholesterol was 12% higher.

Run the same test on people who eat eggs and bacon every morning and the egg makes much less of a mark. Same egg, different headline.

The dose that was tested: how much did people change?

In one trial, 38 teenagers with overweight or obesity, all on a weight-loss diet, were split into two groups. One group was advised to drink eight cups of water a day. The other wasn’t.

Both groups started at about two cups. After six months:

• the water group was drinking 4.8 cups a day
• the other group was drinking 3.5 cups
• only 1 of the 19 teenagers told to drink eight cups got there

Pin

The extra water made no difference to weight. That’s a fair finding, but it’s a finding about one and a half extra cups a day, not eight.

You only see that in the compliance numbers, and they rarely make the headline.

Who Pays for Nutrition Studies

A drug company pays for a drug trial because it can sell the drug afterwards. Large trials of tap water, lentils or a walk after dinner are rare, because there’s nothing to sell at the end.

So some useful questions never get a big trial. Nobody profits from the answer.

Funding shows up in the conclusions, too. The clearest measurement comes from a 2007 review of 206 articles on milk, juice and soft drinks.

Pin

Across all the article types in that review, the odds of a conclusion favorable to the sponsor were about four to eight times higher when industry paid for all of it. That rested on only 76 articles, so the exact figure is loose.

“Doubt is our product”

Some of the confusion is on purpose, and that part is written down. In 1969, a strategy document at the tobacco company Brown & Williamson said it in one line:

“Doubt is our product since it is the best means of competing with the ‘body of fact’ that exists in the mind of the general public.”

The industry never needed to prove smoking was safe. It only needed people to stay unsure.

In 2013, Dr. Margaret Chan, then head of the World Health Organization, told a conference in Helsinki that the same playbook had spread:

“It is not just Big Tobacco anymore. Public health must also contend with Big Food, Big Soda, and Big Alcohol.”

The “Eat Butter” Cover: How One Nutrition Headline Went Wrong

The butter cover is worth going through once, slowly, because the same moves turn up in a lot of “good news about bad food” headlines.

1. The research didn’t say eat butter. Two pooled analyses, from 2010 and 2014, found no clear link between saturated fat and heart disease. Neither said it was good for you. Only the cover told anyone to eat more of it.
2. The 2014 analysis needed corrections. Within days, other researchers found figures taken wrongly from the original studies. One result, for omega-3 fats, went from not significant to significant. The authors said their main conclusion stood.
3. Who paid. The 2010 analysis was supported in part by the National Dairy Council. The 2014 one was funded by public and charitable bodies, which is worth saying too.
4. The cover dropped what replaced the butter. The 2010 authors flagged it in their abstract, and Time‘s own story blamed the refined carbohydrates that filled the gap. The two words on the cover kept neither.

Finland went the other way

In the late 1960s, Finland had the highest death rate from coronary heart disease in the world. It was worst among working-age men in the east.

From 1972, eastern Finland changed how it ate. Fatty milk gave way to skimmed, people ate more vegetables and vegetable oils, and butter fell sharply.

That’s a before-and-after observation, not a trial. Smoking and blood pressure fell too. In the study’s last ten years, cholesterol, smoking and blood pressure together explained about two-thirds of the drop.

On cholesterol, though, diet did most of the work: over 80% of the fall came from food, not drugs.

Two pooled analyses found no clear signal, and a magazine turned that into an instruction. Meanwhile a region that cut its butter, along with its cigarettes and blood pressure, saw heart deaths fall by four-fifths.

Why a Nutrition Study’s Title and Abstract Aren’t the Whole Study

The title and the abstract, the short summary at the top, are usually the only parts of a paper anyone quotes.

Pin

The authors write both. The less flattering details tend to sit lower down.

A Paleo diet study from 2013 has a title saying the diet “causes strong tissue-specific effects on ectopic fat deposition.” “Causes” is a strong word for a study with no control group.

Ten women ate a Paleo-style diet for five weeks. They ate about a quarter fewer calories than before, which on its own produces weight loss on almost any diet. Their liver fat fell by about half.

A few sentences into the abstract comes the part the title leaves out. On the gold-standard test, their whole-body insulin sensitivity, a core problem in type 2 diabetes, didn’t change.

A green tea trial from 2016 gave 102 women either a high dose of green tea extract or a placebo for 12 weeks. The abstract reports that the green tea group’s weight fell from 76.8 to 75.7 kg.

It doesn’t say how that compared with the placebo group. A loss only means something next to the other group. Without the comparison, you can’t tell whether the extract did anything.

Why Conflicting Nutrition Studies Make People Give Up

There’s a second pull that’s harder to spot in yourself. A large review of how people choose what to read found a moderate preference, on average, for information that backs up what they already believe or do.

So the headline saying your usual breakfast is fine reaches you more easily than the one telling you to change it.

How to Read a Nutrition Study in Five Questions

Pin

You don’t need a medical degree for this. A misleading headline will usually fail at least one of these.

Underneath all five sits one more question: is this one study, or the weight of many?

A single surprising result is a reason to look closer. It isn’t a reason to change what’s on your plate.

Where to Start

Next time a nutrition headline lands in your messages, try the first two questions: who paid for it, and what was the other group eating? Between them they catch a lot of “good news about bad food” stories.

If you’d rather spend the time in the kitchen, our guides to lentils and chickpeas cover the two legumes we cook most.

And, always, seek health consciously.

Read Next

• Lentils: Which Kind to Buy, and How to Cook Each One
• Chickpeas: Canned or Dried, and How to Cook Them
• Cooking Without Oil: What Actually Replaces It
• The Four Numbers That Make Plant Food Filling

Venny is a medical doctor and a specialist in Nutrition and Dietetics; Andy holds a master’s degree in psychology. This article explains how to read nutrition research and is general information rather than medical advice. If a study makes you want to change a medication or treatment, talk to your own doctor first.