Diet-Heart: A Problematic Revisit

In 2010, Jeremiah Stamler published the editorial Diet-heart: a problematic revisit in the American Journal of Clinical Nutrition addressing a number of very serious flaws in a meta-analysis paper supported by the National Dairy Council and authored by Siri-Tarino et al. that concluded that there was insufficient evidence from prospective cohort studies to suggest that the intake of saturated fat increases the risk of coronary heart disease and cardiovascular disease.1 A number of researchers including Stamler, who has played a prominent role in the diet-heart hypothesis for over 60 years found that a number of serious flaws in this meta-analysis would have likely biased the association between saturated fat and coronary heart disease towards null.1 2 3

In the editorial Stamler produced a meta-analysis based on the same papers included in the Siri-Tarino et al. meta-analysis and calculated that saturated fat was associated with a 32% increased risk of fatal coronary heart disease, an end point ignored, perhaps intentionally by the authors of the original meta-analysis.1 It would seem almost implausible for anyone citing the Siri-Tarino et al. meta-analysis with an interest in saving lives to fail to mention the findings for this fatal end point, the single most leading cause of death in the world.4 Perhaps the cholesterol skeptics do not share Stamler’s interest of saving lives, explaining why they have chosen to refrain from informing their audience of Stamler’s findings. 

Another shortcoming of the Siri-Tarino et al. meta-analysis paper was the lack of acknowledgement in the assessments and conclusions that major cross-population studies with a prospective (future looking) design, such as the Seven Countries Study found that saturated fat was associated with a significantly increased risk of fatal coronary heart disease (Fig. 1).1 5 Consistent with the trend of the findings from the Seven Countries Studies, the nomadic Kirghiz plainsmen who subsist on a diet of enormous amounts of organic grass-fed milk and meat experience severe vascular disease at a very young age [reviewed previously].

Figure 1. Saturated fat as % of calories and fatal coronary heart disease in 16 cohorts from the Seven Countries Study

In this series of posts I will review the diet-heart hypothesis and the arguments against the hypothesis raised by known cholesterol skeptics. Note that in this review the diet-heart hypothesis refers to the hypothesis that dietary change, such as the substitution of individual dietary fats for carbohydrate influences serum (blood) lipids (including serum total and LDL cholesterol), and therefore at the very least indirectly influences the risk of developing coronary heart disease.

Stephan Guyenet, the author of the Whole Health Source blog has produced some very informative posts dispelling Gary Taubes’s misleading claims regarding carbohydrate metabolism, insulin and obesity.6 Unfortunately, like Taubes rather than embracing the preponderance of evidence that has established the diet-heart hypothesis, Guyenet has chosen to confuse the subject in a series of blog posts. I have previously commented on Guyenet’s blog regarding one such concerning post in May 2011 where I raised my concerns regarding Guyenet’s arguments against the evidence that saturated fat raises serum cholesterol, and increases the risk of coronary heart disease.7  

One of my main concerns I presented in my comments on Guyenet’s blog was his lack of acknowledgement that saturated fat was associated with an increased risk of fatal coronary heart disease in the Health Professional’s Follow-up Study and in Stamler’s meta-analysis. Guyenet was less than appreciative of these comments, stating:

I find it disturbing that you continue to cite the Health Professionals follow-up study to support your position despite the fact that there was no statistically significant association between SFA intake and any measure of CHD after maximum adjustment. If there were really a relationship between the two factors, you wouldn't have to cite non-significant findings to support your position.

In the paper from the Health Professional’s Follow-up Study cited by Guyenet, for men in the top verses the lowest fifth of saturated fat intake the relative risk for fatal coronary heart disease was 1.72 (95% confidence interval 1.01 to 2.90) after maximum adjustment.8 In other words this study found that saturated fat intake was associated with a statistically significant 72% increased risk of fatal coronary heart disease for high compared to low intake. Guyenet avoided directly responding to my comments regarding the finding in Stamler’s meta-analysis for fatal coronary heart disease, and simply referred back to the Siri-Tarino et al. meta-analysis which failed to address this fatal end point.

In this particular post that I commented on, Guyenet made several misleading statements in reference to the findings from the Siri-Tarino et al. meta-analysis: 

Nearly every high-quality (prospective) observational study ever conducted found that saturated fat intake is not associated with heart attack risk. So if saturated fat increases blood cholesterol, and higher blood cholesterol is associated with an increased risk of having a heart attack, then why don't people who eat more saturated fat have more heart attacks?

Apart from the failure to acknowledge that Stamler demonstrated that the cohort studies included in this meta-analysis found that saturated fat intake was actually associated with an increased risk of fatal coronary heart disease, there are several other points in this statement that are problematic that will be addressed separately.

The Problem of Overadjustment 

Guyenet’s suggestion that the Siri-Tarino et al. meta-analysis should have found a positive association between saturated fat and coronary heart disease if saturated fat raises serum cholesterol and serum cholesterol increases the risk of coronary heart disease is misleading. One of the most serious flaws in this meta-analysis was the inclusion of overadjustments for serum lipids and dietary lipids, which would have obscured this diet-cholesterol-heart relationship that Guyenet referred to. Stamler addressed this flaw in the editorial:1

…the issue of whether SFA relates to CHD in univariate analyses is relevant. If findings on this subject are positive but the association is markedly reduced or ceases in multivariate analyses, this may be due to confounding (eg, by dietary cholesterol) and/or overadjustment (eg, by inclusion in analyses of serum total or LDL cholesterol, a major CHD risk factor influenced by SFA intake)… Of 15 studies that unequivocally concern the SFA-CHD relation, 4 did not include other dietary lipids or serum lipids among covariates. Their CHD relative risks (RRs) ranged from 1.22 to 2.77—ie, >1.07, which was the estimated CHD RR in the meta-analysis. Do these larger RRs reflect freedom from confounding and overadjustment?

As Stamler demonstrated, the studies that that did not include overadjustments for dietary and serum lipids were more likely to find a positive association between saturated fat and coronary heart disease, reaffirming that the influence that saturated fat has on coronary heart disease is partly dependent on serum lipids.

The Problem of Dietary Assessment Methods 

Guyenet’s suggestion that the Siri-Tarino et al. meta-analysis found that the majority of high-quality prospective studies failed to find an association between saturated fat and heart disease is also misleading. Another serious flaw in the meta-analysis was the overreliance on poor quality dietary assessment methods, which was addressed by Katan et al.:2

A major weakness of the meta-analysis is the imprecision of dietary assessment methods used in the underlying studies. About half of the studies used 1-d dietary assessments or some other unvalidated method. Food intake varies from day to day, and there is a substantial literature showing that a single 24-h recall provides a poor estimation of the usual dietary intake of an individual. Such methods cannot reliably rank individuals by their long-term intake, especially within populations with a uniformly high saturated fat intake. Such imprecision in the assessment of disease determinants systematically reduces the strength of association of determinants with the disease. This is referred to as attenuation or regression dilution bias.

Stamler noted that the studies included in the Siri-Tarino et al. meta-analysis that used more precise dietary assessment methods were more likely to find a positive association between saturated fat and coronary heart disease:1

...the meta-analysis reported its findings as independent of a quality score including diet assessment. Of the 16 CHD studies, 4 relied on one 24-h dietary recall; the SFA-CHD RR was >1.00 for only one of these studies. Seven used a food-frequency questionnaire (FFQ); the RR was >1.00 in 3 of these studies. Five used dietary history or multiday food record; the RR was >1.00 in all 5 studies, even though 3 were adjusted for serum or dietary lipids. These facts, which were unnoted in the meta-analysis, prompt the question: Did low-level reliability (reproducibility) of dietary SFA data drive RR values toward 1.00 (the regression-dilution bias problem)? No data on SFA reliability are given

It could actually be concluded from this data that the majority of the cohort studies that used ‘high-quality’ dietary assessment methods, in particular those that did not include overadjustments for dietary and serum lipids found that saturated fat was associated with an increased risk of coronary heart disease.

In the Seven Countries Study dietary intake was measured with high quality assessment methods including a seven day food record and for a subsample of participants the diets were also chemically analyzed.5 Another strength of the Seven Countries Study is that dietary intake was assessed between groups of individuals which has been shown to result in less measurement error than assessing dietary intakes between individuals as was done in the cohort studies included in the Siri-Tarino et al. meta-analysis.9 A further strength of the Seven Countries Study was that saturated fat intake ranged from 3% to about 22% of calories, a far greater range than the studies on mostly homogeneous populations included in the Siri-Tarino et al. meta-analysis, providing greater statistical power to detect a significant relationship. 

The Problem of Dietary Modification

Stamler also addressed the problem related to participants making voluntary dietary changes, including the reduction of saturated fat intake in response to elevated serum cholesterol that could have also obscured the findings of the Siri-Tarnio et al. meta-analysis:1

Also, the meta-analysis says nothing about the problem for the 16 studies of possible bias in SFA-CHD findings due to dietary change (eg, reduced SFA intake) in people with higher serum total cholesterol seeking to lower total cholesterol/CHD risk (as occurred for the earliest of the 16 studies).

Even over 50 years ago in the Chicago Western Electric Company study, the earliest of the studies included in the Siri-Tarino et al. meta-analysis, participants were reducing intake of saturated fat and dietary cholesterol in response to unfavourable serum cholesterol concentrations.10 In studies where participants measured their lipid profile and subsequently lowered saturated fat intake in response to unfavourable results before entering the study, the saturated fat intake of these potentially high risk participants measured during the study could have been significantly lower than their lifetime averages. This could have resulted in an artificial increase in number of coronary events in the groups of participants classified as having a low intake of saturated fat. Similarly, the participants who lowered saturated fat intake in response to unfavourable serum lipids after completing their dietary assessment for the study may have artificially lowered the number of coronary events in the groups of participants classified as having high intake.

Few studies included in the Siri-Tarino et al. meta-analysis adequately addressed this problem, with the Health Professionals Follow-up Study perhaps being one of these few.8 In addition to the problem of imprecise dietary assessment methods, this problem further obscures the classification of the participants ranges of usual saturated fat intake potentially biasing the findings further towards null.11 Another problem that could have potentially obscured the findings in these studies, especially those that lasted into the statin era is that participants with higher serum cholesterol as a result of a high saturated fat intake maybe more likely to have received aggressive medical intervention in order to prevent cardiovascular disease. It should be emphasized here that Stamler found in a meta-analysis that saturated fat was associated with a 32% increased risk of fatal coronary heart disease despite such problems.1

The Problem of the Comparison Group

Guyenet’s suggestion that the Siri-Tarino et al. meta-analysis addressed the ‘association’ between saturated fat and coronary heart disease independent of other caloric sources is also misleading, a point that was addressed by Katan et al.:2

First, the notion that there exists such a thing as “the effect of saturated fat” is flawed. A lower intake of saturated fat implies an increased intake of some other source of calories to maintain caloric balance. Different substitutions for saturated fat have different effects on risk of coronary heart disease (CHD) and need to be discussed separately.

One of the greatest contributors to unnecessary confusion in nutritional research has resulted from studies that failed to compare foods or macronutrients with other suitable sources of energy. The majority of participants studied in developed nations typically consume only negligible amounts of whole plant foods, and therefore a lower intake of one particular food typically results in a higher intake of other processed or animal foods.12 Without giving this important fact careful consideration most foods that are less than optimal for human health will appear harmless in studies as they are typically compared with other unhealthy foods. This problem was elaborated on in a research panel including Ronald Krauss, the senior researcher of the Siri-Tarino et al. meta-analysis:13

For example, it may not be useful, as is usually done, to compare a specific food to all other sources of energy, which are usually mainly refined starches, sugars, red meat, and fat-rich dairy products in typical Western diets.

Hu FB and Sun Q, two of the authors of the Siri-Tarino et al. meta-analysis also addressed this shortcoming of the meta-analysis in a paper they co-authored, describing what sources of energy saturated fat was substituted for:14

…however, in this meta-analysis saturated fat was compared with other calorie sources, primarily refined carbohydrates, and high intake of refined carbohydrates has been associated with a high risk of CHD.

As the Siri-Tarino et al. meta-analysis failed to find a lower risk of saturated fat compared primarily to foods rich in refined carbohydrates even after adjusting for serum lipids, these findings hardly justify increasing the intake of saturated fat any more than they do to increasing the intake of refined carbohydrates. As expected from these findings, meta-analysis and systematic reviews that compare foods to all other sources of energy combined have also failed to find a clear association between refined grains and cardiovascular disease and all-cause mortality, even without the inclusion of such significant overadjustments.15 16 If Guyenet and the other cholesterol skeptics applied the same methodology they use to judge the health properties of saturated fat to all foods, they would not be able to justify their recommendation of limiting intake of refined grains in order to reduce the risk of cardiovascular disease and other non-communicable diseases. 

In Guyenet’s post I commented in regards to a pooled analysis of 11 large prospective cohort studies which found that replacing 5% of energy from saturated fat with an equivalent of polyunsaturated fat was associated with a 26% decreased risk of coronary heart disease mortality.17 Here again Guyenet was less than appreciative of such comments, stating: 

That's not how epidemiology works. What you do is you examine if people who eat more SFA have more heart attacks than people who eat less, while controlling for other variables-- and the studies have nearly all found no association. That's how epi works in other disciplines. Moving the goalposts to Keys score, SFA/PUFA ratios and using fancy math to model nutrient substitutions will only fool people who don't know any better or are desperate to believe that there's an association.

It appears that Guyenet is either desperately trying to confuse his audience or is suggesting that a change in saturated fat intake in the general population will not influence the intake of any other sources of energy for which he has provided no evidence for. Examining saturated fat intake is meaningless without considering what sources of energy it is replacing, which is why models of macronutrients substitution is preferred. The study of nutrition epidemiology is different than the study of other exposures such as tobacco smoke in the respect that energy is required in order to maintain life, and therefore it essential in nutrition science to compare one source of calories with suitable alternatives. 

Additional Findings from Observational Studies

Even if one were to judge the health properties of saturated fat on the basis of findings from prospective cohort studies that compared saturated fat intake with all other sources of energy combined, it would still be implausible to conclude that saturated fat is not disease promoting. The finding from Stamler’s meta-analysis that saturated fat intake was associated with a 32% increased risk of fatal coronary heart disease by itself is a cause for concern, however other findings from cohort studies also raise significant concern. 

A meta-analysis of 12 cohort studies of 418,816 women found that saturated fat intake was associated with an increased risk of breast cancer, consistent with the findings from more recent cohort studies of 319,826 and 188,736 women.18 19 20 In addition, a pooled-analysis of 12 cohort studies of 523,217 women found that a high intake of saturated fat was associated with an increased risk of ovarian cancer.21 Furthermore, a large cohort of 525,473 men and women found that saturated fat intake, especially that from animal sources increased the risk of pancreatic cancer and a cohort of 494,000 men and women found that saturated fat intake was associated with a greatly increased risk of small intentional cancer.22 23 Another cohort study of 137,486 women found that saturated fat intake was associated with an increased risk of hip fracture, consistent with other lines of evidence [reviewed previously].24

Siri-Tarino et al. excluded cohort studies of type II diabetics patients from the meta-analysis which should be addressed. Two such studies found a very strong association between saturated fat and cardiovascular disease, including the Nurses’ Health Study which also found a significant association for dietary cholesterol and the Keys score.25 26 Furthermore, although typically considered lower in the hierarchy of evidence than prospective cohort studies, a number of case-control studies have also found a positive association between saturated fat and coronary heart disease.27 28 29 30

In regards to the association between saturated fat intake and the risk of stroke, the Siri-Tarino et al. meta-analysis failed to address the possible influence that blood pressure has on the association between saturated fat and the risk of stroke despite finds from large cohort studies including the Nurses’ Health Study that suggest the association is dependent on blood pressure. Without consideration of these important details the Siri-Tarino et al. meta-analysis should not be considered as providing a clear interpretation of the association between saturated fat intake and the risk of stroke. I have addressed this matter in further detail in Part I and Part II of a review addressing blood pressure, blood cholesterol, diet and the risk of stroke, which also addresses the Northern Manhattan Study which found that saturated fat was associated with a trend towards an increased risk of ischemic stroke that was excluded from the Siri-Tarino et al. meta-analysis despite apparently meeting the requirements for the inclusion criteria.31

The Problem of Reductionism

The disease promoting properties of saturated animal fat cannot be ascribed purely to the substitution of saturated fat for other macronutrients, but also to other nutritional factors including the content of dietary cholesterol, ruminant trans-fat and the lack of dietary fiber and other phytonutrients. For example, a study on an apparently health conscious population included in the Siri-Tarino meta-analysis found that while saturated fat was associated with 2.77 fold increased risk of coronary heart disease which was the value used in the meta-analysis, the association for animal fat was even stronger, a 3.29 fold increased risk.32

In the pooled analysis of 11 large cohort studies, compared to saturated fat, monounsaturated fats which was predominantly derived from animal fat was associated with the greatest increased risk of coronary events out of all the studied macronutrients. Furthermore this pooled analysis adjusted for dietary fiber, dietary cholesterol and possibly ruminant trans-fat, which also needs to be taken into consideration as this could have potentially underestimated the adverse effects of increasing saturated animal fat intake at the expense of whole plant foods.33

As foods contain not only macronutrients but also tens of thousands of different bioactive constituents which can potentially influence health, it would therefore be more informative to compare the effect of substituting different foods rather than isolated macronutrients on disease outcomes.34 35 Arguably the highest quality prospective cohort study to have published a paper addressing the substitution of foods on the risk of coronary heart disease was the Nurses’ Health Study (Fig. 2).14 As suggested by this study, the benefits of replacing animal foods with whole plant foods to lower the risk of coronary heart disease can be explained partly but not entirely by the displacement of saturated fat with other macronutrients.

Figure 2. Coronary heart disease associated with replacement of a major dietary protein source with another in the Nurses' Health Study

Another paper that also addressed the substitution of foods on the risk of coronary heart disease was from the Iowa Women’s Health Study, which found that substituting foods rich in refined carbohydrates with dairy was associated with a increased the risk of fatal coronary heart disease and substitution with red meat was associated with a increased risk of both fatal coronary heart disease and all-cause mortality.36 These findings raise significant doubt towards the cholesterol skeptics claims that certain animal foods appear disease promoting in studies only because they act as a marker of refined food intake. This study actually found that dairy and red meat are disease promoting even when compared to foods rich in refined carbohydrates. 

The Problem of Conflicts of Interests 

It is well documented that the conclusions of studies that receive industry funding, including from the dairy, soda and tobacco industries are far more likely to bias in favor of the invested industry than studies without apparent industry funding.37 The Siri-Tarino et al. meta-analysis was funded by the National Dairy Council and the senior researcher,  Ronald Krauss has reported receiving grants from the National Dairy Council, the National Cattleman’s Beef Association and the Robert C. and Veronica Atkins Foundation. Although such conflicts of interests do not necessarily prove that the meta-analysis is flawed, it does at the very least suggest however that the author’s lack of acknowledgement of the positive association between saturated fat and fatal coronary heart disease and of the very serious flaws in the meta-analysis may have been intentional. 

Like diet, it is notoriously difficult to accurately measure environmental tobacco smoke exposure which has obscured the findings for passive smoking and smoking related diseases in observational studies. In a similar fashion as the dairy industry has done to downplay and distort the relationship between saturated fat and cardiovascular disease, the tobacco industry has taken advantage of measurement error in order to scrutinize the association between passive smoking and lung cancer in part due to the fact that the majority of observational studies failed to find a statistically significant association.38 However, it is clear that when all of the evidence is considered there is convincing evidence that passive smoking increases the risk of lung cancer, just as the substitution of whole plant foods with saturated animal fat increases the risk cardiovascular disease.39

In 2003, tobacco affiliated researchers Enstrom and Kobat published findings from a 39 year follow-up of a prospective cohort study in the British Medical Journal and concluded that exposure to environmental tobacco smoke does not likely significantly influence the development of lung cancer and coronary heart disease. This paper received a lot of attention from the mass media, including the Wall Street Journal, and was used by the tobacco industry to criticize government sponsored ‘junk science’.40

This study was criticized by a number of researchers and by the American Cancer Society which addressed a number of the very serious flaws in the study.41 42 The prominent flaw that was emphasized was the lack of a suitable comparison group. The analysis only took into account whether never smokers who had a smoking spouse were more likely to develop lung cancer and coronary heart disease compared to never smokers without a smoking spouse, and did not account other forms of environmental tobacco smoke. This was an issue because in 1959 when the participants were enrolled there was tobacco smoke virtually everywhere leaving no group unexposed. Furthermore this study only measured the spouses smoking status at study baseline and did not account for whether the spouse quit smoking, ended the marriage or died during the follow-up period. 

To summarize some of the shortcomings of this tobacco industry influenced study, it suffered from a lack of suitable comparison group, lack of  high quality assessment methods to precisely measure exposure, and the lack of assessment of changes to exposure during the follow-up period. These shortcomings remarkably resemble those of the Siri-Tarino et al. meta-analysis. This paper has even been cited in a lawsuit against tobacco companies by the US District Court as ‘a prime example of how nine tobacco companies engaged in criminal racketeering and fraud to hide the dangers of tobacco smoke.’43 

This was unfortunately not the last time that the researchers of an industry influenced study would publish a paper that has the potential to jeopardize the health of so many. What is also unfortunate is that many cholesterol skeptics have also chosen to exploit these findings in an attempt to advocate disease promoting diets to an uninformed audience. Follow-up posts in this series will critically examine other lines of evidence of the diet-heart hypothesis that cholesterol skeptics have chosen to misinterpret and exploit in an attempt to confuse the general population. 

Diet-Heart Posts

Part II - Diet-Heart: Saturated Fat and Blood Cholesterol

Part III - Diet-Heart: The Role of Vegetarian Diets in the Hypothesis
Part IV - Cracking Down on Eggs and Cholesterol
Part V - Cracking Down on Eggs and Cholesterol: Part II


Please post any comments in the Discussion Thread.