Saturated Fat and Heart Disease Meta-Analyses: Were Scientists Wrong?

Was it Ancel Keys or the low carb advocates
who fabricated data on saturated fat?

In a previous review, Clearing up the Confusion Surrounding Saturated Fat, I examined several important flaws and omissions in two meta-analysis studies which are frequently cited as evidence against the hypothesis that dietary saturated fat increases the risk of heart disease. The earlier study was published in 2010 by Siri-Tarino and colleagues in the American Journal of Clinical Nutrition, and the more recent study published by Chowdhury and colleagues in the Annals of Internal Medicine. These studies were both cited in a recent Time magazine article by Bryan Walsh as evidence to "eat butter" for health. In this earlier review I focused primarily on how the studies included in these meta-analyses, as well many other studies actually provide support for the hypothesis that saturated fat increases the risk of fatal forms of heart disease.

Although a number of prominent diet-heart researchers identified many serious flaws and omissions in these meta-analyses that cast doubt on the validity of the researchers conclusions, there are several other critical flaws related to how the estimates were derived for these meta-analyses that have not received adequate attention.1 2 3 4 5 This review will examine how a number of erroneous estimates may have biased the results and primary findings of these meta-analyses. Of particular concern, is in regards to how Siri-Tarino and colleagues derived negative estimates for three studies for which the original peer reviewed publications found positive associations between saturated fat and risk of heart disease. These studies and the estimates published in the meta-analysis by Siri-Tarino and colleagues are described in the table below. 

Study Name                                Risk Ratio (RR)    95% Confidence Interval (95% CI)

Framingham Heart Study6         0.92                       0.68, 1.24

Honolulu Heart Study7               0.86                       0.67, 1.12

Lipid Research Clinics Study8    0.97                       0.80, 1.18

Note: The risk ratio represents high vs low intake of saturated fat in relation to coronary heart disease. An estimate below 1 suggests a benefit of higher intake, and vice-versa. The estimates are considered statistically significant only if both the lower and upper 95% confidence intervals are on either side of 1.

The estimates published in the meta-analysis by Siri-Tarino and colleagues for all three of the abovementioned studies suggest a non-significant trend towards a benefit of saturated fat intake, greatly contrasting the findings from the original peer reviewed publications.9 This warrants an investigation into how Siri-Tarino and colleagues derived these negative estimates.

The Framingham Heart Study

Contrary to other studies included in the meta-analysis, the Framingham study reported estimated risk ratios for low vs high intake, rather than high vs low intake of saturated fat. Among the younger cohort in this study, 10% vs 15% of intake was associated with a 22% decreased risk of coronary heart disease (RR=0.78 [95% CI, 0.61-1.00]).6 Conversely this benefit of low intake was not observed among the older cohort (RR=1.06 [95% CI, 0.86-1.30]). Nevertheless, both of these estimates were controlled for serum cholesterol, which is expected to have biased these findings against showing an adverse effect of saturated fat. 

The tables in the meta-analysis clearly show that rather than converting the risk ratios for low vs high intake to appropriately reflect high vs low intake, Siri-Tarino and colleagues simply reported the estimates for 10% vs 15% of intake as representing 15% vs 10% of intake of saturated fat. The combined estimate for the younger and older groups in this cohort should therefore have been reported as RR=1.09 [95% CI, 0.81-1.47], and not as RR=0.92 [95% CI, 0.68-1.24], which was the estimate reported in the meta-analysis. The estimate provided in the meta-analysis for this study is clearly erroneous and biased the meta-analysis against showing an adverse effect of saturated fat.

The Honolulu Heart Study

In the original peer reviewed publication from the 10 year follow-up of the Honolulu Heart Study, a high intake of saturated fat was associated with a highly statistically significant increased risk of incidence of combined myocardial infarction (heart attack) and coronary heart death.7 This was despite over-adjustment for serum cholesterol. However, when combined with the incidence of the soft end points, angina and coronary insufficiency, the association was only statistically significant in the univariate analysis which did not control for serum cholesterol. The authors of this paper however, warned about misinterpreting the results for these soft end points, asserting:

When interpreting these results about whether the nutrients relate differently to the different manifestations of coronary heart disease, one should keep several problems in mind. The diagnostic certainty of the soft end points (angina pectoris or coronary insufficiency) is much less than that of the hard end points (myocardial infarction or coronary heart disease death).This could result in attenuation of a true relationship.

In the meta-analysis by Siri-Tarino and colleagues, the estimates for the 10 year follow-up of the Honolulu Heart Study were derived not from a peer reviewed publication, but rather from a separate data set obtained from the NHLBI. The primary reason that these estimates provided in the meta-analysis greatly contrasted with that of the peer reviewed publication can likely be explained by difference in the inclusion criteria of the participants. The tables in the meta-analysis show that the data used was based on 1177 cases of coronary heart disease from 8006 participants. However, in the peer reviewed publication there was a total of only 456 cases of coronary heart disease from 7088 participants.  Due to several important factors, the researchers of the original peer reviewed publication excluded 918 of the total 8006 participants from their analysis, asserting:

Men who reported that their intake was atypical for the day covered by the 24 hour recall, or who could not recall their intake, were excluded from the present analyses (n=502). Additionally, all men assessed as being prevalent cases of coronary heart disease (n=301), stroke (n=111), or cancer (n=49) were excluded from the analyses.

It is therefore evident that more than 60% of coronary events included in the meta-analysis for this particular study occurred in participants who had pre-existing disease at study baseline and/or reported unreliable dietary intake. As these additional participants made up less than 12% of the population, yet experienced more than 60% of the total events, this suggests the likelihood that these events occurred primarily among the participants with pre-existing cardiovascular disease, rather than the healthy participants reporting unreliable dietary intake. It should be noted that Siri-Tarino and colleagues clearly stated that one of the inclusion criteria for the meta-analysis was that “study participants were generally healthy adults at study baseline”, suggesting that the data they selected did not meet their own criteria.

When considering the fact that the estimate for this study used in the meta-analysis was negatively associated with coronary heart disease, rather than positively associated as was the case in the peer reviewed publication, this suggests that the participants excluded in the peer reviewed publication who experienced coronary events, largely being those with pre-existing disease, were reporting a lower intake of saturated fat. This is concerning considering the evidence that people who are diagnosed of being at risk of cardiovascular disease tend to limit saturated fat intake post-diagnosis. For example, in the earliest study included in the meta-analysis, participants diagnosed with high cholesterol had a tendency to reduce intake of saturated fat in order to improve risk factors.10 It should therefore not be surprising that participants with established cardiovascular disease likely reported a lower intake of saturated fat in this study, suggesting that a lower reported intake was only a marker of more disease, rather than a cause of it. This phenomenon is known as reverse causality, and is one of the important reasons why participants with pre-existing disease are either excluded, or examined separately from healthy participants in such studies. The inclusion of such data in a meta-analysis could significantly distort the findings. 

It is not entirely clear why a null estimate was reported for this study in the more recent meta-analysis published by Chowdhury and colleagues. However, the sample size described in supplement section of the meta-analysis paper suggests that the estimates were based on the same misleading data used by Siri-Tarino and colleagues, and not the original peer reviewed publication.11

The Lipid Research Clinics Study

In the Lipid Research Clinics study, it was observed that among the younger cohort that a 1% increased intake of energy from saturated fat was associated was a statistically significant 11% increased risk of death from coronary heart disease (RR=1.11 [95% CI, 1.04-1.18]).8 However, similar to the Framingham study, this study also included over-adjustments for serum cholesterol and the adverse effect of a high intake of saturated fat was not apparent among the older cohort (RR=0.96 [95% CI, 0.88-1.05]).

In the meta-analysis by Siri-Tarino and colleagues, the estimate for both the younger and older cohorts combined using the random-effects model was reported as RR=0.97 [95% CI, 0.80-1.18]. It is clear that this estimate is erroneous. When using the random-effects model for two estimates, the estimate with the narrowest confidence intervals should carry more weight, and therefore the combined estimate should have been much closer to the estimate for that of the younger cohort. The estimate for a 1% increase of energy from saturated fat should have been reported as RR=1.04 [95% CI, 0.90-1.19]. Similar to the abovementioned studies, this erroneous estimate biased the meta-analysis against showing an adverse effect of saturated fat.

Reanalysis of the Data

The use of erroneous estimates for several studies included in the meta-analysis by Siri-Tarino and colleagues warrants a reanalysis of the original data. I therefore performed a revised meta-analysis of the same 16 prospective cohort studies included in this meta-analysis. The methods used to derive the estimates have been described previously, and were mostly consistent with those used by Siri-Tarino and colleagues. For the Honolulu Heart Study, I derived the estimates based on only the hard coronary end points due to the concern of the accuracy of the estimates for the soft end points being less certain. As the exact P-value was not available for this publication, and indicated only as being between 0.01 and 0.001, I chose to derive the corresponding standard error using a P-value of 0.01, as this was the most conservative estimate.7

In a meta-analysis based on the 16 studies included in the meta-analysis published by Siri-Tarino and colleagues, dietary saturated fat intake was associated with a statistically significant 16% increased risk of coronary heart disease (Fig. 1).

FIGURE 1. Risk ratios and 95% CIs for fully adjusted random-effects models examining associations between saturated fat intake in relation to incidence of coronary heart disease. ¹Studies that included adjustments for serum or LDL cholesterol. SAT, saturated fat intake.

I also carried out an updated meta-analysis, including prospective cohort studies published up until November 2014 that provided the necessary data to derive risk ratios and the corresponding 95% confidence intervals. If a study was published multiple times, the estimate for the longest period of follow-up was used. A total of 21 studies were included, including recent publications not included in either the meta-analyses by Siri-Tarino and colleagues, and Chowdhury and colleagues.7 8 9 12 13 14 15 16 17 18 19 20 21 22 23 24 25 In a meta-analysis of 21 studies, dietary saturated fat was associated with a statistically significant 15% increased risk of coronary heart disease (Fig. 2).

FIGURE 2. Risk ratios and 95% CIs for fully adjusted random-effects models examining associations between saturated fat intake in relation to incidence of coronary heart disease. ¹Studies that included adjustments for serum or LDL cholesterol. SAT, saturated fat intake.

The findings from these meta-analyses presented here are compatible with the findings from a broad range of evidence described previously. This includes findings from randomized controlled trials showing an adverse effect of saturated fat on blood cholesterol and arterial function, as well as the demonstrated unequivocal causal relationship between diets rich in cholesterol and saturated fat, and the development of atherosclerosis in nonhuman primates, among dozens of other animal species. Furthermore, these findings are supported by numerous ecological studies, including the Seven Countries Study. 

The estimates of this meta-analysis are clearly stronger than that of the Siri-Tarino and Chowdhury meta-analyses, neither of which produced statistically significant estimates. One of the primary reasons for these contrasting estimates was due to the correction of the estimates for several studies described above. These contrasting estimates can also in part be explained by the correction of erroneous estimates for the confidence intervals reported in the Siri-Tarino meta-analysis for the Ireland-Boston Diet Heart Study, and in the Chowdhury meta-analysis for the EPIC-Greece study. It is clear that in the original peer reviewed publications that the estimates in these studies were statistically significant to the 95% confidence level, yet non-statistically significant estimates were reported in the meta-analyses for these studies, biasing against showing an adverse effect of saturated fat.13 20

Another important difference in the meta-analyses presented here, is that the estimates for studies reporting estimates as a 1% increase of energy were transformed to represent a 5% increase of energy from saturated fat. In the Siri-Tarino meta-analysis the researchers either simply reported the estimates for a 1% increase, or transformed the estimates to represent a similarly small increase of energy. Similarly, Chowdhury and colleagues multiplied the estimates from these studies by the power of 2.18, effectively representing the effect of an increase of only a 2.18% increase of energy. The researchers rational for this was that this equation would be expected to show the effect of mean top vs bottom third of intake. While this equation may be suitable for some studies using different scales of measurement, it requires a leap of faith to assume a difference of only 2.18% of energy represents high vs low intake in these studies. Reporting estimates as a 5% increase of energy, as was done here would make differences for high vs low intake much more comparable to that of the other included studies, while still being moderately conservative. As the studies that reported estimates as a 1% increase of energy were more likely to show a positive relationship between saturated fat and coronary heart disease, failing to transform the estimates to represent a sufficient change in intake may result in minimizing the statistical power of these studies, and in-turn biasing a meta-analysis against showing an adverse effect of saturated fat.

Conflicts of Interest

The meta-analyses published by Siri-Tarino and colleagues and Chowdhury and colleagues contained erroneous estimates for several positive studies which in-turn biased against showing an adverse effect of saturated fat. Given the fact that these researchers were well informed in this area of research, it is difficult to accept that they were simply unaware of any of the issues described here. In the meta-analysis by Siri-Tarino and colleagues there was clear evidence of potential conflicts of interest. The meta-analysis was funded in part by the National Dairy Council, and the senior researcher, Ronald Krauss had reported receiving grants from the National Dairy Council, the National Cattleman’s Beef Association and the Robert C. and Veronica Atkins Foundation. Similarly, several of the researchers of the meta-analysis by Chowdhury and colleagues have reported receiving grants from the food industry. Suggestive evidence of these researchers intention to downplay the role of saturated fat on fatal heart disease have also been described previously.

Although receiving grants from a particular industry does not necessarily negate the findings of a study, when errors are made that bias the study results in favor of the concerned industry, the intentions of the researchers should be questioned. The lines of evidence described here lends support to the likelihood that these researchers put their own interests before that of the general public, driving the public to follow dangerous dietary patterns at the hands of fad diet advocates who promote these studies.

Putting Data into Context

It is important to note that the influence that saturated fat has on the risk of disease is not primarily determined by intake per se, but by which foods saturated fat is substituted for. As the intake of dietary fiber was universally low among participants in these studies, this suggests that participants consuming diets lower in saturated fat were substituting saturated fat primarily with lean animal foods and heavily processed foods.26 In addition, as dietary fiber was associated with a decreased risk of coronary heart disease in a number of these studies, the estimates described here therefore could be a significant underestimation of the effect of replacing saturated fat with more healthful, fiber rich foods.26 Furthermore, in the studies included in this meta-analysis, the difference for high vs low intake of saturated fat was relatively low, often only ranging between about 5% and 10% of energy. This suggests that individuals following popular diets which emphasize far greater intakes of saturated fat than recommended levels may be at a much greater risk.

It is also important to note that the effect that a particular food has on the risk of coronary heart disease cannot be fully explained by the amount of one particular nutrient, but rather by multiple nutrients that likely operate together in a complex manner to modify the risk of disease. Therefore, it would be more appropriate to compare the substitution of different foods, rather than focusing entirely on substituting single nutrients. Focusing attention on recommending healthy dietary patterns that are naturally low in saturated fat, while rich in dietary fiber and other beneficial nutrients; primarily, minimally processed, plant-based diets would likely be a more effective measure to improve overall dietary quality, resulting in greater improvements to heart health compared to the more contemporary reductionist approach of focusing on modifying single nutrients. The effectiveness of such a diet was recently demonstrated again by Caldwell Esselstyn in a follow-up of 200 high-risk patients. In this study, coronary artery disease was either arrested or reversed in the great majority of adherent patients, clearly contrasting that of any other peer-reviewed study of similar size.27

This review demonstrates that the conclusions of several meta-analysis studies which suggest that dietary saturated fat unlikely increases the risk of heart disease are misleading, and that the current evidence supports the recommendations to replace foods rich in saturated fat with minimally refined plant based foods. Recommendations based on the findings of these meta-analyses made by the media and low carb advocates to consume more saturated fat rich foods are therefore unsubstantiated and likely dangerous. While it may make an interesting read being told that scientists, such as Ancel Keys have intentionally deceived us into believing that saturated fat-rich foods are unhealthy, it is appears that it may actually be the authors of such articles who lack in the way of honesty. 

Study acronyms: ATBC, Alpha-Tocopherol Beta Carotene Study; BLSA, Baltimore Longitudinal Study of Aging; EPIC-Greece, European Prospective Investigation into Cancer Greece; EUROASPIRE, European Action on Secondary and Primary Prevention through intervention to reduce events; FHS, Framingham Heart Study; GPS, Glostrup Population Study; HHS, Honolulu Heart Study; HLS, Health and Lifestyle Survey; HPFS, Health Professionals' Follow-Up Study; IBDH, Ireland-Boston Diet Heart Study; IIHD, Israeli Ischemic Heart Disease Study; JACC, Japan Collaborative Cohort Study; JPHC, Japan Public Health Center Based Study; KIHD, Kuopio Ischaemic Heart Disease Risk Factor Study; LRC, Lipid Research Clinics; MALMO, Malmo Diet and Cancer Study; NHS, Nurses' Health Study; SHS, Strong Heart Study; WES, Western Electric Study.