Good science, bad interpretation

In 2012, the Journal of the American Medical Association (JAMA) published a study entitled Effects of Dietary Composition on Energy Expenditure During Weight-Loss Maintenance. While I’m guessing most readers have not read this study, I’m pretty sure most of you have heard about the results as it was all over the news this week.

I was fortunate enough to read an embargoed copy two weeks prior to publication with the caveat that I could not speak about it until it was released publicly. Furthermore, I’m friends with one of the reviewers who told me months prior that “a very interesting paper was going to hit a high–impact factor journal very soon.”  Completing my disclosure, I had become acquainted with the senior investigator on this study, Dr. David Ludwig at Harvard.

This study sought to test an important question:

When an overweight or obese person loses weight, how does their choice of macronutrients impact their tendency to regain lost weight?

This is important, of course, because as most of us know that while losing weight is difficult, keeping it off is even more difficult.  In fact, as the authors point out, only about 15% of people who lose 10% of their bodyweight can maintain the weight loss for up to one year.  The obvious question is why?

You’ll recall from this post, that we must always obey the First Law of Thermodynamics.  In other words, we accumulate stored energy (e.g., fat mass) when we are in a positive energy balance and we lose stored energy when we are in a negative energy balance.

Energy balance is a function of two variables:

1. Energy input – what we eat
2. Energy output – what we expend

Furthermore, energy output can be broken down into four sub-components:

1. Resting energy expenditure (REE) – the amount of energy expended to stay alive at rest (e.g., energy required for basic cellular function like ion transport and respiration)
2. Thermic effect of food (TEF) – the amount of energy required to process and digest food (I also include in this category the amount of energy lost as undigested material in stool)
3. Activity energy expenditure (AEE) – the amount of energy expended by exercise and non-exercise movement (I consider these as two forms of expenditure)

The sum of REE, TEF, and AEE is called, appropriately, total energy expenditure (TEE).

Of these, REE is the dominant “sink” of energy output in most people, and it is generally proportional to bodyweight.  I’ll cover the importance of this momentarily.

The traditional model of obesity, the so called “calories-in-calories-out” model, says that obesity is caused by the energy input terms exceeding the energy output terms.  In the words of one prominent obesity researcher, “While it is mathematically true that someone who has gained weight has consumed more energy than they have expended, using the First Law to explain why someone gains weight is of little help.  The First Law is descriptive but not explanative.”

I couldn’t have said that better myself.  The mistake most folks make when using the First Law to explain weight gain (versus using the First Law to describe weight gain) is that they lose sight of the fact that these variables – input, REE, TEF, AEE – are linked.  They are dependent on each other.  They don’t exist in isolation.

Proponents of the Alternative Hypothesis argue that intake (i.e., food) plays a role on hormones and enzymes in the body that have a resulting impact on energy output, and even subsequent input.  For example, eating one food over another can increase or decrease appetite, increase or decrease REE, increase or decrease AEE, and even impact TEF.  While the effect on each of these may be modest in isolation, even small changes over the course of days can result in significant changes over months or years.

What does all of this have to do with this study?

The figure below shows how the study was conducted.  This was a prospective design 3-way crossover study of 21 overweight or obese subjects with an average BMI of 34.4.  Each subject underwent a 20-week run-in phase, which is very common in weight-reduced studies.  During the run-in phase all baseline measurements are collected, including body composition by DEXA, TEE by doubly-labeled water, substrate utilization by respiratory quotient, and plasma levels of various blood markers (e.g., lipids, blood chemistry, hormone levels).  No, unfortunately, lipoprotein particles were not counted.

During the 12 week weight loss phase caloric intake was reduced until each subject lost 12.5% of their starting (stable) weight.  For the final 4 weeks of the run-in phase energy intake was again calibrated to hold their now-reduced-weight stable.

 

The figure below summarizes the data from Table 1 of the paper, showing the breakdown of macronutrients during the run-in phase and the subsequent 3 dietary interventions, each lasting 4 weeks.  Again, each subject did each diet for 4 weeks due to the 3-way cross-over design.  In other words, each subject spent a total of 32 weeks in the study (20 weeks of run-in and 3 x 4 weeks of each intervention diet).

 

How did the diets impact energy expenditure?

The figure below shows the change in REE and TEE measured for all groups.  There was no difference in total physical activity or exercise, so presumably there was no appreciable change in AEE.  I could not find a mention of TEF, suggesting it was not measured.   These figures are a bit ugly, but they convey helpful information.  Each dot represents an individual subject and the lines joining each dot allow you to see the change for each subject across the 3 diets.  The blue box shows the mean change (middle of the box) with the 95% confidence interval above and below.  The height of the box is therefore two standard deviations.

 

 

A few things stand out from these results:

1. The group consuming a very low carbohydrate diet had a higher REE and TEE than the low GI group, which had a higher REE and TEE than the low fat group. In other words, the fewer carbohydrates in the diet, the higher the resting and overall expenditure.  This is actually the sine qua non of the alternative hypothesis: something beyond the actual number of calories is playing a role in how the body expends energy.
2. As expected, given that each subject was starting from a weight-reduced state, the REE was lower for each group, relative to their baseline.  REE is highly (though clearly not entirely) dependent on body mass.
3. There is enormous variation between subjects by diet type.  For example, at least one subject saw a dramatic increase in TEE on the low GI diet versus the other two, while another saw the greatest TEE on the low fat diet.  This speaks to a theme I iterate on this blog: be willing to self-experiment until you find what works for you. 

How did the diets impact metabolic parameters?

The table below shows the changes in hormone levels and metabolic syndrome biomarkers.

One thing that really jumped out at me was that it is quite likely that not one of the subjects in the study met the formal criteria for metabolic syndrome.  MS requires at least 3 of the 5 parameters (blood pressure, waist girth, fasting glucose, HDL-C, and TG) exceed threshold.  The thresholds are as follows:

1. BP > 140/90 [No subject met this at baseline]
2. Waist girth > 40 inches (men), >35 inches [Not reported, but let’s assume at least some subjects met this]
3. Fasting glucose > 100 mg/dL [Not reported, but let’s assume at least some subjects met this]
4. Fasting TG > 150 mg/dL [No subject met this at baseline]
5. HDL-C < 40 mg/dL (men), < 50 mg/dL [No male subject met this, but it’s possible some female subjects did]

This may speak to the age of the subjects, which averaged 30.3 years, but I would have expected a worse set of baseline metabolic parameters. It also speaks to the point that just because someone is obese doesn’t mean they have metabolic syndrome and vice versa.

 

Tables are a bit cumbersome, so I took the liberty of graphing some of these results, mostly because I just can’t resist playing with think-cell (PowerPoint without think-cell is simply a tool for torturing people.)

I’ve explained p-values before, but let me explain the two types of p-values reported above and below.

P_overall is the p-value testing the hypothesis that the mean outcome of the three diets was equal.  The smaller this value, the more likely the differences were not due to chance.  As a general rule, if the p-value is greater than 0.05 we say the difference is “not significant.”  Most use a more stringent requirement of 0.01 to hit the mark of statistical significance.

P_trend is the p-value testing the hypothesis that the mean outcome of the three diets showed a trend from low fat to low GI to low carbohydrate.

 

Not surprisingly, the low fat group experienced a significant reduction in HDL-C.  It’s been documented many times that dietary fat raises HDL-C and dietary carbohydrates reduce HDL-C.

Each group also experienced a reduction in triglyceride (TG) level.  Since we know carbohydrates, not fat, raise TG, you may wonder why this was even the case in the low fat group, which actually increased carbohydrate intake.  I suspect it was a carbohydrate “quality” issue.  I’m guessing the baseline levels reflect more sugar consumption than the low fat phase.  Nevertheless, and again not surprisingly, the high fat-low carbohydrate group experienced the greatest improvement in plasma TG levels.

 

 

Insulin sensitivity was measured according to a protocol in this paper.  The protocol uses time blood draws after an oral glucose challenge.  The higher the index, the greater is the insulin sensitivity.  Each diet improved both hepatic and peripheral insulin sensitivity and both the overall differences and the trends were significant.

If insulin is the most important hormone regulating fat metabolism and accumulation, leptin is certainly a close cousin.  Leptin is a hormone secreted by fat cells that plays an important role in regulating appetite and some metabolic functions.  High levels of circulating leptin can be suggestive of leptin resistance which, like insulin resistance, tends to be a marker for metabolic derangement.  I’ll write a lot about leptin in subsequent posts.   While leptin sensitivity was not directly measured as insulin sensitivity was, the significant reduction in circulating leptin levels suggested it was also improved in all groups, though greatest in the low carbohydrate group.

How did the diets impact inflammation?

Two markers for inflammation were evaluated in this study, C-reactive protein (CRP) and plasminogen activator inhibitor-1 (PAI-1).  Neither is particularly sensitive in the way, say, Lp-PLA2 is (this was discussed in the cholesterol series). Nevertheless, they give us some indication of how much overall inflammation exists in the body.

 

Each group experienced a significant decline in both PAI-1 and CRP, and there was no significant difference between the groups for either marker.  However, the trend was (barely) significant favoring the low carbohydrate group for PAI-1 and favoring the low GI group for CRP.  Sorry low fat, you didn’t win either.

I know what you’re thinking because it was the first thought that ran through my mind when I saw this table:  What?  Is this meaningful or is it an example of statistical chicanery?  I’ll let you decide, but I’m pretty sure I know the answer.

Because I know some of you will ask, I will comment in a subsequent post on the changes seen in TSH, T3, and cortisol.  These topics deserve a post of their own.

What should have been taken away from this study?

This study reproduced a number of results which have been noted for decades:

1. Low carbohydrate, high fat diets reduce TG and raise HDL-C more than other diets.
2. Low carbohydrate, high fat diets improve insulin sensitivity more than other diets.

It never hurts to hammer those findings home again, but the really dramatic finding of the study was the impact of macronutrient balance on REE and TEE.   At previous count (circa 2011), 81 studies over the past 80 years involving 4,094 subjects for more than 1.2 million subject-days have attempted to ask this question – many of them attempting to “prove” that all calories are created equally.  While none (i.e., not one) have refuted the alternative hypothesis, most of them had enough methodologic limitations that it was difficult to know for certain if the type of food – rather than the number of calories – was playing an important role.

This study, while still limited (e.g., short duration, small sample size), makes one of the more compelling cases that all calories are not created equally.

What was taken away from this study?

The embargo on this paper was lifted at 4:00 pm EDT on Tuesday, June 26, 2012.  Within about 30 minutes I was being bombarded with news stories that, if you hadn’t read the study, as virtually no one actually does, would suggest that the low carbohydrate diet was the “worst” of the three diets tested.  This was not universally true, in fairness to the media, but there was no shortage of this sort of commentary:

USA Today

…the authors note a downside to the low-carb diet: it appears to raise some risk factors for heart disease.

Marion Nestle, a nutrition professor at New York University, says longer studies conducted among people in their own environments, not with such controlled meals, have shown “little difference in weight loss and maintenance between one kind of diet and another.”

George Bray, an obesity researcher at Pennington Biomedical Research Center in Baton Rouge who has also studied this topic and who wrote the accompanying editorial in JAMA, says that other studies “show that you can do well on any diet as long as you stick to it. Adherence is the major key for weight loss and maintenance. There is no magic in any diet.”

The New York Times

…the low-carb diet “also had marked problems. It raised levels of CRP (c-reactive protein), which is a measure of chronic inflammation, and cortisol, a hormone that mediates stress.”

The Wall Street Journal

…the low-carb diet had the biggest boost in total energy expenditure, burning about 300 calories more per day than those on the low-fat diet — about the same as an hour of moderate exercise. But that bump came at a cost: increases in cortisol, a stress hormone, and a measure of inflammation called CRP, which can raise the risk of developing heart disease and diabetes.

Some of these comments were patently false (e.g., “the low carbohydrate diet raised levels of CRP” according to the New York Times), reflecting utter incompetence, but most of them commit a different journalistic sin: They miss the forest while machinating on one leaf.

Tragically, most people (unfortunately this includes physicians, dietitians, and politicians) have neither the time nor scientific discipline to wade through these studies and understand their implications.  Instead, they rely on “reputable” journalists to translate for them.