Followup to: Beware of Other-Optimizing

Once upon a time, Seth Roberts (a professor of psychology at Berkeley, on the editorial board of Nutrition) noticed that he'd started losing weight while on vacation in Europe. For no apparent reason, he'd stopped wanting to eat.

Some time later, The Shangri-La Diet swept... the econoblogosphere, anyway. People including some respectable economists tried it, found that it actually seemed to work, and told their friends.

The Shangri-La Diet is unfortunately named - I would have called it "the set-point diet". And even worse, the actual procedure sounds like the wackiest fad diet imaginable:

Just drink two tablespoons of extra-light olive oil early in the morning... don't eat anything else for at least an hour afterward... and in a few days it will no longer take willpower to eat less; you'll feel so full all the time, you'll have to remind yourself to eat.

Why? I'm tempted to say "No one knows" just to see what kind of comments would show up, but that would be cheating. Roberts does have a theory motivating the diet, an elegant combination of pieces individually backed by previous experiments:

Your metabolism has a set point, like the setting on a thermostat: when your weight is below the set point, you feel hungry; when your weight is above the set point, you feel full.
But the set point is not a constant; it is raised and lowered by what you eat.
This mechanism in turn seems to be regulated by a flavor-calorie association. (Possibly as a famine-storage mechanism that tries to store more resources when dense food sources are available.) If you eat something with flavor X, which is followed by your metabolism detecting a large source of calories, flavor X will (a) seem more appealing and taste better, and (b) will raise your set point whenever you eat items with flavor X.
Your set point is always naturally dropping, but is raised by eating; usually these forces are in dynamic balance and your weight stays constant.

I'm not going to go into all the existing evidence that backs up each step of this theory, but the theory is very beautiful and elegant. The actual Shangri-La Diet is painfully simple by comparison: consume nearly tasteless extra-light olive oil, being careful not to associate it with any flavors before or after, to raise your body weight a little without raising your set point. Your body weight goes above your set point, and you stop feeling hungry. Then you eat less... and your weight drops... and your set point drops a little less than that... but then next morning it's time for your next dose of extra-light olive oil, which once again puts your (decreased) weight a bit above the set point. The regular dose of almost flavorless calories tilts the dynamic balance downward. That's the theory.

Many people, including some trustworthy econblogger types, have reported losing 1-2 pounds/week by implementing the actual actions of the Shangri-La Diet, up to 30 pounds or even more in some cases. Without expending willpower.

I tried it. It didn't work for me.

Now here's the frustrating thing: The Shangri-La Diet does not contain an obvious exception for Eliezer Yudkowsky. On the theory as stated, it should just work. But I am not the only person who reports trying this diet (and a couple of variations that Roberts recommended) without anything happening, except possibly some weight gain due to the added calories.

And here's the more frustrating thing: Roberts's explanation felt right. It's one of those insights that you grasp and then so much else snaps into place.

It explained that frustrating experience I'd often had, wherein I would try a new food and it would fill me up for a whole day - and then, as I kept on eating this amazing food in an effort to keep my total intake down, the satiation effect would go away.

It explained why I'd lost on the order of 50-60 pounds - with what, in retrospect, was very little effort - when I first moved out of my parents' house and to a new city and started eating non-Jewish food. In retrospect, I was eating an amazingly little amount each day, like 1200 calories, but without any feeling of hunger. And then my weight slowly started creeping up again, and no amount of exercise - to which (ha!) I'd originally attributed the weight loss - seemed able to stop it.

It's always hard to pick reality out of the gigantic morass of competing dietary theories. One of the elegant charms of Robert's hypothesis is that it helps explain why this is so - the mess of incoherent results. Any new diet will seem to work for a few months or weeks, you're losing weight and everything seems wonderful, you tell all your friends and they buy the same diet book, and then bam the flavor-calorie association kicks back in and you're back to hell. The number-one result of weight-loss science is that 95% of people who lose weight regain it.

(I haven't heard any complaints from people regaining weight they lost on the Shangri-La Diet, however - if it works for you at all, it seems to go on working. Most of the complaints on the forums are from people who suddenly plateau after losing 30 pounds, but who want to lose more. Or people like me, who try it, and find that it doesn't seem to do anything, or that we're gaining weight with no apparent loss of appetite.)

I have a pretty strong feeling - I don't know if I should trust it, since I'm not a dietary scientist - that Roberts's hypothesis is at least partially right. It makes a lot of data snap into focus. The pieces are well-supported individually.

But I don't think that Roberts has the whole story. There's something missing - something that would explain why the Shangri-La Diet lets some people control their weight as easily as a thermostat setting, and why others lose 30 pounds and then plateau well short of their goal, and why others simply find the Shangri-La diet ineffective. The Mystery of Shangri-La is not how the diet works when it does work; Roberts has made an excellent case for that. The question is why it sometimes doesn't work. There is a deeper law, I strongly suspect, that governs both the rule and the exception.

The problem is, though - and here's the really frustrating part - Roberts seems to think he does have the whole answer. If the diet doesn't work at first, his answer is to try more oil... which is a pretty scary answer if you're already gaining weight from the extra calorie intake! I decided not to go down this route because it didn't seem to work for the people on the forums who were reporting that the Shangri-La Diet didn't work for them. They just gained even more weight.

And what really makes this a catastrophe is that this theory has never been analyzed by controlled experiment, which drives me up the frickin' WALL. Roberts himself is a big advocate of "self-experimentation", which I suppose explains why he's not pushing harder for testing. (Though it's not like Roberts is a standard pseudoscientist, he's an academic in good standing.) But with reports of such drastic success from so many observers, some of them reliable, outside dietary scientists ought to be studying this. What the fsck, dietary scientists? Get off your butts and study this thing! NOW! Report these huge results in a peer-reviewed journal so that everyone gets excited and starts studying the exceptions to the rule!

It's awful; it seems like Roberts has gotten so close to burying the scourge Obesity, but the theory is still missing some final element, some completing piece that would explain the rule and the exception, and with that last piece it might be possible to make the diet work for everyone...

If we had a large-sized rationalist community going that had solved the group effort coordination problem, those of us who are metabolically disprivileged would be pooling resources and launching our own controlled study of this thing, and entering every conceivable variable we could report into the matrix, and hiring a professional biochemist to analyze our metabolisms before and afterward, and we would cryopreserve anyone who got in our way. You have no idea.

(Warning: Do not try the Shangri-La diet at home based on only the info here, there's a couple of caveats and I can't think offhand of a good complete description on the 'Net. Also you might want to reconsider the recommendation to use fructose in the sugar water route, because IIRC fructose has been shown to contribute to insulin resistance or something like that - sucrose may actually make more sense, despite the higher glycemic index.)

Continued in: Akrasia and Shangri-La