What got missed in many of the media reports was that the research was incredibly limited. The study ran for only nine days, and involved 43 children — and, importantly, no comparison (or "control") group.
That is Julia Belluz of Vox explaining why you should be skeptical about the childhood obesity study that got so much attention last week. An author claimed "We reversed their metabolic disease in just 10 days, even while eating processed food, by just removing the added sugar and substituting starch, and without changing calories or weight."
Now is not the time to lecture on statistical significance, but think on this: It's hard to show that men are on average heavier than women without a sample size of 100 or more. Presumably something less probable should have a higher burden of proof. There is no rule of thumb, but if you must have one, look for a sample size of many hundred. And always for a credible control group.
On the "adequate sample" but "questionable control group", many of you probably heard last week that red meat causes cancer, and might be as bad as smoking. Anahad O'Connor at the New York Times takes a closer look, and sees ridiculous over-exaggeration of results:
Smoking raises a person's lifetime risk of developing lung cancer by a staggering 2,500 percent. Meanwhile, two daily strips of bacon, based on the associations identified by the W.H.O., would translate to about a 6 percent lifetime risk for colon cancer, up from the 5 percent risk for people who don't enjoy bacon or other processed meats.
I am reminded of Neuroskeptic's 9 circles of scientific hell, where I would like to see a new level for "sample size of 32".
