We turn now to one of my favourite whipping boys of complimentary and alternative medicine (CAM): acupuncture.
More than 3000 research studies into acupuncture are yet to show any kind of effect other than that of an elaborate placebo. Why would they, of course? What possible rationale could there be for sticking needles into your skin to cure your migraine, insomnia, depression or musculoskeletal problems? Obviously I’m ruling out realigning your chi straight off the bat.
Sadly, whenever the latest study of acupuncture is published there is a fresh wave of credible news reports uncritically reporting that acupuncture does, apparently, work. Last week brought us the latest example when the Acupuncture in Medicine Journal, embarrassingly a part of the stable of BMJ journals, published an open access paper that looked at treating colic in babies.
Colic, excessive crying, is not a specific disease, it is merely defined as crying/fussing for more than three hours per day for three or more days per week. It doesn’t seem to do children any harm in the long run but can be very distressing and hard on new parents. There is no ‘cure’ per se, the best you can do is to give the child plenty of attention and tough it out. It can’t be much fun.
The new paper had three experimental arms; one acupuncture group based on the traditional Chinese method, another acupuncture group based on a more ‘modern’ understanding of acupuncture (ha!) and the final group had no acupuncture.
On the face of it this sounds like the trial has been blinded, but it hasn’t, not nearly sufficiently. A vital group missing is an arm with sham acupuncture where everyone involved thinks that acupuncture is being administered but actually it isn’t. When this group is included in studies it performs just as well as ‘real’ acupuncture proving conclusively, one would think, that acupuncture doesn’t work.
A further failure was that parents weren’t blinded and so it isn’t impossible to imagine a scenario where parents that are aware their child has just been stuck full of needles (who are signing their children up to this study?) give the child more attention and comfort in the ensuing days. This, the traditional treatment for colic, could easily confound any results.
With the study complete neither treatment group was found to have results significantly different from that of the group with no treatment. The study was negative, acupuncture doesn’t work be it the old Chinese way or the ‘modern’ way, it really couldn’t be any clearer. Like all good quacks, however, the researchers decided to manipulate the data and move the goalposts until they could just about drag a few measures below the P=0.05 level; P-Hacking at its most cynical.
Having combined the two acupuncture groups and looked at a staggering 23 different measures and outcomes they managed to find a few of significance. There is no indication that the group made any allowances for the fact that they carried out so many comparisons; indeed, at least one of the statistical tests they mention is known to have problems with not accounting for multiple comparisons. Consider: P=0.05 means that there is a 1 in 20 chance that the result observed could be down to random chance as opposed to a real effect. If you are making more than 20 comparisons then you are all but guaranteed that one of them will creep over the 0.05 threshold. This is not evidence of a real effect it is just data dredging.
Bear in mind also that colic sorts itself out spontaneously for no reason at all, the symptoms fade away or regress to the mean. It could be very easy to mistakenly assign this effect to the magic spell you cast over the child a few days previously. One finding the researchers did make which is likely to be legitimate: when you put needles in babies it makes them cry.
After a century of study and more than 3000 trials there is more than enough evidence to come to a conclusion on acupuncture: it doesn’t work, for any indication, ever. Why would it? There is no need for more study, it should be discarded and forgotten about. I look forward to repeating this message a few months from now the next time the press mindlessly regurgitates the latest nonsense from the witchdoctors.