Following in the footsteps of Jack-of-Kent and all good science geeks across the world, I’m afraid I cannot resist the opportunity for a Star Wars reference and, in the case of the recent events surrounding the BCA/Singh case, it even seems rather fitting.
We have an evil empire (the BCA) who with the help of a Dark Lord (Eady) dealt a significant blow to the forces of good (i.e. reason and free speech) and looked set to claim ultimate victory. However, as a result of the BCA’s actions we have now witnessed the growth of a loosely organised alliance of bloggers, science advocates, journalists and supporters of free speech who along with the tenacious rebel leader Singh are all beginning to take the fight back to the BCA.
Well maybe it’s not quite like Star Wars, maybe more like a Star Wars convention but still… it is true that the supporters of science, reason and free speech are now fighting back and are having quite remarkable success.
First, we have the ‘Keep Libel Laws out of Science’ campaign which drew significant support from many high profile figures, has been widely reported on and now has in excess of 10,000 signatures on it’s petition.
Second, we have the reporting in the mainstream media which in this case seems to have been of a rather high standard and also seems to have taken heed of the detailed information available from the blogs. The fact that it’s a journalist being sued for writing an article also likely has something to do with the sympathetic articles… that and Simon is in the right of course!
Third, we have the so called ‘Quacklash’ in which the skeptical and science blogging world has taken the BCA’s members to task for the ‘plethora’ of misleading and prohibited claims that are found throughout their websites and marketing material. The mass of complaints sent to various regulatory bodies has resulted in panicked e-mails from both the BCA and the McTimoney Association (the largest and second largest chiropractic associations in the UK) advising chiropractors to re-examine their claims, stop calling themselves doctors and take down their sites.
Fourth, we have the absolute demolition of the long awaited plethora of studies provided by the BCA in less than a day after it is released. The science bloggers again deserve all the credit here and for anyone who has yet to read the excellent reviews I strongly suggest doing so now (take a look through at the links at the bottom of the roundup).
And so it goes on…
I have not had time to do much blogging lately but my own small contribution to the good fight against the BCA’s quirky campaign of intimadition and silliness follows below:
Issues with the BCA’s ‘plethora’ of evidence
I thought it might be worth considering the studies the BCA replied and the critical analysis offered by bloggers alongside the short summaries of the evidence (or lack thereof) provided by Simon in his defence, which was made publicly avilable (thanks to Jack of Kent) back in November 2008. In it Simon summarised the lack of evidence for six ailments for which the BCA had promoted chiropractic treatments.
The BCA has accused Simon in their press statements of failing to address their evidence so let’s see if it’s true…
- 1. Colic
Nine studies listed in the plethora about colic. David Colquhoun and Ministry of Truth together provided a detailed analysis of the significant problems with each of these ‘studies’ (several are not even trials). MoT somewhat kindly recognised that “Wiberg et al. is perhaps the best designed of the three but still not without significant flaws” and then went on to explain such flaws in detail. Both writers also pointed to a recent well designed double blinded study, that the BCA failed to mention, by Olafsdottir et al (2001) which showed no discernible difference between a placebo treatment and the chiropractic treatment.
Now let’s look at what Simon said all those months ago (as summarised by Jack of Kent):
Simon Singh lists a number of trials where Chiropractic has been shown to be ineffective: Olafsdottir (2001), Ernst (2003), Husereau (2003), Ernst/Canter (2006).
He then provides a detailed critique of the Wiberg (1999) trial cited by the BCA, and in particular where it did not meet most of the standards required and expected of a properly-done clinical trial.
So Simon many months back mentioned better controlled and more recent trials/reviews which the BCA failed to address in their plethora and provided a ‘detailed critique’ of their best study…
And yet, it is the BCA’s contention that Simon failed to properly respond to their evidence…
- 2. Sleeping Problems
There are no trials looking specifically at sleeping problems instead there is one trial (Browning & Miller 2008) comparing two chiropractic treatments for infant colic in which the BCA’s summary mentions both treatments “offered significant benefits including increased sleep”. The problems with this study are discussed in the articles above examining the ‘evidence’ for chiropractic treatments of infant colic because thats what this study was theoretically designed to address.
The trial fails at being a useful study for it’s designated purpose but it doubly fails as evidence for chiropractic as a treatment for sleeping problems as it wasn’t examining this. So the BCA is attempting to use this one study as evidence for chiropractors ability to treat infant colic and evidence for their ability to treat sleeping problems but sadly it shows neither as it is not blinded and has no control group. Also of note is that this study was published after the original article was published so how Simon is expected to have consulted it is beyond me!
So Simon’s summary that:
Simon Singh is unaware of any published clinical trials investigating (let alone supporting) the efficacy of Chiropractic in dealing with sleeping problems.
He points out that the BCA’s reference is to a page in a book (Anrig & Plaugher) discussing two case reports.
Is entirely valid. The BCA has provided no such trials.
- 3. Feeding habits
Here we have a similar situation to the evidence for treating sleeping problems there are no trials specifically looking at chiropractic treatments to improve infant feeding habits and indeed none of the BCA’s comments even make reference to trials improving feeding habits. It’s likely that as with sleeping problems the BCA has found one or two trials looking at another topic in which the parents reported the child had improved eating habits and thus decided that chiropractic treatments are therefore proven to help with this problem. The issue here is once again that there are no trials looking at the topic specifically and that none of the other trials provided are of a high enough quality to draw such conclusions.
So once again Simon’s summary seems entirely fair:
Similarly, Simon Singh is unaware of any published clinical trials investigating (let alone supporting) the efficacy of Chiropractic in dealing with feeding problems.
He points out that the BCA’s reference is to a page in a book (again Anrig & Plaugher) discussing feeding habits of parents, with no research or evidence.
- 4. Frequent Ear infections
Four studies in amongst the plethora which mention ear infections. Gimpy demolishes three of these studies and in particular points out that none include control groups (and one is actually about osteopathy). The fourth (Hawk et. al 2007) is simply a review which includes as the kind of poorly controlled and poorly conducted studies that we see throughout the rest of the plethora and as such it’s conclusions are rather hard to take seriously.
Anyway again back in November we find that:
Simon Singh is also unaware of any published clinical trials investigating (let alone supporting) the efficacy of Chiropractic in dealing with frequent ear infections.
Well it seems that Simon was not aware of at least two of the studies mentioned by the BCA however it is only in the very loosest sense that they can be have said to have ‘investigated’ the efficacy of anything. If these studies amount to the best evidence the BCA can muster for chiropractic treatments of ear infections then I think Simon’s statements remain quite well justified. Extremely poorly designed studies with no control groups do not amount to evidence for efficacy.
- 5. Asthma
Five studies relating to asthma. One is the systematic review discussed above, one is a letter to an editor, two are looking at osteopathic manipulations and the one remaining study is a seriously flawed pilot study. All of these have been dealt with in detail over at Evidence Matters but the point worth emphasising is that the BCA only managed to produce ONE study relating to chiropractic treatments for asthma and even that only claims to be a pilot study.
Simon Singh lists a number of trials where Chiropractic has been shown to not be effective: Balon/Aaker (1998), Balon/Mior (2004), Hondras (2005) and Ernst/Canter (2006)
He then points out that the BCA’s citation (Kukurin) is to a letter to the editor commenting on a pilot study.
Note that again Simon had already pointed out to the BCA the problem with citing a letter to an editor and yet the BCA continues to attempt to use it as evidence for it’s case. As for Simon’s citations:
– Hondras (2005) is a Cochrane review which found that there was not enough evidence from trials to show that ANY manual therapies help asthma.
– Balon / Aaker (1998) is a randomised, blinded and placebo controlled trial available on pubmed that concluded that “In children with mild or moderate asthma, the addition of chiropractic spinal manipulation to usual medical care provided no benefit.”
– Balon / Mior (2004) is a review assessing the evidence for chiropractic care in the management of asthma again it is available on pubmed and features the following conclusion “There is currently no evidence to support the use of chiropractic SMT as a primary treatment for asthma or allergy.”
– Ernst / Canter (2006) is a review of the reviews for spinal manipulation as an effective treatment of any condition. Unsuprisingly the review is easily found on pubmed and concludes that the data examined does “not demonstrate that spinal manipulation is an effective intervention for any condition.”
Compare and contrast the quality of sources and you get a hmmm…
- 6. Prolonged Crying
One ‘study’ in the plethora specifically mentions ‘excessive infant crying’ and the ‘study’ is not even actually a study but simply a proposal for “a framework for chiropractic care”. Presumably the BCA would also argue that there is evidence in the studies that look at other topics that prolonged crying can be reduced but once again this argument would fall flat on it’s face because the other studies are of such a poor quality that drawing wider conclusions from them is just not warranted.
So once again Simon’s assesment seems entirely justified although at least in this case they seem to have dropped the Budgell source that he criticises:
Simon Singh is also unaware of any published clinical trials investigating (let alone supporting) the efficacy of Chiropractic in dealing with prolonged crying.
He points out here that the BCA’s citation is to a non-publicly available paper (Budgell) which looks at the experience of chiropractors and does not look at clinical trials.
So there you have it. The BCA’s plethora of evidence is, as everyone expected, extremely underwhelming. Simon’s original responses offered many months previously also seem to hold up perfectly fine in light of the BCA’s ‘evidence’ despite their claims to the contrary. Perhaps, the compilers of the plethora would have been well served to refer to Simon’s discussion of what counts as good quality evidence contained in his defence back in November 2008:
In brief, the essential features of a high quality trial are:
a) a control intervention that allows an unbiased comparison and conclusions about the scientific effects of the tested intervention per se (rather than non-specific factors such as additional therapeutic attention, for instance);
b) a sufficiently large number of patients to minimise random biases;
c) blinding of patients and those evaluating the results of the tested intervention to minimise expectation bias; and
d) random assignment of patients to a control group or treatments group, to minimise selection bias.
Evidence from poor quality trials is not scientifically reliable and may be worse than no evidence at all, as it has a tendency to be overly positive and might therefore be misleading in suggesting that an intervention is effective when it is not. Case studies, case reports, case series and uncontrolled trials do not constitute evidence/reliable scientific evidence.
Taking such things into account I think the ‘plethora’ quickly reduces itself into less than ‘a jot’.