Scottish heart miracle unspun

For the latest update on this story see here.

"Statistics are like a bikini. What they reveal is suggestive, but what they conceal is vital." Aaron Levenstein



In a previous article ('Publish and be Damned'), I looked at an unpublished heart attack study conducted by Dr Jill Pell and colleagues in Scotland. I said then:

"It is hard to imagine epidemiology sinking much lower than this. So glaring are the flaws in this press release/study that it has since been ridiculed by The Times in its end-of-the-year run down of 'The worst junk stats of 2007', where it appeared in a list of "classic idiocies". The BBC did not fall over itself to debunk the study but it did at least allow an article to appear online entitled 'The facts get in the way of a good story' which left little doubt that all had not been as it seemed when they shouted about the Scottish miracle two months earlier."


Today, ten months after it was first reported in the media, this study has finally been published in the New England Journal of Medicine. What follows is a brief summary of its flaws:


Unusual and selective time-frame

In the previous article, I noted that there was some confusion in the press over whether the period under examination was 10 months or 12 months:

"The country's deputy Chief Medical Officer told the BBC that the study had looked at two 12 month periods but Dr Lawrence Gruer, Pell's colleague at the Greater Glasgow Health Board and also talking to the BBC, referred to two 10 month periods. A small but significant discrepancy: 12 months would be the obvious length of any study of this kind. To pick anything else would suggest cherry-picking; arbitrarily excluding two months immediately raises the suspicion that these months did not contain the kind of data that supported the desired hypothesis."

Now published in the NEJM, it is clear that the researchers did indeed study an unusual 10 month period. The months that were excluded (in both years) were April and May. This is an extraordinary omission, since the ban began in March. The whole premise of the study was that smoking bans immediately cut heart attack incidence and yet they excluded the two months that immediately followed the ban!

Why were they excluded? We know that data was collected for May at least, because the team's mission statement was published in the Journal of Public Health and clearly stated: "Data collection: Continuous May 2005-April 2007" (ref. (pdf)) Perhaps the clue lies in data from ISD Scotland which shows that the decline in heart attack admissions was unusually small in May 2007. Admissions fell by just 1.8%. Excluding that month's data therefore helped to artificially raise the average and kept the headline figure high.


Focus on acute coronary syndrome

In 2007, Pell presented the study to a conference under the name 'Changes in myocardial infarction incidence and mortality following the Scottish smoke-free legislation' and when the same presentation was delivered in Edinburgh a few months later, the BMJ reported: " the Scottish evaluation, by Jill Pell and colleagues, reported a 17% reduction in admissions from acute myocardial infarction in nine hospitals in the six months after the ban."(ref.)

In fact, as we can now see, Pell's team studied acute coronary syndrome (ACS), which is rather different. But why? When the study was presented in Edinburgh, it went under the name 'Testing the Montana Hypothesis: Results from Scotland'. The 'Montana hypotheisis' is a reference to a study of heart attack admissions in the town of Helena, Montana which showed a fall in admissions of 40%. The study was ludicrous and has been long-since discredited but it did, at least, confine itself to heart attacks ie. acute myocardial infarction. This was no secret, the study was called "Reduced incidence of admissions for myocardial infarction associated with public smoking ban: before and after study" If it was Pell's intention to 'test' the 'Montana hypothesis', she should have looked at the data for myocardial infarction, which was avilable to her. Instead she used figures for acute coronary syndrome. It is tempting to conclude that she did so because the figures for ACS happened to show a sharper drop than those of AMI.


Exposure

Previous studies of this kind have been criticised for not asking patients whether they were exposed to secondhand smoke - quite an important question, one would have thought, considering the hypothesis. One advantage of the Scottish study is that it made an effort to find out what, if any, exposure, heart patients had experienced. The results are very interesting. After smoking became illegal in pubs and clubs, very few of the patients (33) said they had been exposed to secondhand smoke in these venues. No surprise there. What is remarkable, however, is that of the 1,614 nonsmoking patients who were admitted before the ban, only 383 had been exposed to any tobacco smoke in pubs or clubs. If, as is clearly being suggested, secondhand smoke in bars was one of the principle causes of acute coronary syndrome before the ban, one would expect a far larger proportion of ACS patients to have been exposed to it. The reality was that 76% of nonsmoking cases never went to smoky pubs or clubs before the ban, and 96% were not exposed to any secondhand smoke at work.


Comparison with England

The Pell study uses England as a control group because, until July 2007, smoking was permitted by law in most public places. Pell reports that admissions for acute coronary syndrome fell by 4% in England during this period and uses the contrast with Scotland to support her hypothesis. Inadvertently, she does quite the reverse. Earlier this year, a study appeared in England showing that ACS admissions had fallen by 3% since the smoking ban. Assuming both figures to be true, and using Pell's logic, we can conclude that the English smoking ban 'caused' the rate of decline for ACS admissions to slow down by 25%.

At the very least, the fact that the smoking ban in England had no effect on heart admissions shows that the 'Montanan hypothesis' is erroneous.


Effect on smokers

No one doubts that there was a significant fall in ACS admissions in Scotland in 2006, as part of a long-running downward trend. But the notion that it was caused by the smoking ban is severely undermined by the fact that incidence amongst smokers also fell sharply. The effect of secondhand smoke on active smokers is, of course, negligible. The 14% drop in ACS admissions amongst smokers cannot therefore be attribituted to the smoking ban. Some other factors must have been responsible. And since other factors were clearly responsible for the 14% drop amongst smokers, it is reasonable to surmise that the same factors were responsible for the supposed 20% amongst nonsmokers.

Conclusion

Pell and her team picked their own timeframe, excluded four crucial months, used a selective sample group and studied a different medical condition to the one studied in the paper that it was attempting to verify. By so doing, they exaggerated a pre-existing downward trend in acute coronary syndrome and arbitraily attributed it to a piece of legislation.

This is not science. Using the same techniques I can 'prove' that the Scottish smoking ban led to an increase in skin cancer incidence and a rise in oil prices. Nonsensical though this study is, its publication in a prestigious medical journal will no doubt give it another ill-deserved moment in the sun. At the time of writing, The Times, The Guardian and the BBC - who had their fingers burnt with the story last time - have declined to report it again, but it is being covered by, amongst others, Reuters, The Vancouver Sun, The Herald, USA Today. According to The Herald: "The findings of a major study into the smoking ban in Scotland supports calls for a worldwide ban of the practice in public places, health officials said today."

In USA Today, Tom Glynn of the American Cancer Society calls the study "virtually flawless." This is breath-taking chutzpah. What, one wonders, does a flawed study look like?*



*I suppose it might look like this. Or perhaps like this.



Christopher J. Snowdon is the author of Velvet Glove, Iron Fist: A History of Anti-Smoking

If you have any comments or corrections please send them to author@velvetgloveironfist.com