Scottish miracle: The final verdict

Many readers will be familiar with Dr Jill Pell's controversial study of hospital admissions. Released to the media in September 2007, it was finally published in the New England Journal of Medicine in July 2008.

Pell and her team were always leaving a hostage to fortune by claiming that there was a 17% drop in acute coronary syndrome (ACS) across Scotland after - and as a result of - the smoking ban. NHS Scotland employs professional statisticians to monitor hospital admissions and, since they publish the full nationwide data on the web, it was inevitable that the veracity of their claim would be tested eventually. The latest hospital admission statistics will not be available on the ISD Scotland website until the end of October but, under the Freedom of Information Act, I have been given access to them.

Before the Pell study was published, there was some confusion about what was being studied. It is now apparent that the team did not look solely at heart attacks, but at acute coronary syndrome. Acute coronary syndrome is a broader category of heart conditions which includes heart attacks (acute myocardial infarction (AMI) (ICD-10 code I21-22)) as well as angina (ICD-10 code I20).

Because of the way Pell conducted her study, it will never be possible for her findings to be tested retrospectively. This is because she did not use the NHS Scotland figures or the conventional definition of Acute Coronary Syndrome. Instead, she studied patients in nine hospitals and diagnosed ACS according to a tropinin test. Because the test for troponin can detect even small amounts of heart muscle damage, many of her 'ACS' patients would normally have been diagnosed with lesser conditions such as chest pain (ICD-10 code R074) and atherosclerotic disease (ICD-10 code I251).

This was an idiosyncratic approach but, it should be said, there is nothing wrong with it per se. The problem, as Dr Michael Siegel has pointed out, is that historically ACS has not been measured in this way and so we cannot tell if a 17% fall is unusual or not. Pell claims that the previous ten years had seen ACS admissions fall at an average rate of 3% per year but she is not comparing like with like. She used the NHS Scotland figures which were not based on a tropinin test and simply included - as is the norm - AMI and angina (ICD-10 code I20-22).

She also claimed that her control group of England saw a 3% decline in ACS admissions. But again, and for the same reason, this is of no relevance since NHS England measure ACS the conventional way. They, too, include all AMI and angina admissions.

In other words, the tropinin test may be a better way of diagnosing ACS (then again, it may not) but it is pointless using it in an epidemiological study unless the control group is subject to the same test.*

And so, while it is impossible to verify or challenge Pell's findings using her own methodology, we can see whether there was an exceptional drop in ACS admissions using the standard diagnosis of ACS that is traditionally used in medicine. We also have the advantage of being able to see the trend in all hospitals, not just nine of them.

I have collected four years of data from ISD Scotland. The first graph shows the number of admissions in all Scottish hospitals for both acute myocardial infarction and angina in the 48 months between January 2004 and December 2007. The smoking ban began on March 26 2006 and is indicated with an arrow. There was a total of 16,690 admissions for acute coronary syndrome (heart attacks and angina) in the year before the ban (April 05 to March 06). In the same period after the ban there was 15,133 admissions. This translates to a fall of 9.3%, a long way off the 17% claimed in the Pell study**.

The second graph shows the numbers of admissions for ACS, with AMI and angina broken down separately. AMI incidence fell by 9.5% and angina fell by 9.2% in the year before the ban compared to the same period after it.

In both graphs, the long-term downward trend is clearly visible and the rate of decline does not sharpen noticeably after the smoking ban. In fact, there is a clear rise in angina cases in the second half of 2007 which suggests that the long-term fall may be bottoming out.

Of course, there will still be some people who persist in saying that - even though the Pell study exaggerated the post-ban fall massively - ACS admissions fell by 9.3% because of the smoking ban. They are clutching at straws. A decline of this size is by no means unusual. AMI admissions fell by 10.2% in 1999-2000 and angina admissions fell by 10.5% in 2005-2006 (there were also falls of 11.6%, 11% and 12% in previous years).***

I don't know what to add to the data I have presented here. When I started writing Velvet Glove nearly three years ago, I never imagined that it would lead me to making Freedom of Information requests to the Scottish government! I have done so out of pure, morbid curiosity (this episode will not make it into the book). I do not pretend that what you have read above is a "study" or an epidemiological paper of any sort. What I have presented here are simply the facts about acute coronary syndrome admissions in Scotland as given to me by those who collate them. These facts indicate that there was not a fall in acute coronary syndrome of anything like the rate claimed by Dr Jill Pell and that, although there clearly was a fall, it was part of a long-term downward trend and was certainly not exceptional.

There are all sorts of questions to be answered about how the Pell study was conducted but they are largely academic. An epidemiological study has little value once the real statistics are revealed. Pell claimed that ACS incidence fell by 17%. It did not. The newspapers said that heart attacks fell by 17%. They did not. That is the bottom line. But a lie, a Mark Twain said, can travel halfway round the world while the truth is putting on its shoes. Never was this more true than with the claim that heart attacks fell in Scotland because of the smoking ban. It is a very peculiar affair but it is one that has been repeated elsewhere in the world and, no doubt, will be repeated again.






Christopher J. Snowdon is the author of Velvet Glove, Iron Fist: A History of Anti-Smoking

If you have any comments or corrections please send them to author@velvetgloveironfist.com



With thanks to Brian Bond for his patience and invaluable knowledge.

* "Only 52% of our subjects received a clinical diagnosis of ICD-10 121 (acute myocardial infarction)", wrote Pell in her NEJM paper. The remainder were diagnosed with "unstable angina" (I200), "chest pains, unspecified" (R074) and "predominantly I248 (other forms of ischemic heart disease)". Clearly then, Pell was not comparing like with like. It was very poor practice to compare this broad sample group with an English control group which only included one specific type of heart ailment.

While R074 ("chest pains, unspecified" is sometimes included under the umbrella of acute coronary syndrome, I248 ("other forms of ischemic heart disease") never is. (See 'Definition of acute coronary syndrome' and elsewhere.) To see how very unusual the practice is, try typing 'acute coronary syndrome'+ I248 or 'acute coronary syndrome'+R074 into Google. You will very nearly have a googlewhack; the Pell study comes top of the list in each case.

** This is the first time the monthly figures have been shown. The annual figures have been available online for some time and, as has been noted elsewhere, they showed that ACS discharges fell by 8%. When the monthly figures are added up, they show a fall of 9.3%. This discrepancy is probably due to the annual figures only including Scottish residents while the monthly figures include all admissions. NB. the data for April-Dec '07 is provisional.

*** The year-on-year changes in AMI and angina incidence are based on Scotland's hospital discharges. These can be viewed at the ISD Scotland website.

This is the third of three articles about the 'Scottish heart miracle'. See also Publish and be damned and Scottish heart miracle unspun.

And still they come! See Game over

For the story of the heart attack study that was repeatedly rejected for publication, click here.