Ah, but on the other hand, as the prosecution revealed at her trial, Lucia did like tarot. And she does sound a bit weird in her private diary, excerpts from which were read out. So she might have done it anyway.
But the strangest thing of all is this. In generating his obligatory, spurious, Meadowesque figure—which this time was ‘one in 342 million’—the prosecution’s statistician made a simple, rudimentary mathematical error. He combined individual statistical tests by multiplying p-values, the mathematical description of chance, or statistical significance. This bit’s for the hardcore science nerds, and will be edited out by the publisher, but I intend to write it anyway: you do not just multiply p-values together, you weave them with a clever tool, like maybe ‘Fisher’s method for combination of independent p-values’.
If you multiply p-values together, then harmless and probable incidents rapidly appear vanishingly unlikely. Let’s say you worked in twenty hospitals, each with a harmless incident pattern: say p=0.5. If you multiply those harmless p-values, of entirely chance findings, you end up with a final p-value of 0.5 to the power of twenty, which is p < 0.000001, which is extremely, very, highly statistically significant. With this mathematical error, by his reasoning, if you change hospitals a lot, you automatically become a suspect. Have you worked in twenty hospitals? For God’s sake don’t tell the Dutch police if you have.
15 Health Scares
In the previous chapter we looked at individual cases: they may have been egregious, and in some respects absurd, but the scope of the harm they can do is limited. We have already seen, with the example of Dr Spock’s advice to parents on how their babies should sleep, that when your advice is followed by a very large number of people, if you are wrong, even with the best of intentions, you can do a great deal of harm, because the effects of modest tweaks in risk are magnified by the size of the population changing its behaviour.
It’s for this reason that journalists have a special responsibility, and that’s also why we will devote the last chapter of this book to examining the processes behind two very illustrative scare stories: the MRSA swabs hoax, and MMR. But as ever, as you know, we are talking about much more than just those two stories, and there will be many distractions along the way.
The Great MRSA Hoax
There are many ways in which journalists can mislead a reader with science: they can cherry-pick the evidence, or massage the statistics; they can pit hysteria and emotion against cold, bland statements from authority figures. The MRSA stings of 2005 come as close to simply ‘making stuff up’ as anything I’ve stumbled on so far.
I first worked out what was going on when I got a phone call from a friend who works as an undercover journalist for television. ‘I just got a job as a cleaner to take some MRSA swabs for my filthy hospital superbug scandal,’ he said, ‘but they all came back negative. What am I doing wrong?’ Happy to help, I explained that MRSA doesn’t survive well on windows and doorknobs. The stories he had seen elsewhere were either rigged or rigged. Ten minutes later he rang back, triumphant: he had spoken to a health journalist from a well-known tabloid, and she had told him exactly which lab to use: ‘the lab that always gives positive results’ were the words she used, and it turned out to be Northants-based Chemsol Consulting, run by a man called Dr Christopher Malyszewicz. If you have ever seen an undercover MRSA superbug positive swab scandal, it definitely came from here. They all do.
Microbiologists at various hospitals had been baffled when their institutions fell victim to these stories. They took swabs from the same surfaces, and sent them to reputable mainstream labs, including their own: but the swabs grew nothing, contrary to Chemsol’s results. An academic paper by eminent microbiologists describing this process in relation to one hospital—UCLH—was published in a peer-reviewed academic journal, and loudly ignored by everyone in the media.
Before we go any further, we should clarify one thing, and it relates to the whole of this section on health scares: it is very reasonable to worry about health risks, and to check them out carefully. Authorities are not to be trusted, and in this specific case, plenty of hospitals aren’t as clean as we’d like them to be. Britain has more MRSA than many other countries, and this could be for any number of reasons, including infection control measures, cleanliness, prescribing patterns, or things we’ve not thought of yet (I’m talking off the top of my head here).
But we’re looking at the issue of one private laboratory, with an awful lot of business from undercover journalists doing MRSA undercover swab stories, that seems to give an awful lot of positive results.
I decided to ring Dr Chris Malyszewicz and ask if he had any ideas why this should be.
He said he didn’t know, and suggested that the hospital microbiologists might be taking swabs from the wrong places at the wrong times. They can often be incompetent, he explained. I asked why he thought the tabloids always chose his lab (producing almost twenty articles so far, including a memorable ‘mop of death’ front page in the Sunday Mirror). He had no idea. I asked why various microbiologists had said he refused to disclose his full methods, when they only wanted to replicate his techniques in their own labs in order to understand the discrepancy. He said he’d told them everything (I suspect in retrospect he was so confused that he believed this to be true). He also mispronounced the names of some very common bacteria.
It was at this point that I asked Dr Malyszewicz about his qualifications. I don’t like to critique someone’s work on the basis of who they are, but it felt like a fair question under the circumstances. On the telephone, being entirely straight, he just didn’t feel like a man with the intellectual horsepower necessary to be running a complex microbiology laboratory.
He told me he had a BSc from Leicester University. Actually it’s from Leicester Polytechnic. He told me he has a PhD. The News of the World called him ‘respected MRSA specialist Dr Christopher Malyszewicz’. The Sun called him ‘the UK’s top MRSA expert’ and ‘microbiologist Christopher Malyszewicz’. He was similarly lauded in the Evening Standard and the Daily Mirror. On a hunch, I put a difficult question to him. He agreed that his was a ‘non-accredited correspondence course PhD’ from America. He agreed that his PhD was not recognised in the UK. He had no microbiology qualifications or training (as many journalists were repeatedly told by professional microbiologists). He was charming and very pleasant to talk to: eager to please. What was he doing in that lab?
There are lots of ways to distinguish one type of bacteria from another, and you can learn some of the tricks at home with a cheap toy microscope: you might look at them, to see what shape they are, or what kinds of dyes and stains they pick up. You can see what shapes and colours the colonies make as they grow on ‘culture media’ in a glass dish, and you can look at whether certain things in the culture media affect their growth (like the presence of certain antibiotics, or types of nutrient). Or you can do genetic fingerprinting on them. These are just a few examples.
I spoke to Dr Peter Wilson, a microbiologist at University College London who had tried to get some information from Dr Malyszewicz about his methods for detecting the presence of MRSA, but received only confusing half stories. He tried using batches of the growth media that Dr Malyszewicz was using, which he seemed to be relying on to distinguish MRSA from other species of bacteria, but it grew lots of things equally well. Then people started trying to get plates from Dr Malyszewicz which he claimed contained MRSA. He refused. Journalists were told about this. Finally he released eight plates. I spoke to the microbiologists who tested them.
On six of the eight, where Dr Malyszewicz PhD believed he had found MRSA, the lab found none at all (and these plates were subjected to meticulous and forensic microbiological analyses, including PCR, the technology behind ‘genetic fingerprinting’). On two of the plates there was indeed MRSA; but it was a very unusual strain. Microbiologists have huge libraries of the genetic make–up of different types of infectious agents, which are used to survey how different disease
s are travelling around the world. By using these banks we can see, for example, that a strain of the polio virus from Kano province in northern Nigeria, following their vaccine scare, has popped up killing people on the other side of the world (see page 277).
This strain of MRSA had never been found in any patient in the UK, and it had only ever been seen rarely in Australia. There is very little chance that this was found wild in the UK: in all likelihood it was a contaminant, from the media work ChemSol had done for Australian tabloids. On the other six plates, where Malyszewicz thought he had MRSA, there were mostly just bacilli, a common but completely different group of bacteria. MRSA looks like a ball. Bacilli look like a rod. You can tell the difference between them using 100x magnification—the ‘Edu Science Microscope Set’ at Toys’R’Us for £9.99 will do the job very well (if you buy one, with the straightest face in the world, I recommend looking at your sperm: it’s quite a soulful moment).
We can forgive journalists for not following the science detail. We can forgive them for being investigative newshounds, perhaps, even though they were repeatedly told—by perfectly normal microbiologists, not men in black—that Chemsol’s results were improbable, and probably impossible. But was there anything else, something more concrete, that would have suggested to these journalists that their favourite lab was providing inaccurate results?
Perhaps yes, when they visited Malyszewicz’s laboratory, which had none of the accreditation which you would expect for any normal lab. On just one occasion the government’s Inspector of Microbiology was permitted to inspect it. The report from this visit describes the Chemsol laboratory as ‘a freestanding, single storey wooden building, approximately 6m x 2m in the back garden’. It was a garden shed. They go on to describe ‘benching of a good household quality (not to microbiology laboratory standards’). It was a garden shed with kitchen fittings.
And we should also mention in passing that Malyszewicz had a commercial interest: ‘Worried about MRSA? The perfect gift for a friend or relative in hospital. Show them how much you care for their health by giving a Combact Antimicrobial Hospital Pack. Making sure they come out fighting fit.’ It turned out that most of Chemsol’s money came from selling disinfectants for MRSA, often with bizarre promotional material.
How did the papers respond to the concerns, raised by senior microbiologists all over the country, that this man was providing bogus results? In July 2004, two days after Malyszewicz allowed these two real microbiologists in to examine his garden shed, the Sunday Mirror wrote a long, vitriolic piece about them: ‘Health Secretary John Reid was accused last night of trying to gag Britain’s leading expert on the killer bug MRSA.’ Britain’s leading expert who has no microbiology qualifications, runs his operation from a shed in the garden, mispronounces the names of common bacteria, and demonstrably doesn’t understand the most basic aspects of microbiology. ‘Dr Chris Malyszewicz has pioneered a new method of testing for levels of MRSA and other bacteria,’ it went on. ‘They asked me a lot of questions about my procedures and academic background,’ said Dr Malyszewicz. ‘It was an outrageous attempt to discredit and silence him,’ said Tony Field, chairman of the national MRSA support group, who inevitably regarded Dr Malyszewicz as a hero, as did many who had suffered at the hands of this bacterium.
The accompanying editorial in the Sunday Mirror heroically managed to knit three all-time classic bogus science stories together, into one stirring eulogy:
Whistle-blowers appear to bring out the very worst in this Government.
NO WAY TO TREAT A DEDICATED DOCTOR
First, Frankenstein foods expert Arpad Puzstai felt Labour’s wrath when he dared to raise the alarm over genetically-modified crops. Then Dr Andrew Wakefield suffered the same fate when he suggested a link between the single-jab MMR vaccine and autism. Now it’s the turn of Dr Chris Malyszewicz, who has publicly exposed alarmingly high rates of the killer bug MRSA in NHS hospitals.
Dr Chris Malyszewicz should get a medal for his work. Instead he tells the Sunday Mirror how Health Secretary John Reid sent two senior advisers to his home to ‘silence him’.
The Sunday Mirror was not alone. When the Evening Standard published an article based on Malyszewicz’s results (‘Killer Bugs Widespread in Horrifying Hospital Study’), two senior consultant microbiologists from UCH, Dr Geoff Ridgway and Dr Peter Wilson, wrote to the paper pointing out the problems with Malyszewicz’s methods. The Evening Standard didn’t bother to reply.
Two months later it ran another story using Malyszewicz’s bogus results. That time Dr Vanya Gant, another UCH consultant microbiologist, wrote to the paper. This time the Standard did deign to reply:
We stand by the accuracy and integrity of our articles. The research was carried out by a competent person using current testing media. Chris Malyszewicz…is a fully trained microbiologist with eighteen years’ experience…We believe the test media used…were sufficient to detecl the presence of pathogenic type MRSA.
What you are seeing here is a tabloid journalist telling a department of world-class research microbiologists that they are mistaken about microbiology. This is an excellent example of a phenomenon described in one of my favourite psychology papers: ‘Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments’, by Justin Kruger and David Dunning. They noted that people who are incompetent suffer a dual burden: not only are they incompetent, but they may also be too incompetent to assay their own incompetence, because the skills which underlie an ability to make a correct judgement are the same as the skills required to recognise a correct judgement.
As has been noted, surveys repeatedly show that a majority of us consider ourselves to be above average at various skills, including leadership, getting on with other people, and expressing ourselves. More than that, previous studies had already found that unskilled readers are less able to rate their own text comprehension, bad drivers are poor at predicting their own performance on a reaction-time test, poorly performing students are worse at predicting test performance, and most chillingly, socially incompetent boys are essentially unaware of their repeated faux pas.
Perceived logical reasoning ability and test performance as a function of actual test performance
Kruger and Dunning brought this evidence together, but also did a series of new experiments themselves, looking at skills in domains like humour and logical reasoning. Their findings were twofold: people who performed particularly poorly relative to their peers were unaware of their own incompetence; but more than that, they were also less able to recognize competence in others, because this, too, relied on ‘meta-cognition’, or knowledge about the skill.
That was a pop-psych distraction. There is also a second, more general point to be made here. Journalists frequently flatter themselves with the fantasy that they are unveiling vast conspiracies, that the entire medical establishment has joined hands to suppress an awful truth. In reality I would guess that the 150,000 doctors in the UK could barely agree on second-line management of hypertension, but no matter: this fantasy was the structure of the MMR story, and the MRSA swab story, and many others, but it was a similar grandiosity that drove many of the earlier examples in this book where a journalist concluded that they knew best, including ‘cocaine use doubles in the playground’.
Frequendy, journalists will cite ‘thalidomide’ as if this was investigative journalism’s greatest triumph in medicine, where they bravely exposed the risks of the drug in the face of medical indifference: it comes up almost every time I lecture on the media’s crimes in science, and that is why I will explain the story in some detail here, because in reality—sadly, really—this finest hour never occurred.
In 1957, a baby was born with no ears to the wife of an employee at Grunenthal, the German drug company. He had taken their new anti-nausea drug home for his wife to try while she was pregnant, a full year before it went on the market: this is an illustration both of how slapdash things were, and of how
difficult it is to spot a pattern from a single event.
The drug went to market, and between 1958 and 1962 around 10,000 children were born with severe malformations, all around the world, caused by this same drug, thalidomide. Because there was no central monitoring of malformations or adverse reactions, this pattern was missed. An Australian obstetrician called William McBride first raised the alarm in a medical journal, publishing a letter in the Lancet in December 1961. He ran a large obstetric unit, seeing a large number of cases, and he was rightly regarded as a hero—receiving a CBE—but it’s sobering to think that he was only in such a good position to spot the pattern because he had prescribed so much of the drug, without knowing its risks, to his patients.*
≡ Many years later William McBride turned out to be guilty, in an unfortunate twist, of research fraud, falsifying data, and he was struck off the medical register in 1993, although he was later reinstated.
By the time his letter was published, a German paediatrician had noted a similar pattern, and the results of his study had been described in a German Sunday newspaper a few weeks earlier.
Almost immediately afterwards, the drug was taken off the market, and pharmacovigilance began in earnest, with notification schemes set up around the world, however imperfect you may find them to be. If you ever suspect that you’ve experienced an adverse drug reaction, as a member of the public, I would regard it as your duty to fill out a yellow card form online at yellowcard.mhra.gov.uk: anyone can do so. These reports can be collated and monitored as an early warning sign, and are a part of the imperfect, pragmatic monitoring system for picking up problems with medications.
No journalists were or are involved in this process. In fact Philip Knightley—a god of investigative journalism from the Sunday Times’ legendary Insight team, and the man most associated with heroic coverage on thalidomide—specifically writes in his autobiography about his shame over not covering the thalidomide story sooner. They covered the political issue of compensation, rather well (it’s more the oeuvre of journalists after all) but even that was done very late in the day, due to heinous legal threats from Grunenthal throughout the late 1960s and early 1970s.