What doctors don't know about the drugs they prescribe - Ben Goldacre

Key Takeaways

[Music] [Music] [Applause] hi uh so um this chap here he thinks he can tell you the future his name is Nostradamus although here the son have made him look a little bit like Sean conery and like most of you I suspect I don't really believe that people can see into the future I don't believe in precognition and every now and then you hear that somebody has been able to predict something that happened in the future and that's probably because it was a fluke and we only hear about the flukes and about the freaks we don't hear about all the times that people got stuff wrong now we expect that to happen with silly stories about precognition but the problem is we have exactly the same problem in Academia and in medicine and in this environment it costs lives so firstly thinking just about precognition as it turns out just last year a researcher called Daryl bam conducted a piece of research where he found evidence of precognitive powers in undergraduate students and this was published in a peer-reviewed academic journal and most of the people who read this just said okay well fair enough but I think that's a flute that's a freak because I know that if I did a study where I found no evidence that undergraduate students had precognitive powers it probably wouldn't get published in a journal and in fact we know that that's true because several different groups of research scientists tried to replicate the findings of this precognition study and when they submitted it to the exact same journal the journal said no we're not interested in publishing replication we're not interested in your negative data so this is already evidence of how in the academic literature we will see a biased sample of the true picture of all of the scientific studies that have been conducted but it doesn't just happen in the dry academic field of psychology it also happens in for example cancer research so in March 2012 just one month ago some researchers reported in the journal Nature how they had tried to replicate 53 different basic science studies looking at potential treatment Targets in cancer and out of those 53 studies they were only able to successfully replicate six 47 out of those 53 were unreplicable and they say in their discussion that this is very likely because freaks get published people will do lots and lots and lots of different studies and the occasions when it works they will publish and the ones where it doesn't work they won't and their first recommendation of how to fix this problem because it is a problem because it sends us all down blind alleys their first recommendation of how to fix this problem is to make it easier to publish negative results in science and to change the incentives so that scientists are encouraged to post more of their negative results in public but it doesn't just happen in the very dry world of preclinical basic science cancer research it also happens in the very real Flesh and Blood of academic medicine so in 1980 some researchers did a study on a drug called lenide and this was an anti arthic drug a drug that suppresses abnormal heart rhythms and the idea was after people have had a heart attack they're quite likely to have abnormal heart rhythms so if we give them a drug that suppresses abnormal heart rhythms this will increase the chances of them surviving early on in its development they did a very small trial just under a 100 patients 50 patients got lenide and of those patients 10 died another 50 patients got a dummy Placebo sugar pill with no active ingredient and only one of them died so they rightly regarded this drug as a failure and its commercial development was stopped and because its commercial development was stopped this trial was never published unfortunately over the course of the next five 10 years other companies had the same idea about drugs that would prevent arhythmia in people who've had heart attacks these drugs were brought to Market they were prescrib very widely because heart attacks are a very common thing and it took so long for us to find out that these drugs also caused an increased rate of death that before we detected that safety signal over a 100,000 people died unnecessary neily in America from the prescription of anti-ar rythmic drugs now actually in 1993 the researchers who did that 1980 study that early study published a mayor Cula an apology to the scientific community in which they said when we carried out our study in 1980 we thought that the increased death rate that occurred in the lenide group was an effect of chance the development of lenide was abandoned for commercial reasons and this study was never published it's now a good example of publication bias that's the technical term for the phenomenon where unflattering data gets gets lost gets unpublished is left missing in action and they say the results described here might have provided an early warning of trouble ahead now these are stories from basic science these are stories from 20 30 years ago the academic publishing environment is very different now there are academic journals like trials the Open Access Journal which will publish any trial conducted in humans regardless of whether it has a positive or A negative result but this problem of negative results that go missing in action is still very prevalent in fact it's so prevalent that it cuts to the core of evidence-based medicine so this is a drug called reboxetine and this is a drug that I myself have prescribed it's an anti-depressant and I'm a very nerdy doctor so I read all of the studies that I could on this drug I read the one study that was published that showed that robotin was better than Placebo and I read the other three studies that were published that showed that rootine was just as good as any other anti-depressant and because this patient hadn't done well on those other antidepressants I thought well robotin just as good it's one to try but it turned out that I was misled in fact seven trials were conducted comparing reboxetine against a dummy Placebo sugar pill one of them was positive and that was published but six of them were negative and they were left unpublished three trials were published comparing robotin against other anti-depressants in which rootine was just as good and they were published but three times as many patients worth of data was collected which showed that robotin was worse than those other treatments and those trials were not published I felt misled now you might say well that's an extreme and unusual example and I wouldn't want to be guilty of the same kind of cherry-picking and selective referencing that I'm accusing other people of but it turns out that this phenomenon of public bias has actually been very very well studied so here is one example of how you approach it the classic Model is you get a bunch of studies where you know that they've been conducted and completed and then you go and see if they've been published anywhere in the academic literature so this took all of the trials that had ever been conducted on antidepressants that were approved over a 15-year period by the FDA they took all of the trials which were submitted to the FDA as part of the approval package so that's not all of the trials that were ever conducted on these drugs because we can never know if we have those but it is the one that were conducted in order to get the marketing authorization and then they went to see if these trials had been published in the peer-reviewed academic literature and this is what they found it was pretty much a 50/50 split half of these trials were positive half of them were negative in reality but when they went to look for these trials in the peer-reviewed academic literature what they found was a very different picture only three of the negative trials were published but all but one of the positive trials were published now if we just flick back and forth between those two you can see what a staggering difference there was between reality and what doctors patients commissioners of health services and academics were able to see in the peer-reviewed academic literature we were misled and this is a systematic flaw in the core of medicine in fact there have been so many studies conducted on publication bias now over a hundred that they've been collected in a systematic review published in 2010 that took every single study on publication the bias that they could find publication bias affects every field of medicine about half of all trials on average go missing in action and we know that positive findings are around twice as likely to be published as negative findings this is a cancer at the core of evidence-based medicine if I flipped a coin 100 times but then withheld the results from you you from half of those tosses I could make it look as if I had a coin that always came up heads but that wouldn't mean that I had a two-headed coin that would mean that I was a chancer and you were an idiot for letting me get away with it but this is exactly what we blindly tolerate in the whole of evidence-based medicine and to me this is research misconduct if I conducted one study and I withheld half of the data points from that one study you would rightly accuse me essentially of research fraud and yet for some reason if somebody conducts 10 studies but only publishes the five that give the result that they want we don't consider that to be research misconduct and when that responsibility is diffused between a whole network of researchers academics industry sponsors Journal editors for some reason we find it more acceptable but the effect on patients is damning and this is happen happening right now today this is a drug called Tamiflu Tamiflu is a drug which governments around the world have spent billions and billions of dollars on stockpiling and we've stockpiled Tamiflu in panic in the belief that it will reduce the rate of complications of influenza complications is a medical euphemism for pneumonia and death now when the cockrane systematic reviewers were trying to collect together all of the data from all of the trials that had ever been conducted on whether tamilo actually did this or not they found that several of those trials were unpublished the results were unavailable to them and when they started obtaining the writeups of those trials through various different means through Freedom of Information at requests through harassing various different organizations what they found was inconsistent and when they tried to get hold of the clinical study reports the 10,000 Page Long documents that have the the the best possible rendition of the information they were told they weren't allowed to have them and if you want to read the full correspondence and the excuses and the explanations given by the drug company you can see that written up in this week's edition of plus medicine and the most staggering thing of all of this to me is that not only is this a problem not only do we recognize that this is a problem but we've had to suffer fake fixes we've had people pretend that this is a problem that been fixed first of all we had trials registers and everybody said oh it's okay we'll get everyone to register their trials well they'll post the protocol they'll say what they're going to do before they do it and then afterwards we'll be able to check and see if all the trials which have been conducted and completed have been published but people didn't bother to use those registers and so then the international Committee of medical journal editors came along and they said oh well we will hold the line we won't publish any journals we won't publish any trials unless they've been registered before they began but they didn't hold the line in 2008 a study was conducted which showed that half of all the trans published by journals edited by members of the icmj weren't properly registered and a quarter of them weren't registered at all and then finally the FDA Amendment Act was passed a couple of years ago saying that everybody who conducts a trial must post the results of that trial within one year and in the bmj in the first edition of January 2012 you can see a study which looks to see if people kept to that ruling and it turns out that only one in five have done so this is a disaster we cannot know the true effects of the medicines that we prescribe if we do not have access to all of the information and this is not a difficult problem to fix we need to force people to publish all trials conducted in humans including the older trials because the FDA Amendment Act only asks that you publish the trials conducted after 200 and I don't know what world it is in which we're only practicing medicine on the basis of trials that completed in the past two years we need to publish all trials in humans including the older trials for all drugs in current use and you need to tell everyone you know that this is a problem and that it has not been fixed thank you very [Applause] [Music] much