Drug Safety and FDA Approval Times: It Is MUCH More Complicated Than the HEALTH AFFAIRS Study or the FORBES Response

In my blog post of March 26, 2014, I commented on a New England Journal of Medicine article authored by Darrow, Avorn, and Kesselheim that focused on the serious safety issues arising from the FDA’s accelerated drug approval programs for “breakthrough” drugs.  It is clear that the expedited approval of drugs based on surrogate endpoints can result in marketing approval for drugs with questionable risk/benefit ratios.  However, in the past week a different controversy has arisen over the relationship between drug safety and FDA approval times, sparked by an article in Health Affairs by Cassie Frank and others entitled Era Of Faster FDA Drug Approval Has Also Seen Increased Black-Box Warnings And Market Withdrawals.  The Frank article prompted a response from John R. Graham in Forbes with a title that evidences his disagreement with Frank’s group: Faster FDA Approvals Have Not Caused More Drug Safety Problems.  So who is right?  Actually, my answer is “Neither article sheds much light on the FDA role in drug safety.” It is complicated, like so many problems in pharmaceutical policy.

 Download PDF

The hypothesis of the Frank group’s research was that the shorter FDA review times mandated by the  Prescription Drug User Fee Act (PDUFA) might increase the number of drugs approved that are later found to have serious safety issues.  To explore the correlation between PDUFA and drugs discovered to have serious safety problems after FDA approval, Frank’s group did an empirical study that used two markers of drug safety:  black-box warnings and drugs withdrawn from the market because of safety concerns. To determine the effect of PDUFA on those variables, the investigators tracked the appearance of new black-box warnings and withdrawals from the market for safety reasons for all new molecular entities (NMEs—drugs with new active ingredients) approved from 1975-1999 (pre-PDUFA) and from 2000 to 2010 (post-PDUFA).  As the authors noted, previous studies had investigated the relationship between approval times and drug safety, but none had used both black-box warnings and market withdrawals.  The conclusion of the study was that drugs approved after PDUFA yielded faster FDA review times had an increased likelihood of later requiring a black-box warning or being withdrawn from the market for safety reasons:

By twenty-five years after their introduction to the market, for every hundred drugs introduced to the market, there were thirty-four withdrawals or black-box warnings (Exhibit 1) [Exhibit 1 not included here]. Half of the new warnings appeared within twelve years of the drugs’ introduction; half of the withdrawals occurred within five years (data not shown). Drugs approved after the enactment of PDUFA were significantly more likely to receive a black-box warning or withdrawal than drugs approved before PDUFA’s enactment (26.7 out of 100.0 drugs versus 21.2 out of 100.0 drugs; odds ratio: 1.35; p<.05) at up to sixteen years of follow-up….

Graham’s analysis in Forbes of the Frank study data is interesting in examining the total number of black-box warnings and safety withdrawals prior to and after PDUFA.  Graham found that there were a slightly higher number of black-box warnings and safety withdrawals for the drugs approved before PDUFA compared with drugs approved after PDUFA.  However, Graham seems somehow to miss the fact that the Frank study calculated those events as a ratio of safety events per 100 drugs, so that the total number of such events is irrelevant.  So I will state, for the record: I believe that the Frank study’s data and the conclusion quoted above are valid, but the variables selected are not sufficiently informative about changes in drug safety to shed significant light on the issue of drug safety and the FDA approval process.

The first and most significant problem with the Frank study is that it is unclear whether or not the FDA has consistently applied the same standard for requiring black-box warnings over time.  Not much can be learned from variations over time in a variable that is not objectively determinable and consistent over time.  The standard for imposing a back-box warning may even vary between different FDA Offices at any particular time.  There are no bright-line criteria that define a “serious risk requiring a black-box warning” that can be uniformly applied to such diverse groups as: NSAIDS (such as ibuprofen, naproxen, and Celebrex) which all now have a warning concerning cardiovascular risk; SSRI antidepressants (such as Lexapro, Paxil and Zoloft) which all have a black-box warning concerning the risk of suicidal thinking and behavior in adolescents; and, monoclonal antibodies for cancer (such as Erbitux [black-box warning concerning infusion reactions and cardiopulmonary risks] or Avastin for colorectal cancer [a very lengthy black-box warning concerning numerous very serious and relatively likely adverse events]).  In addition, drug safety surveillance has been improving over time, albeit too slowly, making it more likely that adverse events will be detected and warnings issued.

The second problem with the Frank study is that requiring a black-box warning, (unlike withdrawing a drug from the market because of safety concerns) indicates that, despite the new safety concern, the overall risk-benefit ratio of the drug merits its continuing marketing and use. Graham also makes this point.  In other words, patients treated with the drug before the warning might not have been fully apprised of the drug’s risks or, as is more likely, their prescribing physicians may not have been, but they were receiving a drug for which the benefits outweigh the risks.  However, when a drug is withdrawn from the market for safety reasons it is indeed a determination that the drug’s risks exceeded the benefits and patients who took it prior to withdrawal were exposed to an inappropriate degree of risk.  However, the number of such withdrawals for safety prior to PDUFA (14-- according to Appendix Table 3 of the Frank study) or after PDUFA (18—also according to Appendix Table 3) is simply too small to provide strong conclusions about the impact of drug review times on drug safety.

Mary K. Olson, cited in the Frank study, investigated the relationship between serious adverse events and drug approval times and found that there is an apparent correlation -- “drugs receiving faster reviews are associated with increased counts of serious adverse drug reactions.” The Institute of Medicine (IOM), also cited in the Frank study, recommended that all new drugs carry a special symbol on their labels and all promotional materials for two years, because such a symbol “may help to increase awareness of the nature of newly approved therapies, for example, the incompleteness of information on safety.” While the IOM proposal has considerable merit, the FDA rejected that recommendation. 

So what can we learn from the latest round in the debate about drug safety?  First, it is impossible to know all the risks of a drug before it is approved (but we already knew that).  Second, we need better tools to assess drug safety and do post-market surveillance (but we already knew that too).  And third, drug safety is complicated--really, really complicated.