Thursday, June 21, 2007

Surreality in noninferiority trials: Advanced Life "statistically not inferior"

Noninferiority trials are trials that aim to show that one drug (or, more generically, factor) is not worse than another, older active drug. This is accomplished by setting a "noninferiority margin," running the trial, taking the difference between the two treatment effects, and finding out where the lower end of the 95% confidence interval lies. If the lower 95% confidence limit is higher than the noninferiority margin, then the new drug is noninferior to the other one.

These trials are useful (in pharmaceuticals) when placebo controlled trials are considered unethical or otherwise infeasible, and there are other treatments on the market, and are used all the time in anti-infectives (especially antibiotics). And, of course, they have their problems, like how to choose noninferiority margins (and I hope to shed a little darkness on this issue at an upcoming talk at the Joint Statistical Meetings). Don't look to the FDA for guidance; they've retracted all guidances and says that all margins must be justified statistically and clinically on a case-by-case basis.

And then there is this problem. The short of it goes like this: Advanced Life's new antibiotic treatment showed an effect that was slightly smaller than the active control, yet the lower 95% confidence limit still fell above the noninferiority margin. (This is the meaning of "statistically not inferior to ... Biaxin, although the latter drug technically performed better.") Of course, the consequences of this seemed pretty bad, and the press release language is certainly bizarre.

Noninferiority trials, in my opinion, are one of the failings of statisticians. We simply haven't figured out, at least in the classical statistics camp, how to effectively do this kind of analysis. The way we have it set up right now, we can have a drug that is slightly inferior slip under the radar, and if we approve a slightly inferior drug, and then use that as an active control against another drug that is slightly inferior to that, and so forth, you can end up with a truly worthless drug coming through with shining colors.

I haven't explored the Bayesian version of noninferiority, but it seems better to evaluate a drug on the basis of a posterior probability that the new drug is at least as good as the control than it does to set an arbitrary margin as see where the confidence interval of the difference falls. Unless we can come up with a better solution based on classical statistics.

Thursday, June 14, 2007

Look at the size of that trial! Avandia fallout? Let's hope not.

Usually, when we determine the sample size of a clinical trial, we calculate based on efficacy first. The International Committee on Harmonisation (ICH) has drug exposure guidelines which define the minimum. And if our efficacy sample sizes aren't quite enough to examine the safety issues outlined in the guidelines, we bump the sample size up.

Via Kevin, MD, I found someone who is sharing my concern that, in the recent swing of focus to drug safety, we are probably passing up drugs that may have ugly side effects, but are necessary in treating ugly diseases. Of course, no one wants to risk liver damage to quell a toothache (or do we? Hey, do you take acetaminophen?).

It goes like this: Steve Nissen's meta-analysis took over 42 trials and 28,000 patients to detect a statisticially significant result in cardiovascular risk for Avandia. If we want to be absolutely sure that a drug is safe and doesn't even have a small risk, this is the patient population size you will have to enroll in a development program.

I pretty much agree with the assessment that big pharma is to blame with dubious marketing practices and the blockbuster mentality. An FDA with the authority and will to enforce post-marketing commitments and safety surveillance will go a long way to identifying safety issues more quickly and identifying patients most at risk of adverse drug effects, much like the way that apolipoprotein E (ApoE) genotyping may identify the patients most at risk of cardiovascular side effects of Avandia.

That is, if we can keep our head on straight and remember the goal of drug research — to help people.