Sunday, January 27, 2008

Is it ever OK to change the primary endpoint?

One of the recent points for discussion in the recent ENHANCE trial was whether it was ethical to change the primary endpoint of the trial. Ed Silverman writes in his Pharmalot blog
We’ve written this before, and we will write this again: changing a primary endpoint - and doing so without consulting the lead investigator - is inappropriate. And failing to provide more meaningful updates to a widely anticipated trial much sooner in the process only caused skepticism and suspicison. And naming a so-called independent panel without releasing names compounds the matter, especially when some panel members aren’t independent. If Merck and Schering-Plough execs are upset over “mischaracterization,” they have only themselves to blame.

I want to focus in on one statement: "changing a primary endpoint ... is inappropriate."

There are a few points here I want to quickly discuss. In the opinion of this statistician, it is sometimes appropriate to change primary endpoints. The conditions under which this radical change is appropriate may or may not all have been met in the ENHANCE trial. However, while a change in primary endpoint ought to be enough to raise suspicions (and such a change is not to be done lightly), it should not be, by itself, enough to sink a study.

So, without further ado, circumstances where it may be appropriate to change primary endpoints:

Circumstance and discussionMet in ENHANCE?
The change must be decided before the study is unblinded. Making the change after the study is unblinded is enough to sink a study, even if an independent board is the only group who is unblinded.
Yes. The study was unblinded on Dec 31, 2007 (if we believe the press release, but the FDA should be able to audit this).
It would be very useful for the primary investigator (PI) to be involved in the decision. While statistically not necessary, from an operations point a view the PI has been trusted with the scientific responsibility of the study, and so should have input on the primary endpoint.
No, and, as Silverman points out, this casts further doubt on an already suspicious act. The composition and independence of the board who made the decision is unclear, and this may be an issue in the coming weeks.
There should be a good scientific reason for preferring the new endpoint over the old. Sometimes the reason is statistical (for example, an old endpoint may be much less powerful than a new one) or operational (eg. recruitment projects were way off target), but in any case the scientific justification of the new endpoint needs to be well established.
This is unclear. The claim is that there were difficulties in interpreting the old endpoint - the intima media thickness (IMT) - which is essentially the thickness of artery walls which must be determined from ultrasound images. Determining medical measures for clinical trials from imaging is a difficult task, even for areas such as arthritis where the measures are now standard.
Sometimes, there may be a plan to select a primary endpoint based on the data, but the algorithm for this needs to be specified in advance and the operating characteristics of the procedure, such as Type I error and power, need to be understood and specified. If this is the case, the primary endpoint can be chosen after unblinding, but the algorithm should be programmed before unblinding and should adhere to the plan exactly. Indeed, this is a tricky situation, and such a plan should only be used in extenuating circumstances.
I don't think so. I think if ENHANCE had an adaptive hypothesis we would know that by now (but this is not guaranteed - don't want to place too much weight on an appeal to consequence). At any rate, this is auditable, since the plan has to be written and signed.
The study is a pilot trial and the sponsor is prepared to deal with a selection bias.
No, ENHANCE was not a pilot trial. Instead, as can be seen from the news and stock, this trial had major financial and medical consequences.

Personally, I'm not quite ready to point my finger and yell "j'accuse!" at Schering-Plough and Merck quite yet, at least over attempting to change their primary endpoint in ENHANCE. I certainly will follow the facts that bubble up with great interest, though.