How can "functional unblinding work"?

So, continuing in a series on blinding (i.e. the hiding of treatment group assignments from study personnel until all the data have been collected and cleaned and analysis programs written), I will talk about possible ways to do so-called "functional unblinding" -- that is, effectively getting an answer on a drug's treatment effect before treatment codes are released for analysis. Here I will assume that treatment codes are actually remaining sealed (so we aren't talking about "cheating"). This kind of cheating is a serious breach of ethics and merits its own discussion, but I'm saving that for another time.

Also, this post was inspired by the ENHANCE trial and the fallout from it, but I'm not going to make any further comment about it except to say that there are a lot of other features to that situation that make it, at best, appear suspicious. (And to the wrong people.)

So, on to the question, "Is it possible to determine a treatment effect without unblinding the trial?" My answer: a very risky yes, and in some circumstances. I think it is going to be very difficult to show that no treatment effect exists, while a huge treatment effect will be clear. Since I'll be throwing several statistical tools at the problem, this post will only be the first in a series.

The first method is graphical and is called kernel density estimation. This method has the nice feature that it can quickly be done in R or SAS (and I think most other statistical packages) and shows nice graphs. Here I simulated 3 possible drug trials. The first one, the treatment had no effect whatsoever. In the second one, the drug had what I would consider a moderate effect (equal to the standard deviation of the outcome). In the third one, the drug had a huge effect (and probably what would not commonly be seen in drug trials today--3 times the standard deviation of the outcome). I ran the default kernel density estimate in R (using the density() function with defaults), and came up with the image accompanying the post. The top graph looks like a normal distribution graph, as one would expect. The middle graph also looks like a normal distribution, but it is more spread out than the top one. The third one clearly shows two groups.

Identifying huge effects seems to be pretty easy, at least by this method. Identifying moderate effects is a whole lot harder, and distinguishing them from no effect is a bit risky.

However, this isn't the only method of analyzing this problem, and so I will talk about some other methods next time.