Wednesday, January 16, 2008

Notable addition to the blogroll

I just found a new biostatistics blog called Epidemiological Methods. The posts I've looked at so far are high quality, and the blogroll is high quality as well.

Oh, and via that, I found a Bayesian Stats blog, too.

Monday, January 14, 2008

Not "ENHANCE"-ing the public image of science, but it does show the power of the scientific method

The recent brouhaha over the Merck/Schering-Plough ENHANCE study can be pointed to as a triumph of science. SP clearly tried to pull a fast one by convening a "panel of experts" to determine the endpoint of the ENHANCE trial long after data was collected. Statistically, of course, this amounts to data dredging and is rather dishonest and unethical, not to mention unscientific. The power of science, after all, lies in our ability to predict outcomes based on our present data and current understanding of how things work, not picking outcomes on the basis of past data.

Our current safeguards have turned back a major assault on our sensibilities. I do feel bad for the Vytorin team because, well, they've gone this far into the development process and had a major setback. But as for the geniuses who decided to try to break the rules, I hope they find a different line of work. Having applied research help the public depends on a certain degree of trust, and shenanigans like this only serve to erode that trust.

As for me, I'm glad we've come this far in our ethical development as scientists to at least catch these blatant cases. But Congress is already trying to bang down our doors calling for more controls. I'm afraid there's more of our act to clean up.

As a note, Derek Lowe has a nice analysis of the clinical and statistical issues.

Sunday, January 6, 2008

A level-headed skeptical assessment of echinacaea

It seems that almost everyone either loves or hates echinacaea. Or, at least, there's a vocal core group of people that espouses its miraculous wonders as a cold remedy, and another equally vocal crowd that wants the first to shut up.

At any rate, the research on echinacaea has been mixed and confusing, which is par for the course for an extract of a natural product.

Stats.org just released a commentary on some shoddy reporting of the research, and a resource at the Mayo Clinic shows just how confusing the research has been on echinacaea in particular.

One note about the Stats.org commentary is worthy of a "lying with statistics" article. Articles from Bloomberg, NTY, and LA Times all reported that prophylactic use echinacaea reduced the rate of getting colds by about 65 percent. However, this number is the reduction in the odds of getting a cold, not a reduction in the probability of getting a cold (about 30 percent). Statistically, the concepts of odds and probability are way different, and the reporting of the odds in this case made echinacaea look way better than the reference study would indicate.

That's not the only issue, but I refer you to the Stats.org article for more ways of inoculating yourself against questionable reporting of statistics.

One last comment: it doesn't seem the author is against echinacaea or thinks that it is ineffective, but is simply evaluating the quality of the evidence and the reporting of the evidence.

Friday, December 7, 2007

Dynamic graphs in any web page via Google


Figure 1: what theoretical respondents drink right before producing pie charts

Google now has an API for generating dynamic graphs from web site. All you need to do is put in the HTML! You can generate line charts, bar charts, pie charts, Venn diagrams, and scatterplots. Full instructions here.

(via Information Aesthetics)

Saturday, November 24, 2007

Quick lesson: how does SAS calculate confidence intervals?

Occasionally, I'll go through the referrals to this site. One referral I found was "how does SAS calculate confidence intervals." Here's a brief explanation.

1. What kind of confidence intervals are you talking about? Regression coefficients, sample means, sample variances, prediction intervals, differences of sample means, ratios?
2. Once you've got that figured out, go to the SAS manual. If you have SAS, it's in the help online. Otherwise, you can probably find it on the web. Go to the documentation for the right procedure (depends on what confidence interval you are calculating). In there, there is a section on details and computational considerations. In those sections SAS details the statistical theory they use for all of their procedures as well as any methods they use to make the procedure more efficient. If that isn't enough, they give copious references.

Back to work for me.

Wednesday, November 14, 2007

Bayes is big

When I was in graduate school, Bayesian statistics was a small, but important, part of my statistical inference curriculum. When I graduated, I all but forgot it. But a couple of years ago, I saw the storm on the horizon, and started furious self-study, including theory and computation. A few months ago, that storm hit shore. At the Joint Statistical Meetings 2007, I got raised eyebrows when I told former colleagues that life was turning me into a Bayesian, but I also met some prominent figures in the Bayesian biostatistics movement.

And now it's happening again.

About a year ago, I predicted over at Derek Lowe's excellent blog that a drug development program based on a full Bayesian approach would be 10 years off, though drug safety would probably see the largest immediate application. I was wrong. The storm is creeping inland. Be ready for it.

Wednesday, November 7, 2007

When graphics fail

I love graphical methods, and think that biostatisticians ought to use them more (in many sectors of my industry, they are using graphical methods more and are advancing the field). However, there are times when graphics deceive us even if they are done correctly, such as when one is trying to compare two overlaid time series.

Junk charts shows a recent example from the NYT. I think this example also shows that creating truly illuminating graphics is both an art and a science.

Wednesday, September 19, 2007

My O'Brien-Fleming design is not the same as your O'Brien-Fleming design

I know this discussion is a little technical, and nonstatisticians can probably skip this, but I hope that a statistician struggling with the O'Brien-Fleming design and its implementation in SAS/IML (notably the SEQ, SEQSHIFT, and SEQSCALE functions) can find this from a search engine and save hours of headache.

There are two ways of designing an O'Brien-Fleming design, a popular design for conducting interim analyses of clinical trials. The first method is to use an error (or alpha) spending function, which essentially gives you a "budget" of error you can spend at each interim analysis. The second is to realize that, if you are looking at cumulative sums in the trial, the O'Brien-Fleming design terminates if you cross a constant threshhold. In the popular design programs LD98 and PASS 2007, the spending function approach is used. In the book Analysis of Clinical Trials using SAS, (a book I highly recommend, by the way), the cumulative sum approach is used at the design stage (the spending function is used at the monitoring stage). When interim analyses are equally spaced, the two approaches give the same answer. When interim analyses are not equally spaced, the two approaches seem to give different answers. What's more, the spending function for O'Brien-Fleming as implemented in LD98 and PASS are different from what they show you in the books. They use:

4 - 4*PHI(z(1-alpha/4)/sqrt(tau))

for two-sided designs.

They don't tell you these things in school. Or in the books.

Update: Steve Simon's post on the topic has moved as of 11/21/2008. Please see the third comment below.

Monday, September 17, 2007

He makes the data sing their story

Hans Rosling gave a TED talk in 2006. If you love to work with data, you must watch this.



By the way, I am a statistician, and I love it. Yeah, this is all observational and "hypothesis-generating," as we like to say, but letting the data sing their story tells us where to concentrate our efforts.

Saturday, September 1, 2007

Bias in group sequential designs - site effect and Cochran-Mantel-Hanszel odds ratio


It is well known that estimating treatment effects from a group sequential design results in a bias. When you use the Cochran-Mantel-Haenszel statistic to estimate an odds ratio, the number of patients within each site affects the bias in the estimate of the odds ratio. I've presented the results of a simulation study, where I created a hypothetical trial and then resampled from this trial 1000 times. I calculated the approximate bias in the log odds ratio (i.e. log of the CMH odds ratio estimate) and plotted that versus the estimated log odds ratio. The line is cubic smoothing spline, made by the statement symbol i=sm75ps in SAS. The actual values are underprinted in light gray circles just to get some idea of the variability.