Earlier this morning I put a link to press release by Stanford University relating a new paper by John Ioannnidis and Shanil Ebrahim, published yesterday at JAMA. I did not read the paper yet, but I am concerned with what was communicated in the PR, and hope that te message in the paper is different.
The PR title says that “Re-analysis of clinical trial data can change conclusions“, and these are old news. It is true for any data: analyze it using a different statistical method, get a different result. A really nasty and evil statistician can even introduce many sources of bias to get whatever results he wants to present.
Ioannidis’ solution: release the clinical data to the public so anyone will be able to analyze it. And I add to that: so everyone will be able to tweak analyses and introduce any bias they like in order to present any result they want.
There is a reason why the statistical methods for a clinical trial are set in the trial’s protocol, before the first subject is even screened. Protocols also include sensitivity analyses, and all the results, including the data are eventually reviewed by the regulatory authorities, who perform they own sensitivity analyses. To my opinion, this is sufficient to maintain the integrity of the trial results. Letting everyone to introduce bias will just harm the credibility of clinical trials and science.