A new automated tool, the “Trial Tracker”, has been launched by two academic researchers working at the Evidence-Based Medicine Data Lab, University of Oxford, UK. The tool provides an automated software algorithm designed to identify clinical trial sponsors with the best and worst record for sharing trial results. The underlying premise is that a failure to publish clinical trial results leads to a publication bias, which the authors argue harms patients, researchers and doctors, and ultimately harms clinical care. The initiative is supported by the Laura and John Arnold Foundation to conduct work on research integrity. A number of the best and worst performers include sponsors of trials on new treatments for retinal disorders.
The tool, available at https://trialstracker.ebmdatalab.net/, ranks trial sponsors, both industry and academic, who fail to publish their research results following completion of clinical trial studies. Failure to publish leads to issues of “missing data” which may lead to direct harm to the patient and specific examples of such harm are well document by one of the tool’s designers, Dr. Ben Goldacre. According to Dr. Goldacre, evidence from earlier reviews showed that “studies with ‘negative’ or non-significant results are twice as likely to be left unpublished” and, despite legislation from the FDA to mandate sponsors to publish results within 12 months of study completion, such rules have largely been ignored and breaches appear to go un-sanctioned, despite the FDA’s ability to impose $10,000 per day fines for non-compliance. According to at least one estimate, the FDA could have collected an estimated $25 billion from drug companies alone in the past seven years if non-compliance were properly enforced.
At present, the Oxford university generated tool tracks interventional studies registered at the US clinicaltrials.gov database with a status of either “completed”, or with a completion date more than 24 months ago and after Jan 1 2006. The tool then searches for results both within clincaltrials.gov and PubMed and, if no results are found, lists the study results as unreported. Output data is then shared through an interactive website and, for transparency, all code for the tool is shared openly as an iPython notebook. Results to date show that the dataset currently comprises 25,927 eligible trials, of which 11,714 (45.2%) have failed to make results available. In summary, 2390/8799 (27.2%) of industry trials were identified as unreported, while 122/470 (26.0%) of trials with sponsors classed as “US Fed” were unreported and, 361/996 (36.2%) of trials with sponsors classed as “NIH” were unreported. 8841/15662 (56.4%) of trials with sponsors classed as “other” were additionally identified as unreported. Of significance, the authors of the initiative highlight that 8.7 million patients were enrolled in trials identified by the trial tracker as unreported. Given that many patients volunteer their bodies and time for altruistic reasons, many in the hope that their participation will benefit themselves or others, such failure to honour the patients’ commitment through publishing the results is definitively unacceptable. The purpose behind the initiative is to increase the level of access to data from clinical trials by highlighting those companies and institutes that fail to publish. The authors conclude their study stating, “we have also previously argued that medicine has an ‘information architecture’ problem; all publicly accessible documents and data on all clinical trials should be aggregated and indexed for comparison and gap identification, and that good knowledge management and better use of trial identifiers will facilitate this. At present, medicine faces serious shortcomings in this area. With 75 trials and 11 systematic reviews being published every day on average better knowledge management must be a priority.”