Is Spontaneous Data Dead?
Regardless, if the analytical method is qualitative or quantitative, there is one constant which is the data source itself. Adverse Events also known as "Spontaneous Data" is increasingly coming into question by industry professionals in terms of its value due to accuracy and specificity issues. Spontaneous data is considered dubious due its very nature due to several factors:
Reporting Sources: Adverse Events can be submitted by anyone including consumers, health care professionals, pharmacists, attorneys and even literature references.
Collection Source: AEs are collected by numerous parties including sponsors, insurance companies, 3rd party call centers and even received directly by the FDA itself.
As evidenced in the FDA Adverse Event Reporting System (AERS) available via the Freedom of Information Act (FOI) this situation creates duplicate reports, incomplete data, inconsistent terminology and data quality issues (e.g. misspellings). Worse yet, the lack of reporting control can easily create a scenario whereby the number of events can be artificially inflated due to "solicitation" by legal firms and consumer advocacy groups e.g. 1-800-BAD-DRUG.
Over the past 10 years, I have met with numerous industry professionals in various roles associated with a product's safety profile including; Drug Safety, Risk Management and Epidemiology. During those meetings opinions are many and passions run high when it comes to the topic of how safety should be assessed and the value of source data. On one hand, no one can dispute the fact that spontaneous data is flawed. Drug Safety often makes the argument while Spontaneous Data is not perfect it is the industry standard and "doing nothing is not an option". Thus we must do everything in our power to maintain high quality in terms of the analytical process and the data itself. In the opposite camp, I have witnessed Epidemiologists openly call the Drug Safety process "poppycock" and even "voodoo" due to Spontaneous data and how it is collected. While both camps have a personal and professional dog in this hunt, they both have valid points. Further, they will both agree that they often have common adversaries in Legal and Marketing who are very conservative when it comes to classifying something as a "safety concern". So if everyone acknowledges that Spontaneous Data is inherently flawed, what if any are the alternatives? Over the past few years, the use of longitudinal patient data has increase dramatically. Often called "Observational Data", it is the collection of several data sources including patient health records, prescription data and health insurance claims. A relatively new and expensive data source made possible through the advent of electronic health records and sales data the use of observational data was usually reserved for formal Epidemiology studies due to higher quality resulting from greater historical information on a given drug and/or disease. Lately, more and more Safety Scientists are increasing their use of observational data even if for just "Signal Strengthening" which is another way of saying testing of hypotheses generated through the mining of spontaneous data.
The much anticipated CIOMS Working Group VIII report on "Practical Aspects of Signal Detection in Pharmacovigilance is expected to provide guidance if not a solution to this dilemma as it specifically addresses approaches to signal detection including both traditional statistical data mining methods and interpretation of results as well as discussion on limitations and challenges of spontaneous data. According to the June 2010 CIOMS Newsletter, the final report is in press and is expected to be publicly available shortly. However, according to the newsletter "The report aims primarily to provide a comprehensive resource for those considering how to strengthen their pharmacovigilance systems and practices and to give practical advice. But the report does not specify instant solutions. These will inevitably be situation specific and require careful consideration taking into account local needs." Sounds like a punt to me.
The publication goes on to state; "Finally, in looking ahead the report anticipates a number of ongoing developments, including techniques with wider applicability to other data forms than individual case reports. The ultimate test for pharmacovigilance systems is the demonstration of public health benefit and it is this test which signal detection methodologies need to meet if the expectations of all stake holders are to be fulfilled."
While I will reserve judgment until I can read the final report, it sounds like no one is willing to formally declare spontaneous data dead due to the enormous investments made over the years to collect it by both the FDA and manufacturers and quite possibly risk their own professional reputations as well due to previous advocacy. However, they are clearly leaving the door open to the use of alternative data sources such as observational data or others. In my professional opinion based on my experience over the past two years, I will also not declare Spontaneous Data dead at this time. But it is definitely on life support. It should also be noted that a number of valuable components have been created in support of spontaneous data that will likely continue to provide value in the next generation of Pharmacovigilance namely the MedDRA dictionary. Further, my direct experience analyzing spontaneous data supports the conclusion that trends and outliers can be identified in spontaneous data either through statistical analysis of disproportionality or through simple trending and visualization. In my opinion the key to successful signal detection is the consistent application of one's own training and experience when it comes to forming a conclusion based any data source. Analysis is a human, cognitive skill that will not soon be replaced by technology. My objective is to create technology solutions that allows one to apply their knowledge in a simple and consistent manner.
