ENCePP Guide on Methodological Standards in Pharmacoepidemiology

11.2. Methods of statistical signal detection

Quantitative analysis of spontaneous adverse drug reaction reports is routinely used in drug safety research. Several articles have been published on statistical signal detection. Quantitative signal detection using spontaneous ADR reporting (Pharmacoepidemiol Drug Saf. 2009;18(6):427-36) describes the core concepts behind the most common methods, the proportional reporting ratio (PRR), reporting odds ratio (ROR), information component (IC) and empirical Bayes geometric mean (EBGM). The authors also discuss the role of Bayesian shrinkage in screening spontaneous reports and the importance of changes over time in screening the properties of the measures.

Additionally, they discuss major areas of controversy (such as stratification and evaluation and implementation of methods) and give some suggestions as to where emerging research is likely to lead. Data mining for signals in spontaneous reporting databases: proceed with caution (Pharmacoepidemiol Drug Saf. 2007;16(4):359–65) reviews data mining methodologies and their limitations and provides useful points to consider before incorporating data mining as a routine component of any pharmacovigilance program. Disproportionality Analysis for Pharmacovigilance Signal Detection in Small Databases or Subsets: Recommendations for Limiting False-Positive Associations (Drug Saf. 2020;43(5):479-87) evaluates the impact of database size on the performance of disproportionality analysis, with regards to limiting spurious associations.

Methods such as multiple logistic regression have the theoretical capability to reduce masking and confounding by co-medication and underlying disease. Regression-Adjusted GPS Algorithm describes the use of regression to increase the discriminatory power of the Gamma Poisson Shrinkage (GPS) algorithm. Data-Driven Prediction of Drug Effects and Interactions (Sci Transl Med. 2012 Mar 14; 4(125): 125ra31) describes the application of regression methods to correct for synthetic associations caused by hidden, or unmeasured, covariates as well as those from indication and concomitant drug use. The letter Logistic regression in signal detection: another piece added to the puzzle (Clin Pharmacol Ther. 2013;94(3):312) highlights the variability of results obtained in different studies based on this method and the daunting computational task it requires. More work is needed on its value for pharmacovigilance in the real-world setting.

A more recent proposal involves a broadening of the basis for computational screening of individual case safety reports, by considering multiple aspects of the strength of evidence in a predictive model. This approach combines disproportionality analysis with features such as the number of well-documented reports, the number of recent reports and geographical spread of the case series (Improved statistical signal detection in pharmacovigilance by combining multiple strength-of-evidence aspects in vigiRank, Drug Saf. 2014;37(8):617–28). In a similar spirit, logistic regression has been proposed to combine a disproportionality measure with a measure of unexpectedness for the time-to-onset distribution (Use of logistic regression to combine two causality criteria for signal detection in vaccine spontaneous report data, Drug Saf. 2014;37(12):1047-57). In A prediction model‐based algorithm for computer‐assisted database screening of adverse drug reactions in the Netherlands (Pharmacoepidemiol Drug Saf. 2018;27(2):199-205), five relevant characteristics (number of reports, disproportionality, Naranjo score, proportion of MAH reports, proportion of HCP reports) were chosen as potential predictors in the model and tested against the presence in the Summary of Product Characteristics (SmPC) of each unique drug‐ADR association at the time of the analysis. All candidate predictors were included into the final model with an increased screening efficiency. The authors comment that the choice of candidate predictors may depend on each spontaneous report databases but that the method of generating a prediction model‐based priority list of signals could be useful in other databases.

Methods for statistical signal detection tend to classify reports based on reported adverse event terms considered one at time. Broader categories such as High-Level Terms or Standardized MedDRA Queries are sometimes used to group similar adverse events and improve sensitivity. However, this may be at the expense of specificity. Consensus clustering for case series identification and adverse event profiles in pharmacovigilance (Artif Intell Med, 2021; 122:1-9) proposes a different approach where cluster analysis attempts to identify case series describing similar clinical conditions, accounting for the complete sets of signs, symptoms, and diagnoses on each report.

Disproportionality methods are usually calculated on the cumulative data and therefore do not provide a direct insight into temporal changes in frequency of reports. Methodologies to monitor changes in the frequency of reporting over time have been developed with the focus to enhance pharmacovigilance when databases are small, when drugs have established safety profiles and/or when product quality defects, medication errors and cases of abuse or misuse are of concern. Automated method for detecting increases in frequency of spontaneous adverse event reports over time (J Biopharm Stat. 2013; 23(1):161-77) presents a regression method with both smooth trend and seasonal components, while An algorithm to detect unexpected increases in frequency of reports of adverse events in EudraVigilance (Pharmacoepidemiol Drug Saf. 2018;27(1):38-45) presents the testing of a model based on a negative binomial time-series regression model on thirteen historical concerns. Additionally, a modification of the Information Component to screen for spatial-temporal disproportionality is described in Using VigiBase to Identify Substandard Medicines: Detection Capacity and Key Prerequisites (Drug Saf. 2015; 38(4): 373–82). Despite the promising results of these methods, and even if theoretically they seem appealing, limited work has been performed to assess their effectiveness.

« Back