Resize text 
Resize text
16.6.1. Introduction
16.6.2. Definitions
16.6.3. Use of real-world evidence in medicines evaluation
16.6.4. Real-world evidence vs. clinical trials
16.6.5. Real-world evidence and pharmacoepidemiology
The pharmacoepidemiology community has a long tradition of producing, evaluating, and interpreting observational data to provide evidence on the use, safety and effectiveness of medicines. The increasing ability to electronically capture and store data from routine healthcare systems and transform it into additional knowledge has opened up new opportunities for investigators to conduct studies. The terms real-world data (RWD) and real-world evidence (RWE) have been increasingly used since the early 2000’s to denote evidence generated from observational data collected during routine patient-level healthcare interactions. In medicines evaluation, evidence relying on RWD is now frequently submitted across the lifecycle of a product to complement and contextualise clinical trial knowledge with information from the routine healthcare setting, but the place of RWD in regulatory decision-making is still a subject of debate (see for example Replacing RCTs with real world data for regulatory decision making: a self-fulfilling prophecy? BMJ. 2023:380:e073100). Contribution of Real-World Evidence in European Medicines Agency’s Regulatory Decision Making (Clin Pharmacol Ther. 2023;113(1):136-51) reports that RWD/RWE was considered not supportive or was not further addressed in the regulatory evaluation report for 15 of 26 applications submitted to EMA in 2018-2019, where RWD/RWE was included to support efficacy pre-authorisation. Many issues discussed in the evaluation reports with respect to RWE were weaknesses related to methodological aspects, highlighting the need for adequate pharmacoepidemiological and statistical expertise in the generation of RWE.
There is currently no internationally agreed definition of RWD and RWE. Real World Evidence – Where Are We Now? (N Engl J Med. 2022;386(18):1680-2) emphasises that these terms are being used inconsistently and sometimes interchangeably across different health domains. Although evolving, a consistent terminology is yet to be established.
This chapter discusses ENCePP’s views on definitions of RWD and RWE, their role in medicines approval and evaluation, their relation to evidence generated by clinical trials, and why pharmacoepidemiological methods remain essential for the generation and assessment of RWD and RWE.
The FDA’s Real-World Evidence website defines RWD as “the data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources” and RWE as “clinical evidence regarding the usage and potential benefits or risks of a medical product derived from analysis of RWD”. These definitions are widely used and have been adopted by other regulatory agencies. There is however a debate about the appropriateness of including both the nature of the data and the way it is collected (“routinely”) in the definition of RWD. RWD is commonly understood as observational data from various origins (e.g., electronic healthcare records, claims data, registries) but Marketing Authorization Applications Made to the European Medicines Agency in 2018-2019: What was the Contribution of Real-World Evidence? (Clin Pharmacol Ther. 2021;111(1):90-7) illustrates the difficulty of applying definitions of RWD in authorisation applications, notably when RWD is included as an element of clinical trials. Real-World Trends in the Evaluation of Medical Products (Am J Epidemiol. 2023;192(1):1-5) states that there is room for interpretation as to the data considered as RWD, for example data collected outside of health care settings for research purposes such as those collected through patient self-report outside of clinical encounters, or data collected through new technologies such as wearable biometric devices. This comment also applies to genetic data that are often collected outside the context of routine care or clinical trials but are generally considered as RWD. It is also noted that the data quality frameworks developed for RWD (see Chapter 13.2) examine how accurately the data represent the original information and how suitable they are but not how routinely they have been collected.
The view of ENCePP is that the specificity of RWD in comparison to any other observational data lies in the requirement for a true representation of the “real-world” patient characteristics (i.e., data with a high external validity) without influence of any specific study conditions. An assessment and validation of this real-world attribute, e.g., by external validation or benchmarking, is needed to provide assurance that it applies, or at least to evaluate and understand the deviation that may exist. A simpler definition of RWD could therefore only refer to patient data in contemporary clinical practice.
RWE is information derived from the analysis of RWD using sound epidemiological and statistical practices. The term RWE does not refer to specific methodologies and overlaps with pharmacoepidemiology, although it only partially overlaps with traditional classification of clinical research such as randomised vs. observational, prospective vs. retrospective or primary data collection vs. secondary use of data. The term RWE is nevertheless useful to state that the evidence originates from RWD, in the same way as the term experimental evidence is sometimes used to state that the evidence is based on experimental data.
16.6.3. Use of real-world evidence in medicines evaluation
There are many examples where RWD and RWE can be submitted to support medicines evaluation and regulatory decision-making. Three main objectives are identified in EMA’s DARWIN EU®: Multi-stakeholder information webinar (2022; slides 14-21):
to support the planning and validity of applicant studies, for example to inform the recruitment in pre- and post-authorisation studies, to examine the impact of planned inclusion/exclusion criteria, to measure the representativeness of the CT population (treatment and control arm) vs. the real-world target population and to evaluate whether the standard of care used in the control arm of a CT is comparable with the current real-word standard of care;
to understand the clinical context, for example to evaluate the incidence, prevalence and characteristics of diseases, to generate evidence on the actual clinical standards of care and compare them in different populations, and to characterise real-world drug use (incidence, prevalence, amount, duration, switching patterns);
to investigate associations and impact, for example to investigate the association between treatment exposure and either effectiveness or safety outcomes (including use of RWD as external control group), and to monitor the implementation and the effectiveness of risk minimisation measures.
Several studies have recently attempted to measure the frequency of use of RWD or RWE in marketing authorisation applications and the extent to which these data were actually utilised for decision-making, see, for example:
Use of real-world evidence in postmarketing medicines regulation in the European Union: a systematic assessment of European Medicines Agency referrals 2013-2017. BMJ Open 2019;9(10):e028133
Marketing Authorization Applications Made to the European Medicines Agency in 2018-2019: What was the Contribution of Real-World Evidence? Clin Pharmacol Ther. 2021;111(1):90-7
The Role of Real- World Evidence in FDA- Approved New Drug and Biologics License Applications. Clin Pharmacol Ther. 2022;111(1):133-44;
Use of Real-World Data and Evidence in Drug Development of Medicinal Products Centrally Authorized in Europe in 2018–2019. Clin Pharmacol Ther. 2022;111(1):310-20.
Due to variability in definitions, data sources, study designs and acceptability of RWD by regulatory decision-making bodies, very different estimates were found in these studies, with percentages of authorisation applications including RWE ranging from 39.9% to 100%.
How to enhance the suitability and acceptability of RWD/RWE to support authorisation applications is a matter of discussion and several publications have made proposals:
Contribution of Real-World Evidence in European Medicines Agency’s Regulatory Decision Making (Clin Pharmacol Ther. 2023;113(1):136-51) provides an in-depth analysis of the actual contribution of RWE in the decision-making on marketing authorisation approvals of applications submitted to EMA in 2018-2019, why such information was not considered supportive in some cases and how it contributed to the approval decision in other cases. It discusses suggestions to enable broader use of RWE in medicines development, including provision of data on mechanisms of action where RWE is used to extrapolate efficacy data from adults to children, previous experience with the medicinal product outside the EU application, description of the disease population and natural course of the disease, and early interactions (such as through scientific advice) between applicants and regulators to discuss the expected value of RWD to answer a specific research question, their limitations and how they could be minimised.
Harnessing Real-World Evidence to Advance Cancer Research (Curr. Oncol. 2023;30(2):1844-59) proposes a strategy with four steps: 1) to identify meaningful and well-defined clinical questions answerable with available RWD rather than scenarios for which RCTs are necessary and feasible; 2) to rely on high-quality RWD representative of the population of interest and contemporary clinical practice and with documented data completeness and provenance; 3) to use appropriate study designs accounting for data limitations, bias, confounding and sensitivity analyses; 4) to use clear, transparent and replicable study methodology to increase the confidence in the results.
Assessing and Interpreting Real-World Evidence Studies: Introductory Points for New Reviewers (Clin Pharmacol. 2022;111(1):145-9) details three aspects: the research question evaluated in the RWE study must align with the question of interest, with a recommendation to break it down according to the Population, Intervention, Comparator Outcome and Timing (PICOT) framework; the study design must use valid methods minimising selection bias, information bias and confounding, with a recommendation to use the target trial framework to help plan and design the RWE study; and the data must be suitable to address the research question, with elements of reliability (incl. plausibility and missingness) and relevance.
When Can We Rely on Real‐World Evidence to Evaluate New Medical Treatments? (Clin Pharmacol Ther. 2021;111(1):30-4) recommends that decisions regarding use of RWE in the evaluation of new treatments should depend on the specific research question, characteristics of the potential study settings and characteristics of the settings where study results would be applied, and take into account three dimensions in which RWE studies might differ from traditional clinical trials: use of RWD, delivery of real-world treatment and real-world treatment assignment.
Real-world evidence to support regulatory decision making: New or expanded medical product indications (Pharmacoepidemiol Drug Saf. 2021;30(6):685-93) reviews more specifically study designs used to generate RWE, including pragmatic trials, externally controlled trials and non-randomised healthcare database studies, among others.
Real-World Data for Regulatory Decision Making: Challenges and Possible Solutions for Europe (Clin Pharmacol Ther. 2019; 106(1):36-9) specifies four criteria for acceptability of RWE for regulatory purposes: it should be derived from data sources of demonstrated good quality, valid (with both internal and external validity), consistent (or heterogeneity should be explained) and adequate in terms of amount of information provided.
When and How Can Real World Data Analyses Substitute for Randomized Controlled Trials? (Clin Pharmacol. Ther. 2017;102(6):924-33) suggests that RWE is likely to be preferred over RCTs when studying a highly promising treatment for a disease with no other available treatments, where ethical considerations may preclude randomising patients to placebo, particularly if the disease is likely to result in severely compromised quality of life or mortality. In these cases, RWE could support product regulation by providing evidence on the safety and effectiveness of the therapy against the typical disease progression observed in the absence of treatment.
Reporting to Improve Reproducibility and Facilitate Validity Assessment for Healthcare Database Studies V1.0 (Pharmacoepidemiol Drug Saf. 2017;26(9):1018-32) highlights that substantial improvement in reproducibility, rigor and confidence in RWE generated from healthcare databases could be achieved with greater transparency about study parameters used to create analytic datasets from longitudinal healthcare databases and provides lists of specific parameters to be reported to increase reproducibility of studies.
Regulatory agencies have also published methodological recommendations to medicines developers on the submission of RWD/RWE within their applications to support their evaluation and acceptability:
Among other guidance available on the FDA’s Real-World Evidence website, a draft FDA guidance for industry provides Considerations for the Use of Real-World Data and Real-World Evidence to Support Regulatory Decision-Making for Drug and Biological Products(2021), the draft guidance Real-World Data: Assessing Electronic Health Records and Medical Claims Data To Support Regulatory Decision-Making for Drug and Biological Products (2021) provides recommendations in three domains: data sources (relevance of data source and data capture), study design elements (time period, study population, exposure, outcome, covariates) and data quality, and the draft guidance Considerations for the Design and Conduct of Externally Controlled Trials for Drug and Biological Products (2023) provides recommendations to sponsors and investigators considering the use of externally controlled clinical trials to provide evidence of the safety and effectiveness of a drug product.
The MHRA guidance on the use of real-world data in clinical studies to support regulatory decisions (2021) emphasises the importance of the quality of the data source, including its accuracy, validity, variability, reliability and provenance, with areas of consideration prior to submitting the study protocol. The MHRA guideline on randomised controlled trials using real-world data to support regulatory decisions (2021) provides points to consider when planning a prospective randomised trial using RWD sources with the intention of using the trial to support a regulatory decision, together with examples of scenarios, endpoints and designs.
Health Canada’s Elements of Real-World Data/Evidence Quality throughout the Prescription Drug Product Life Cycle (2019) provides overarching principles to guide the generation of RWE and an overview of some of the elements that should be addressed in protocol development and documentation of data quality within submissions containing RWE.
The Guidance for Reporting Real-World Evidence (2023) lays the foundation for the use of RWE in regulatory approval and Health Technology Assessment (HTA) in Canada; it focusses on guiding the evaluation of a study, providing core reporting standards and prioritising transparency in reporting while accounting for practical challenges related to RWD and RWE.
The EMA’s Guideline on registry-based studies (2021) provides recommendations on key methodological aspects that are specific to the use of patient registries by marketing authorisation applicants and holders planning to conduct registry-based studies for regulatory purposes.
16.6.4. Real-world evidence vs. clinical trials
The value of RWE to provide valid evidence on medicinal products as compared to clinical trials is a frequent subject of debate in the context of regulatory assessments, especially for medicines effectiveness where a departure from traditional clinical trials has been called on to speed-up their pace, reduce their cost and increase their generalisability. While RCTs are the gold standard for demonstrating the efficacy of medicinal products, they rarely measure the benefits and risks of an intervention when used in contemporary clinical practice and the current thinking is moving away from the long-held position that RWE is always inferior due to the likelihood of bias. Randomized Controlled Trials Versus Real World Evidence: Neither Magic Nor Myth (Clin Pharmacol Ther. 2021;109(5):1212–8) illustrates that the question is not about RCTs vs. RWE but about RCTs and RWE. In other words, use of observational evidence should generally not be considered to replace RCT information, except in specific circumstances, but both are complementary, as RWE may provide additional data, such as on longer follow-up of interventions and on treatment effects in populations not included in RCTs. Real World Evidence – Where Are We Now? (N Engl J Med. 2022;386(18):1680-2) suggests that randomised, non-randomised interventional and non-randomised non-interventional studies may rely on RWD for different objectives and therefore generate RWE, as illustrated by the following diagram:
Reliance on RWD in Representative Types of Study Design. RCT denotes randomized, controlled trial; RWD real-world data; and RWE real-world evidence. Source: Concato J, Corrigan-Curay JD. Real World Evidence – Where Are We Now? (N Engl J Med. 2022;386(18):1680-2).
Statistical Considerations When Using RWD and RWE in Clinical Studies for Regulatory Purposes: A Landscape Assessment (Statistics in Biopharmaceutical Research 2023;15:1,3-13) discusses examples of when RWD can be incorporated into the design of various study types, including RCTs and purely observational studies, and reviews biostatistical challenges and methods for the use of RWE for medicinal product development.
A current domain of research is the assessment of whether non-interventional RWE studies can provide the same results as RCTs performed for the same research question. Emulation of Randomized Clinical Trials With Nonrandomized Database Analyses: Results of 32 Clinical Trials (JAMA 2023;329(16):1376-85) concludes that RWE studies can reach similar conclusions as RCTs when design and measurements can be closely emulated, but this may be difficult to achieve. Concordance in results varied depending on the agreement metric. Emulation differences, chance, and residual confounding can contribute to divergence in results and are difficult to disentangle.
16.6.5. Real-world evidence and pharmacoepidemiology
All the elements cited above to generate valid and reliable RWE using RWD are related to fundamental principles of pharmacoepidemiology. The widespread use of the concept of RWD/RWE has stimulated the use, accessibility and quality control of data sources as well as methodological developments to prevent and control bias and confounding, for example confounding by indication. Pharmacoepidemiologists should therefore take a leadership role and embrace this concept as a domain of research supporting regulatory decisions on medicinal products and public health in general. The following list includes areas of pharmacoepidemiological expertise that ENCePP considers important to develop and disseminate:
- Knowledge about RW data source metadata and its characteristics
- Knowledge about appropriate methods to establish meaningful RW evidence
For the UK, as from 1.1.2021, EU Law applies only to the territory of Northern Ireland (NI) to the extent foreseen in the Protocol on Ireland/NI
