Print page Resize text Change font-size Change font-size Change font-size High contrast

Home > Standards & Guidances > Methodological Guide

ENCePP Guide on Methodological Standards in Pharmacoepidemiology


15.6. Real-world evidence and pharmacoepidemiology


15.6.1. Introduction


The pharmacoepidemiology community has a long tradition of producing, evaluating, and interpreting observational data to provide evidence on the use, safety and effectiveness of medicines. The increasing ability to electronically capture and store data from routine healthcare systems and transform it into new knowledge has opened up new opportunities for investigators to conduct studies. The concept of “real-world data” (RWD) and “real-world evidence” (RWE) has been increasingly used since the early 2000s to denote evidence generated from observational data collected during routine patient-level healthcare interactions. Its scope is wider than medicines evaluation as it is also applied in other domains, such as health technology assessment, health economics, patient-reported outcomes and disease epidemiology.


The concept of RWD and RWE is sometimes presented as a distinct scientific discipline (as illustrated by expressions like “expertise in RWE”) despite the absence (as for now) of a firm theoretical foundation and specific body of knowledge. However, there is currently no agreed definition of RWD and RWE and Real World Evidence – Where Are We Now? (N Engl J Med. 2022;386(18):1680-2) emphasises that these terms are being used inconsistently and sometimes interchangeably. There is also a lack of clarity on what data or information should be considered as RWD or RWE and how they relate to pharmacoepidemiology.


This chapter discusses definitions of RWD and RWE, their role in medicines evaluation, their relation to evidence generated by clinical trials, and why pharmacoepidemiological methods remain essential for the generation and assessment of RWD and RWE.


15.6.2. Definitions


The recency of the terms RWD and RWE may explain the lack of internationally agreed definitions. Amongst existing definitions, Real-World Data for Regulatory Decision Making: Challenges and Possible Solutions for Europe (Clin Pharmacol Ther. 2019;106(1):36-9) defines RWD as “routinely collected data relating to a patient’s health status or the delivery of health care from a variety of sources other than traditional clinical trials”, and RWE as “the information derived from the analysis of RWD”. The FDA’s Real-World Evidence website defines RWD as “the data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources” and RWE as “clinical evidence regarding the usage and potential benefits or risks of a medical product derived from analysis of RWD. RWE can be generated by different study designs or analyses, including but not limited to, randomized trials, including large simple trials, pragmatic trials, and observational studies (prospective and/or retrospective)”. The first definition seems to exclude many clinical trials and the second one seems to include them when the design uses observational data. The scope of RWE in the second definition seems also focussed on medicines evaluation.


The concept of RWD and RWE, as used today, only partially overlaps with traditional classification of clinical research such as randomised vs. non-randomised, prospective vs. retrospective or primary vs. secondary data collection. Marketing Authorization Applications Made to the European Medicines Agency in 2018-2019: What was the Contribution of Real-World Evidence? (Clin Pharmacol Ther. 2021;111(1):90-7) illustrates the difficulties of applying operational definitions of RWD and RWE, notably when they are included as elements of clinical trials in authorisation applications. Randomized, observational, interventional, and real-world-What's in a name? (Pharmacoepidemiol. Drug Saf. 2020;29(11):1514-7) considers the terminology regarding RWE commonly used in the scientific community. It concludes that, as regards use of RWE for regulatory decisions, the “randomised trial versus observational study” dichotomy is overly simplistic and clarity is needed regarding interventional or non-interventional design, primary collection or secondary use of data, and characteristics of comparison group(s), as well as an assessment of cause-effect association.


In summary, the term “real-world” is a descriptor of the source of the data and of the evidence, but the question is what does it describe exactly. RWD is commonly understood as observational data which are most often secondary data from various origins (e.g., electronic healthcare records, claims data, registries) but may also originate from primary data collection (e.g., data collected in an observational study, data collected with digital wearable devices or patients’ or physicians’ surveys) or a combination of both. The term RWE is commonly used as the result of the analysis of RWD using well validated and appropriate methods that may combine design elements of observational studies and clinical trials. There is however no consensus on this understanding and the term RWE sometimes simply means inferences made by exploring unstructured data. This chapter does not attempt to solve the differences between these definitions.


15.6.3. Use of real-world evidence in medicines evaluation


There are many examples where RWD and RWE are submitted to support medicines evaluation and regulatory decision-making. Three main objectives are identified in EMA’s DARWIN EU®: Multi-stakeholder information webinar (2022; slides 14-21): to support the planning and validity of applicant studies, to understand the clinical context and to investigate associations and impact, with sub-categories for each of these objectives. Real-world evidence to support regulatory decision making: New or expanded medical product indications (Pharmacoepidemiol Drug Saf. 2021;30(6):685-93) reviews more specifically study designs used to generate RWE, including pragmatic trials, externally controlled trials and non-randomised healthcare database studies, among others.


Several studies have recently attempted to measure the frequency of use of RWD or RWE in marketing authorisation applications and the extent to which these data were actually utilised for decision-making (see, for example, Marketing Authorization Applications Made to the European Medicines Agency in 2018-2019: What was the Contribution of Real-World Evidence? Clin Pharmacol Ther. 2021;111(1):90-7; The Role of Real- World Evidence in FDA- Approved New Drug and Biologics License Applications. Clin Pharmacol Ther. 2022;111(1):133-44; Use of Real-World Data and Evidence in Drug Development of Medicinal Products Centrally Authorized in Europe in 2018–2019. Clin Pharmacol Ther. 2022;111(1):310-20). Due to variability in definitions, data sources and study designs, very different estimates were found in these studies, with percentages of applications including RWE ranging from 39.9% to 100%.


More work is necessary for an in-depth analysis of the actual contribution of RWE in the decision-making on marketing authorisation approvals, why such information was not considered in some cases and how it contributed to the approval decision in other cases. This information would help complement existing recommendations to medicines developers published by regulatory agencies on the submission of RWE within their applications. Among other guidance available on the FDA’s Real-World Evidence website, a draft FDA guidance for industry provides Considerations for the Use of Real-World Data and Real-World Evidence to Support Regulatory Decision-Making for Drug and Biological Products(2021) and the draft guidance Real-World Data: Assessing Electronic Health Records and Medical Claims Data To Support Regulatory Decision-Making for Drug and Biological Products (2021) provides recommendations in three domains: data sources (relevance of data source and data capture), study design elements (time period, study population, exposure, outcome, covariates) and data quality. The MHRA guidance on the use of real-world data in clinical studies to support regulatory decisions (2021) emphasises the importance of the quality of the data source, including its accuracy, validity, variability, reliability and provenance, with areas of consideration prior to submitting the study protocol. The MHRA guideline on randomised controlled trials using real-world data to support regulatory decisions (2021) provides points to consider when planning a prospective randomised trial using RWD sources with the intention of using the trial to support a regulatory decision, together with examples of scenarios, endpoints and designs. Health Canada’s Elements of Real-World Data/Evidence Quality throughout the Prescription Drug Product Life Cycle (2019) provides overarching principles to guide the generation of RWE and an overview of some of the elements that should be addressed in protocol development and documentation of data quality within submissions containing RWE. The EMA’s Guideline on registry-based studies provide recommendations on key methodological aspects that are specific to the use of patient registries by marketing authorisation applicants and holders planning to conduct registry-based studies for regulatory purposes.


15.6.4. Real-world evidence vs. clinical trials


The value of RWE to provide unbiased evidence on medicinal products as compared to clinical trials is a frequent subject of debate in the context of regulatory assessments, especially for medicines efficacy or effectiveness where departure from traditional clinical trials has been called to speed-up their pace, reduce their cost and increase their generalisability. While RCTs are the gold standard for demonstrating the efficacy of medicinal products, they rarely measure the benefits and risks of an intervention when used in routine clinical practice and the current thinking is moving away from the long-held position that RWE is always inferior due to the likelihood of bias. Randomized Controlled Trials Versus Real World Evidence: Neither Magic Nor Myth (Clin Pharmacol Ther. 2021;109(5):1212–8) illustrates that the question is no more about RCTs vs. RWE but about RCTs and RWE, not only for the assessment of safety but also effectiveness. It also highlights that, in the era of precision medicine, some small treatment effects cannot be described either by RCTs or RWE alone.


It is now widely accepted that use of observational evidence is generally not appropriate to replace RCT information, except in specific circumstances, but that both are complementary to generate optimal evidence. However, Real World Evidence – Where Are We Now? (N Engl J Med. 2022;386(18):1680-2) suggests that RWD/RWE and RCTs are not quite different concepts as randomised or non-randomised interventional studies may rely on RWD for different objectives and therefore generate RWE, as illustrated by the following diagram:


Reliance on RWD in Representative Types of Study Design. RCT denotes randomized, controlled trial; RWD real-world data; and RWE real-world evidence. Source: Concato J, Corrigan-Curay JD. Real World Evidence – Where Are We Now? (N Engl J Med. 2022;386(18):1680-2).


Examples of use cases of RWE in medicines development is presented in EMA’s DARWIN EU®: Multi-stakeholder information webinar (2022; slides 14-21). When Can We Rely on Real‐World Evidence to Evaluate New Medical Treatments? (Clin Pharmacol Ther. 2021;111(1):30-4) recommends that decisions regarding use of RWE in the evaluation of new treatments should depend on the specific research question, characteristics of the potential study settings and characteristics of the settings where study results would be applied, and take into account three dimensions in which RWE studies might differ from traditional clinical trials: use of RWD, delivery of real-world treatment and real-world treatment assignment. As regards the appropriateness of RWE for clinical questions, When and How Can Real World Data Analyses Substitute for Randomized Controlled Trials? (Clin Pharmacol. Ther. 2017;102(6):924-33) suggests that RWE is likely to be preferred over RCTs when studying a highly promising treatment for a disease with no other available treatments, where ethical considerations may preclude randomising patients to placebo, particularly if the disease is likely to result in severely compromised quality of life or mortality. In these cases, RWE could support product regulation by providing evidence on the safety and effectiveness of the therapy against the typical disease progression observed in the absence of treatment. This comparator disease trajectory may be assessed from historical controls that were diagnosed prior to the availability of the new treatment, or other sources.


A current domain of research is the assessment of whether RWE studies can provide the same results as RCTs performed for the same research question, such as in Emulating Randomized Clinical Trials With Nonrandomized Real-World Evidence Studies: First Results From the RCT DUPLICATE Initiative (Circulation 2021;143(10):1002-13). Such research does not aim to show that RWE can replace RCTs but it may give confidence in the validity of RWE studies based on health care data if they can consistently match the results of published trials and even predict the results of ongoing trials.


15.6.5. Real-world evidence and pharmacoepidemiology


Use of RWE to support regulatory decision-making depends on several factors and several publications describe components of RWE that determine whether it is relevant and acceptable in this context. Data quality frameworks documenting the suitability of RWD and RWE for regulatory purposes and other research questions are described in Chapter 12.2. Real-World Data for Regulatory Decision Making: Challenges and Possible Solutions for Europe (Clin Pharmacol Ther. 2019; 106(1):36-9) specifies four criteria for acceptability of RWE for regulatory purposes: it should be derived from data sources of demonstrated good quality, valid (with both internal and external validity), consistent (or heterogeneity should be explained) and adequate in terms of amount of information provided. FDA’s draft Real-World Data: Assessing Electronic Health Records and Medical Claims Data To Support Regulatory Decision-Making for Drug and Biological Products (2021) discusses three domains: data sources, study design elements and data quality. Assessing and Interpreting Real-World Evidence Studies: Introductory Points for New Reviewers (Clin Pharmacol. 2022;111(1):145-9) details three aspects: the research question evaluated in the RWE study must align with the question of interest, with a recommendation to break it down according to the Population, Intervention, Comparator Outcome and Timing (PICOT) framework; the study design must use valid methods minimising selection bias, information bias and confounding, with a recommendation to use the target trial framework to help plan and design the RWE study; and the data must be suitable to address the research question, with elements of reliability (incl. plausibility and missingness) and relevance. Reporting to Improve Reproducibility and Facilitate Validity Assessment for Healthcare Database Studies V1.0 (Pharmacoepidemiol Drug Saf. 2017;26(9):1018-32) highlights that substantial improvement in reproducibility, rigor and confidence in RWE generated from healthcare databases could be achieved with greater transparency about operational study parameters used to create analytic datasets from longitudinal healthcare databases and provides lists of specific parameters to be reported to increase reproducibility of studies.


All the elements cited above represent fundamental principles of pharmacoepidemiology that are described in detail in this ENCePP Guide. While RWD refers to the availability of a large amount of data from different sources, RWE relies on the application of sound epidemiological methods to analyse such data. It should nevertheless be acknowledged that the concept of RWD and RWE has stimulated better visibility, accessibility and quality control of data sources as well as methodological developments to prevent and control bias and confounding, for example confounding by indication. Given the importance taken by RWD and RWE, especially in the context of the SARS-CoV-2 pandemic, pharmacoepidemiologists should embrace it as a domain of research supporting regulatory decisions on medicinal products and public health in general.


As the concept of RWD and RWE is based on essential principles of pharmacoepidemiology, it is important to determine how pharmacoepidemiologists can best support RWE studies. The following list includes areas of pharmacoepidemiological expertise that ENCePP considers important to develop and disseminate:

  • Understanding of different data types (e.g., primary care, specialist care, hospital care, disease registries, claims data, longitudinal drug prescription, dispensing or other drug utilisation data).

  • Understanding of the context in which the data are collected, which should include – but not be limited to – local diagnostic criteria, local prescribing practices, local prescribing formularies, local coding practices, reimbursement policies, etc.

  • Understanding of real-world data sources, including:

    • common coding terminologies for drug exposure and clinical events,

    • common data models,

    • assessment of data quality (incl. data quality metrics, data quality frameworks, misclassification and missingness, benchmarking),

    • their limitations and the statistical approaches to address them.

  • Expertise in epidemiological study designs, including traditional designs as well as case-only and case-population designs; studies with primary data collection vs. secondary use of data; prevalent-user vs. incident-user designs, positive and negative control exposures and outcomes; use of active exposure vs. non-exposure comparator groups.

  • Knowledge of mechanisms of bias in observational studies (information bias, selection bias, confounding) and methods to address them at the design and analytical stages (incl. restriction, matching, stratification, modelling, use of propensity score methods); methods to address unmeasured confounding and time-dependent confounding.

  • Knowledge in handling effect modification and interaction in observational studies.

  • Expertise in assessing and validating different exposures, outcomes and covariates in observational studies.

  • Knowledge in causal inference methods (incl. missing data handling, target trial emulation and interplay with ICH E9 (R1)).

  • Knowledge in meta-analysis and data pooling.

  • Experience in assessing a statistical analysis plan for a RWE study.

15.6.6. Conclusion


Real-world data and real-world evidence have become important components of the scientific information supporting medicines evaluation, regulatory decision-making, health technology assessment and public health in general. Although they have been used in the fields of drug utilisation, disease epidemiology and drug safety for decades (without being labelled as such), their application to the field of medicines efficacy or effectiveness, especially in the context of the SARS-CoV-2 pandemic, and their integration into different types of design, including RCTs, have led to increased attention and methodological scrutiny of their strengths and limitations, especially regarding data quality and validity of the evidence. Further developments would require international agreement on definitions and methodological standards required to support regulatory decisions alongside clinical trials.


Pharmacoepidemiology is a core scientific discipline sustaining the generation and assessment of valid and reliable real-world evidence. Pharmacoepidemiologists should take a leadership role in the development and testing of methods, design and conduct of studies and adequate reporting of the evidence.



« Back