The concept of performing more pragmatic clinical trials is gaining momentum. Recently, the National Institutes of Health (NIH) published a website described as a “living textbook” to explain and promote the concept of pragmatic clinical trials.1 Separately, the Food and Drug Administration (FDA)
recently published guidance for investigators and sponsors entitled, “Use of Real-World Evidence to Support Regulatory Decision-Making for Medical Devices.”2

This document discusses what pragmatic clinical trials are, and how they are designed and intended to answer important questions in health care.

What Are Pragmatic Clinical Trials and Why Are They of Interest?

Califf and Sugarman defined pragmatic clinical trials as research that is “…designed for the primary purpose of informing decision-makers regarding the comparative balance of benefits, burdens and risks of a biomedical or behavioral health intervention at the individual or population level.”3

The motivation for pragmatic clinical trials comes from two observations. First, many treatments and procedures frequently used in the practice of medicine have never been rigorously studied with a clinical trial. Therefore, most of the healthcare that is delivered in the United States is based on anecdotal evidence and is not what is referred to as “evidence-based medicine.” Second, there is a high degree of variability in the type of care delivered. A patient can get surgery, drugs, or watchful waiting depending upon which physician they choose to see.

Which of these options will lead to the best outcomes? Randomized clinical trials could potentially answer this question. On the other hand, it is hard to turn back the clock, and perform randomized clinical trials on procedures generally assumed to be safe and effective. The concept of a pragmatic clinical trial was developed to create a trial that can practically answer questions like, “Is the healthcare we are delivering really safe and effective?” and “Which of the healthcare options should we use in practice to provide the best care to the patient?”

Future Trends in Clinical Trials

Parsing out the definition of “pragmatic clinical trial” can be difficult. There is overlap between the classic randomized clinical trial and the pragmatic clinical trial. Instead of focusing on the definitions and dividing clinical trials into two baskets, it can be better to think about how the concept of a pragmatic clinical trial will affect future trends in the conduct of all clinical trials. In this regard, there are two major differences. Pragmatic clinical trials are focused on care delivered to the patient versus focused on a specific treatment or procedure. Pragmatic clinical trials are also focused on effectiveness versus efficacy.

Care-Focused Versus Intervention-Focused

Clinical trials are classically conducted to evaluate a specific intervention, such as a new drug, a new device, or new surgical procedure. Payors, government agencies, and researchers are putting greater emphasis on clinical trials that focus on health care in a broader sense. Today, a clinical trial might evaluate whether a new drug lowers hemoglobin A1c without evaluating the acceptability of the treatment to the patient or the effect of non-pharmacologic interventions. In the future, we can expect more trials that evaluate whether a combination of diet, drug treatment, education, and
supervision lowers hemoglobin A1c in a manner that is readily adopted by the patient.

Another trend is to have study questions developed by clinicians and patients versus researchers and sponsors. Currently, a clinical trial is usually focused on gathering data to obtain approval from regulatory agencies. In the future, we might see more clinical trials that gather data to answer frequent questions of patients and clinicians, sometimes while also designed to support regulatory approvals. A typical patient question might be, “If I have lung disease, what level of supplemental oxygen needed at home indicates that it is no longer safe for me to fly?” A typical clinician question might be, “If I treat this disease aggressively, which patients will live longer with an improved quality of life, and which patients will just live longer?” In this sense, the trend is for clinical trials to become more patient focused rather than disease-focused, and to compare and evaluate real-world situations and alternatives rather than comparing an intervention to a placebo.

Focus on Effectiveness Versus Efficacy

Although most clinical trials start out with a statement that the purpose of the research is to evaluate the safety and effectiveness of an intervention, clinical trials typically evaluate the safety and efficacy of an intervention.

Efficacy is the performance of an intervention under ideal circumstances, whereas effectiveness is the performance of an intervention under real-world circumstances. In the setting of a classic clinical trial, many variables are controlled to maximize the probability of getting clear answers to a few, narrowly defined research questions, and to minimize the probability of adverse outcomes. Staff are carefully trained in the intervention and the use of the intervention is highly controlled. Not infrequently, an intervention will be efficacious but will fail to be effective for reasons such as steep learning curve for clinician competency, or inability of patients to comply with the treatment. Sometimes, the inclusion criteria of a protocol demonstrating efficacy for a disease are so strict that the study results don’t apply to most patients with that disease. In this regard, the trend is for clinical trials that embrace heterogeneous subject populations to avoid studying a homogeneous population of no practical application.

Along with the trend to focus on effectiveness versus efficacy, is a trend for research that studies data gathered by clinicians versus data gathered by investigators. The goal is to capture data that the clinicians feel is necessary to evaluate their patients, rather than collecting structured data. The focus is on data collection dictated by healthcare versus data collection dictated by the protocol. There is also resignation to the fact that actual medical practice cannot be rigorously structured without the intensive resources used in many of today’s clinical trials. Therefore, there is a trend to allow protocol flexibility versus a machine-like rigidity. The data gathered by clinicians is often called “real-world data” and enormous amounts of these data are embedded in electronic medical records. These data are much less structured than the data collected in a classic protocol, but the hope is that machine learning techniques (“big data” or “artificial intelligence”) can be brought to bear to discover a strong signal despite the added noise.


In the United States, healthcare costs are skyrocketing while objective measures of quality lag far behind other industrialized nations. This has led clinicians, medical administrators, and thought leaders to seek objective answers to questions about what forms of healthcare are cost-effective. Pragmatic clinical trials have been proposed to get these answers. We can expect to see more research in the future that is focused on real-world care, designed to compare the effectiveness of combinations of interventions without conducting placebo-controlled trials, and are focused on effectiveness in the real-world rather than efficacy under artificial conditions.

  1. “Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials”

  3. Califf RM, Sugarman J. 2015. Exploring the ethical and regulatory issues in pragmatic clinical trials. Clin Trials. 12:436–441. Doi:10.1177/1740774515598334.