Pragmatic Free Trial Meta
Pragmatic Free Trial Meta is a non-commercial open data platform and infrastructure that supports research on pragmatic trials. It is a platform that collects and shares clean trial data and ratings using PRECIS-2 which allows for multiple and varied meta-epidemiological studies that examine the effects of treatment across trials that employ different levels of pragmatism as well as other design features.
Background
Pragmatic trials are increasingly acknowledged as providing evidence from the real world to support clinical decision-making. However, the use of the term "pragmatic" is inconsistent and its definition as well as assessment requires further clarification. The purpose of pragmatic trials is to guide clinical practice and policy decisions, rather than confirm the validity of a clinical or physiological hypothesis. A pragmatic study should strive to be as close as is possible to the real-world clinical practice which include the recruiting participants, setting, designing, delivery and execution of interventions, determining and analysis outcomes, and primary analyses. This is a major distinction between explanatory trials as described by Schwartz and Lellouch1 which are designed to test the hypothesis in a more thorough manner.
Truly pragmatic trials should not blind participants or clinicians. This can result in bias in the estimations of the effect of treatment. Pragmatic trials will also recruit patients from various health care settings to ensure that the outcomes can be compared to the real world.
Furthermore, trials that are pragmatic must be focused on outcomes that matter to patients, such as quality of life and functional recovery. This is especially important in trials that require the use of invasive procedures or could have dangerous adverse impacts. The CRASH trial29 compared a 2 page report with an electronic monitoring system for hospitalized patients with chronic heart failure. The catheter trial28 on the other hand, used symptomatic catheter associated urinary tract infection as the primary outcome.
In addition to these features pragmatic trials should also reduce the procedures for conducting trials and requirements for data collection to reduce costs and time commitments. Furthermore, pragmatic trials should seek to make their results as applicable to clinical practice as possible by making sure that their primary analysis is based on the intention-to-treat method (as described in CONSORT extensions for pragmatic trials).
Many RCTs which do not meet the criteria for pragmatism, but contain features in opposition to pragmatism, have been published in journals of various types and incorrectly labeled pragmatic. This could lead to false claims of pragmatism, and the usage of the term should be standardized. The development of a PRECIS-2 tool that offers an objective and standardized assessment of pragmatic features is a good start.
Methods
In a pragmatic research study it is the intention to inform clinical or policy decisions by showing how an intervention can be integrated into routine treatment in real-world situations. Explanatory trials test hypotheses regarding the cause-effect relation within idealized settings. In this way, pragmatic trials can have lower internal validity than explanatory studies and be more prone to biases in their design as well as analysis and conduct. Despite these limitations, pragmatic trials can be a valuable source of information for decision-making in the context of healthcare.
The PRECIS-2 tool assesses the level of pragmatism that is present in an RCT by scoring it across 9 domains ranging from 1 (very explicit) to 5 (very pragmatic). In this study the areas of recruitment, organization and flexibility in delivery, flexible adherence, and follow-up received high scores. However, the principal outcome and method of missing data scored below the pragmatic limit. This suggests that a trial could be designed with effective practical features, yet not harming the quality of the trial.
It is, however, difficult to judge how pragmatic a particular trial is, since the pragmatism score is not a binary attribute; some aspects of a study can be more pragmatic than others. Additionally, logistical or protocol modifications during the course of a trial can change its score in pragmatism. Koppenaal and colleagues discovered that 36% of 89 pragmatic studies were placebo-controlled or conducted prior to the licensing. Most were also single-center. They aren't in line with the usual practice, and can only be called pragmatic if their sponsors agree that the trials aren't blinded.
A typical feature of pragmatic research is that researchers attempt to make their findings more meaningful by studying subgroups of the trial sample. This can lead to unbalanced comparisons with a lower statistical power, increasing the likelihood of missing or incorrectly detecting differences in the primary outcome. This was the case in the meta-analysis of pragmatic trials due to the fact that secondary outcomes were not corrected for covariates' differences at baseline.
In addition, pragmatic studies may pose challenges to collection and interpretation safety data. It is because adverse events are usually self-reported and are susceptible to delays, errors or coding variations. It is important to increase the accuracy and quality of the results in these trials.
Results
Although the definition of pragmatism does not require that all trials be 100 percent pragmatic, there are some advantages to incorporating pragmatic components into clinical trials. These include:
Increasing sensitivity to real-world issues, reducing the size of studies and their costs and allowing the study results to be faster translated into actual clinical practice (by including routine patients). However, pragmatic trials may also have disadvantages. The right type of heterogeneity for instance could allow a study to expand its findings to different patients or settings. However, the wrong type can reduce the sensitivity of an assay, and therefore lessen the power of a trial to detect small treatment effects.
A number of studies have attempted to classify pragmatic trials with a variety of definitions and scoring systems. Schwartz and Lellouch1 developed an approach to distinguish between explanation-based trials that support the clinical or physiological hypothesis and pragmatic trials that help in the selection of appropriate therapies in the real-world clinical setting. The framework was composed of nine domains assessed on a scale of 1-5 with 1 being more explanatory while 5 was more practical. The domains included recruitment of intervention, setting up, delivery of intervention, flexible adhering to the program and primary analysis.
The original PRECIS tool3 was based on a similar scale and domains. Koppenaal and colleagues10 created an adaptation of the assessment, called the Pragmascope, that was easier to use for systematic reviews. They discovered that pragmatic reviews scored higher in all domains, but scored lower in the primary analysis domain.
This distinction in the primary analysis domain can be explained by the way that most pragmatic trials approach data. Some explanatory trials, however, do not. The overall score for systematic reviews that were pragmatic was lower when the domains of organization, flexible delivery, and following-up were combined.
It is important to remember that a pragmatic study should not mean a low-quality trial. In fact, there is a growing number of clinical trials which use the word 'pragmatic,' either in their abstract or title (as defined by MEDLINE however it is neither precise nor sensitive). The use of these words in abstracts and titles could suggest a greater awareness of the importance of pragmatism, however, it is not clear if this is manifested in the contents of the articles.
Conclusions
As the importance of evidence from the real world becomes more widespread and pragmatic trials have gained popularity in research. They are clinical trials randomized that compare real-world care alternatives instead of experimental treatments under development. They include patients which are more closely resembling those treated in routine care, they use comparisons that are commonplace in practice (e.g., existing medications) and rely on participant self-report of outcomes. This method is able to overcome the limitations of observational research for example, the biases associated with the reliance on volunteers, and the lack of codes that vary in national registers.
Other advantages of pragmatic trials include the possibility of using existing data sources, and a greater chance of detecting meaningful changes than traditional trials. However, these tests could have some limitations that limit their effectiveness and generalizability. The participation rates in certain trials could be lower than expected due to the health-promoting effect, financial incentives, or competition from other research studies. A lot of pragmatic trials are restricted by the necessity to enroll participants on time. Practical trials aren't always equipped with controls to ensure that the observed differences aren't due to biases that occur during the trial.
The authors of the Pragmatic Free Trial Meta identified RCTs that were published between 2022 and 2022 that self-described themselves as pragmatic. The PRECIS-2 tool was used to determine the pragmatism of these trials. It includes domains such as eligibility criteria and flexibility in recruitment as well as adherence to interventions and follow-up. 프라그마틱 무료체험 메타 discovered that 14 of the trials scored highly or pragmatic sensible (i.e., scoring 5 or more) in any one or more of these domains and that the majority of these were single-center.
Trials with a high pragmatism score tend to have higher eligibility criteria than traditional RCTs, which include very specific criteria that are not likely to be found in the clinical setting, and contain patients from a broad variety of hospitals. The authors claim that these characteristics can help make pragmatic trials more meaningful and applicable to everyday practice, but they do not guarantee that a pragmatic trial is free of bias. Moreover, the pragmatism of trials is not a definite characteristic; a pragmatic trial that does not contain all the characteristics of an explanatory trial can yield valid and useful results.