Shifting the Paradigm to Pragmatic Evidence Generation

By: Richard Gliklich, MD, OM1 & Sonja Wustrack, MPH, OM1

Integrated Evidence Generation

Integrated evidence generation (IEG) is a framework for generating evidence to support decision-making in healthcare. IEG leverages a) existing real-world data sources; and b) new data collection to generate evidence that is ‘fit-for-purpose’[i]. This means that the real-world data (RWD) and the evidence that will be generated from it will meet the evidentiary standard for the intended use case. Regulatory use cases have different requirements and expectations than evidence for peer-reviewed research publications. As early as possible in the product life cycle, it is valuable to map out different evidentiary goals for understanding markets, gaining approvals, meeting post-marketing commitments, driving provider awareness and market access, and supporting commercialization in order to develop a comprehensive strategy. Inevitably, these strategies will require both existing and new data. While it is difficult to get different teams to collaborate on data needs across an organization, the earlier a strategy is developed, the more robust and cost-effective it will be. Occasionally, one may find that existing real-world data from electronic health records and medical claims is sufficient to perform a retrospective analysis to answer a research question. But, not infrequently, missing data is an issue. If that analysis requires data elements not routinely collected in real-world practice, such as clinician or patient-reported outcomes, biomarkers or medical device identifiers, then ancillary data collection will be required to meet the purpose.

Further, if the intent of IEG is to submit data for regulatory purposes, such as for an external control arm for a clinical trial or to meet a post-marketing commitment, then that evidence generation will need to meet additional requirements such as transparency of processes, traceability of data elements, compliance of data collection systems and auditability of data sources. Similar requirements may also become applicable for the collection and validation of outcomes for programs under the Inflation Reduction Act. A well developed strategy that both leverages data from existing networks and collects ancillary information to fill in gaps in a regulatory compliant way, is likely to fulfill many, many needs. As such, an IEG strategy considers broader goals, across multiple company departments and over time to create a streamlined, cost-effective approach over the long-term.

Reagan-Udall Report on Evidence Generation

In his recent remarks on Reagan-Udall’s report and recommendations on Evidence Generation(ii), FDA Commissioner Dr. Robert Califf said, “The premarket evaluation of drugs, biologics and devices does its job, but leaves many questions on the table after initial approval as these medical products move into clinical use…..Unfortunately, the clinical trials enterprise has not differentiated these different purposes for trials in a manner that enables a robust, fit for purpose post-marketing evidence generation system.” Dr. Califf argues that there is tremendous opportunity to simplify and digitize data collection both pre- and post-approval to create more pragmatic and less burdensome approaches to evidence generation. “Technology is no longer our limitation.”

Data and Study Automation

As noted above, automation is emerging as the key to lowering the burden and simplifying participation in both pre- and post-marketing studies. Some of the advantages of automation include:

● Accelerating study timelines and reducing costs
● Recruiting higher volumes of patients
● Minimizing burden to sites and patients
● Increasing the inclusion of more diverse patients
● Reducing bias in enrollment

Today, validated systems such as OM1’s automated study platforms aggregate data directly from integrated delivery systems and practices and process electronic health records and other data into normalized data sets. This includes the transformation of unstructured clinical narratives and reports into standardized and validated data points. These systems not only extract information from clinical text, they also utilize artificial intelligence to identify computable phenotypes and to estimate and amplify key outcomes. For example, these systems are trained to ‘read’ a biopsy report and extract key information, and isolate diagnoses such as treatment resistant depression that are not well coded, or will amplify outcomes such as the New York Heart Association (NYHA) classification in heart failure patients. These efforts are behind the scenes, leveraging technology that is designed both for research and regulatory submission purposes.

Data collection can now be broken down into active and passive. Active data collection means collecting specifically for the intended purpose because the data needed does not sufficiently exist in clinical records. Passive data collection refers to data that can be extracted from contemporaneous real-world sources such as electronic health records, claims databases and other systems and linked to the actively collected data. By leveraging automation, active data collection can be limited to the minimum data necessary to be collected, resulting in simpler studies from the provider and patient perspectives. These are pragmatic evidence generation studies and registries(iii) because they limit burden only to that which cannot otherwise be captured automatically. However, these studies are far richer because they combine active data with linkage to EMR, claims, mortality, social determinants and other relevant data sources for the same patients. Further, these automated studies can meet the requirements for regulatory submission of data. Finally, because these are technology-driven processes, these programs scale tremendously generating dramatic savings. In fact, evidence generation programs that combine custom active and linked passive data collection will typically be the most cost effective approach across product life cycle needs.

As an example, running more than seven years, a sponsored OM1 automated registry that uses these principles has enrolled more than 1 million patients. It has been used extensively to understand not only product performance and comparative effectiveness, but access to care based on social determinants of health.

Conclusion

Pharmaceutical companies are beginning to recognize the benefits of an IEG strategy for most products in development through commercialization. As Dr. Califf points out, the time is ripe to modernize and simplify data collection. Combining these imperatives, a well-defined evidence strategy and next-generation data automation, provides a cost-effective recipe for success in pragmatic evidence generation.

 

[i] https://www.fda.gov/science-research/science-and-research-special-topics/real-world-evidence last accessed 12/3/2023

[ii] https://www.fda.gov/news-events/speeches-fda-officials/remarks-dr-robert-califf-reagan-udall-foundations-report-and-recommendations-evidence-generation last accessed 12/3/2023

[iii] Gliklich RE, Leavy MB, Dreyer NA, editors. Registries for Evaluating Patient Outcomes: A User’s Guide [Internet]. 4th edition. Rockville (MD): Agency for Healthcare Research and Quality (US); 2020 Sep. Chapter 1, Patient Registries. Available from: https://www.ncbi.nlm.nih.gov/books/NBK562581/