Abstract

BACKGROUND: Randomized clinical trials (RCTs) define evidence-based medicine, but quantifying their generalizability to real-world patients remains challenging. We propose a multidimensional approach to compare individuals in RCT and electronic health record (EHR) cohorts by quantifying their representativeness and estimating real-world effects based on individualized treatment effects (ITE) observed in RCTs. METHODS: We identified 65 pre-randomization characteristics of an RCT of heart failure with preserved ejection fraction (HFpEF), the Treatment of Preserved Cardiac Function Heart Failure with an Aldosterone Antagonist Trial (TOPCAT), and extracted those features from patients with HFpEF from the EHR within the Yale New Haven Health System. We then assessed the real-world generalizability of TOPCAT by developing a multidimensional machine learning-based phenotypic distance metric between TOPCAT stratified by region including the United States (US) and Eastern Europe (EE) and EHR cohorts. Finally, from the ITE identified in TOPCAT participants, we assessed spironolactone benefit within the EHR cohorts. RESULTS: There were 3,445 patients in TOPCAT and 8,121 patients with HFpEF across 4 hospitals. Across covariates, the EHR patient populations were more similar to each other than the TOPCAT-US participants (median SMD 0.065, IQR 0.011-0.144 vs median SMD 0.186, IQR 0.040-0.479). At the multi-variate level using the phenotypic distance metric, our multidimensional similarity score found a higher generalizability of the TOPCAT-US participants to the EHR cohorts than the TOPCAT-EE participants. By phenotypic distance, a 47% of TOPCAT-US participants were closer to each other than any individual EHR patient. Using a TOPCAT-US-derived model of ITE from spironolactone, all patients were predicted to derive benefit from spironolactone treatment in the EHR cohort, while a TOPCAT-EE-derived model predicted 13% of patients to derive benefit. CONCLUSIONS: This novel multidimensional approach evaluates the real-world representativeness of RCT participants against corresponding patients in the EHR, enabling the evaluation of an RCT's implication for real-world patients.