An official website of the United States government
Using internally developed risk models to assess heterogeneity in treatment effects in clinical trials.
Pubmed ID: 24425710
Pubmed Central ID: PMC3957096
Journal: Circulation. Cardiovascular quality and outcomes
Publication Date: Jan. 1, 2014
Affiliation: Departments of Neurology.
MeSH Terms: Humans, Cardiovascular Diseases, Clinical Trials as Topic, Risk Assessment, Multivariate Analysis, Models, Statistical, Reproducibility of Results, Decision Support Techniques, Precision Medicine, Bias
Grants: 1K08NS082597, K08 NS082597, P30 DK092926
Authors: Kent DM, Burke JF, Hayward RA, Nelson JP
Cite As: Burke JF, Hayward RA, Nelson JP, Kent DM. Using internally developed risk models to assess heterogeneity in treatment effects in clinical trials. Circ Cardiovasc Qual Outcomes 2014 Jan;7(1):163-9. Epub 2014 Jan 14.
Studies:
- Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial (ALLHAT)
- Aspirin-Myocardial Infarction Study (AMIS)
- Atrial Fibrillation Follow-Up Investigation of Rhythm Management (AFFIRM)
- Beta-Blocker Evaluation in Survival Trial (BEST)
- Beta-Blocker Heart Attack Trial (BHAT)
- Bypass Angioplasty Revascularization Investigation (BARI)
- Digitalis Investigation Group (DIG)
- Enhancing Recovery in Coronary Heart Disease Patients (ENRICHD)
- Hypertension Detection and Follow-Up Program (HDFP)
- Lipid Research Clinics (LRC) Coronary Primary Prevention Trial (CPPT)
- Multiple Risk Factor Intervention Trial for the Prevention of Coronary Heart Disease (MRFIT)
- Prevention of Events With Angiotensin-Converting Enzyme Inhibitor Therapy (PEACE)
- Studies of Left Ventricular Dysfunction (SOLVD)
- Systolic Hypertension in the Elderly Program (SHEP)
- Thrombolysis in Myocardial Ischemia Trial II (TIMI II)
Abstract
BACKGROUND: Recent proposals suggest that risk-stratified analyses of clinical trials be routinely performed to better enable tailoring of treatment decisions to individuals. Trial data can be stratified using externally developed risk models (eg, Framingham risk score), but such models are not always available. We sought to determine whether internally developed risk models, developed directly on trial data, introduce bias compared with external models. METHODS AND RESULTS: We simulated a large patient population with known risk factors and outcomes. Clinical trials were then simulated by repeatedly drawing from the patient population assuming a specified relative treatment effect in the experimental arm, which either did or did not vary according to a subject's baseline risk. For each simulated trial, 2 internal risk models were developed on either the control population only (internal controls only) or the whole trial population blinded to treatment (internal whole trial). Bias was estimated for the internal models by comparing treatment effect predictions to predictions from the external model. Under all treatment assumptions, internal models introduced only modest bias compared with external models. The magnitude of these biases was slightly smaller for internal whole trial models than for internal controls only models. Internal whole trial models were also slightly less sensitive to bias introduced by overfitting and less sensitive to falsely identifying the existence of variability in treatment effect across the risk spectrum compared with internal controls only models. CONCLUSIONS: Appropriately developed internal models produce relatively unbiased estimates of treatment effect across the spectrum of risk. When estimating treatment effect, internally developed risk models using both treatment arms should, in general, be preferred to models developed on the control population.