Abstract

The external validity of the estimated treatment effect from a clinical trial is in doubt when there are effect modifiers whose distribution in the target population differs from that in the trial. Adjusting an estimated treatment effect from a trial to predict its likely value for the target population is a process known as generalization. We review classical and contemporary approaches to this problem. The traditional method is post-stratification, or the reweighting of stratum-specific treatment effect estimates by population distribution proportions. Contemporary methods employ stratification or weighting techniques based on estimates of the probability that an individual is included in the trial, akin to the propensity score. These methods are more flexible in that they readily accommodate continuous effect modifiers. Estimating the probabilities, however, requires in principle that one have individual-level population data, which are seldom available for pharmaceutical trials. When the effect modifiers are all discrete, the post-stratification and probability-weighting approaches give essentially the same estimates. Naïvely computed standard errors with the contemporary methods may be inflated. We illustrate and compare generalization methods in a simulation and using data from the Lipids Research Clinics Coronary Primary Prevention Trial and the New York School Choice Experiment.