Reweighting the RCT for generalization: finite sample analysis and variable selection

Bénédicte Colnet (1, 2, 3) , Julie Josse (2) , Gaël Varoquaux (1) , Erwan Scornet (3)

1 SODA - Méthodes computationnelles et mathématiques pour comprendre la société et la santé à partir de données
2 PREMEDICAL - Médecine de précision par intégration de données et inférence causale
3 CMAP - Centre de Mathématiques Appliquées - Ecole Polytechnique

The limited scope of Randomized Controlled Trials (RCT) is increasingly under scrutiny, in particular when samples are unrepresentative. Indeed, some RCTs over-or under-sample individuals with certain characteristics compared to the target population, for which one want to draw conclusions on treatment effectiveness. Re-weighting trial individuals to match the target population helps to improve the treatment effect estimation. Such procedures require an estimation of the ratio of the two densities (trial and target distributions). In this work, we establish the exact expressions of the bias and variance of such reweighting procedures-also called Inverse Propensity of Sampling Weighting (IPSW)-in presence of categorical covariates for any sample size. Such results allow us to compare the theoretical performance of different versions of IPSW estimates. Besides, our results show how the performance (bias, variance and quadratic risk) of IPSW estimates depends on the two sample sizes (RCT and target population). A by-product of our work is the proof of consistency of IPSW estimates. A refined analysis also shows that IPSW performances are improved when the trial probability to be treated is estimated (rather than using its oracle counterpart). In addition, we study how including covariates that are unnecessary to a proper estimation of the weights may impact the asymptotic variance. We illustrate all the takeaways twice: in a toy and didactic example, and on a semi-synthetic simulation inspired from critical care medicine.

Mots clés: Average treatment effect (ATE); Sampling bias; External validity; Transportability; Distributional shift; IPSW

Domaines: Statistiques [math.ST]

Bénédicte Colnet, Julie Josse, Gaël Varoquaux, Erwan Scornet. Reweighting the RCT for generalization: finite sample analysis and variable selection. 2022. ⟨hal-03822662v1⟩