Unmeasured confounding and misclassification in studies estimating vaccine effectiveness against hospitalisation and death using electronic health records (EHRs): an evaluation of a multi-country European retrospective cohort study
Humphreys, James Nicolay, Nathalie Braeye, Toon Van Evercooren, Izaak Hansen, Christian Holm Moustsen-Helms, Ida Rask Sacco, Chiara Mateo-Urdiales, Alberto Castilla, Jesús Martínez-Baz, Iván
Humphreys, James
Nicolay, Nathalie
Braeye, Toon
Van Evercooren, Izaak
Hansen, Christian Holm
Moustsen-Helms, Ida Rask
Sacco, Chiara
Mateo-Urdiales, Alberto
Castilla, Jesús
Martínez-Baz, Iván
Series / Report no.
Open Access
Type
Journal Article
Article
Article
Language
en
Date
2025-12-17
Research Projects
Organizational Units
Journal Issue
Title
Unmeasured confounding and misclassification in studies estimating vaccine effectiveness against hospitalisation and death using electronic health records (EHRs): an evaluation of a multi-country European retrospective cohort study
Translated Title
Published in
BMC Med Res Methodol 2025; 26(1):9
Abstract
BACKGROUND: Electronic health record (EHR)-based observational studies can rapidly provide real-world data on vaccine effectiveness (VE), though EHR data may be prone to misclassification and unmeasured confounding.
METHODS: In VEBIS-EHR, a retrospective multi-country COVID-19 VE cohort study, we examined unmeasured confounding using a negative control outcome (death not related to COVID-19) and misclassification due to timing of data extraction. The evaluation spanned two periods (November-December 2023, January-February 2024), encompassing up to 18.7 million individuals across six EU/EEA countries. Vaccine confounding-adjusted hazard ratios (aHRs) were pooled using random-effects meta-analysis.
RESULTS: aHRs against non-COVID-19 mortality ranged from 0.35 (95% CI: 0.28-0.44) to 0.70 (0.66-0.73) when comparing vaccinated versus unvaccinated. Delaying EHR data extraction modestly increased the capture of outcome and exposure events, with some variation by vaccination status. Site-level fluctuations in aHRs did not meaningfully alter the overall pooled VE, suggesting stable estimates despite misclassification related to extraction timing.
CONCLUSIONS: We observed some evidence of unmeasured confounding when using non-COVID-19 deaths as a negative outcome, though the specificity of our negative control must be considered. This result may suggest overestimation of VE, but also the need for further analysis with more specific negative control outcomes and confounding-adjustment techniques. Addressing such confounding using richer data sources and more refined approaches remains critical to ensure accurate, timely VE estimates based on retrospective cohorts constructed using registry data. Extending the delay between the end of observation and data extraction modestly improves the completeness of exposure and outcome data, with limited effect on pooled VE estimates.