Effects of Confounding Bias in Coronavirus Disease 2019 (COVID-19) and Influenza Vaccine Effectiveness Test-Negative Designs Due to Correlated Influenza and COVID-19 Vaccination Behaviors

TLDR

The test‑negative design is widely used to estimate influenza and COVID‑19 vaccine effectiveness, but correlated vaccination behaviors can introduce confounding bias when controls include the other vaccine‑preventable acute respiratory illness. The authors quantified this bias by simulating study populations with varying vaccination probabilities, COVID‑19 VE, influenza VE, and proportions of controls from the alternate ARI. They calculated mean bias as the difference between estimated and true VE, classifying it as low (<10 %), moderate (10–20 %), or high (≥20 %) and found that positive correlation of vaccination probabilities leads to underestimation of VE. Simulations revealed that bias remains low when influenza controls are ≤25 % of COVID‑19 studies or SARS‑CoV‑2 controls ≤10 % of influenza studies, but increases with higher conditional vaccination probability, low efficacy of the target vaccine, and high efficacy of the confounding vaccine, making influenza VE estimates more susceptible and underscoring the need for researchers to account for this bias.

Abstract

The test-negative design is commonly used to estimate influenza and coronavirus disease 2019 (COVID-19) vaccine effectiveness (VE). In these studies, correlated COVID-19 and influenza vaccine behaviors may introduce a confounding bias where controls are included with the other vaccine-preventable acute respiratory illness (ARI). We quantified the impact of this bias on VE estimates in studies where this bias is not addressed.We simulated study populations under varying vaccination probabilities, COVID-19 VE, influenza VE, and proportions of controls included with the other vaccine-preventable ARI. Mean bias was calculated as the difference between estimated and true VE. Absolute mean bias in VE estimates was classified as low (<10%), moderate (10% to <20%), and high (≥20%).Where vaccination probabilities are positively correlated, COVID-19 and influenza VE test-negative studies with influenza and severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) ARI controls, respectively, underestimate VE. For COVID-19 VE studies, mean bias was low for all scenarios where influenza represented ≤25% of controls. For influenza VE studies, mean bias was low for all scenarios where SARS-CoV-2 represented ≤10% of controls. Although bias was driven by the conditional probability of vaccination, low VE of the vaccine of interest and high VE of the confounding vaccine increase its magnitude.Where a low percentage of controls is included with the other vaccine-preventable ARI, bias in COVID-19 and influenza VE estimates is low. However, influenza VE estimates are likely more susceptible to bias. Researchers should consider potential bias and its implications in their respective study settings to make informed methodological decisions in test-negative VE studies.