Jesca Birungi post the following in LinkedIn:
A variable has a p-value of 0.32. Should we exclude it from our multivariable regression model?
In many epidemiological studies, variable selection follows a familiar approach:
1. Run bivariable analyses.
2. Select variables with p < 0.20 or p < 0.25.
3. Enter the selected variables into a multivariable regression model.
4. Retain variables that meet a specified significance threshold.
But here is a question worth considering.
Can a variable be an important confounder even when its bivariable association with the outcome is not statistically significant?
Yes.
A variable’s role in confounding is not determined by its p-value alone.
Consider a hypothetical study investigating the relationship between an exposure and a health outcome. Age is a plausible confounder based on existing epidemiological knowledge, but its bivariable association with the outcome has a p-value of 0.32.
Should we automatically exclude age?
Not necessarily.
If age is a common cause of the exposure and outcome, excluding it could introduce confounding bias into our effect estimate.
Interestingly, this debate has been discussed in the methodological literature for decades.
Mickey and Greenland (1989) examined different approaches to confounder selection, including significance testing and change-in-estimate criteria. Their findings highlighted limitations of conventional significance thresholds for confounder control.
More recently, VanderWeele (2019) discussed principles for selecting adjustment variables based on their causal relationships with the exposure and outcome.
The important distinction is this
A variable can be statistically non-significant yet still be important for confounding control.
This does not mean that every available variable should automatically be included in a regression model.
Adjusting for mediators, colliders or inappropriate variables can also introduce bias.
Instead, decisions about confounder adjustment should be informed by the research question, existing evidence, subject-matter knowledge and, where appropriate, causal diagrams.
Statistical criteria may still have a role in model development, particularly when the number of candidate variables is large. But they should not replace careful consideration of the underlying causal relationships.
As biostatisticians and epidemiologists, perhaps we need to ask a different question:
Not simply, “Is this variable statistically significant?” but “Why should this variable be included in the model?”