Best Subsets Logistic Regression

TLDR

Covariate selection is critical in regression, and best‑subset algorithms—originally developed for linear models and extended by Lawless and Singhal for nonnormal regressions—provide an efficient screening approach now available in software. This note demonstrates that logistic regression can be analyzed with the Lawless‑Singhal best‑subset method using any case‑weight‑capable linear regression program. The approach follows Gilks’s general model‑selection framework and employs Mallows’s predictive‑error statistic to compare variable subsets.

Abstract

Selection of covariates is an important step in any regression modeling situation. An effective strategy when the model is a linear regression with normal errors is to use a software package that implements a best subsets selection algorithm. A number of years ago Lawless and Singhal (1978) proposed a method for efficiently screening nonnormal regression models, thus providing the basis for best subsets nonlinear regression. Recently Lawless and Singhal (1987a, 1987b) have made available a software package that implements their method. The purpose of this note is to illustrate that for one of the more frequently used nonnormal regression models, logistic regression, we may perform the Lawless-Singhal analysis with any best subsets linear regression program that allows for case weights. The methods presented may also be obtained from the general method for model selection proposed by Gilks (1986). We also discuss the use and interpretation of Mallows's measure of predictive squared error as a statistic for comparing models with different subsets of variables.