The Impact of Model Selection on Inference in Linear Regression

Abstract

Abstract Model selection and inference are usually treated as separate stages of regression analysis, even though both tasks are performed on the same set of data. Once a model has been selected, one typically proceeds as though one has a fresh data set generated by the selected model. Here, we present Monte Carlo results on the coverage rates of confidence regions for the regression parameters, conditional on the selected model order. The conditional coverage rates are much smaller than the nominal coverage rates, obtained by assuming that the model was known in advance. Furthermore, the overall coverage rate is much smaller than the nominal value. A possible remedy based on data splitting is suggested.