Simple linear regression and correlation

Abstract

One of the simplest and yet a commonly occurring data analytic problem is exploring the relationship between two numerical variables. In many applications, one of the variables may be regarded as a response variable and the other as a predictor variable and the goal is to find the best fitting relationship between the two. The simplest equation one can fit to bivariate numerical data is a straight line. This chapter considers the problem of fitting a straight line by using the least squares method. It discusses statistical inferences for a simple linear regression model. There are four assumptions implicit in the simple linear regression model. They are normality, constant variance (called homoscedasticity), independence, and linearity. The chapter describes correlation analysis by assuming two random variables with a joint distribution and discusses statistical inference on population correlation coefficient by assuming a probability model for the joint distribution.