Linear Regression Models (lm()) in R

Linear Regression Models (lm()) in R

In the fields of statistical analysis and data science, linear regression is one of the most fundamental and widely applied tools for exploring relationships between variables. Leveraging its powerful statistical computing library stats, R provides the comprehensive lm() function to build linear models. This article delves into how to use lm() for model fitting, result interpretation, and diagnostics, helping beginners master this core skill.
The lm() function is the core tool in R for fitting linear regression models. Its most basic invocation form is model <- lm(response ~ predictor, data = dataset). Here, response is the response variable (dependent variable), predictor is the predictor variable (independent variable), and dataset is the object containing the relevant data.

The function is highly flexible; besides handling simple single-variable regression, it also supports multiple regression, polynomial regression, and the inclusion of non-linear terms. For instance, to examine the effect of height on weight while considering age as a covariate, one can construct the following model:

model <- lm(weight ~ height + age, data = dataset)

In the formula, the ~ symbol represents "predicted by," with the left side being the response variable and the right side containing the predictor variables. To include interaction terms or non-linear relationships (such as quadratic terms), one can add product terms to the formula or wrap polynomial expressions using the I() function, such as height^2 or height * age.

Model Fitting and Result Interpretation

After calling lm(), R returns an object containing a wealth of information. To view the summary statistics of the model, the most common approach is to print the object directly or use the summary() function. The summary output typically includes several key components:

  • Coefficients: Show the direction and magnitude of the effect of each predictor on the response variable. For example, if a coefficient is positive, it implies that an increase in that variable tends to result in an increase in the response variable.
  • Standard Error (Std. Error): Measures the precision of the coefficient estimate; smaller values generally indicate more reliable estimates.
  • t-value and P-value: Used to test whether a coefficient is significantly different from zero. If the P-value is less than the significance level (typically 0.05), we reject the null hypothesis and conclude that the variable is statistically significant.
  • R-squared (R²): Represents the proportion of variance in the data explained by the model, with a range between 0 and 1; values closer to 1 indicate a better fit.

Here is a simple code example to view the results:

summary(model)

Through the output above, researchers can quickly identify which variables are important and preliminarily assess the rationality of the model.

Model Diagnostics and Residual Analysis

Fitting a model is not the end of the work; verifying whether the model assumptions align with the data characteristics is the crucial step to ensure reliable results. Linear regression typically relies on four basic assumptions: linearity, homoscedasticity, normality, and independence. Among these, residual analysis is the most direct diagnostic method.

Residuals are the differences between observed values and predicted values ($e_i = y_i - \hat{y}_i$). We can check model quality by plotting residuals. In R, plot(model) automatically generates four subplots, displaying:

  1. Residuals vs Fitted: Used to detect non-linear relationships and heteroscedasticity (i.e., variance of errors changing with predicted values). Ideally, points should be randomly distributed above and below the zero line without any obvious trend or fan-shaped distribution.
  2. Normal Q-Q Plot: Used to test if residuals follow a normal distribution. If points roughly lie on the diagonal line, the normality assumption is met; if there is obvious curvature or deviation, this assumption may be violated.
  3. Residuals vs Order: Used to detect correlations among predictor variables (especially important for cross-sectional data).
  4. Histogram of Standardized Residuals: Visually displays the distribution shape of residuals.

If residual plots show clear non-linear trends, polynomial terms might need to be introduced, or data transformations considered; if heteroscedasticity is detected, one might consider using weighted least squares (the weights parameter) or Generalized Linear Models (GLM).

Practical Case: Housing Price Prediction Analysis

To illustrate the application of lm() more intuitively, we construct a simplified housing price prediction case. Assume we have a dataset housing_data containing house area, age, and final selling price. Our goal is to build a model to predict housing prices.

First, load the data and view the first few rows:

# Assume housing_data is already loaded
head(housing_data)

Construct a linear regression model with "price" as the response variable and "area" and "age" as predictors:

price_model <- lm(price ~ area + age, data = housing_data)
summary(price_model)

Assume the summary() output shows that the coefficient for "area" is 1500 with a very small P-value, and the coefficient for "age" is -200 with a small P-value. This means that for every additional square meter, the average house price increases by 1500 yuan, while for every additional year of age, the average house price decreases by 200 yuan.

Further residual diagnostics:

plot(price_model)

If observation reveals no abnormal patterns in the residual plots and a high R² (e.g., 0.85), it indicates that this linear model can well explain the patterns of housing price changes. At this stage, we can utilize the predict() function to calculate the expected price for new houses:

new_housing <- data.frame(area = 120, age = 10)
predicted_price <- predict(price_model, newdata = new_housing)
print(predicted_price)

The output will provide the predicted price for the house based on the current model, offering a quantitative basis for actual decision-making.

Summary and Advanced Recommendations

The lm() function is the cornerstone of statistical inference in R. Mastering it means not only being able to run code but also understanding the statistical logic behind the data. In practical operations, do not blindly trust the model output; strictly combine it with residual plots for diagnostics. Furthermore, for complex non-linear relationships or categorical response variables, R offers more advanced tools like glm() (Generalized Linear Models), which are worth exploring in further study. Through the combination of theory and practice, you will be able to more efficiently use R to solve various data analysis problems.