Multiple Linear Regression Model

Linear regression is a linear approach to modeling the relationship between a scalar response (or dependent variable) and one or more independent variables. The case of one independent variable is called simple linear regression. For more than one independent variable, the process is called multiple linear regression.

Given a data set {y_i, x_i1, ..., x_ip} where i = 1,...,n, a linear regression model assumes that the relationship between the dependent variable y and the p-vector of regressors x is linear. This relationship is modeled through a disturbance term or error variable ε - an unobserved random variable that adds noise to the linear relationship between the dependent variable and regressors. Thus the model takes the form:

y_i = β₀ + β₁x_i1 +...+ β_px_ip + ε_i = x_i^T + ε_i, i = 1,...,n

where β₀ is the y-intercept, ^T is the transpose operator and x_i^Tε is the inner product between vectors x_i and ε. These n equations can be written together in matrix notation as:

Y = Xβ + ε

This system of equations can be solved using the singular value decomposition (SVD) algorithm. The routine used in QtiPlot is adapted from svdecomp.c in XLISP-STAT 2.1 by Luke Tierney and David Betz.