The Method of Least Squares

The Method of Least Squares

  • Regression line (best fit) passes through \((\bar{x},\bar{y})\).
  • \(e_1,e_2,\ldots,e_6\) represent the \(y\) errors.
  • Aim to minimise the values of the errors,

    \(\sum e_k^2=e_1^2+e_2^2+\cdots+e_6^2\)

  • Regression line of \(y\) on \(x\) obtained.
  • Regression line (best fit) passes through \((\bar{x},\bar{y})\).
  • \(e_1,e_2,\ldots,e_6\) represent the \(x\) errors.
  • Aim to minimise the values of the errors,

    \(\sum e_k^2=e_1^2+e_2^2+\cdots+e_6^2\)

  • Regression line of \(x\) on \(y\) obtained.

Important notes:

  1. With nonzero standard deviations and \(0<|r|<1\), the \(x\)-on-\(y\) line is steeper in absolute slope when both are plotted with \(x\) horizontal and \(y\) vertical. They coincide when \(|r|=1\). When \(r=0\), the lines are \(x=\bar{x}\) and \(y=\bar{y}\).
  2. The point\(\left( \bar{x},\bar{y} \right)\) lies on both the regression lines of “\(y\) on \(x\)” and “\(x\) on \(y\)”.

Video Solution:

Video Solution

Video solution locked

A free account saves your progress; this video needs tuition enrolment.

Solution:

Solution locked

680+ questions here come with free worked solutions. A free account saves your questions and progress; this one needs tuition enrolment.

Create a free account to practise similar questions.

Answer:The \(y\)-on-\(x\) regression minimises squared \(y\)-residuals; the \(x\)-on-\(y\) regression minimises squared \(x\)-residuals. Both pass through \((\bar{x},\bar{y})\). Choose according to the variable being predicted. Interpolation stays within the observed range; strong correlation alone does not guarantee reliable extrapolation.

Need help? Join our JC Math tuition classes.

Learn more