Least-Squares Geometry Explorer

Fit real-world data with a line or curve, then inspect the residuals that least squares is trying to shrink.

Worked example — Least-Squares Geometry Explorer

Points	6
Fitted model	42.7143+5.9143x
Sum of squared errors (SSE)	1.3714
RMSE	0.4781
Largest-point SSE share	37.2%

These are the values the calculator shows when the page opens. Change any input above to recompute them.

Loading interactive simulation...

Lesson

The theory — Least-Squares Geometry Explorer

“Least squares” names what is being minimised: the sum of squared residuals. Geometrically the fitted values are the closest point to your data within the set of curves the model can produce — closest in the ordinary straight-line sense, which is why the leftover residual vector ends up perpendicular to everything the model could have done.

What each symbol means

residual: one gap between a data point and the fitted curve, measured vertically. The readout lists every one before squaring it.
SSE: the sum of squared errors — add up the squares of all the residuals. 1.3714 for the straight-line fit here.
RMSE: the root mean square error, √(SSE/n) = 0.4781. Unlike SSE it is in the same units as y, so it is the one worth quoting.
model: the family of curves allowed. Switching from line to quadratic widens that family from two free numbers to three.

Where the formula comes from

Each candidate curve produces a list of fitted values. Treat that list as a single point in n-dimensional space, and treat your data as another point there.
SSE is then literally the squared distance between those two points. Minimising SSE means finding the reachable point closest to the data — a projection, not a search.
And the defining property of a projection is that the error is perpendicular to everything reachable. That is why the residuals of a best fit always sum to zero and show no leftover correlation with x: any such pattern would be a direction the model could still have moved in.

How to read what you see

The point count, then the fitted model written out, then SSE shown term by term — −0.7143² + 0.3714² + 0.4571² + ⋯ = 1.3714 — so you can see which point is hurting the fit before the squares hide the signs. RMSE comes last as the human-readable summary.

Assumes: Vertical gaps only, so x is treated as exact, and every point weighted equally. The model must also be linear in its parameters — which a quadratic still is, despite the curve, because x² enters multiplied by a constant.
Breaks when: A smaller SSE does not mean a better model, and you can prove it here in one click. Switch from line to quadratic and SSE falls from 1.3714 to 0.4071 — it was always going to fall, because a quadratic contains every straight line as the special case where the x² coefficient is zero. A model with more freedom cannot fit worse, so SSE alone can never tell you whether the extra term is real or is fitting noise.

The best-fit line always passes through the average point 🖖

Whatever your data looks like, the least-squares line is guaranteed to go through (x̄, ȳ) — the point made from the mean of the x values and the mean of the y values. This is not a tendency; it falls straight out of the condition that the residuals sum to zero, so it holds for every dataset without exception. It gives you a free sanity check: mark the average point, and any candidate line missing it is not the least-squares fit. It also explains the method’s weakness. Because the gaps are squared, a point twice as far away pulls four times as hard, so one distant outlier can swing the line more than a dozen well-behaved points holding it steady.

Why we square the gaps 🖖

The tool adds up the squares of the vertical gaps between each point and the line, then hunts for the line that makes that total as small as possible. Squaring means a gap twice as wide counts four times as much, so the fit works hard to avoid any single large miss. Drag one point far off and watch the whole line lunge toward it — that pull is the direct price of squaring.

Regression is not symmetric 🖖

Because least squares only shrinks the vertical gaps, it quietly assumes y is the target and x is exact. Swap the axes and refit, and you get a different line — not the mirror image you would expect. Both lines pass through the data's average point (x̄, ȳ), but they coincide only when the points sit perfectly on a line. The two slopes even multiply out to r², the squared correlation.

Practice

Check yourself

Predict the answer first, then use the controls above to find out. Reveal only after you have committed to a guess — that is what makes it practice.

Press braking with line: six points, a straight fit, SSE 84. Now press braking with curve — the same six points, fitted with a quadratic. Predict the new SSE.

Show answer

0. Not nearly zero: zero, with RMSE 0 and a fitted curve of ŷ = 8 − 0.25x + 0.015x². Those six points came off that parabola exactly, and once the family of allowed curves contains the truth there is nothing left over to square. A falling SSE on its own proves nothing — a wider family can never fit worse. A fall all the way to the floor is different in kind, because there is no residual left for any further model to remove.
Press course retention, which fits a line with SSE 1.3714 and a largest-point SSE share of 37.2%. Switch the model selector to Quadratic. SSE will fall — predict what the largest-point share does.

Show answer

It goes up, to 56.3%, while SSE falls to 0.4071. A share is not an amount: the extra freedom pulled the curve towards the points it could reach, so the one it still cannot reach now makes up a bigger fraction of a smaller total. One warning while you are here — pressing any preset snaps the selector back to Line, so these numbers only mean this while Quadratic is still showing.
Back to braking with line. The readout shows five of the six residuals before its ⋯: 5, −1, −4, −4, −1. What is the sixth, and what do the six together tell you that SSE 84 does not?

Show answer

+5. The residuals of a least-squares fit always sum to zero and the five shown add to −5, so the missing one is forced. The residual plot draws exactly that shape: two tall bars up at the ends, four dipping below through the middle, a clean U. Summing to zero and leaving no tilt against x is the whole of what least squares guarantees, and a symmetric U satisfies both — its residuals really do sum to zero, and it leans neither way — so a straight line can neither absorb it nor notice it. SSE says the fit is loose; only the pattern says it is loose in one direction at a time.

Problem solved in full

Six points with a sum of squared errors of 1.3714 5 steps
Six points, a fitted line, and a sum of squared errors of 1.3714. Work out how much of that total comes from a single point — and prove a fact about the residuals that holds for every least-squares fit ever computed.
1. The coefficients come from the same two sums as any simple regression: how x and y vary together, over how x varies alone.
  
  $\overset{x}{ˉ} = 2.5, \overset{y}{ˉ} = \frac{345}{6} = 57.5$
2. That gives the line the panel reports. Nothing about this data is special — it is a nearly straight run of six points with a little scatter.
  
  $b = \frac{S _{x y}}{S _{xx}} = \frac{103.5}{17.5} = 5.9143, a = 57.5 - 5.9143 (2.5) = 42.7143$
3. The residuals are what the fit could not explain. Compute all six and one of them is conspicuous: the point at x = 0 is off by −0.7143, roughly double the typical miss.
  
  $r_{i} = - 0.7143, 0.3714, 0.4571, 0.5429, - 0.3714, - 0.2857$
4. Before using them, notice they sum to zero — not approximately, exactly. That is forced by the intercept: the second normal equation is the statement that the residuals average to zero, so it is true of every least-squares line regardless of the data.
  
  $\sum r_{i} = 0 exactly, SSE = \sum r_{i}^{2} = 1.3714$
5. Squaring destroys that balance, and that is the point. The −0.7143 becomes 0.5102, which is 37.2% of the total SSE of 1.3714 — one point in six carrying more than a third of the penalty.
  
  $RMSE = \frac{1.3714}{6} = 0.4781, \frac{( - 0.7143 ) ^{2}}{1.3714} = 37.2%$
Answer

The tool prints SSE = 1.3714, RMSE = 0.4781 and a largest-point share of 37.2%. Both facts matter and they pull against each other. The residuals summing to zero says the line is balanced; the 37.2% says the fit is not, because squaring weights a double-sized miss four times as heavily. That is exactly why least squares is so sensitive to outliers, and why the residuals-sum-to-zero property is no protection at all: a single bad point cannot shift the average residual, and can still dominate what the line is trying to minimise.

$\sum r_{i} = 0 by construction; one point of six carries 37.2% of SSE$

Learning path

Fitting a line to data

Leads to Linear regression the fitted line as a projection, and the reason its residuals are perpendicular to the model.

References (1)

Least squares as an orthogonal projection, which is what makes the residuals perpendicular to the model: G. Strang, Introduction to Linear Algebra, 5th ed., chapter 4 (“Orthogonality”). Wellesley-Cambridge Press, 2016. ISBN 978-0-9802327-7-6.

By the 42Math team · Updated 2026-07-29

Reviewed against published sources. Corrections welcome. Editorial policy · Corrections

All Mathematics lessons →

Example problems

course retention - Six points, SSE 1.3714 and R² 0.9978. This is the worked problem's dataset, and its line y = 42.71 + 5.91x runs through the mean point (2.5, 57.5).
campaign outlier - The first five points sit on a slope of exactly 5. The sixth drags the fit to 8.54 and carries 47.5% of the SSE by itself - one point in six holding nearly half the error.
braking with line - A straight line through parabolic data: SSE 84.0 and R² 0.9618, which looks respectable until you read the residuals. They curve rather than scatter.
braking with curve - The same six points fitted with a quadratic: y = 8 - 0.25x + 0.015x², and the SSE is exactly 0. The data lies on that parabola to the last digit, where the line left 84.
housing prices - Seven flats at 2.74 thousand per square metre, R² 0.998, SSE 88.39. Real-looking data that is still almost perfectly straight.