Este guia está disponível atualmente em inglês.
Technology / Technical guide
Least-Squares Fitting: Residuals, Weights & Input Errors
Least squares selects model parameters by minimising a defined set of squared discrepancies. It gives a precise optimisation objective, but it does not prove that the chosen model describes the underlying mechanism. Residuals, observation weights and errors in the input variable determine what the fitted parameters can represent.
A small fitting error answers a specific question.
The model, variables and weights define that question; goodness of fit and physical validity remain different judgements.
01 / Objective
The residual depends on both observation and model.
For a response y[i] and model f(x[i], β), define r[i] = y[i] − f(x[i], β).
For a response y[i] and model f(x[i], β), define r[i] = y[i] − f(x[i], β). Ordinary least squares minimises Σr[i]² over β. The parameter vector changes during the mathematical optimisation; the stated observations remain the data being represented.
Squaring makes positive and negative residuals contribute positively and gives larger departures more influence. This objective differs from minimising absolute error or maximum error. A minimum belongs to the selected criterion, rather than to every possible interpretation of accuracy.
02 / Linear meaning
Linear in parameters is different from linear in input.
A model β0 + β1x is linear in its unknown coefficients.
A model β0 + β1x is linear in its unknown coefficients. So is β0 + β1x + β2x², even though the graph is curved. Conversely, placing an unknown parameter inside an exponential generally produces a different optimisation problem.
A unique coefficient solution in linear least squares requires the represented coefficient columns to be independent. Repeating the same relationship in another column does not add an independent parameter. This algebraic condition differs from claiming that a physical model’s parameters are identifiable.
03 / Toy line
One invented data set illustrates the criterion.
Take hypothetical pairs (x, y) = (0, 1), (1, 2), (2, 4), with x dimensionless and y in arbitrary units.
Take hypothetical pairs (x, y) = (0, 1), (1, 2), (2, 4), with x dimensionless and y in arbitrary units. The equally weighted fitted line has slope 1.5 units and intercept 5/6 unit, rather than passing through every point.
The three residuals are 1/6, −1/3 and 1/6 unit. Their squared sum is 1/6 unit². This is an exact arithmetic example; it is not a sensor calibration, an observed linearity result or evidence for extrapolating beyond the invented input values.
04 / Weights
Unequal variability changes the stated fitting objective.
NIST describes weighted least squares as minimising Σw[i]r[i]², allowing observations to influence the result differently.
NIST describes weighted least squares as minimising Σw[i]r[i]², allowing observations to influence the result differently. All weights must be nonnegative for this interpretation. Multiplying every positive weight by the same constant leaves the minimising parameter unchanged.
Inverse-variance weights have a particular statistical motivation when response errors are independent, their variances are represented correctly and inputs are treated as known. Weighting does not automatically fix an incorrect model or an erroneous predictor. The reason for the weights belongs alongside the fitted result.
05 / Input error
A noisy predictor changes the relationship being fitted.
Consider a hypothetical centred population: the true predictor X has variance 4, observed x = X + e has independent zero-mean error of variance 1, and y = 2X.
Consider a hypothetical centred population: the true predictor X has variance 4, observed x = X + e has independent zero-mean error of variance 1, and y = 2X. The population ordinary-least-squares slope is Cov(x, y)/Var(x) = 8/5 = 1.6.
The slope differs from the true coefficient 2 because the observed predictor contains extra variation without corresponding response variation. The example assumes independence and finite moments. Assigning response weights alone does not remove this predictor-error effect or establish a valid correction.
06 / Scope
Fitted agreement is not independent model validation.
The same observations used to select coefficients can show a small residual by construction.
The same observations used to select coefficients can show a small residual by construction. A more flexible model can reduce that objective while representing accidental features. Agreement outside the fitted conditions is a separate question, as is the causal meaning of each coefficient.
No MARSTON fitting algorithm, calibration mapping or coefficient is asserted. The residual guide explains how prediction discrepancies differ from a confirmed fault, without giving calibration or live-control procedures.
Related technical guides: Estimator Bias, Variance & Mean-Squared Error · Parameter Identifiability: Unique Models & Hidden Ambiguity · Model Residuals: Prediction Gaps & Ambiguous Causes.
General background: NIST: weighted least-squares criterion. These references explain industry concepts; they are separate from the applicable MARSTON product and project documents.
Key concepts and conditions.
Read each quantity with its reference, units and stated conditions. A general relation does not establish a model-specific value.
| Concept / quantity | Meaning / relation | Conditions / limits |
|---|---|---|
| Residual r | Observed response minus model response | Same units and stated model |
| Squared objective | Σr² or Σwr² | Optimised over declared parameters |
| Parameter linearity | Linear dependence on coefficients | Can include nonlinear functions of known input |
| Observation weight | Relative influence on the objective | Statistical interpretation requires justified error assumptions |
| Input uncertainty | Error in the predictor itself | Response weights do not automatically correct it |
Engineering questions.
Must a least-squares line pass through each point?
No. It minimises the specified aggregate squared discrepancy.
Can a polynomial model be linear?
It can be linear in its unknown coefficients while nonlinear in input.
Does scaling every weight change the fit?
A common positive scale leaves the minimising parameters unchanged.
When are inverse-variance weights meaningful?
Under a justified response-error model with the stated independence and known-input assumptions.
Can input errors bias the fitted slope?
Yes. The hypothetical population example illustrates one such mechanism.
Does a small residual prove the model is physical?
No. Optimisation and independent validity answer different questions.
Technical enquiry
Discuss the technical question.
Least squares selects model parameters by minimising a defined set of squared discrepancies. Share the model and applicable operating or design information for an engineering discussion.