Este guia está disponível atualmente em inglês.
Technology / Technical guide
Estimator Bias, Variance & Mean-Squared Error
An estimate can vary little and still miss its target systematically. It can also be unbiased while varying substantially between hypothetical repetitions. Bias and variance describe these separate behaviours. Mean-squared error combines them for a fixed target, making the assumptions behind averaging easier to see.
A stable estimate is not necessarily a correct estimate.
The target, repetition model and shared errors determine whether reducing visible variation improves the quantity being estimated.
01 / Target
Bias is defined against a particular quantity.
Let θ be a fixed target and T an estimator whose result varies across repetitions of a stated statistical model.
Let θ be a fixed target and T an estimator whose result varies across repetitions of a stated statistical model. Its bias is b = E[T] − θ. This expectation concerns that repetition model, not merely a short sequence of readings from one realised situation.
Changing the target changes the interpretation. Estimating a fixed mean differs from following a time-varying signal. A delay can create disagreement with the current signal even if an estimate correctly describes an earlier interval; calling all disagreement “bias” hides that distinction.
02 / Spread
Variance measures variation around the estimator’s own mean.
Variance is E[(T − E[T])²], in squared units of T.
Variance is E[(T − E[T])²], in squared units of T. It describes spread around the estimator’s average result. It does not measure the distance between that average and θ. A narrow distribution can therefore sit away from the target.
MIT’s estimation notes distinguish bias, variance and mean-squared error. This statistical language differs from a finite repeatability record: repeated values provide evidence about one process, while the expectation and variance definitions refer to the declared population model.
03 / Squared error
The MSE decomposition separates two contributions.
When the required second moments exist, MSE = E[(T − θ)²] = Var(T) + b².
When the required second moments exist, MSE = E[(T − θ)²] = Var(T) + b². Expanding T − θ around E[T] leaves a cross-term with zero expectation. Both contributions have squared target units, so they can be added consistently.
Consider two hypothetical estimators. A has zero bias and variance 9 unit², giving MSE 9 unit². B has bias 1 unit and variance 4 unit², giving MSE 5 unit². B has lower MSE in this specified model despite being biased.
04 / Independent mean
Averaging reduces one term while retaining another.
Suppose Y[i] = θ + b + ε[i], where b is fixed and errors ε[i] are independent, zero-mean and have variance σ².
Suppose Y[i] = θ + b + ε[i], where b is fixed and errors ε[i] are independent, zero-mean and have variance σ². The mean of N readings retains bias b and has variance σ²/N. Its MSE is b² + σ²/N.
As N increases, the independent spread decreases, but the fixed offset remains. The result does not say that every average becomes accurate. Correlation, drift and a changing target can invalidate the simple repetition model even when the number of readings is large.
05 / Shared variation
Common errors create an averaging floor.
Extend the hypothetical model to Y[i] = θ + b + C + ε[i], where C is a zero-mean shared random contribution independent of the ε terms.
Extend the hypothetical model to Y[i] = θ + b + C + ε[i], where C is a zero-mean shared random contribution independent of the ε terms. The average has variance Var(C) + Var(ε)/N. The common term is present in every reading.
For b = 2 units, Var(C) = 1 unit², Var(ε) = 4 unit² and N = 4, the average has variance 2 unit² and MSE 6 unit². These invented values illustrate common variation; they are not a sensor uncertainty specification.
06 / Scope
Statistical improvement depends on the defined objective.
MSE penalises squared departures, so it is one criterion among possible objectives.
MSE penalises squared departures, so it is one criterion among possible objectives. An estimator with smaller MSE need not have a smaller maximum possible error. The example comparison also does not establish the same ordering under a different repetition model.
No MARSTON estimator accuracy or averaging function is claimed. The fitting guide addresses a separate objective: selecting model parameters to minimise discrepancies in a set of observations.
Related technical guides: Numerical Derivatives: Signal Slope & Noise Amplification · Least-Squares Fitting: Residuals, Weights & Input Errors · Model Residuals: Prediction Gaps & Ambiguous Causes.
General background: MIT OpenCourseWare: estimation bias, variance and MSE. These references explain industry concepts; they are separate from the applicable MARSTON product and project documents.
Key concepts and conditions.
Read each quantity with its reference, units and stated conditions. A general relation does not establish a model-specific value.
| Concept / quantity | Meaning / relation | Conditions / limits |
|---|---|---|
| Target θ | Fixed quantity in the comparison | Its meaning and time boundary must be declared |
| Bias b | E[T] minus θ | Needs a repetition model rather than only one result |
| Variance | Spread around E[T] | Second moment exists and units are squared |
| MSE | Variance plus squared bias | Uses the same fixed target and finite moments |
| Mean variance | Independent and shared contributions | Correlation and drift change the averaging relationship |
Engineering questions.
Can an estimate be precise but biased?
Yes. Small spread does not establish a mean equal to the target.
Is an unbiased estimate always preferable?
Not under every objective; the hypothetical MSE comparison includes variance.
Does averaging remove a fixed offset?
No. The fixed offset remains in the average.
What if errors share a common random term?
That term remains instead of decreasing as independent noise does.
Does MSE have the same unit as the target?
No. MSE has squared target units.
Does the example describe a MARSTON sensor?
No. It is a declared statistical model with invented values.
Technical enquiry
Discuss the technical question.
An estimate can vary little and still miss its target systematically. Share the model and applicable operating or design information for an engineering discussion.