The size of the smallest step an instrument can show: on a digital display it is one step of the last digit, on an analog dial it depends on scale spacing and pointer width. It enters the budget as a rectangular component with a half-width of half the resolution, giving a standard uncertainty of about 0.29 times the resolution. High resolution says nothing about accuracy: a 7-digit display can be wrong in the third digit.
Gauge resolution is 0.01 mm and the tolerance is 0.02, you can't calibrate that meaningfully.
Customers buy digits; metrologists buy stability. Extra resolution on an unstable instrument only displays the noise more clearly.
Also heard as
- display resolution
- readability
Related terms
Counts
The steps of the last digit on a digital display, used in specs such as plus or minus 2 counts.
Rectangular distribution
A distribution giving equal likelihood to every value between two limits, used when only the bounds are known.
Uncertainty budget
A table listing every uncertainty source in a measurement with its size, distribution, divisor and contribution to the total.
Accuracy
How close a measured value comes to the true value, a qualitative idea rather than a reported number.
Double counting
Budget slang for entering the same physical effect twice under different names, which inflates the uncertainty.