National Physical Laboratory

Is it acceptable to apply rounding to tolerances? (FAQ - Length)

For example, if a specification is defined as 0.087 mm ± 0.001 mm, then could a measured value of 0.088 2 mm be rounded down to 0.088 mm on account of the tolerance only being quoted to the 3rd decimal place?

In short the answer is no. What you do need to take into account is the uncertainty of measurement. This topic is covered in ISO 14253-2:1998 Geometrical Product Specifications (GPS) -- Inspection by measurement of workpieces and measuring equipment -- Part 1: Decision rules for proving conformance or non-conformance with specifications. Once you follow these rules I suspect that you will find that a value of 0.088 2 mm definitely does not conform. The expanded uncertainty of measurement of your system should be better that 1/10th of the tolerance zone. Given a 0.002 mm tolerance zone you will need an expanded uncertainty of measurement of 0.000 2 mm. If the resolution of your equipment is only 0.000 1 mm you are going to have trouble achieving this uncertainty. If your expanded uncertainty of measurement is indeed 0.000 2 mm then the result you have measured is outside the tolerance zone by an amount equal to the expanded uncertainty (or put another way 1/10th of the tolerance zone). This is a special case - neither conformance nor non-conformance with specification can be proved. ISO 14253 states that you should reduce your conformance zone by an amount equal to your expanded uncertainty of measurement. Conversely the non-conformance zone should be extended by an amount equal to the expanded uncertainty of measurement. For the zone in the middle neither conformance nor non-conformance with specification can be proved. This may seem strange but remember it is up to the supplier to prove conformance and the customer to prove non-conformance.

Last Updated: 1 Dec 2010
Created: 1 Dec 2010

Registration

Please note that the information will not be divulged to third parties, or used without your permission

Login