The Museum of HP Calculators

HP Forum Archive 21

 [WP 34s] Pressure Conversion FactorsMessage #1 Posted by Timothy Roche on 3 Nov 2013, 11:49 p.m. WP 34s (3.2T 3375) Pressure conversion factors implemented in the WP 34s calculator seem to be based on the published values found in the NIST Guide to SI Units. (So are all the others, for that matter. I think.) This is a good source for conversion factors (reliable and free), particularly the exact conversion factors highlighted in bold face in the document., There are, however, many listed conversion factors which are approximations in the NIST Guide; results from conversions carried out using these approximated factors must be restricted to the number of given significant figures in the value from the table. As the given approximate conversion factors contain six or seven significant figures (usually) and most people are looking for answers with two, three, or four sig-figs (at least in engineering calculations in my experience), these conversion factors are more than adequate for the job. EG: The NIST Guide gives the conversion factor from units of mmHg to units of Pa as 1.333 224 E+02. So, given a pressure 760 mmHg the converted result (760 * 133.3224) would be 101 325.024 Pa. Because the factor has seven significant digits the answer should be rounded to no more than seven significant figures: therefore, our rounded answer to seven significant digits is 101 325.0 Pa. You could argue that the answer should not have more than two significant digits based on the given value 760 mmHg, which is true if you did not implicitly mean the number 760 exactly. Now to my point: Some of the included approximate conversion factors in the NIST Guide have exact conversions which were not included in the publication (NIST is emphasizing the uncertainty of certain measured values like barometric pressures). In particular, the conversions for many of the pressure units: those listed as conventional (mmHg, cmHg, ftHg, mmH2O, cmH2O, inH2O, ftH2O) and those that are related to these conventional pressure units (torr). All these conventional units are based on the pressure unit of atmosphere and the equivalent in these other units. For instance, mmHg-to-Pa factor is based on the exact definition 1 atm = 101 325 Pa and 1 atm = 760 mmHg (conventional); so the listed value in the NIST Guide 1.333 224 E+02 is the approximation of (101 325/760). I think that the pressure conversion factors implemented in the WP 34s should use the exact definitions of these conventional pressure units. This would eliminate unexpected and slightly wrong conversions displayed on the calculator such as 760 mmHg = 101 325.024 Pa?! Here are a list of suggested adjusted pressure conversion factors. 760 mmHg = 101 325 Pa, therefore: 1 mmHg = (101 325/760) Pa = 133.322 368 421 0526 Pa 1 torr = 133.322 368 421 0526 Pa 29.921 259 842 519 69 inHg = 101 325 Pa, therefore: 1 inHg = 3 386.388 157 894 736 Pa 1 lbm = 0.453 592 37 kg (exact) 1 lbf = (0.453 592 37 kg)(9.806 65 m s^-2) = 4.448 221 615 260 500 N 1 (lbf/in2) = 1 psi = (1 lbf in^-2)(4.448 221 615 260 500 N lbf^-1)[(39.370 078 740 157 48)^2 in^2 m^-2] 1 psi = 6 894.757 293 168 361 Pa Tim Edited: 3 Nov 2013, 11:52 p.m.

 Re: [WP 34s] Pressure Conversion FactorsMessage #2 Posted by Walter B on 4 Nov 2013, 1:05 a.m.,in response to message #1 by Timothy Roche Tim, Thanks for your elaborate post. AFAICS, you're right. Now let's go to the consequences. Quote: As the given approximate conversion factors contain six or seven significant figures (usually) and most people are looking for answers with two, three, or four sig-figs (at least in engineering calculations in my experience), these conversion factors are more than adequate for the job. Concurs with my experience :-) BTW, I don't know any pressure-meter (?) being accurate to five figures or more. And please see the very first page about CONV in the manual. Summing up, 90% of CONV is a crutch for those folks who still can't get accustomed to SI - and to my experience, those folks don't work in precision businesses to any extent coming close to the precision you request. d:-)

 Re: [WP 34s] Pressure Conversion FactorsMessage #3 Posted by Timothy Roche on 4 Nov 2013, 1:59 a.m.,in response to message #2 by Walter B Understood. Your calculator has the highest precision on the market (one of the reasons I purchased the physical unit from Eric and donated to the project). The answers it displays ought to reflect that precision (I think); even if the extent of that precision is not required. When I use the CONV function I expect it to work to the same standard as the rest of the calculator. Should it not give the right answer to a conversion calculation? I really like the WP 34s and plan to continue using it -- whether you choose to fix the approximated conversion factors or not. It is of minor significance. All the best, Tim

 Re: [WP 34s] Pressure Conversion FactorsMessage #4 Posted by Walter B on 4 Nov 2013, 2:19 a.m.,in response to message #3 by Timothy Roche Thanks. From a "real world" point of view, I vote for leaving CONV as is. d:-)

 Re: [WP 34s] Pressure Conversion FactorsMessage #5 Posted by Lars Bergström on 4 Nov 2013, 12:07 p.m.,in response to message #1 by Timothy Roche Somewhat off-topic: > So, given a pressure 760 mmHg the converted result (760 * 133.3224) would be 101 325.024 Pa Using only 3 digits as input value (760), only 3 digits are relevant in the answer... 101 *10^3 Pa :-) Edited: 4 Nov 2013, 12:08 p.m.

 Re: [WP 34s] Pressure Conversion FactorsMessage #6 Posted by Walter B on 4 Nov 2013, 12:14 p.m.,in response to message #5 by Lars Bergström Quote:Quote: So, given a pressure 760 mmHg the converted result (760 * 133.3224) would be 101 325.024 Pa Using only 3 digits as input value (760), only 3 digits are relevant in the answer... 101 *10^3 Pa Usually true though not here: 760 is an exact value, i.e. 760.000000... Please see the OP. HTH. d:-) Edited: 4 Nov 2013, 12:16 p.m.

 Re: [WP 34s] Pressure Conversion FactorsMessage #7 Posted by Lars Bergström on 4 Nov 2013, 2:12 p.m.,in response to message #6 by Walter B Just curious: Is there any know conventiont to indicate an exact value, in contrast to a rounded value? Eg 760=760 in contrast to 759,5.....760,4 = 760

 Re: [WP 34s] Pressure Conversion FactorsMessage #8 Posted by Walter B on 4 Nov 2013, 3:51 p.m.,in response to message #7 by Lars Bergström I'm not aware of any mathematical symbol indicating such an exactitude. IIRC, the torr scale is simply defined by the fact that 760 torr = p0, like the Celsius scale by 0°C = T0, with 760, p0, 0 and T0 being exact values. Without knowing the definition, you can't know the precision. d:-/

 Re: [WP 34s] Pressure Conversion FactorsMessage #9 Posted by Dave Shaffer (Arizona) on 4 Nov 2013, 7:17 p.m.,in response to message #7 by Lars Bergström As we told the students in physics class, 760. would be the proper way to express 760 itself to 3 significant figures. For infinite precision, you could use an overscore/overline: Per Wikipedia, the overline can be used to represent a repeating decimal: 3.¯3 = 3.333333333333... (I couldn't copy, nor figure out how to put in, the overscore directly above the number!) Thus, in this notation 760.¯0 would indicate infinite precision of 760

Go back to the main exhibit hall