WP-34S galUS-> l Precision

[Joe Horn]

(09-07-2016 11:56 PM)Paul Dale Wrote:  We used the conversions tables by NIST for our constants. That's about as canonical as can be got.

All the conversions listed on that page are rounded to a maximum of 7 significant digits, even when exact conversion factors are well established but require more than 7 significant digits. This is even mentioned in NIST Guide to the SI, Appendix B: Conversion Factors: "The factors given in Secs. B.8 and B.9 are written as a number equal to or greater than 1 and less than 10, with 6 or fewer decimal places."

Bottom line: It might be good enough for government work, but I wouldn't consider THAT particular table to be a "canonical" list of conversion factors at all. What we need is the original table of factors which were rounded off to create this table.