Difference between revisions of "Talk:High precision analog"
Jump to navigation
Jump to search
(Created page with "I can't understand the first proposal, can the author provide us with examples please? I don know if you are thinking what I am thinking. I'm thinking about a solution along...") |
(No difference)
|
Revision as of 06:06, 3 January 2015
I can't understand the first proposal, can the author provide us with examples please? I don know if you are thinking what I am thinking.
I'm thinking about a solution along the lines of a BCD value, with around 10 digits of precision (the highest resolution multimeters in existance have 8 and a half digits of resolution), with a base 10 exponent. Math is possible, but slower than normal and for complex operations, a conversion to floating point might be convenient.