Bug 1335

Summary: Analog data under cursor rounded to 1 digit, eg. 3V
Product: PulseView Reporter: chrysn
Component: Data displayAssignee: Nobody <nobody>
Status: RESOLVED FIXED    
Severity: normal CC: soeren, spamMayGoHere, uwe
Priority: Normal    
Version: unreleased development snapshot   
Target Milestone: ---   
Hardware: All   
OS: All   
Attachments: Set precision from 0 to 2

Description chrysn 2018-11-29 13:35:07 CET
Data in the analog view is rounded to 0 decimal places in the "number under cursor" view. For examaple, a voltage that visibly fluctuates in the plot between 3.1 and 3.5 V shows as "[3 V] 2 V/div" no matter where I place the cursor.

For comparison, I've patched a locally checked out version to pass 2 instead of 0 to the `format_value_si` invocation in `pv/views/trace/analogsignal.cpp`, and now I get usable readings. (Other readings now say 745.12mV instead of 745mV).

I propose two ways to proceed with this:

* Just change the 0 to a 2.

  I can commit that to a fetchable location if you prefer, but you're probably faster just editing it.

  That variation would be good enough for my purposes, and is what I recommend.

* Change the semantics of format_value_si's precision argument from "digits after the decimal point in whichever unit was chosen" to "number of non-zero digits". (No other code in pulseview appears to use that function).

  That variation would be more correct from a significant-digits point of view (but we don't have that information to tell how many are significant, and'd need to guess). I'd offer to write the few lines of log10-round-and-clip code if that's what you prefer.
Comment 1 chrysn 2019-01-08 13:44:39 CET
Created attachment 504 [details]
Set precision from 0 to 2

This is the trivial solution, ready for application to git revision 0adee2d934 (or aa78b2df)
Comment 2 Soeren Apel 2019-03-07 08:26:28 CET
Thanks for the bug report and the patch, Uwe will commit it when he gets the chance. There was a reason I used 0 originally, I think it was because of readability reasons - or maybe because it should depend on the zoom level.

Either way, it doesn't matter at the moment because your point is valid and this is a simple way to provide something that works.
Comment 3 Uwe Hermann 2019-03-09 15:46:38 CET
Fixed in 58cd5b584f5bcb19d7c9bd28391c53dd2488fc59, thanks a lot!
Comment 4 Soeren Apel 2019-03-24 16:17:28 CET
*** Bug 1368 has been marked as a duplicate of this bug. ***