Bug 1335 - Analog data under cursor rounded to 1 digit, eg. 3V
Summary: Analog data under cursor rounded to 1 digit, eg. 3V
Status: RESOLVED FIXED
Alias: None
Product: PulseView
Classification: Unclassified
Component: Data display (show other bugs)
Version: unreleased development snapshot
Hardware: All All
: Normal normal
Target Milestone: ---
Assignee: Nobody
URL:
Keywords:
: 1368 (view as bug list)
Depends on:
Blocks:
 
Reported: 2018-11-29 13:35 CET by chrysn
Modified: 2019-03-24 16:17 CET (History)
3 users (show)



Attachments
Set precision from 0 to 2 (1.02 KB, patch)
2019-01-08 13:44 CET, chrysn
Details

Note You need to log in before you can comment on or make changes to this bug.
Description chrysn 2018-11-29 13:35:07 CET
Data in the analog view is rounded to 0 decimal places in the "number under cursor" view. For examaple, a voltage that visibly fluctuates in the plot between 3.1 and 3.5 V shows as "[3 V] 2 V/div" no matter where I place the cursor.

For comparison, I've patched a locally checked out version to pass 2 instead of 0 to the `format_value_si` invocation in `pv/views/trace/analogsignal.cpp`, and now I get usable readings. (Other readings now say 745.12mV instead of 745mV).

I propose two ways to proceed with this:

* Just change the 0 to a 2.

  I can commit that to a fetchable location if you prefer, but you're probably faster just editing it.

  That variation would be good enough for my purposes, and is what I recommend.

* Change the semantics of format_value_si's precision argument from "digits after the decimal point in whichever unit was chosen" to "number of non-zero digits". (No other code in pulseview appears to use that function).

  That variation would be more correct from a significant-digits point of view (but we don't have that information to tell how many are significant, and'd need to guess). I'd offer to write the few lines of log10-round-and-clip code if that's what you prefer.
Comment 1 chrysn 2019-01-08 13:44:39 CET
Created attachment 504 [details]
Set precision from 0 to 2

This is the trivial solution, ready for application to git revision 0adee2d934 (or aa78b2df)
Comment 2 Soeren Apel 2019-03-07 08:26:28 CET
Thanks for the bug report and the patch, Uwe will commit it when he gets the chance. There was a reason I used 0 originally, I think it was because of readability reasons - or maybe because it should depend on the zoom level.

Either way, it doesn't matter at the moment because your point is valid and this is a simple way to provide something that works.
Comment 3 Uwe Hermann 2019-03-09 15:46:38 CET
Fixed in 58cd5b584f5bcb19d7c9bd28391c53dd2488fc59, thanks a lot!
Comment 4 Soeren Apel 2019-03-24 16:17:28 CET
*** Bug 1368 has been marked as a duplicate of this bug. ***