Bug 1500

Summary:	timestamp column is for information only, does not control consumption
Product:	libsigrok	Reporter:	Gerhard Sittig <Gerhard.Sittig>
Component:	Input: csv	Assignee:	Nobody <nobody>
Status:	CONFIRMED ---
Severity:	normal	CC:	alexander.dyke, easfreitas
Priority:	Normal
Version:	unreleased development snapshot
Target Milestone:	---
Hardware:	All
OS:	All

Description Gerhard Sittig 2020-02-18 18:51:48 CET

The recently introduced support for a "timestamp" column type in the CSV input 
module does not _control_ how the input data gets processed, instead it's only 
used for information, as a fallback approach to somehow get the input samplerate 
when the user did not specify one.

It might be desirable to use the timestamp column (when specified) as a means 
to repeatedly feed the most recent set of samples to the session when the 
timestamp difference suggests that more time has passed than one step in the 
sampling sequence.

This could be low hanging fruit for somebody with a personal interest in seeing 
this feature supported in mainline. :)  The source is in src/input/csv.c.  The 
process_buffer() routine's queue_*_samples() calls need to get repeated when 
the parse_timestamp() routine determined how many sample sets to feed for the 
current line of CSV input.

Comment 1 Gerhard Sittig 2020-10-13 19:02:38 CEST

*** Bug 1620 has been marked as a duplicate of this bug. ***

Comment 2 Gerhard Sittig 2020-10-17 12:02:03 CEST

See bug 1625 for more information on internal constraints. Automatic 
samplerate detection from incompletely received input data may not be 
possible at all. And the submission of "missing" sample points' values 
which were omitted during generation of the input file will depend on 
having _some_ samplerate available right from the start of processing 
any of the input data. So the need to specify the samplerate remains 
for those situations where the input data is not suitable for the task.

Comment 3 Alex 2020-10-19 16:03:15 CEST

(In reply to Gerhard Sittig from comment #2)
> See bug 1625 for more information on internal constraints. Automatic 
> samplerate detection from incompletely received input data may not be 
> possible at all. And the submission of "missing" sample points' values 
> which were omitted during generation of the input file will depend on 
> having _some_ samplerate available right from the start of processing 
> any of the input data. So the need to specify the samplerate remains 
> for those situations where the input data is not suitable for the task.

Would it be feasible to pre-process the CSV to calculate the smallest gap between timestamps, and use that as the samplerate? Even if that was not the true samplerate of the captured data, would that value be sufficient for display?

Comment 4 Gerhard Sittig 2020-11-01 13:04:45 CET

The file format supporting input module receives the data in a strict linear 
sequence. It cannot seek, and shall not. Being able to work in a pipeline 
is an important feature that must not be broken without necessity. The only 
exceptions in the current code base are those formats which don't lend 
themselves to sequential processing in the first place (ZIP files, header 
information kept in foot sections, or similar constraints).

The libsigrok using applications should not bother implementing specific 
hacks or assuming specific properties of individual file formats. They 
should just feed the input data to the library as is, and not interpret 
the content in any way. Doing so violates API layers, introduces additional 
complexity and redundancy, and burdens maintenance.

The current implementation of CSV input just expects to see a continuous 
stream of sample data. It's easy enough to provide that. The timestamp 
column is an optional feature, and support for its automatic processing 
is limited as is stated above.

If you consider removing that limitation you are welcome to do that as is 
outlined above. Though you need to have a samplerate for that (as was also 
mentioned above). If the input data doesn't provide the rate, specify it. 
It's easy enough to have that information when you are the creator of the 
input data. It's of limited use to process the input data without knowing 
the capture's most essential properties.

You can always do such pre-processing outside of sigrok applications if you 
want to automatically determine a file's properties, or convert not supported 
cases to supported types of input. Or pick a better match for your specific 
needs from the set of supported input formats.

Comment 5 Soeren Apel 2020-12-24 20:43:36 CET

*** Bug 1652 has been marked as a duplicate of this bug. ***

Comment 6 Edu 2020-12-24 21:32:44 CET

So if I'm understand correct, all import from .csv will consider the same time between each line? it will not consider the time passed between each line. Am I understanding correct? strange is the it take the time between line 2 and line 3 (disconsidering the header) and use for all others. So if I have a PWM imported will always presented as 50% duty, independent of the time between lines, Am I understanding correct?