Bug 1364 - DS1074Z/DS1104Z memory acquisition not working
Summary: DS1074Z/DS1104Z memory acquisition not working
Alias: None
Product: libsigrok
Classification: Unclassified
Component: Driver: rigol-ds (show other bugs)
Version: unreleased development snapshot
Hardware: All All
: Normal normal
Target Milestone: ---
Assignee: Nobody
Depends on:
Reported: 2019-03-11 10:49 CET by Mariusz Białończyk
Modified: 2020-02-13 14:09 CET (History)
2 users (show)

sigrok-cli obtaining 1 frame (23.69 KB, text/x-log)
2019-03-11 10:51 CET, Mariusz Białończyk
sigrok-cli scan output (1.46 KB, text/x-log)
2019-03-11 10:52 CET, Mariusz Białończyk
patch for hardcoding the datasource (1.34 KB, patch)
2019-03-11 10:53 CET, Mariusz Białończyk
patch for more frames then 1 (1.08 KB, patch)
2019-03-11 10:53 CET, Mariusz Białończyk
PCAP file with obtaining full memory waveform using DSRemote tool (59 bytes, text/plain)
2019-03-11 11:00 CET, Mariusz Białończyk
Patch correct read completion (761 bytes, patch)
2020-02-12 12:09 CET, Hugo Laloge

Note You need to log in before you can comment on or make changes to this bug.
Description Mariusz Białończyk 2019-03-11 10:49:40 CET
Hi Community,
Recently I am discovering my shiny new Rigol DS1074Z (hacked to DS1104Z MHz) and its sigrok support. The oscilloscope has a 00.04.04.SP3 firmware (originally installed). I have it connected using LAN connection (and below only applies to this connection type).

I have to admit that he support seems really broken for me despite the "supported" status for this model on the wiki page :(

The Live acquisition triggering is working weird (above some time base value it is even not resulting any results until clicking Stop) and its resolution is weak (maybe this is the specific of this acquisition mode), but in this bug I will focus on obtaining the waveform from oscilloscope's memory.

So let's just start from beginning:
I am starting the scope and trigger interesting waveform in Single mode. The scope is stopped (RUN/STOP button on the scope front panel is RED).
I am starting PulseView, selecting and configuring TCP-RAW connection and IP address of the scope. The scope is detected properly.
Next I am selecting a "Memory" as a source. I assume this should obtain the full wave form from memory to pulseview. Clicking "Run" is doing two bad things:
1. Obtaining probably a Live instead of Memory buffer (the waveform is different every time)
2. Starting oscilloscope to running! (instead of staying in STOP state)

This leads me to start tracking the problem from sigrok-cli.
I've launched:
sigrok-cli -d rigol-ds:conn=tcp-raw/ -l 5 --frames 1 -O analog
The results are in my attachment: frame1-output.log

As you can see there is a command ":RUN" to the scope and the ":WAV:MODE NORM" instead of ":WAV:MODE RAW".
It's probably because sigrok-cli source was not select, so I was trying to pass the 'Memory' source to the driver somehow eg with '-d rigol-ds:data_source=Memory...' with no luck. Documentation is not stating anything about it :( How to select the source using sigrok-cli?
In this case I just do the ugly hack and hardcoded the source in the code
with the attached patch: mem_hardcode.patch
... but there was still problems:
1. the data seems different with every call
2. if I pass eg "--frames 10" then I have
rigol-ds: Starting data capture for frameset 1 of 10

But it is stopping on frame 1 with:
sr: [00:01.191605] rigol-ds: Block has been completed
sr: [00:01.191610] rigol-ds: Read should have been completed
sr: [00:01.191614] session: bus: Received SR_DF_FRAME_END packet.
cli: Received SR_DF_FRAME_END.
sr: [00:01.191620] hwdriver: rigol-ds: Stopping acquisition.
sr: [00:01.191624] session: bus: Received SR_DF_END packet.
cli: Received SR_DF_END.
g_variant_unref: assertion 'value->ref_count > 0' failed

My second attempt to this problem was the patch: more_frames.patch.
This time it was at least trying to fetch another frames. But the resulted samples also doesn't seem to be acquired from scope memory. I gave up at this point and started to write the support for writing output to binary data file from a tool called: DSRemote (https://gitlab.com/Teuniz/DSRemote).

This tool is fetching the full waveform from this scope perfectly fine.
After loading the data manually in PulseWave I am finally able to attach decoders and it is working fine. But of course we need this in sigrok, not using external tools! :)

I am attaching you a full pcap from this hoping you can also fix this in sigrok.
Full pcap is: dsremote-fullwave.pcap.bz2

Related tip: The block size is sigrok seems to be relatively low (1200, computed with min() function), while the DSRemote is using 250000. The difference in visible, fetching the full memory data is dramatically faster (the scope max mem size is 24M samples).

Tip #2: The DSRemote is saying that I cannot obtain a memory waveform data when the acquisition memory in scope is set to Auto (which is the scope default) so I've set it to 24M samples and I had it set like this for the whole above tests.
It would be great is also sigrok informs about this (or at least throw some informative error that the 'scope mem is set to auto, you cannot acquire memory waveform')...

btw: if I change the "-O analog" to "-O binary" in sigrok-cli it does change nothing, while if I set it to CSV output than is crashing :(

Reference for users reading this bug: Until this bug is fixed, I will write a quick tutorial how to save a waveform from DSRemote to being able to manually load and analyze it in PulseView:

All the tests was done on recent git versions (from yesterday) of libsigrok, libsigrokdecode, pulseview and sigrok-cli.
Comment 1 Mariusz Białończyk 2019-03-11 10:51:28 CET
Created attachment 507 [details]
sigrok-cli obtaining 1 frame
Comment 2 Mariusz Białończyk 2019-03-11 10:52:05 CET
Created attachment 508 [details]
sigrok-cli scan output
Comment 3 Mariusz Białończyk 2019-03-11 10:53:12 CET
Created attachment 509 [details]
patch for hardcoding the datasource
Comment 4 Mariusz Białończyk 2019-03-11 10:53:34 CET
Created attachment 510 [details]
patch for more frames then 1
Comment 5 Mariusz Białończyk 2019-03-11 11:00:01 CET
Created attachment 511 [details]
PCAP file with obtaining full memory waveform using DSRemote tool
Comment 6 Hugo Laloge 2020-02-10 15:05:56 CET
Can confirm with DS1104Z, rigol_ds_receive stops systematically after acquired one frame.
Comment 7 Hugo Laloge 2020-02-12 12:09:14 CET
Created attachment 591 [details]
Patch correct read completion

I might have a fix: the rigol_ds_receive prodedure while reading exactly what it was expecting, and the called `sr_scpi_read_complete`. However, `sr_scpi_read_complete` return false if the last read received exactly the number of requested byte.

A simple fix is to ask for one more byte at the last read, so when the driver ask to read the trailing \n, it now ask for two bytes.
Comment 8 Mariusz Białończyk 2020-02-12 12:48:09 CET
Great you are working on this. Tell me if your patch is for reading memory or live "data source" from the scope?
Comment 9 Hugo Laloge 2020-02-12 13:52:53 CET
Actually, the patch fixes reading the samples in multiple time. In LIVE mode, all the data can be retrieved in one time, so the bug is not noticeable (although there was the error "Read should have been completed").

In memory, the number of samples is too large to retrieve them at once, so the bug is encountered.

So this fix is for both mode, but the result is mainly noticeable when reading from memory.

Note that I still encounter some other bugs, not related to the data "reading". For instance, the driver does not yet specify the sample rate in memory mode.