34C3

From sigrok
Jump to navigation Jump to search

We're getting together at the annual CCC conference, 34C3, for a sigrok hackathon. In addition to the usual "whatever we feel like hacking on", we also have several architectural decision to make, and doing this in person is a lot easier. If you want to be part of the conversation and decision-making process, show up!

New sigrok file format

File format:sigrok/v3

What's missing to make it happen?

Discuss concepts for sparse data loading presented here: https://sourceforge.net/p/sigrok/mailman/message/25153496/

[2017-12-28]

Martin, Uwe and Soeren discussed the current state of the spec. In general it's fine and can be implemented, some remarks:

  • All packets will have a 1-byte version field to allow changing the spec on a per-packet level without the need for new UUIDs (Rationale: an older client will silently ignore packets with unknown UUIDs but a known-UUID packet with an unrecognized version allows the client to produce a meaningful error message to the user)
  • A field that is mandatory is added to the relevant packet, a field that's optional receives its own packet (Rationale: when changing the meaning of a field, increasing the version of the packet it belongs to would result in an unreadable file for older clients if the packet is mandatory. If a client encounters an optional packet with an unknown version, it can simply be skipped and the file remains readable)
  • PV will use custom UUIDs to store UI/acquisition/device state with saved session files
  • PV will use the same file format to store UI/acquisition/device state, just without any device/channel/data packets

Handling of RLE data

What is needed to make libsigrok able to handle RLE data? Should there be a common format that drivers have to convert custom formats to? Should there be a new session bus packet type? What are the implications?

See also the discussion from 30C3 here.


Analog I/O for libsigrokdecode

What are the requirements/use cases? What's needed to make it happen?


Flowgraph architecture

How do we achieve this step-by-step? It won't ever happen if it's a huge change :) Currently, plans are to make PV have a modularized internal structure and use flowgraphs internally to gather experience and have something working

[2017-12-27/28]

This is what we have:

This is what we want:

  • libsigrokflow as a core library that provides the basic flowgraph architecture elements (potentially re-usable by different projects)
  • The same library allows libsigrok and libsigrokdecode to be tied into the flowgraph architecture
  • Library code in C/C++ for speed and flexibility to allow blocks to be written in any language later on
  • numpy BitArray-like data type for uncompressed data, allowing blocks to hand over BitArray objects if they don't change it ("it's a view on the raw data")
  • One thread per source block, downstream blocks are called within the same thread
  • FIFO blocks that act like a buffer+source block, creating a new thread to allow decoupling
  • Signal (connection between two blocks) types are modeled by hierarchical C++ classes, e.g. Analog;Logic->Uncompressed/RLE;Decode to allow type checking at runtime and easy "don't care" block inputs: if a block accepts Logic, can just tell the Logic base class to hand over RLE data without needing to worry about any conversions required to obtain it
  • libsigrokgui as a library that contains Qt-based UI elements used to build a basic sigrok client UI; at least a sigrok flowgraph UI window (including all necessary libsigrokflow interfacing) and trace drawing widgets

This is how we'll do it:

  • We'll work on top and bottom levels at the same time
  • First, Martin and Soeren will work on a python-based PoC, including a throwaway UI to test out the API
  • When the architecture is stable, Martin will work on the C++ framework for libsigrokflow and Soeren on libsigrokgui and its PV integration
  • Uwe will set up the subproject on git once the library is in a usable state (no longer PoC status)

Extra thoughts:

  • Blocks have key/value properties, keys being strings allowing the object to provide descriptions as well
  • This way, we can get the https://sigrok.org/wiki/Improved_configuration_enumeration for free
  • libsigrokgui is needed because otherwise, we will never see full-featured UIs - the UI code would be too tightly coupled to PV; if not doing this, PV will be regarded as the universal sigrok client and see more and more feature creep

This needs more thinking:

  • How would a plugin mechanism work for the blocks?
  • How can blocks be made available at runtime? (i.e. device blocks)
  • How do we handle device discovery?
  • Do new signal types need to be addable at runtime, e.g. for PD blocks that provide binary data (e.g. images or EEPROM dumps) or do we simply use a "binary" type with metadata as properties, e.g. a MIME type to identify the contents?
  • The current concept implies a push architecture where data is pushed out by the source and all attached blocks are only executed once they have enough data to operate on. However, for even moderately fast USB3 acquisition devices (think 400 MS/s per analog channel), this model breaks down. In this scenario, it would make more sense to let the acq driver buffer the data in the device's native format for performance reasons (data processing would break acquisition) and then let the consumers of the data (view, PD, etc.) pull the desired data from the connected blocks which in turn pull the data from the source, which then converts the device's native data to standardized data as it's requested. We need to decide which approach works better in the long run and determine how the decision affects our design.

PV: Definition of scope

What is PV supposed to be? What is it not supposed to be?

We should define this so we can determine a sane architecture and prevent feature creep. The architecture for PV would need major extension if we also want to support power supplies and frequency generators to be able to do (cool) stuff like network analysis, bode plots, load tests, pattern/signal generation and similar. However, the question is: what is realistic and what is better left to more specialized clients? Turning PV into a one-size-fits-all client will be too much work for one person and progress will slow down.

[2017-12-29]

For now, PV will only cover devices that deliver sampled data in regular intervals. Other devices (e.g. AWGs, pattern gens, PSUs) will be supported through libsigrokflow by means of input/output blocks.


PV: Scripting

Is it useful to have scripting available in PV? What use cases are there? Determine what would need to happen to support it.

[2017-12-29]

As there are more pressing issues in the near future, this isn't going to be on the roadmap for a while. However, it is worthwhile to check how scopy did it and implement this for PV as well since it appears to be pretty much free if Qt properties are attached to objects.

PV: Decode table view

The decode table view isn't going to be very usable at the moment because it lacks metadata that libsigrokdecode should provide. We should discuss what the decode table view should offer in terms of functionality and what's needed in libsigrokdecode to make it happen.

[2017-12-29]

Uwe and Soeren discussed use cases and the resulting requirements on the libsigrokdecode side:

  • Simple key/value (key=annotation class, e.g. CRC) pairs are useful but insufficient
  • PDs that output analog data must provide metadata for creation of libsigrok analog packets (e.g. unit)
  • PDs should be able to define a "desired output format" for each annotation class - e.g. ASCII for integer annotations
  • PDs could provide hierarchy information by referencing annotations by ID; this is a good way to deal with annotations which are generated as part of a packet but have a more global meaning and shouldn't be grouped with the packet itself

There is more discussion needed to determine how to proceed.

PV: Scope UI

PV is getting closer and closer to being usable for analog scopes. We should discuss ideas how a UI could look like that allows for full remote-control operation. There are good and bad UI examples to draw ideas from.


PV: Create "wishlist" wiki page

Users don't know what features are planned and when. Soeren has his own (non-public) to-do list and there are some items in the bugzilla. This should be unified so that users can see what's planned and what's not on our radar yet.