Difference between revisions of "Protocol decoder API"
Uwe Hermann (talk | contribs) m |
|||
Line 1: | Line 1: | ||
This page describes how [[libsigrokdecode]] '''Protocol Decoders (PDs)''' work. | This page describes how [[libsigrokdecode]] '''Protocol Decoders ([[Protocol decoders|PDs]])''' work. | ||
See also [[Protocol decoder HOWTO]] for a quick introduction of how to write your own decoders. | See also [[Protocol decoder HOWTO]] for a quick introduction of how to write your own decoders. | ||
Line 31: | Line 31: | ||
* '''<tt>OUTPUT_ANN</tt>''' | * '''<tt>OUTPUT_ANN</tt>''' | ||
<blockquote> | <blockquote> | ||
A constant used to register annotation output, used as argument to the '''<tt>[[#register-function|register()]]</tt>''' function. [[Sigrok-cli|sigrok-cli]] shows the annotation output of a decoder stack's topmost decoder (per default), [[PulseView]] shows annotation output as graphical boxes or as circles (if the duration of the annotation is zero). | |||
</blockquote> | </blockquote> | ||
* '''<tt>OUTPUT_PYTHON</tt>''' | * '''<tt>OUTPUT_PYTHON</tt>''' | ||
<blockquote> | <blockquote> | ||
A constant used to register python output, used as argument to the '''<tt>[[#register-function|register()]]</tt>''' function. Python output is passed as input to a decoder that is stacked onto the current decoder. The format of the data that is given to the '''<tt>[[#put-function|put()]]</tt>''' function is specific to a certain PD and should be documented for the authors of the higher level decoders, for example with a comment at the top of the decoder's source file. | |||
</blockquote> | </blockquote> | ||
* '''<tt>OUTPUT_BINARY</tt>''' | * '''<tt>OUTPUT_BINARY</tt>''' | ||
<blockquote> | <blockquote> | ||
A constant used to register binary output, used as argument to the '''<tt>[[#register-function|register()]]</tt>''' function. The format of the data that is outputted is not specified, it's up to the author of the decoder to choose one (or multiple) appropriate format(s). For example, the [[Protocol_decoder:Uart|UART]] decoder outputs the raw bytes that it decodes, the [[Protocol_decoder:I2s|I²S]] decoder outputs the audio in WAV format, but the output could also be an image (JPG, PNG, other) file for a decoder that decodes a display protocol, a PCAP file for network/USB decoders, or one of [[Protocol decoder output|many other]] formats. [[Sigrok-cli|sigrok-cli]] can be used to redirect the binary output of a decoder into a file (or to pipe it into other applications), see the documentation of its '''--protocol-decoder-binary''' option. | |||
</blockquote> | </blockquote> | ||
* '''<tt>OUTPUT_META</tt>''' | * '''<tt>OUTPUT_META</tt>''' | ||
<blockquote> | <blockquote> | ||
A constant used to register metadata output, used as argument to the '''<tt>[[#register-function|register()]]</tt>''' function. An example for a PD that outputs metadata is the SPI decoder that uses it to output the detected bitrate. See [[Protocol decoder output]] for various other possible examples. | |||
</blockquote> | </blockquote> | ||
* <div id="put-function">'''<tt>put(startsample, endsample, output_id, data)</tt>'''</div> | * <div id="put-function">'''<tt>put(startsample, endsample, output_id, data)</tt>'''</div> | ||
<blockquote> | <blockquote> | ||
This is used to provide the decoded data back into the backend. | This is used to provide the decoded data back into the backend. '''<tt>startsample</tt>''' and '''<tt>endsample</tt>''' specify the absolute sample numbers of where this item (e.g. an annotation) starts and ends. '''<tt>output_id</tt>''' is an output identifier returned by the '''<tt>[[#register-function|register()]]</tt>''' function. The '''<tt>data</tt>''' parameter's contents depend on the output type ('''<tt>output_id</tt>'''). | ||
'''<tt>output_id</tt>''' is an output identifier returned by the '''<tt>[[#register-function|register()]]</tt>''' function. | |||
The '''<tt>data</tt>''' parameter's | |||
</blockquote> | </blockquote> | ||
Line 59: | Line 57: | ||
* '''<tt>start(self)</tt>''' | * '''<tt>start(self)</tt>''' | ||
<blockquote> | <blockquote> | ||
This function is called before the beginning of the decoding. This is the place to '''<tt>[[#register-function|register()]]</tt>''' the output types. | This function is called before the beginning of the decoding. This is the place to '''<tt>[[#register-function|register()]]</tt>''' the output types, check the user-supplied PD options for validity, and so on. | ||
</blockquote> | </blockquote> | ||
* <div id="decode-function">'''<tt>decode(self, startsample, endsample, data)</tt>'''</div> | * <div id="decode-function">'''<tt>decode(self, startsample, endsample, data)</tt>'''</div> | ||
<blockquote> | <blockquote> | ||
This is a function that | This is a function that is called by the [[libsigrokdecode]] backend whenever it has a chunk of data for the protocol decoder to handle. | ||
{| border="0" style="font-size: smaller;" class="alternategrey sortable sigroktable" | {| border="0" style="font-size: smaller;" class="alternategrey sortable sigroktable" | ||
Line 84: | Line 82: | ||
* Raw samples ('''<tt>inputs = ['logic']</tt>'''): | * Raw samples ('''<tt>inputs = ['logic']</tt>'''): | ||
<blockquote> | <blockquote> | ||
'''<tt>data</tt>''' is a list of tuples containing the sample number and the channels of that sample: '''<tt>[(samplenum, channels), (samplenum, channels), ...]</tt>'''.<br> | '''<tt>data</tt>''' is a list of tuples containing the (absolute) sample number and the channels of that sample: '''<tt>[(samplenum, channels), (samplenum, channels), ...]</tt>'''.<br /> | ||
'''<tt>samplenum</tt>''' is the number of the sample, an integer that takes the values from '''<tt>startsample</tt>''' to '''<tt>endsample</tt>''' - 1.<br> | '''<tt>samplenum</tt>''' is the (absolute) number of the sample, an integer that takes the values from '''<tt>startsample</tt>''' to '''<tt>endsample</tt>''' - 1.<br /> | ||
The type of '''<tt>channels</tt>''' is [https://docs.python.org/3/library/stdtypes.html#typebytes '''<tt>bytes</tt>'''], | The type of '''<tt>channels</tt>''' is [https://docs.python.org/3/library/stdtypes.html#typebytes '''<tt>bytes</tt>'''], | ||
a sequence type whose length is the sum of the lengths of '''<tt>channels</tt>''' and '''<tt>optional_channels</tt>''' | a sequence type whose length is the sum of the lengths of '''<tt>channels</tt>''' and '''<tt>optional_channels</tt>''' | ||
( | (in other words, '''<tt>channels</tt>''' contains a byte for every channel/optional channel).<br /> | ||
The order of the bytes is the same as the order of the channels in '''<tt>channels</tt>''' and '''<tt>optional_channels</tt>'''.<br> | The order of the bytes is the same as the order of the channels in '''<tt>channels</tt>''' and '''<tt>optional_channels</tt>'''.<br> | ||
The individual bytes take the values '''<tt>0</tt>''' or '''<tt>1</tt>''', or some other value for optional channels that aren't supplied to the decoder.<br> | The individual bytes take the values '''<tt>0</tt>''' or '''<tt>1</tt>''', or some other value for optional channels that aren't supplied to the decoder.<br> | ||
The [[Protocol_decoder_HOWTO#channels_.26_optional_channels|Protocol decoder HOWTO | The [[Protocol_decoder_HOWTO#channels_.26_optional_channels|Protocol decoder HOWTO]] page contains an example how the data can be processed. | ||
</blockquote> | </blockquote> | ||
Line 103: | Line 101: | ||
</blockquote> | </blockquote> | ||
=== Optional | ==== Optional functions ==== | ||
* '''<tt>metadata(self, key, value)</tt>''' | * '''<tt>metadata(self, key, value)</tt>''' |
Revision as of 22:41, 24 July 2014
This page describes how libsigrokdecode Protocol Decoders (PDs) work.
See also Protocol decoder HOWTO for a quick introduction of how to write your own decoders.
Architecture
The frontend gets input from the user on which PDs to use in an acquisition session. It then configures these into the session with session_pd_add(). As the first PD is added, the session sets up an additional datafeed callback to itself, which it uses as input to the first PD in the stack. The output of that is sent to the frontend, along with its original datafeed, as well as fed into the next PD in the stack.
The frontend thus gets the raw datafeed as well as a feed from every PD in the stack. Which of these different feeds is actually displayed to the user is a matter of configuration or selection by the user; it should be possible, for example, to have sigrok-cli print only the top of the PD stack's output on stdout.
- All PDs are written in Python (>= 3.0).
- Every PD registers its name, description, capabilities, etc.
- PDs will be stacked together, so the user can construct a decoding pipeline. The control of communication to/from PDs is done by the PD controller code in libsigrokdecode.
- The data feed into the PDs will be streamed, so they will run in real time as the data comes in from the hardware (or from a file).
- In order to keep PDs simple, they don't have to deal with the intricacies of the datafeed packets. Instead, the PD controller will hide the details from the Python code:
- When receiving a DF_HEADER packet going to a PD, the controller intercepts the packet and instead generates a DF_HEADER "coming from" that PD across the session bus, with that PD's output characteristics (which it derived from the PD's register dictionary).
- Data packets get translated into a bytestream, which the PDs access through an API: the function get() is a blocking call into the controller, which only returns when a datafeed packet has arrived, and its payload queued up for the PD.
- DF_END packets are translated into a call to decode() with an empty data set.
API
Backend library
A Python module called sigrokdecode is provided. Every protocol decoder must import this. It contains the following items:
- the Decoder object
Every protocol decoder must subclass this object.
- OUTPUT_ANN
A constant used to register annotation output, used as argument to the register() function. sigrok-cli shows the annotation output of a decoder stack's topmost decoder (per default), PulseView shows annotation output as graphical boxes or as circles (if the duration of the annotation is zero).
- OUTPUT_PYTHON
A constant used to register python output, used as argument to the register() function. Python output is passed as input to a decoder that is stacked onto the current decoder. The format of the data that is given to the put() function is specific to a certain PD and should be documented for the authors of the higher level decoders, for example with a comment at the top of the decoder's source file.
- OUTPUT_BINARY
A constant used to register binary output, used as argument to the register() function. The format of the data that is outputted is not specified, it's up to the author of the decoder to choose one (or multiple) appropriate format(s). For example, the UART decoder outputs the raw bytes that it decodes, the I²S decoder outputs the audio in WAV format, but the output could also be an image (JPG, PNG, other) file for a decoder that decodes a display protocol, a PCAP file for network/USB decoders, or one of many other formats. sigrok-cli can be used to redirect the binary output of a decoder into a file (or to pipe it into other applications), see the documentation of its --protocol-decoder-binary option.
- OUTPUT_META
A constant used to register metadata output, used as argument to the register() function. An example for a PD that outputs metadata is the SPI decoder that uses it to output the detected bitrate. See Protocol decoder output for various other possible examples.
- put(startsample, endsample, output_id, data)
This is used to provide the decoded data back into the backend. startsample and endsample specify the absolute sample numbers of where this item (e.g. an annotation) starts and ends. output_id is an output identifier returned by the register() function. The data parameter's contents depend on the output type (output_id).
Decoder class functions
Required functions
- start(self)
This function is called before the beginning of the decoding. This is the place to register() the output types, check the user-supplied PD options for validity, and so on.
- decode(self, startsample, endsample, data)
This is a function that is called by the libsigrokdecode backend whenever it has a chunk of data for the protocol decoder to handle.
Argument Description startsample The absolute samplenumber of the first sample in this chunk of data. endsample The absolute samplenumber of the last sample in this chunk of data. data A list containing the data to decode. Depending on whether the decoder decodes raw samples or is stacked onto another decoder, this argument is:
- Raw samples (inputs = ['logic']):
data is a list of tuples containing the (absolute) sample number and the channels of that sample: [(samplenum, channels), (samplenum, channels), ...].
samplenum is the (absolute) number of the sample, an integer that takes the values from startsample to endsample - 1.
The type of channels is bytes, a sequence type whose length is the sum of the lengths of channels and optional_channels (in other words, channels contains a byte for every channel/optional channel).
The order of the bytes is the same as the order of the channels in channels and optional_channels.
The individual bytes take the values 0 or 1, or some other value for optional channels that aren't supplied to the decoder.
The Protocol decoder HOWTO page contains an example how the data can be processed.
- Stacked decoder (inputs = ['<id of some other decoder>']):
data is the OUTPUT_PYTHON output of the decoder this PD is stacked upon. Its format depends on the implementation of the underlying decoder and should be documented there.
Optional functions
- metadata(self, key, value)
Used to pass the decoder metadata about the data stream. Currently the only value for key is sigrokdecode.SRD_CONF_SAMPLERATE, value is then the sample rate of the data stream in Hz.
Decoder registration
A PD's Decoder class must contain a few attributes specifying metadata about the PD. The following keys can be used:
Key Description api_version The libsigrokdecode API version which this module uses. This is currently 2. id A short unique identifier for this protocol decoder. It should be all-lowercase, and only contains a-z, 0-9 and underscores. This must match the PD's Python module name (subdirectory name in the decoders directory). The sigrok-cli tool uses this to specify PDs on the command-line. Examples: 'jtag', 'sdcard_spi', 'uart'. name The name of the decoder. Used when listing available PDs. Examples: 'JTAG', 'SD card (SPI mode)', 'UART'. longname The (long) name of the decoder. Used when listing available PDs. Example: 'Joint Test Action Group (IEEE 1149.1)', 'Secure Digital card (SPI mode)', 'Universal Asynchronous Receiver/Transmitter'. desc A freeform one-line description of the decoder. Used when listing available PDs. Should end with a full stop. Example: 'Protocol for testing, debugging, and flashing ICs.', 'Secure Digital card (SPI mode) low-level protocol.', 'Asynchronous, serial bus.'. license The license under which the module is provided. This must be either gplv2+ (meaning the GNU General Public License 2 or later), or gplv3+ (GNU General Public License 3 or later). No other licenses for modules are permitted in libsigrokdecode. inputs The list of types of input this decoder needs. If the decoder takes input from a logic analyzer driver, this should be set to logic, which maps to SR_DF_LOGIC, the datafeed type. If it takes input from another PD, it should be set to the value of the outputs key of that PD. It should conform to the same rules as the id key (lowercase, no spaces, and so on). outputs The list of types of output this decoder produces. If this decoder can feed decoded data back into the datafeed stream, its outputs will be identified with this key's value. It should conform to the same rules as the id key. If not specified, this decoder cannot feed data back into the stream. This typically means it only does analysis on the whole stream, producing a report at the end of acquisition. channels This key lists the channels (pins) the hardware should feed to this PD. The channels in this list MUST be present in the data; the PD will not be able to work without them. For example, an SPI decoder has to know which chanel has the clock, which has the chip select, and so on. This key contains a list of channel entries, where each entry is of the Python dict with the keys id, name, and desc. Example: {'id': 'rx', 'name': 'RX', 'desc': 'UART receive line'}. optional_channels The list of channels the PD can make use of, but are not strictly required. The list is in the same format as that of the channels key, above. This list is allowed to be empty if the respective protocol has no optional channels. options A dictionary with options for this decoder. The keys should follow the same rules as for the id key above, and each value is a list consisting of a short freeform description of the option, and the default value for that option. For example, an SPI decoder might have an entry with key cpol, and value ['Clock polarity', 0]. This list can be empty, if the PD has no options. annotations A list of annotation types this protocol decoder can output. Elements of this list are tupes consisting of an identifier string and a human readable description string. The identifier string can be used in the options of sigrok-cli to select the specific annotation type, and should therefore not contain whitespace or special characters annotation_rows Annotation rows are used to group multiple annotation types together. The elements of this list are three element tuples consisting of:
- An identifier string.
- A human readable description string.
- A tuple containing the indices of the the annotation types in the annotations list.
See the example on the Protocol decoder HOWTO page for more information on this attribute.
binary A list of binary types this protocol decoder can output, same format as the annotations list.
- register(output_type)
This function is used to register the output that will be generated by the decoder, its argument should be one of the OUTPUT_... constants described above. The function returns an identifier that can then be used as the output_id argument of the put() function.
See the Protocol decoder HOWTO#pd.py for an example.
File structure
Code
See Protocol decoder HOWTO#Files.
Source code and copyright
The module MUST come with source code in the form of .py files. No pre-compiled code should be present, Python or otherwise. The module must not use any helpers that are not provided as source code under the same license as the module itself.
The Decoder class must have a license declaration (see above), stating the license under which all the contents in the module directory are provided.
Example files
Every protocol decoder module MUST come with example input and output files that put the decoder through its paces. All corner cases the decoder can handle must be triggered by the example files. Please submit the example files for inclusion in the sigrok-dumps repository.