Difference between revisions of "Protocol decoder API"

From sigrok
Jump to navigation Jump to search
m
(Update parts of the page to the current API and implementation.)
Line 7: Line 7:
The frontend thus gets the raw datafeed as well as a feed from every PD in the stack. Which of these different feeds is actually displayed to the user is a matter of configuration or selection by the user; it should be possible, for example, to have [[sigrok-cli]] print only the top of the PD stack's output on stdout.
The frontend thus gets the raw datafeed as well as a feed from every PD in the stack. Which of these different feeds is actually displayed to the user is a matter of configuration or selection by the user; it should be possible, for example, to have [[sigrok-cli]] print only the top of the PD stack's output on stdout.


* All PDs are written in Python.
* All PDs are written in Python (>= 3.0).
* Every PD registers its name, description, capabilities, etc by populating a hash (dictionary) in its own main namespace called <tt>register</tt>.
* Every PD registers its name, description, capabilities, etc.
* PDs will be stacked together, so the user can construct a decoding pipeline. The control of communication to/from PDs is done by the PD controller code in libsigrokdecode.
* PDs will be stacked together, so the user can construct a decoding pipeline. The control of communication to/from PDs is done by the PD controller code in libsigrokdecode.
* The data feed into the PDs will be streamed, so they will run in real time as the data comes in from the hardware (or from a file).
* The data feed into the PDs will be streamed, so they will run in real time as the data comes in from the hardware (or from a file).
Line 20: Line 20:
=== Backend library ===
=== Backend library ===


A module called '''<tt>sigrokdecode</tt>''' is provided. Every protocol decoder must import this. It contains the following items:
A Python module called '''<tt>sigrokdecode</tt>''' is provided. Every protocol decoder must import this. It contains the following items:


* '''<tt>the Decoder object</tt>'''
* '''<tt>the Decoder object</tt>'''
Line 31: Line 31:
* '''<tt>OUTPUT_BINARY</tt>'''
* '''<tt>OUTPUT_BINARY</tt>'''


* '''<tt>put(time, duration, data)</tt>'''
* '''<tt>put(startsample, endsample, output_type, data)</tt>'''
<blockquote>
<blockquote>
This is used to provide the decoded data back into the backend. It takes the same arguments as the '''<tt>decode()</tt>''' function.
This is used to provide the decoded data back into the backend. It takes the same arguments as the '''<tt>decode()</tt>''' function.
Line 44: Line 44:
==== Required functions ====
==== Required functions ====


* '''<tt>decode(time, duration, data)</tt>'''
* '''<tt>decode(startsample, endsample, data)</tt>'''
<blockquote>
<blockquote>
This is a function that will be called by the PD controller whenever it has a chunk of data for the module to handle.
This is a function that will be called by the PD controller whenever it has a chunk of data for the module to handle.


{| border="0"
{| border="0" style="font-size: smaller;" class="alternategrey sortable sigroktable"
|- bgcolor="#6699ff"
|-
!style="width: 8em;" | Argument
!style="width: 8em;" | Argument
!Description
!Description


|- bgcolor="#eee"
|-
| '''<tt>time</tt>'''
| '''<tt>startsample</tt>'''
| The time when this set of samples was captured, expressed as an offset in picoseconds since the session's starttime as returned by the meta() function.
| The absolute samplenumber of the first sample in this chunk of data.


|- bgcolor="#eee"
|-
| '''<tt>duration</tt>'''
| '''<tt>endsample</tt>'''
| The time it took, in picoseconds, for all the data in this set to be captured.
| The absolute samplenumber of the last sample in this chunk of data.


|- bgcolor="#eee"
|-
| '''<tt>data</tt>'''
| '''<tt>data</tt>'''
| A list containing items of data, such as samples or output from protocol decoders lower on the stack. Each item consists of an [http://docs.python.org/py3k/library/array.html array] of type '''B''', which is an unsigned char. The length of this array corresponds with the '''<tt>unitsize</tt>''' value returned by the '''<tt>get_meta()</tt>''' call.
| A list containing items of data, such as samples or output from protocol decoders lower on the stack. Each item consists of an [http://docs.python.org/py3k/library/array.html array] of type '''B''', which is an unsigned char. The length of this array corresponds with the '''<tt>unitsize</tt>''' value returned by the '''<tt>get_meta()</tt>''' call.
Line 69: Line 69:


==== Optional functions ====
==== Optional functions ====
* '''<tt>report()</tt>'''
* '''<tt>report()</tt>'''
<blockquote>
<blockquote>
Line 76: Line 77:
== Module registration ==
== Module registration ==


A PD must contain at least a populated dictionary '''register''' which defines the name, capabilities etc of the PD. The following keys can be used:
A PD's '''Decoder''' class must contain a few attributes specifying metadata about the PD. The following keys can be used:


<blockquote>
<blockquote>
{| border="0"
{| border="0" style="font-size: smaller;" class="alternategrey sortable sigroktable"
|- bgcolor="#6699ff"
|-
 
!style="width: 8em;" | Key
!style="width: 8em;" | Key
!Required
!Description
!Description


|- bgcolor="#6699ff"
|-
| colspan="3" | '''Protocol Decoder information'''
| '''<tt>api_version</tt>'''
| The libsigrokdecode API version which this module uses. This is currently 1.


|- bgcolor="#eee"
|-
| '''<tt>id</tt>'''
| '''<tt>id</tt>'''
| yes
| A short unique identifier for this protocol decoder. It should be all-lowercase, and only contains a-z, 0-9 and underscores. The [[sigrok-cli]] tool uses this to specify PDs on the command-line. Example: "jtag".
| A short unique identifier for this protocol decoder. It should be all-lowercase, and only contains a-z, 0-9 and underscores. The [[sigrok-cli]] tool uses this to specify PDs.
 
|- bgcolor="#ddd"
| '''<tt>description</tt>'''
| yes
| A freeform one-line description of the decoder. Used when listing available PDs.


|- bgcolor="#eee"
|-
| '''<tt>author</tt>'''
| '''<tt>name</tt>'''
| no
| The name of the decoder. Used when listing available PDs. Example: "JTAG".
| Name of the author of this PD.


|- bgcolor="#ddd"
|-
| '''<tt>email</tt>'''
| '''<tt>longname</tt>'''
| no
| The (long) name of the decoder. Used when listing available PDs. Example: "Joint Test Action Group (IEEE 1149.1)".
| Email address of the author of this PD.


|- bgcolor="#eee"
|-
| '''<tt>version</tt>'''
| '''<tt>desc</tt>'''
| no
| A freeform one-line description of the decoder. Used when listing available PDs. Example: "Protocol for testing, debugging, and flashing ICs.".
| Version of this PD.


|- bgcolor="#ddd"
|-
| '''<tt>apiversion</tt>'''
| yes
| The sigrok API version which this module uses. This is currently 1.
 
|- bgcolor="#eee"
| '''<tt>license</tt>'''
| '''<tt>license</tt>'''
| yes
| The license under which the module is provided. This must be either '''<tt>gplv2+</tt>''' (meaning the GNU General Public License 2 or later), or '''<tt>gplv3+</tt>''' (GNU General Public License 3 or later). No other licenses for modules are permitted in libsigrokdecode.
| The license under which the module is provided. This must be either '''<tt>gplv2+</tt>''' (meaning the GNU General Public License 2 or later), or '''<tt>gplv3+</tt>''' (GNU General Public License 2 or later). No other licenses for modules are permitted in sigrok.


|- bgcolor="#ddd"
|-
| '''<tt>in</tt>'''
| '''<tt>inputs</tt>'''
| yes
| The list of types of input this decoder needs. If the decoder takes input from a logic analyzer driver, this should be set to '''<tt>logic</tt>''', which maps to SR_DF_LOGIC, the datafeed type. If it takes input from another PD, it should be set to the value of the '''<tt>outputs</tt>''' key of that PD. It should conform to the same rules as the '''<tt>id</tt>''' key (lowercase, no spaces, and so on).
| The type of input this decoder needs. If the decoder takes input from a logic analyzer driver, this should be set to '''<tt>logic</tt>''', which maps to DF_LOGIC, the datafeed type. If it takes input from another PD, it should be set to the value of the '''<tt>out</tt>''' key of that PD. It should conform to the same rules as the '''<tt>id</tt>''' key (lowercase, no spaces, and so on).


|- bgcolor="#eee"
|-
| '''<tt>out</tt>'''
| '''<tt>outputs</tt>'''
| no
| The list of types of output this decoder produces. If this decoder can feed decoded data back into the datafeed stream, its outputs will be identified with this key's value. It should conform to the same rules as the '''<tt>id</tt>''' key. If not specified, this decoder cannot feed data back into the stream. This typically means it only does analysis on the whole stream, producing a report at the end of acquisition.
| If this decoder can feed decoded data back into the datafeed stream, its output will be identified with this key's value. It should conform to the same rules as the '''<tt>id</tt>''' key. If not specified, this decoder cannot feed data back into the stream. This typically means it only does analysis on the whole stream, producing a report at the end of acquisition.


|- bgcolor="#6699ff"
|-
| colspan="3" | '''Protocol Decoder configuration'''
 
|- bgcolor="#eee"
| '''<tt>probes</tt>'''
| '''<tt>probes</tt>'''
| no
| This key lists the probes (pins) the hardware should feed to this PD. The probes in this list MUST be present in the data; the PD will not be able to work without them. For example, an SPI decoder has to know which probe has the clock, which has the chip select, and so on. This key contains a list of probe entries, where each entry is of the Python dict with the keys '''id''', '''name''', and '''desc'''. Example: '''<tt>{'id': 'rx', 'name': 'RX', 'desc': 'UART receive line'}</tt>'''.
| This key lists the probes (pins) the hardware should feed to this PD. If this key is specified, the probes in this list MUST be present in the data; the PD will not be able to work without them. For example, an SPI decoder has to know which probe has the clock, which has the chip select, and so on. This key contains a list of probe entries, where each entry can be either a string with the probe name ('''<tt>"SCLK"</tt>''') or a list with the probe name and description, such as '''<tt>["SCLK", "Clock"]</tt>'''.


|- bgcolor="#ddd"
|-
| '''<tt>extra_probes</tt>'''
| '''<tt>optional_probes</tt>'''
| no
| The list of probes the PD can make use of, but are not strictly required. The list is in the same format as that of the <tt>probes</tt> key, above. This list is allowed to be empty if the respective protocol has no optional probes.
| If present, this is a list of probes the PD can make use of, but are not strictly required. The list is in the same format as that of the <tt>probes</tt> key, above.


|- bgcolor="#eee"
|-
| '''<tt>options</tt>'''
| '''<tt>options</tt>'''
| no
| A dictionary with options for this decoder. The keys should follow the same rules as for the '''<tt>id</tt>''' key above, and each value is a list consisting of a short freeform description of the option, and the default value for that option. For example, an SPI decoder might have an entry with key '''<tt>cpol</tt>''', and value '''<tt>['Clock polarity', 0]</tt>'''. This list can be empty, if the PD has no options.
| A dictionary with options for this decoder. The keys should follow the same rules as for the '''<tt>id</tt>''' key above, and each value is a list consisting of a short freeform description of the option, and the default value for that option. For example, an SPI decoder might have an entry with key '''<tt>cpol</tt>''', and value '''<tt>['Clock polarity', 0]</tt>'''.


<!--
|-
|- bgcolor="#eee"
| '''<tt>annotations</tt>'''
|
| A list of annotation types this protocol decoder can output.
|
|
 
|- bgcolor="#ddd"
|
|  
|  
-->


|}
|}
Line 170: Line 139:
Here's an example for an SPI decoder:
Here's an example for an SPI decoder:


  register = {
  class Decoder(srd.Decoder):
  'id': 'spi',
    api_version = 1
  'desc': 'Serial peripheral interface',
    id = 'spi'
  'author': 'sigrok project',
    name = 'SPI'
  'email': 'sigrok-devel@lists.sourceforge.net',
    longname = 'Serial Peripheral Interface'
  'version': '1.0',
    desc = 'Full-duplex, synchronous, serial bus.'
  'apiversion': 1,
    license = 'gplv2+'
  'license': 'gplv3+',
    inputs = ['logic']
  'in': 'logic',
    outputs = ['spi']
  'out': 'spi',
    probes = [
  'probes': [
        {'id': 'miso', 'name': 'MISO',
    ['sclk', 'Clock, also known as SCK, or CLK'],
        'desc': 'SPI MISO line (Master in, slave out)'},
    'mosi',
        {'id': 'mosi', 'name': 'MOSI',
    'miso',
        'desc': 'SPI MOSI line (Master out, slave in)'},
    ['ncs', 'Chip Select, also known as CS, NCS, or CE'],
        {'id': 'sck', 'name': 'CLK', 'desc': 'SPI clock line'},
  ],
        {'id': 'cs', 'name': 'CS#', 'desc': 'SPI CS (chip select) line'},
  'options': {
    ]
    'cpol': ['Clock polarity', 0],
    optional_probes = []
    'cpha': ['Clock phase', 0],
    options = {
    'wordsize': ['Word size (in bits)', 8],
        'cs_polarity': ['CS# polarity', 'active-low'],
  },
        'cpol': ['Clock polarity', 0],
}
        'cpha': ['Clock phase', 0],
 
        'bitorder': ['Bit order within the SPI data', 'msb-first'],
The <tt>start</tt> function is called by libsigrokdecode at the start of a session which has this PD in its pipeline. It should implement a loop calling '''<tt>get()</tt>''' and '''<tt>put()</tt>''', and only return when an EOF is detected in that loop.
        'wordsize': ['Word size of SPI data', 8],
    }
    annotations = [
        ['Hex', 'SPI data bytes in hex format'],
    ]


== File structure ==
== File structure ==
Line 199: Line 172:
==== Code ====
==== Code ====


A protocol decoder always has its own directory, named after the PD's ID, corresponding to the '''<tt>id</tt>''' field in the '''<tt>register</tt>''' dictionary. The directory should have a '''<tt>__init__.py</tt>''' file in it. Like any other python module, this is the file that is responsible for organizing the rest of the module's code into its namespace.
See [[Protocol decoder HOWTO#Files]].
 
The '''<tt>register</tt>''' dictionary and the '''<tt>run</tt>''' fucntion are the only symbols that absolutely MUST be in the module's main namespace.
Here's an example minimal module:
 
In file '''<tt>__init__.py</tt>'''
  from foo import register, run
 
In file '''<tt>foo.py</tt>'''
 
  register = {
    'id': 'bar',
  [...]
  }
 
  def run():
    [...]


==== Source code and copyright ====
==== Source code and copyright ====


The module MUST come with source code in the form of .py files. No pre-compiled code should be present, python or otherwise. The module must not use any helpers that are not provided as source code under the same license as the module itself.
The module MUST come with source code in the form of .py files. No pre-compiled code should be present, Python or otherwise. The module must not use any helpers that are not provided as source code under the same license as the module itself.


The '''<tt>register</tt>''' dictionary must have a license declaration (see above), stating the license under which all the contents in the module directory are provided.
The '''<tt>Decoder</tt>''' class must have a license declaration (see above), stating the license under which all the contents in the module directory are provided.


==== Example files ====
==== Example files ====


Every protocol decoder module MUST come with example input and output files that put the decoder through its paces. All corner cases the decoder can handle must be triggered by the example files.  
Every protocol decoder module MUST come with example input and output files that put the decoder through its paces. All corner cases the decoder can handle must be triggered by the example files. Please submit the example files for inclusion in the [http://sigrok.org/gitweb/?p=sigrok-dumps.git;a=summary sigrok-dumps] repository.
 
 
* '''*.in''': example input as it would be received from sigrok, either raw samples from a hardware device, or decoded data from an upstream PD. In other words, the data in these files corresponds in format to the '''<tt>in</tt>''' field of this PD's '''<tt>register</tt>''' struct. Just as with live data in a real session, the module should not assume it will be fed this file at any particular speed, or in same-sized chunks.
 
* '''*.out''': this must correspond '''exactly''' with what this PD will output, having been fed the input file with the same filename and '''.in' extension. No more, no less.
 
These files must be present in the module's '''<tt>examples</tt>''' directory.

Revision as of 19:33, 1 January 2013

This page describes how Protocol Decoders (PDs) work in sigrok.

Architecture

The frontend gets input from the user on which PDs to use in an acquisition session. It then configures these into the session with session_pd_add(). As the first PD is added, the session sets up an additional datafeed callback to itself, which it uses as input to the first PD in the stack. The output of that is sent to the frontend, along with its original datafeed, as well as fed into the next PD in the stack.

The frontend thus gets the raw datafeed as well as a feed from every PD in the stack. Which of these different feeds is actually displayed to the user is a matter of configuration or selection by the user; it should be possible, for example, to have sigrok-cli print only the top of the PD stack's output on stdout.

  • All PDs are written in Python (>= 3.0).
  • Every PD registers its name, description, capabilities, etc.
  • PDs will be stacked together, so the user can construct a decoding pipeline. The control of communication to/from PDs is done by the PD controller code in libsigrokdecode.
  • The data feed into the PDs will be streamed, so they will run in real time as the data comes in from the hardware (or from a file).
  • In order to keep PDs simple, they don't have to deal with the intricacies of the datafeed packets. Instead, the PD controller will hide the details from the Python code:
    • When receiving a DF_HEADER packet going to a PD, the controller intercepts the packet and instead generates a DF_HEADER "coming from" that PD across the session bus, with that PD's output characteristics (which it derived from the PD's register dictionary).
    • Data packets get translated into a bytestream, which the PDs access through an API: the function get() is a blocking call into the controller, which only returns when a datafeed packet has arrived, and its payload queued up for the PD.
    • DF_END packets are translated into a call to decode() with an empty data set.

API

Backend library

A Python module called sigrokdecode is provided. Every protocol decoder must import this. It contains the following items:

  • the Decoder object

Every protocol decoder must subclass this object.

  • OUTPUT_ANN
  • OUTPUT_PROTO
  • OUTPUT_BINARY
  • put(startsample, endsample, output_type, data)

This is used to provide the decoded data back into the backend. It takes the same arguments as the decode() function.

The data parameter's arrays must be of whatever length is appropriate to this PD's output. In other words, it must match the PD's native unitsize.

Decoder module functions

Required functions

  • decode(startsample, endsample, data)

This is a function that will be called by the PD controller whenever it has a chunk of data for the module to handle.

Argument Description
startsample The absolute samplenumber of the first sample in this chunk of data.
endsample The absolute samplenumber of the last sample in this chunk of data.
data A list containing items of data, such as samples or output from protocol decoders lower on the stack. Each item consists of an array of type B, which is an unsigned char. The length of this array corresponds with the unitsize value returned by the get_meta() call.

Optional functions

  • report()

If provided, this is a function which returns a report in freeform text, describing some analysis done on the stream. For example, PDs for counting logic level transitions, bit rate analysis and so on would use this to report back. This function is typically only called at the end of the stream, so the whole stream is analyzed. However, the function may be called at any time -- possibly giving an analysis of data received up until that point.

Module registration

A PD's Decoder class must contain a few attributes specifying metadata about the PD. The following keys can be used:

Key Description
api_version The libsigrokdecode API version which this module uses. This is currently 1.
id A short unique identifier for this protocol decoder. It should be all-lowercase, and only contains a-z, 0-9 and underscores. The sigrok-cli tool uses this to specify PDs on the command-line. Example: "jtag".
name The name of the decoder. Used when listing available PDs. Example: "JTAG".
longname The (long) name of the decoder. Used when listing available PDs. Example: "Joint Test Action Group (IEEE 1149.1)".
desc A freeform one-line description of the decoder. Used when listing available PDs. Example: "Protocol for testing, debugging, and flashing ICs.".
license The license under which the module is provided. This must be either gplv2+ (meaning the GNU General Public License 2 or later), or gplv3+ (GNU General Public License 3 or later). No other licenses for modules are permitted in libsigrokdecode.
inputs The list of types of input this decoder needs. If the decoder takes input from a logic analyzer driver, this should be set to logic, which maps to SR_DF_LOGIC, the datafeed type. If it takes input from another PD, it should be set to the value of the outputs key of that PD. It should conform to the same rules as the id key (lowercase, no spaces, and so on).
outputs The list of types of output this decoder produces. If this decoder can feed decoded data back into the datafeed stream, its outputs will be identified with this key's value. It should conform to the same rules as the id key. If not specified, this decoder cannot feed data back into the stream. This typically means it only does analysis on the whole stream, producing a report at the end of acquisition.
probes This key lists the probes (pins) the hardware should feed to this PD. The probes in this list MUST be present in the data; the PD will not be able to work without them. For example, an SPI decoder has to know which probe has the clock, which has the chip select, and so on. This key contains a list of probe entries, where each entry is of the Python dict with the keys id, name, and desc. Example: {'id': 'rx', 'name': 'RX', 'desc': 'UART receive line'}.
optional_probes The list of probes the PD can make use of, but are not strictly required. The list is in the same format as that of the probes key, above. This list is allowed to be empty if the respective protocol has no optional probes.
options A dictionary with options for this decoder. The keys should follow the same rules as for the id key above, and each value is a list consisting of a short freeform description of the option, and the default value for that option. For example, an SPI decoder might have an entry with key cpol, and value ['Clock polarity', 0]. This list can be empty, if the PD has no options.
annotations A list of annotation types this protocol decoder can output.

Here's an example for an SPI decoder:

class Decoder(srd.Decoder):
   api_version = 1
   id = 'spi'
   name = 'SPI'
   longname = 'Serial Peripheral Interface'
   desc = 'Full-duplex, synchronous, serial bus.'
   license = 'gplv2+'
   inputs = ['logic']
   outputs = ['spi']
   probes = [
       {'id': 'miso', 'name': 'MISO',
        'desc': 'SPI MISO line (Master in, slave out)'},
       {'id': 'mosi', 'name': 'MOSI',
        'desc': 'SPI MOSI line (Master out, slave in)'},
       {'id': 'sck', 'name': 'CLK', 'desc': 'SPI clock line'},
       {'id': 'cs', 'name': 'CS#', 'desc': 'SPI CS (chip select) line'},
   ]
   optional_probes = []
   options = {
       'cs_polarity': ['CS# polarity', 'active-low'],
       'cpol': ['Clock polarity', 0],
       'cpha': ['Clock phase', 0],
       'bitorder': ['Bit order within the SPI data', 'msb-first'],
       'wordsize': ['Word size of SPI data', 8],
   }
   annotations = [
       ['Hex', 'SPI data bytes in hex format'],
   ]

File structure

Code

See Protocol decoder HOWTO#Files.

Source code and copyright

The module MUST come with source code in the form of .py files. No pre-compiled code should be present, Python or otherwise. The module must not use any helpers that are not provided as source code under the same license as the module itself.

The Decoder class must have a license declaration (see above), stating the license under which all the contents in the module directory are provided.

Example files

Every protocol decoder module MUST come with example input and output files that put the decoder through its paces. All corner cases the decoder can handle must be triggered by the example files. Please submit the example files for inclusion in the sigrok-dumps repository.