Protocol decoder output

From sigrok
Revision as of 18:32, 8 November 2013 by Uwe Hermann (talk | contribs) (→‎Reports: PD report use-cases.)
Jump to navigation Jump to search

NOTE: This page is currently work-in-progress.

A protocol decoder (PD) can output various different things that libsigrokdecode frontends (or other protocol decoders) can use for different purposes.

The following types of protocol decoder output are currently implemented or planned to be implemented later: annotations, binary output, reports, proto output.

Annotations

Annotations (the OUTPUT_ANN output type) are meant to be displayed to the user via a frontend (GUI, command-line, or other).

They are specifically not meant for any usage other than displaying, i.e. no further processing is done on them, no dumping to file formats, or anything similar to that.

While it is technically possible to a certain degree to e.g. parse the decoder annotation output that (command-line or other) frontends print, and do further processing on that (using sed, awk, grep, perl, or other tools), that is risky since the format of the respective frontend's output and the PD output itself can change at any time and is specifically not designed to be easily processable programmatically in any way, shape, or form.

There is a better facility for this kind of usage in libsigrokdecode though, see Binary output below.

Every protocol decoder can output an arbitrary number of different annotation types. Not all protocol decoders have to output annotations, though (this is optional). There are also various PDs that are used for several other purposes (not for generating output to be displayed in frontends), and they thus don't output annotations at all.

Binary output

Binary output (the OUTPUT_BINARY output type) is not meant for direct displaying to the user by a frontend. Instead, it is some kind of representation of decoded data, in some specific format, that can be saved to file and/or loaded by (or piped into) other applications and/or be further processed via other tools.

Every protocol decoder can output an arbitrary number of different "binary" output formats. Not all protocol decoders have to have binary output, though (this is optional).

Use cases

There are many, many possible use-cases for the binary output feature of protocol decoders. This is just a very short collection of random things that can be done with this feature. Users can easily add support for their own special-purpose output formats that are useful for their specific task at hand.

Memory

  • The decode of some SD card traffic could yield a dump of the contents of an SD card in various formats, e.g. individual files, or a "raw dump" containing the full filesystems (ext3, FAT, or any other) that could be written on another SD card via dd, or mounted locally via e.g. mount -t vfat -o loop,ro,noexec img.dd /mnt, and so on.
  • The decode of the LPC traffic when a PC mainboard reads the contents of the (LPC) BIOS chip, could yield a backup file of the BIOS chip's contents. This can be used simply for backup purposes, or used by tools like flashrom or the mainboard vendor's flash tools, and so on.
  • The decode of some microcontroller reading the contents of an I2C/Microwire/1-Wire/SPI/other EEPROM could yield a binary file containing the raw EEPROM contents. This can be further processed by other tools, e.g. hexdump, perl, or others.
  • The decode of the communication of something reading the EEPROM of an FTDI chip could yield a configuration file for the ftdi_eeprom tool (or similar vendor tools), e.g. to be used to configure another FTDI device to exactly the same settings as the one in the logic analyzer capture.
  • The decode of the communication when a PC reads the Serial Presence Detect (SPD) data from an EEPROM on an SDRAM module could yield a file in the format that tools such as decode-dimm (part of i2c-tools) can use.
  • The decode of some game catridge protocol of a gaming console or handeld device could be used to yield files suitable for use in one of the many arcade game emulators such as MAME.

Audio

  • The decode of an I2S audio stream could yield a WAV file (and/or any other audio file format). That can then be played via an audio player (aplay, mplayer, etc), viewed in an audio editor (e.g. audacity), and so on.
  • The decode of some MIDI data could yield the music being played on the respective instrument in the form of a MIDI file. And/or a list of modes the instrument has been put into or the configuration that has been performed on some other MIDI-capable gear, in a format suitable for some MIDI PC software.
  • The decode of the communication seen on some Bluetooth IC of and audio headset could yield a WAV file of the spoken words transmitted over the Bluetooth link. And/or it could (with some speech recognition magic) yield a text transcript of the audio.
  • The decode of some DECT communication on some of the involved ICs could yield a WAV file of the audio during a phone call.
  • The decode of a digital FM radio chip (e.g. the TEA5767 or others) could yield a WAV file of the radio program being received by the IC.

Graphics and video

  • The decode of the traffic of some microcontroller talking to some LCD (or 7-segment display, or OLED, or any other type of display) could yield a PNG (or BMP or JPG or other file format) of the contents being shown on that display. If multiple images are contained in the decoded data, it could also yield a video (in various formats) of the data being displayed. If the logic analyzer data also contains an audio stream (that the device with the display is playing while it shows stuff on the LCD), that video could even have audio as well.
  • The decode of data captured from a PC querying the attached monitor for EDID data could yield a file suitable for use with tools such as read-edid, Powerstrip, xrandr and others.

Firmware files

  • A decode of the Atmel AVR ISP communication (for flashing AVR 8-bit microcontrollers) could yield an Intel Hex (or Motorola S-record, or some other) file containing the microcontroller firmware that was flashed. This can then be used by various tools, e.g. to flash it into another microcontroller, to convert it into another format and so on. It could also yield a file that contains the fuse settings or other config that was performed on the microcontroller, e.g. to be used by the flashing software avrdude or other tools.
  • The decode of some firmware/bootloader/kernel being flashed onto an embedded device, e.g. via XMODEM or some other protocol, could yield a file containing that firmware/bootloader/kernel binary. This could be used for flashing it onto other devices, or for usage/analysis with other tools, e.g. u-boot-tools, xburst-tools, and others.

USB

  • The decode of some USB traffic could yield a pcap file for use with Wireshark. And/or it could yield a file in the format expected by the software of other USB analyzers such as the TotalPhase Beagle.

CAN

Bluetooth

  • The decode of some OBEX protocol going via some Bluetooth IC in a phone could yield a calendar entry file (e.g. iCal or xCal) suitable for import in various calendar software such as KOrganizer, Thunderbird, iCal, Ximian, and others. And/or it could yield a file with a person's contact details in some format that various email clients can import.

Networking

  • The decode of some network related communication to or from Ethernet-related chips could yield a pcap file for use with Wireshark. And/or other files for use with various other network related tools.

Keyboard and mouse

  • The decode of some PS/2 (or USB) keyboard/mouse traffic could yield an ASCII text file of all the keys that were pressed on the keyboard. And/or it could yield a video (or list file in some format) of the mouse movements that were made during the time the data was capture with a logic analyzer.

Positioning

  • The decode of some NMEA 0183 communication of a GPS device could yield a GPX file suitable for use with various geographical tools such as gpsbabel, gpsdrive and many other tools. And/or it could yield a KML file, for use with Google Earth, Marble and many other tools.

Medical

  • The decode of the digital output (e.g. SPI, or something else) of some ADC used in an ECG device could yield a file in the EDF format for use by medical visualization and analysis tools such as edfbrowser, Biosignal Tools, and many other tools.
  • The decode of Bluetooth low energy communication of some medical device (ECG, blood pressure meter, thermometer, or anything else) could yield a file for use with various medical analysis tools.

SDR

  • The decode of the digital output of some chips for wireless communications of various forms could yield a file suitable for further use/processing in SDR software like GNU Radio, rtl-sdr based tools (e.g. dump1090, rtl_433, FS20_decode) and various other SDR related tools.

Visualization

  • The decode of some temperature sensor(s) such as an LM75 or 1-Wire based DS18B20 could yield a gnuplot file that visualizes the temperature(s) over time. And/or a CSV file for further processing with other tools. And/or a WAV file for displaying in audacity. Or various other formats for use with additional visualization tools.
  • The decode of the data from various sensors could yield an RRD file for use with rrdtool, (k)collectd, ddraw and similar tools.

Other

  • The data decoded from a DCF77 clock could yield a simple file/string that contains the date and time as received via DCF77 in a format suitable for use in e.g. the date or hwclock UNIX command (for example to set a PC's date/time). And/or it could be some format suitable for use with NTP or random other tools that deal with date/time.
  • The decode of some IR protocol (e.g. for various remote controls) could yield files suitable for use with IR software such as LIRC, IRMP, and others.
  • The decode of some snippet of data a DAC outputs could yield a config file suitable to be imported into a function generator such as a the Siglent SDG1010, so that the function generated can output the exact same signal that was captured.

Reports

Reports are certain pieces of additional information a protocol decoder can output, for a frontend to display to a user.

In contrast to annotations, this information is not specific to a certain subset of samples. Instead, it is information that applies to the full sample set (the full input that is decoded). Reports are only generated when the decoding run is finished.

Every protocol decoder can output an arbitrary number of different types of reports (each of them applies to the whole dataset). Not all protocol decoders have to have report output, though (this is optional).

Use cases

  • A can (or usb_packet or ...) decoder could report the number (and/or percentage) of packets that have an incorrect/invalid CRC (and/or are otherwise invalid/incomplete packets).
  • A uart decoder could report the number (and/or percentage) of frames with parity errors (and/or frame errors).
  • A $variousprotocols decoder could report the number (and/or percentage) of collisions on the bus.
  • A can decoder could report the percentage of the CAN bus capacity/utilization (the more devices/frames are on the bus, the higher this will usually be).
  • A $someLCD decoder could report the update rate of the display in certain cases (i.e. how often per second a new image is displayed).
  • A mlx90614 (or lm75 or maxim_ds18b20) temperature sensor decoder could report the min/max/avg/mean of the temperature values seen.
  • A ir_nec (IR remote control protocol) decoder could report the number of buttons pressed on the remote. And/or the number of different (unique) buttons pressed. And/or the mix/max/avg/mean duration between two button presses.
  • A usb decoder could report the percentage (max/min/avg/mean) of the bus capacity/utilization, i.e., how much of a (micro)frame is still free to be used for additional USB packets.
  • A onewire (or i2c, or can) decoder could report the number of different slaves seen on the bus/network. This could be handy e.g. when analyzing a sensor network with many 1-Wire sensors (or other devices) that are all on the same bus. And/or it could report a list of slave IDs/addresses. And/or a list of slave types (e.g. "EERPOM", "Temperature sensor", "RTC", and so on), where possible.
  • A midi decoder could report the total number of notes/keys being pressed (and/or released). And/or a list of vendor names for the instruments seen.
  • A guess_bitrate decoder could report the bitrate or baudrate that has been detected/guessed. And/or a "confidence" number which specifies how likely the PD thinks it is that the reported value will be correct.
  • A guess_dmm_chip decoder could guess the type of multimeter IC used, based on the data seen. While this will not be 100% reliable in all cases (and is completely impossible in some other cases), it could still be useful for the cases where it does work. E.g. the decoder could be able to determine whether a DMM uses a Cyrustek ES519xx or Fortune Semiconductor FS9922* or Metex 14-byte ASCII protocol chip, and report the name of the guessed chip (as string).
  • A gps decoder could report the distance traveled by the person/car carrying the GPS receiver (by comparing the first reported position to the last one). And/or it could report the duration of the travel. And/or it could report a list of cities that the person was driving by etc. etc.
  • The avago_adns mouse sensor decoder could report whether or not the left mouse button was clicked. And/or the right mouse button. And/or whether the mouse was moved or not.
  • An ethernet decoder could report all MAC addresses seen in the traffic (list of strings). And/or the packet loss percentage. And/or the number of network clients seen.
  • An FM radio chip with RDS support decoder could report a list of song titles, radio program titles (or similar things) seen. And/or the number of different songs.
  • A jtag decoder could report the IDs/names of all the chips (CPUs/microcontrollers, NAND flash chips, others) in the JTAG chain (as observed in the data), e.g. ["ST STM32F1...", "Atmel AT91...", "Numonyx ..."] and so on.

Note: This list is not exhaustive, and it is not yet finished.

Specific PD examples

UART

The following reports (and possibly more) might potentially be defined for the UART protocol decoder:

Report name Report description Type of output
Number of frames Total number of frames seen in the traffic Integer number
Frame errors Number/percentage of frame errors encountered Integer number, float number
Parity errors Number/percentage of parity errors encountered Integer number, float number
Baudrate error Percentage of the deviation of the actual baudrate from the nominal baudrate List of float numbers (min/max/avg/mean)
Baudrate jitter Percentage of jitter in the baudrate deviation from the nominal baudrate List of float numbers (min/max/avg/mean)
Interbyte delay The delay/time between two consecutive bytes in nanoseconds List of integer numbers (min/max/avg/mean)
Protocol overhead Percentage of protocol overhead compared to payload data Float number

Example reports: 

Number of frames: 2997
Frame errors: [0, 0%]
Parity errors: [7, 0.23%]
Baudrate error: [0.3%, 0.4%, 1.7%, 1.6%]
Baudrate jitter: [1.0%, 3.0%, 1.3%, 2.9%]
Interbyte delay: [300ns, 1299ns, 450ns, 440ns]
Protocol overhead: 12%

MIDI PD example

The following reports (and possibly more) might potentially be defined for the MIDI protocol decoder:

Report name Report description Type of output
Number of notes Total number of "Note on" messages seen on the bus Integer number
Vendor list List of MIDI instrument/gear vendors seen on the bus List of strings
Number of channel messages Number of MIDI messages of the "channel message" category seen Integer number
Number of SysEx messages Number of MIDI messages of the "system exclusive message" category seen Integer number

Example reports: 

Number of notes: 408
Vendor list: ["Rogers Instrument Corp.", "Ad Lib, Inc.", "ESS Technology", "Yamaha"]
Number of channel messages: 395
Number of SysEx messages: 13

Ethernet PD example

The following reports (and possibly more) might potentially be defined for the Ethernet protocol decoder:

Report name Report description Type of output
Number of frames Total number of Ethernet frames seen Integer number
MAC address list List of unique MAC addresses seen List of strings
Payload length The length of the Ethernet payload field (in bytes) List of integer numbers (min/max/avg/mean)
FCS errors Number + percentage of frames which have an incorrect Frame Check Sequence field. Integer number, list of float numbers (min/max/avg/mean)
Total number of payload bytes Total number of data transferred over the link in any direction, from any source to any other source (in bytes) Integer number
Packet loss Percentage of the packet loss encountered. List of float numbers (min/max/avg/mean)

Example reports: 

Number of frames: 976622
MAC address list: ["68:13:c4:44:70:10", "11:22:33:44:55:66", "aa:bb:cc:dd:ee:ff"]
Payload length: [5, 1500, 470, 400]
FCS errors: 126, [0.2%, 4.7%, 1.3%, 1.2%]
Total number of payload bytes: 89324737
Packet loss: [0.2%, 4.7%, 1.3%, 1.2%]

Temperature sensor network example

The following reports (and possibly more) might potentially be defined for the $sometemperaturesensor protocol decoder (where there can be many such sensors on the same bus):

Report name Report description Type of output
Number of sensors Total number of sensors/slaves seen Integer number
Sensor ID/address list List of unique sensor slave IDs/addresses seen List of strings (or integers, depending on protocol)
Latency Time between a temperature request from the host and the reply of the respective sensor (in nanoseconds) List of integers (min/max/avg/mean)
Temperature Temperature reported by any of the sensors (in degrees Celsius) List of float numbers (min/max/avg/mean)

Example reports: 

Number of sensors: 5
Sensor ID/address list: [0x50, 0x52, 0x8f, 0x9a, 0x33] (depends on protocol used, e.g. 1-Wire, I2C, or various others)
Latency: [1840, 2400, 2030, 2028]
Temperature: [21.5, 28.7, 26.1, 25.9]

Common patterns for types of things a report might output

  • Integer numbers
    • Absolute number/count of certain packets/things/events
    • Relative number/count of certain packets/things/events
      • Percentage of certain packets/things/events (e.g. packet with incorrect CRC, packets with various other properties)
      • Min of certain packets/things/events
      • Max of certain packets/things/events
      • Average of certain packets/things/events
      • Mean of certain packets/things/events
  • Booleans
    • Flags/notifiers of whether or not certain packets/things/events have been observed in the data.
  • Strings
    • Certain things that have been seen in the data that are strings (vendor names, device names, many other things).
  • Lists of either of the above types
    • List of MAC addresses (list of strings, or list of list of integers where each MAC address is a list of 6 bytes)
    • List of vendor names / device names (list of strings)
    • List of slave IDs/addresses/names (list of integers or list of strings)
    • List of yes/no items to be reported (list of boolean values)
  • Possibly more complex types (nested lists, struct-like items)
  • ...

Note: This list is not exhaustive, and it is not yet finished.

Proto output

Proto output (the OUTPUT_PROTO output type) is used by PDs to pass data on to other (stacked) PDs. It is specifically not used in any way, shape, or form by frontends (only by other PDs). It is not passed to a frontend at all.

On the implementation level libsigrokdecode passes arbitrary Python objects from one (Python) decoder to the next (stacked one). The contents of those objects are entirely up to the lower-level PD, and every PD uses a different, suitable format that is useful for the specific protocol at hand, and easy to use by stacked decoders. There is no standard format for this output (and there cannot be any). Every PD that provides OUTPUT_PROTO documents the format of its output, so that authors of stacked PDs know how to use it.

Every protocol decoder can output OUTPUT_PROTO data for other PDs. Not all protocol decoders have to have OUTPUT_PROTO output, though (this is optional). There are protocol decoders that, for example, only output annotations, or only output binary formats, and don't (yet) have any OUTPUT_PROTO data to be used by stacked decoders. Decoders without OUTPUT_PROTO output cannot be used by any stacked PDs, of course.

Retrieved from "https://sigrok.org/w/index.php?title=Protocol_decoder_output&oldid=7620"