scpi-dmm: prefer double over float data types more often

The 'float' data type uses 23 bits for the mantissa, which can express
some 6.9 decimal places. Some meters claim "6 digits" when in fact they
are 6.5 or so. Averaging over higher numbers of power line cycles (NPLC)
could additionally raise a meter's precision. Prefer double over float
for 6-digit devices already, just to err on the safe side.

The previous implementation only started using the double data type for
devices with 7 and more digits, and left a theoretical gap that's fully
closed now.

diff --git a/src/hardware/scpi-dmm/protocol.c b/src/hardware/scpi-dmm/protocol.c
index f413fcd730fbcaad79d58be38fbc2de3ed74d4fd..a13db8415162e6f4ea1b89314997edd28a46eedb 100644 (file)
--- a/src/hardware/scpi-dmm/protocol.c
+++ b/src/hardware/scpi-dmm/protocol.c
@@ -396,7 +396,7 @@ SR_PRIV int scpi_dmm_get_meas_agilent(const struct sr_dev_inst *sdi, size_t ch)
        if (ret != SR_OK)
                return ret;
        g_strstrip(response);
-       use_double = devc->model->digits > 6;
+       use_double = devc->model->digits >= 6;
        ret = sr_atod_ascii(response, &info->d_value);
        if (ret != SR_OK) {
                g_free(response);