Overview of Available Suites
D-Cube currently supports three benchmark suites. Each suite is specified by: (i) a given hardware platform, (ii) a given application scenario, and (iii) a set of performance metrics. We describe next the benchmark suites on a high level:
- SkyDC_1 (Tmote Sky Data Collection v1)
This benchmark resembles the first category of the EWSN 2019 dependability competition.- Hardware platform: Tmote Sky.
- Application scenario: In this benchmark, a fixed number of source nodes (at most eight) communicate to a single destination node over a multi-hop network. This benchmark hence focuses on multipoint-to-point traffic. In particular, each destination node collects sensor data of different length transmitted by the different source nodes and allows out-of-order delivery.
- Performance metrics: The measured performance metrics are: (i) the reliability of transmissions, i.e., the number of messages correctly reported to each intended destination, (ii) the average end-to-end latency in communicating each message to its intended destination(s), and (iii) the average energy consumption on all nodes in the network. Note that, on each run, during the first 60 seconds no data is generated and the energy consumption is not measured, so to allow the firmware under test to bootstrap the network.
- SkyDD_1 (Tmote Sky Data Dissemination v1)
This benchmark resembles the second category of the EWSN 2019 dependability competition.- Hardware platform: Tmote Sky.
- Application scenario: In this benchmark, a fixed number of source nodes needs to disseminate actuation commands of different length to a specific set of destination nodes over a multi-hop network. This benchmark hence focuses on point-to-multipoint traffic. In particular, each source node is associated with a specific set of destinations (at most eight), which will be injected as an input parameter into the firmware under test. A destination can receive messages from only a single source node, cannot act as a source at the same time, and allows out-of-order delivery.
- Performance metrics: The measured performance metrics are: (i) the reliability of transmissions, i.e., the number of messages correctly reported to each intended destination, (ii) the average end-to-end latency in communicating each message to its intended destination(s), and (iii) the average energy consumption on all nodes in the network. Note that, on each run, during the first 60 seconds no data is generated and the energy consumption is not measured, so to allow the firmware under test to bootstrap the network.
- nRFDC_1 (nRF52840 Timely Data Collection v1) This benchmark runs on the nRF52840 platform and resembles a data collection with bounded delays within a multi-hop network. This benchmark hence focuses on multipoint-to-point traffic. In particular, in this benchmark, up to 48 source nodes generate raw sensor values of different length, which should be communicated to the same destination. The latter may be located several hops away from a given source node, even when making use of the coded PHY layers available on the nRF52. The messages containing the raw sensor values should be forwarded to the intended destination as efficiently as possible within a maximum per-message delay bound ∂, i.e., the end-to-end delay of every message from its generation to its reception at the sink should be lower than ∂ (which is fixed and applies to all messages exchanged during a test run). If a message has been received with an end-to-end delay greater than ∂, it is considered to be lost. The measured performance metrics are: (i) the reliability of transmissions, i.e., the number of messages correctly reported to each intended destination, and (ii) the average energy consumption on all nodes in the network. Note that, on each run, during the first 60 seconds no data is generated and the energy consumption is not measured, so to allow the firmware under test to boostrap the network. The results of this benchmark are available here.