Difference between revisions of "Vision & Roadmap"

 
(14 intermediate revisions by the same user not shown)
Line 2: Line 2:
 
{|
 
{|
 
|D-Cube is one of the outcomes of the [https://www.iotbench.ethz.ch/ IoTBench initiative] gathering academics and industrial practitioners from the low-power wireless networking community towards a better evaluation and comparison of the performance of low-power wireless communication protocols.  
 
|D-Cube is one of the outcomes of the [https://www.iotbench.ethz.ch/ IoTBench initiative] gathering academics and industrial practitioners from the low-power wireless networking community towards a better evaluation and comparison of the performance of low-power wireless communication protocols.  
As an increasing number of IoT systems imposing strict dependability requirements on network performance is being developed and commercialized, the demand for dependable communication protocols delivering information in a reliable, efficient, and timely manner is raising. In response to this need, many low-power wireless protocols have been proposed by both industry and academia over the last decade. However, the lack of a standardized methodology to evaluate protocol performance often leads to a high divergence across experimental setups, which makes it hard, if not impossible, to compare results obtained by different authors. As a consequence, there is an [https://home.deib.polimi.it/mottola/papers/boano18bench.pdf|increasing need to rigorously benchmark low-power wireless systems under the exact same settings].  
+
As an increasing number of IoT systems imposing strict dependability requirements on network performance is being developed and commercialized, the demand for dependable communication protocols delivering information in a reliable, efficient, and timely manner is raising. In response to this need, many low-power wireless protocols have been proposed by both industry and academia over the last decade. However, the lack of a standardized methodology to evaluate protocol performance often leads to a high divergence across experimental setups, which makes it hard, if not impossible, to compare results obtained by different authors. As a consequence, there is an [http://www.carloalbertoboano.com/documents/boano18benchmark.pdf increasing need to rigorously benchmark low-power wireless systems under the exact same settings].  
 
|[[File:empty.png|left|275px]]
 
|[[File:empty.png|left|275px]]
 
|[[File:Logo_iotbench.png|875px|right]]
 
|[[File:Logo_iotbench.png|875px|right]]
 
|}
 
|}
  
To address this issue, we have started in 2016 the [https://iot.ieee.org/newsletter/march-2017/ewsn-dependability-competition-experiences-and-lessons-learned EWSN Dependability Competition Series] as a first attempt to rigorously benchmark the performance of low-power wireless systems in harsh RF environments. To support the competition, we have created D-Cube, a benchmarking infrastructure that allows to accurately measure key dependability metrics such as end-to-end delay, reliability, and power consumption, as well as to graphically visualize their evolution in real-time.  
+
To address this issue, we have started a [https://iot.ieee.org/newsletter/march-2017/ewsn-dependability-competition-experiences-and-lessons-learned Dependability Competition Series] as a first attempt to rigorously benchmark the performance of low-power wireless systems under the same settings in harsh RF environments. To support the competition, we have created D-Cube, a benchmarking infrastructure that allows to accurately measure key dependability metrics such as end-to-end delay, reliability, and power consumption, as well as to graphically visualize their evolution in real-time. The dependability competition has been co-located with the International Conference on Embedded Wireless Systems and Networks in [http://ewsn2016.tugraz.at/cms/index.php%3Fid=49.html 2016], [http://ewsn2017.it.uu.se/dependability-competition1.html 2017], [https://ewsn2018.networks.imdea.org/competition-program.html 2018], and [http://ewsn2019.thss.tsinghua.edu.cn/call-for-competitors.html 2019].
  
Across the years, D-Cube’s hardware, software, and back-end has been [[http://www.carloalbertoboano.com/documents/schuss18benchmark.pdf|upgraded and enriched]] with features enabling an automatic benchmarking of protocol performance in order to support the following editions of the EWSN dependability competition. These features include binary patching to decouple the traffic pattern and node identities from the firmware under test, as well as the ability to generate reproducible Wi-Fi interference [[testbed capabilities|(more info)]].
+
Across the years, D-Cube’s hardware, software, and backend has been [http://www.carloalbertoboano.com/documents/schuss18benchmark.pdf upgraded and enriched] with features enabling a remote evaluation of protocol performance. These features include binary patching to decouple the traffic pattern and node identities from the firmware under test, an improved power and GPIO profiling unit, an automated computation of the performance metrics, as well as the ability to generate reproducible interference using off-the-shelf Wi-Fi devices ([[Special Features|more info]]).
 +
 
 +
<br/>
 +
In November 2019, we have decoupled the availability of D-Cube from the dependability competition, i.e., the testbed facility is available (in a best-effort fashion) to any team that is willing to benchmark the performance of its solutions also beyond the timespan of the EWSN contest. In other words, D-Cube becomes a low-power wireless benchmark open to the community as a common yardstick to compare the performance of various protocols, as well as a way to raise the bar in the quality of networking experiments. We believe that D-Cube can be a useful tool to provide researchers and engineers in both academia and industry with an objective view and quantitative understanding of the strengths and weaknesses among existing protocols. For more information about how to use D-Cube, please refer to [[Testbed Access|this page]].

Latest revision as of 18:35, 25 November 2019


D-Cube is one of the outcomes of the IoTBench initiative gathering academics and industrial practitioners from the low-power wireless networking community towards a better evaluation and comparison of the performance of low-power wireless communication protocols.

As an increasing number of IoT systems imposing strict dependability requirements on network performance is being developed and commercialized, the demand for dependable communication protocols delivering information in a reliable, efficient, and timely manner is raising. In response to this need, many low-power wireless protocols have been proposed by both industry and academia over the last decade. However, the lack of a standardized methodology to evaluate protocol performance often leads to a high divergence across experimental setups, which makes it hard, if not impossible, to compare results obtained by different authors. As a consequence, there is an increasing need to rigorously benchmark low-power wireless systems under the exact same settings.

Empty.png
Logo iotbench.png

To address this issue, we have started a Dependability Competition Series as a first attempt to rigorously benchmark the performance of low-power wireless systems under the same settings in harsh RF environments. To support the competition, we have created D-Cube, a benchmarking infrastructure that allows to accurately measure key dependability metrics such as end-to-end delay, reliability, and power consumption, as well as to graphically visualize their evolution in real-time. The dependability competition has been co-located with the International Conference on Embedded Wireless Systems and Networks in 2016, 2017, 2018, and 2019.

Across the years, D-Cube’s hardware, software, and backend has been upgraded and enriched with features enabling a remote evaluation of protocol performance. These features include binary patching to decouple the traffic pattern and node identities from the firmware under test, an improved power and GPIO profiling unit, an automated computation of the performance metrics, as well as the ability to generate reproducible interference using off-the-shelf Wi-Fi devices (more info).


In November 2019, we have decoupled the availability of D-Cube from the dependability competition, i.e., the testbed facility is available (in a best-effort fashion) to any team that is willing to benchmark the performance of its solutions also beyond the timespan of the EWSN contest. In other words, D-Cube becomes a low-power wireless benchmark open to the community as a common yardstick to compare the performance of various protocols, as well as a way to raise the bar in the quality of networking experiments. We believe that D-Cube can be a useful tool to provide researchers and engineers in both academia and industry with an objective view and quantitative understanding of the strengths and weaknesses among existing protocols. For more information about how to use D-Cube, please refer to this page.