ExPECA: An Experimental Platform for Trustworthy: Experimental Testbed Validation

Table of Links

• Abstract & Introduction
• Testbed Design and Architecture
• Experimental Testbed Validation
• Supported Experimentation
• Acknowledgements & References

III. EXPERIMENTAL TESTBED VALIDATION

This section aims to substantiate two key assertions about ExPECA: its efficiency in facilitating end-to-end experiments and its ability to ensure experimental reproducibility[1].

To validate the first claim, we employ OpenRTiST [7], an open-source application that is computationally demanding, bandwidth-intensive, and latency-sensitive. OpenRTiST uses machine learning filtering to overlay various artistic styles onto video frames, creating an augmented reality experience. In this experiment, the OpenRTiST client connects to the server either at the edge, specifically the testbed servers, or in the cloud via a server located at Carnegie Mellon University (CMU), in Pittsburgh, Pennsylvania, through a COTS 5G system and the adv-03 node. In Figure 4, we depict the corresponding critical performance indicators such as frames per second (FPS) and round-trip time (RTT) measured using the testbed. In Figure 4a, we plot the Cumulative Distribution Function (CDF) of FPS, and in Figure 4a we plot the Complementary CDF (CCDF) of RTT. As we move from edge to cloud, RTT significantly increases which results in lower refresh rates and less responsive video stream. The performance is benchmarked against different video frame sizes to analyze network and compute capacity.

For the second claim, we concentrate on wireless network latency, a metric inherently challenging to model due to its stochastic nature at the packet level. While latency can be influenced by factors like transmit power and channel interference, it also depends on wireless protocol settings such as modulation schemes, bandwidth, and antenna configurations. ExPECA’s isolated location minimizes variability in channel quality, mitigating potential interference. In Figure 5, we present latency measurements across different radio links over several days, demonstrating the reproducibility level in the case of COTS 5G wireless links. The close similarity between these CCDFs confirms the reproducibility of wireless latency measurements achieved through ExPECA.

[1] Experiments available as Python Jupyter notebooks at https://github.com/KTH-EXPECA/examples