Skip to content

T
tsn-measures
Commit 6e3e5c63 authored by Joanne Hugé's avatar Joanne Hugé
Browse files
Options

Add ignored metadata array and ignore board names

parent e84136dc
Show whitespace changes
Inline Side-by-side
Showing with 49 additions and 46 deletions
measure-analysis/measure-analysis.py
View file @ 6e3e5c63
...@@ -30,6 +30,7 @@ class MeasureSetHandler: ...@@ -30,6 +30,7 @@ class MeasureSetHandler:
graphs_dir = "{}/graphs".format(measures_dir) graphs_dir = "{}/graphs".format(measures_dir)
measure_sets_file_name = "measure_sets.json" measure_sets_file_name = "measure_sets.json"
measure_sets_path = measures_dir + "/" + measure_sets_file_name measure_sets_path = measures_dir + "/" + measure_sets_file_name
ignored_metadata = ["board"]
def __init__(self): def __init__(self):
with open(MeasureSetHandler.measure_sets_path) as measure_sets_file: with open(MeasureSetHandler.measure_sets_path) as measure_sets_file:
...@@ -233,6 +234,7 @@ class MeasureSetHandler: ...@@ -233,6 +234,7 @@ class MeasureSetHandler:
# Generate the metadata mask, by grouping the identical metadata # Generate the metadata mask, by grouping the identical metadata
for measure in measures[1:]: for measure in measures[1:]:
for metadata_name in measure.metadata: for metadata_name in measure.metadata:
if metadata_name not in self.ignored_metadata:
# If it is not already in the metadata mask # If it is not already in the metadata mask
if metadata_name not in metadata_mask: if metadata_name not in metadata_mask:
# If there are two different metadata, they are added to the mask # If there are two different metadata, they are added to the mask
...@@ -244,6 +246,7 @@ class MeasureSetHandler: ...@@ -244,6 +246,7 @@ class MeasureSetHandler:
# Compute the identical metadata # Compute the identical metadata
common_metadata = [] common_metadata = []
for metadata_name in first_metadata: for metadata_name in first_metadata:
if metadata_name not in self.ignored_metadata:
if metadata_name not in metadata_mask: if metadata_name not in metadata_mask:
if metadata_name in MeasureSet.abbreviations: if metadata_name in MeasureSet.abbreviations:
common_metadata.append("{}: {}".format(MeasureSet.abbreviations[metadata_name], first_metadata[metadata_name])) common_metadata.append("{}: {}".format(MeasureSet.abbreviations[metadata_name], first_metadata[metadata_name]))
...@@ -267,6 +270,7 @@ class MeasureSetHandler: ...@@ -267,6 +270,7 @@ class MeasureSetHandler:
report.write("### Description of the metadata associated to each test\n\n") report.write("### Description of the metadata associated to each test\n\n")
for abbr_name in MeasureSet.metadata_descriptions: for abbr_name in MeasureSet.metadata_descriptions:
if abbr_name not in self.ignored_metadata:
report.write("* **{}**: {}\n".format(abbr_name, MeasureSet.metadata_descriptions[abbr_name])) report.write("* **{}**: {}\n".format(abbr_name, MeasureSet.metadata_descriptions[abbr_name]))
report.write("\n") report.write("\n")
......
measure-analysis/measure-report.md
View file @ 6e3e5c63
...@@ -12,7 +12,6 @@ Measures measuring the same propriety are grouped together in tables and graphs, ...@@ -12,7 +12,6 @@ Measures measuring the same propriety are grouped together in tables and graphs,
* **ker**: Linux kernel version used * **ker**: Linux kernel version used
* **prio**: Priority of the real-time thread * **prio**: Priority of the real-time thread
* **i**: Interval between each packet transmission * **i**: Interval between each packet transmission
* **board**: Name of the board used, one of three : Emerald, Slate or Onyx. (All boards are identical)
* **boot_p**: Boot Parameters passed to u-boot * **boot_p**: Boot Parameters passed to u-boot
* **delta**: ETF qdisc delta value * **delta**: ETF qdisc delta value
* **load**: External device and processor load during the test * **load**: External device and processor load during the test
...@@ -27,12 +26,12 @@ Uses cyclictest from the rt-tests test suite to measure wake-up latency. A real- ...@@ -27,12 +26,12 @@ Uses cyclictest from the rt-tests test suite to measure wake-up latency. A real-
**Common test metadata:** Linux kernel version: 4.19, Boot Parameters: isolcpus, Interval: 200us, ETF qdisc delta: 200us, Task priority: 99, qdisc: pfifo_fast, etf_offset: none, route: switch **Common test metadata:** Linux kernel version: 4.19, Boot Parameters: isolcpus, Interval: 200us, ETF qdisc delta: 200us, Task priority: 99, qdisc: pfifo_fast, etf_offset: none, route: switch
Metadata | Minimum | Maximum | Average | Standard deviation Metadata | Minimum | Maximum | Average | Standard deviation
----------------------------------- | ------------------- | ------------------- | ------------------- | ------------------- ---------------------------- | ------------------- | ------------------- | ------------------- | -------------------
**board, duration, load** | **wake-up latency** | **wake-up latency** | **wake-up latency** | **wake-up latency** **duration, load** | **wake-up latency** | **wake-up latency** | **wake-up latency** | **wake-up latency**
Slate, 21h22, hackbench (384342932) | 14.0000us | 65.0000us | 21.2779us | 2.1746us 21h22, hackbench (384342932) | 14.0000us | 65.0000us | 21.2779us | 2.1746us
Emerald, 21h22, hackbench (384614861) | 14.0000us | 112.0000us | 19.5352us | 2.0136us 21h22, hackbench (384614861) | 14.0000us | 112.0000us | 19.5352us | 2.0136us
Onyx, 46h, hackbench (1269804578) | 14.0000us | 72.0000us | 20.5580us | 1.9820us 46h, hackbench (1269804578) | 14.0000us | 72.0000us | 20.5580us | 1.9820us
Emerald, 24h, ssh (303042302) | 12.0000us | 80.0000us | 15.4101us | 1.1091us 24h, ssh (303042302) | 12.0000us | 80.0000us | 15.4101us | 1.1091us
![alt text](measures/graphs/cyclictest_wake-up_latencywake-up latency.png "cyclictest_wake-up_latency Graph") ![alt text](measures/graphs/cyclictest_wake-up_latencywake-up latency.png "cyclictest_wake-up_latency Graph")
...@@ -43,12 +42,12 @@ Similar to packet_recv_timestamps, but on the transmitting board. ...@@ -43,12 +42,12 @@ Similar to packet_recv_timestamps, but on the transmitting board.
**Common test metadata:** Linux kernel version: 4.19, Boot Parameters: isolcpus, Task priority: 99, Device and processor load: ssh, etf_offset: none, route: switch **Common test metadata:** Linux kernel version: 4.19, Boot Parameters: isolcpus, Task priority: 99, Device and processor load: ssh, etf_offset: none, route: switch
Metadata | Minimum | Maximum | Average | Standard deviation Metadata | Minimum | Maximum | Average | Standard deviation
------------------------------------------------------ | ----------------------------- | ----------------------------- | ----------------------------- | ----------------------------- ------------------------------------------------ | ----------------------------- | ----------------------------- | ----------------------------- | -----------------------------
**board, i, duration, delta, qdisc** | **user_space - kernel_space** | **user_space - kernel_space** | **user_space - kernel_space** | **user_space - kernel_space** **i, duration, delta, qdisc** | **user_space - kernel_space** | **user_space - kernel_space** | **user_space - kernel_space** | **user_space - kernel_space**
Onyx, 100000us, 0h17, 200us, pfifo_fast (10459, 10458) | 2.0000us - 42.0000us | 12.0000us - 145.0000us | 2.3099us - 51.8681us | 0.5212us - 7.5021us 100000us, 0h17, 200us, pfifo_fast (10459, 10458) | 2.0000us - 42.0000us | 12.0000us - 145.0000us | 2.3099us - 51.8681us | 0.5212us - 7.5021us
Slate, 1000us, 1h00, 200us, pfifo_fast (5069781, 5069781) | 1.0000us - 32.0000us | 12.0000us - 222.0000us | 2.0360us - 36.6218us | 0.2221us - 3.0259us 1000us, 1h00, 200us, pfifo_fast (5069781, 5069781) | 1.0000us - 32.0000us | 12.0000us - 222.0000us | 2.0360us - 36.6218us | 0.2221us - 3.0259us
Slate, 1000us, 0h2, 200us, pfifo_fast (111540, 111540) | 1.0000us - 32.0000us | 12.0000us - 118.0000us | 1.9341us - 36.6011us | 0.2771us - 1.9293us 1000us, 0h2, 200us, pfifo_fast (111540, 111540) | 1.0000us - 32.0000us | 12.0000us - 118.0000us | 1.9341us - 36.6011us | 0.2771us - 1.9293us
Onyx, 1000us, 72h24, 160us, etf (1311, 1311) | 1.0000us - 33.0000us | 65.0000us - 110.0000us | 1.9916us - 37.1342us | 1.7622us - 3.6937us 1000us, 72h24, 160us, etf (1311, 1311) | 1.0000us - 33.0000us | 65.0000us - 110.0000us | 1.9916us - 37.1342us | 1.7622us - 3.6937us
![alt text](measures/graphs/packet_send_timestampsuser_space.png "packet_send_timestamps Graph") ![alt text](measures/graphs/packet_send_timestampsuser_space.png "packet_send_timestamps Graph")
...@@ -61,11 +60,11 @@ An UDP packet is periodically sent from one board to another using a real time t ...@@ -61,11 +60,11 @@ An UDP packet is periodically sent from one board to another using a real time t
**Common test metadata:** Linux kernel version: 4.19, Boot Parameters: isolcpus, Task priority: 99, Device and processor load: ssh, etf_offset: none, route: switch **Common test metadata:** Linux kernel version: 4.19, Boot Parameters: isolcpus, Task priority: 99, Device and processor load: ssh, etf_offset: none, route: switch
Metadata | Minimum | Maximum | Average | Standard deviation Metadata | Minimum | Maximum | Average | Standard deviation
------------------------------------------------------- | ----------------------------- | ----------------------------- | ----------------------------- | ----------------------------- ------------------------------------------------ | ----------------------------- | ----------------------------- | ----------------------------- | -----------------------------
**i, duration, board, delta, qdisc** | **user_space - kernel_space** | **user_space - kernel_space** | **user_space - kernel_space** | **user_space - kernel_space** **i, duration, delta, qdisc** | **user_space - kernel_space** | **user_space - kernel_space** | **user_space - kernel_space** | **user_space - kernel_space**
100000us, 0h17, Slate, 200us, pfifo_fast (10419, 10418) | 0.0000us - 44.0000us | 26.0000us - 142.0000us | 0.1151us - 55.4212us | 0.4057us - 7.6346us 100000us, 0h17, 200us, pfifo_fast (10419, 10418) | 0.0000us - 44.0000us | 26.0000us - 142.0000us | 0.1151us - 55.4212us | 0.4057us - 7.6346us
1000us, 16h48, Slate, 200us, pfifo_fast (2406205, 2406204) | 0.0000us - 35.0000us | 41.0000us - 233.0000us | 0.0036us - 41.5560us | 0.2282us - 5.1991us 1000us, 16h48, 200us, pfifo_fast (2406205, 2406204) | 0.0000us - 35.0000us | 41.0000us - 233.0000us | 0.0036us - 41.5560us | 0.2282us - 5.1991us
1000us, 72h24, Emerald, 160us, etf (2606916, 2606915) | 0.0000us - 41.0000us | 42.0000us - 291.0000us | 0.0235us - 50.7618us | 0.3294us - 7.4393us 1000us, 72h24, 160us, etf (2606916, 2606915) | 0.0000us - 41.0000us | 42.0000us - 291.0000us | 0.0235us - 50.7618us | 0.3294us - 7.4393us
![alt text](measures/graphs/packet_recv_timestampsuser_space.png "packet_recv_timestamps Graph") ![alt text](measures/graphs/packet_recv_timestampsuser_space.png "packet_recv_timestamps Graph")
...@@ -78,17 +77,17 @@ An UDP packet is periodically sent from one board to another using a real time t ...@@ -78,17 +77,17 @@ An UDP packet is periodically sent from one board to another using a real time t
**Common test metadata:** Linux kernel version: 4.19, Boot Parameters: isolcpus, Task priority: 99, Device and processor load: ssh **Common test metadata:** Linux kernel version: 4.19, Boot Parameters: isolcpus, Task priority: 99, Device and processor load: ssh
Metadata | Minimum | Maximum | Average | Standard deviation Metadata | Minimum | Maximum | Average | Standard deviation
------------------------------------------------------------- | ----------- | ---------- | ---------- | ------------------ ------------------------------------------------------ | ----------- | ---------- | ---------- | ------------------
**i, duration, board, delta, qdisc, etf_offset, route** | **jitter** | **jitter** | **jitter** | **jitter** **i, duration, delta, qdisc, etf_offset, route** | **jitter** | **jitter** | **jitter** | **jitter**
100000us, 0h17, Slate, none, pfifo_fast, none, switch (10417) | -139.0000us | 147.0000us | 10.4314us | 15.8981us 100000us, 0h17, none, pfifo_fast, none, switch (10417) | -139.0000us | 147.0000us | 10.4314us | 15.8981us
1000us, 16h48, Slate, none, pfifo_fast, none, switch (2406203) | -434.0000us | 237.0000us | 6.9930us | 9.0448us 1000us, 16h48, none, pfifo_fast, none, switch (2406203) | -434.0000us | 237.0000us | 6.9930us | 9.0448us
1000us, 72h24, Emerald, 160us, etf, 1000000, switch (2606914) | -397.0000us | 369.0000us | 7.4514us | 12.4505us 1000us, 72h24, 160us, etf, 1000000, switch (2606914) | -397.0000us | 369.0000us | 7.4514us | 12.4505us
1000us, 0h15, Onyx, 150us, etf, 400, switch (953805) | -535.0000us | 125.0000us | 10.5255us | 12.8854us 1000us, 0h15, 150us, etf, 400, switch (953805) | -535.0000us | 125.0000us | 10.5255us | 12.8854us
1000us, 0h12, Emerald, 150us, etf, 400, E2E (743922) | -692.0000us | 129.0000us | 9.7386us | 12.4607us 1000us, 0h12, 150us, etf, 400, E2E (743922) | -692.0000us | 129.0000us | 9.7386us | 12.4607us
1000us, 0h11, Onyx, none, pfifo_fast, none, E2E (711176) | -327.0000us | 121.0000us | 13.9827us | 19.7918us 1000us, 0h11, none, pfifo_fast, none, E2E (711176) | -327.0000us | 121.0000us | 13.9827us | 19.7918us
1000us, 0h39, Onyx, none, pfifo_fast, none, switch (2367379) | -167.0000us | 212.0000us | 6.5787us | 9.1972us 1000us, 0h39, none, pfifo_fast, none, switch (2367379) | -167.0000us | 212.0000us | 6.5787us | 9.1972us
1000us, 0h50, Onyx, 160us, etf, 400, switch (3027426) | -151.0000us | 126.0000us | 9.0267us | 11.8804us 1000us, 0h50, 160us, etf, 400, switch (3027426) | -151.0000us | 126.0000us | 9.0267us | 11.8804us
1000us, 0h32, Onyx, 160us, etf, 400, E2E (1977063) | -593.0000us | 157.0000us | 9.7129us | 11.8025us 1000us, 0h32, 160us, etf, 400, E2E (1977063) | -593.0000us | 157.0000us | 9.7129us | 11.8025us
![alt text](measures/graphs/packet_jitterjitter.png "packet_jitter Graph") ![alt text](measures/graphs/packet_jitterjitter.png "packet_jitter Graph")
...@@ -96,7 +95,7 @@ Metadata | Minimum | Ma ...@@ -96,7 +95,7 @@ Metadata | Minimum | Ma
An UDP packet is periodically sent from one board to another and sent back as soon as it arrives using a real time thread. The round trip time is computed with clock_gettime on the transmitting board. An UDP packet is periodically sent from one board to another and sent back as soon as it arrives using a real time thread. The round trip time is computed with clock_gettime on the transmitting board.
**Common test metadata:** Board name: Onyx, Linux kernel version: 4.19, Boot Parameters: isolcpus, Interval: 10000us, ETF qdisc delta: 200us, Task priority: 99, Device and processor load: ssh, Test duration: 0h4, qdisc: pfifo_fast, etf_offset: none, route: switch **Common test metadata:** Linux kernel version: 4.19, Boot Parameters: isolcpus, Interval: 10000us, ETF qdisc delta: 200us, Task priority: 99, Device and processor load: ssh, Test duration: 0h4, qdisc: pfifo_fast, etf_offset: none, route: switch
Minimum | Maximum | Average | Standard deviation Minimum | Maximum | Average | Standard deviation
---------- | ---------- | ---------- | ------------------ ---------- | ---------- | ---------- | ------------------
...@@ -107,7 +106,7 @@ Minimum | Maximum | Average | Standard deviation ...@@ -107,7 +106,7 @@ Minimum | Maximum | Average | Standard deviation
### packet_jitter_tcpdump results ### packet_jitter_tcpdump results
**Common test metadata:** Board name: Onyx, Linux kernel version: 4.19, Boot Parameters: isolcpus, Interval: 1000, Task priority: 99, Device and processor load: ssh **Common test metadata:** Linux kernel version: 4.19, Boot Parameters: isolcpus, Interval: 1000, Task priority: 99, Device and processor load: ssh
Metadata | Minimum | Maximum | Average | Standard deviation Metadata | Minimum | Maximum | Average | Standard deviation
--------------------------------------------- | ---------- | ---------- | ---------- | ------------------ --------------------------------------------- | ---------- | ---------- | ---------- | ------------------
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7