adafruit-mirror/zephyr
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 9

samples: zbus: Add benchmark sample

Browse source 
The sample measures the time to transfer 256KB from the
producer to the consumers.

Signed-off-by: Rodrigo Peixoto <rodrigopex@gmail.com>
This commit is contained in:
Rodrigo Peixoto 2022-09-05 15:02:53 -03:00 committed by Anas Nashif
parent 078d561bf6
commit 27ec69eca1
8 changed files with 521 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
samples/subsys/zbus/benchmark/CMakeLists.txt Normal file
View file

@ -0,0 +1,8 @@
# SPDX-License-Identifier: Apache-2.0
cmake_minimum_required(VERSION 3.20.0)
find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
project(benchmark)
FILE(GLOB app_sources src/*.c)
target_sources(app PRIVATE ${app_sources})

16
samples/subsys/zbus/benchmark/Kconfig Normal file
View file

@ -0,0 +1,16 @@
# Copyright (c) 2022 Rodrigo Peixoto <rodrigopex@gmail.com>
# SPDX-License-Identifier: Apache-2.0
config BM_MESSAGE_SIZE
int "Message size"
default 256
config BM_ONE_TO
int "Number of consumers"
default 8
config BM_ASYNC
bool "Consuming in asynchronous mode"
default false
source "Kconfig.zephyr"

132
samples/subsys/zbus/benchmark/README.rst Normal file
View file

@ -0,0 +1,132 @@
.. _zbus-benchmark-sample:
Benchmark sample
################
This sample implements an application to measure the time for sending 256KB from the producer to the consumers.
Building and Running
********************
.. zephyr-app-commands::
:zephyr-app: samples/subsys/zbus/dyn_channel
:host-os: unix
:board: qemu_cortex_m3
:gen-args: -DCONFIG_BM_MESSAGE_SIZE=1 -DCONFIG_BM_ONE_TO=1 -DCONFIG_BM_ASYNC=0
:goals: build run
Notice we have the following parameters:
* **CONFIG_BM_MESSAGE_SIZE** the size of the message to be transferred;
* **CONFIG_BM_ONE_TO** number of consumers to send;
* **CONFIG_BM_ASYNC** if the execution must be asynchronous or synchronous. Use y to async and n to sync;
Sample Output
=============
The result would be something like:
.. code-block:: console
*** Booting Zephyr OS build zephyr-v3.2.0 ***
I: Benchmark 1 to 8: Dynamic memory, ASYNC transmission and message size 256
I: Bytes sent = 262144, received = 262144
I: Average data rate: 1872457.14B/s
I: Duration: 140ms
@140
Running the benchmark automatically
===================================
There is a Robot script called ``benchmark_256KB.robot`` which runs all the input combinations as the complete benchmark.
The resulting file, ''zbus_dyn_benchmark_256KB.csv`` is generated in the project root folder. It takes a long time to execute. In the CSV file, we have the following columns:
+------------+---------------------+--------------------------+---------------+-------------+-------------+
| Style | Number of consumers | Message size (bytes) | Duration (ms) | RAM (bytes) | ROM (bytes) |
+============+=====================+==========================+===============+=============+=============+
| SYNC/ASYNC | 1,2,4,8 | 1,2,4,8,16,32,64,128,256 | float | int | int |
+------------+---------------------+--------------------------+---------------+-------------+-------------+
The complete benchmark command using Robot framework is:
.. code-block:: console
robot --variable serial_port:/dev/ttyACM0 -d /tmp/benchmark_out benchmark_256KB.robot
An example of execution using the ``hifive1_revb`` board would generate a file like this:
.. code-block::
SYNC,1,1,8534.0,6856,17434
SYNC,1,2,4469.0,6856,17440
SYNC,1,4,2362.3333333333335,6856,17438
SYNC,1,8,1307.6666666666667,6864,17448
SYNC,1,16,768.6666666666666,6872,17478
SYNC,1,32,492.0,6888,17506
SYNC,1,64,391.0,6920,17540
SYNC,1,128,321.0,6984,17600
SYNC,1,256,258.0,7112,17758
SYNC,2,1,4856.666666666667,6880,17490
SYNC,2,2,2464.0,6880,17494
SYNC,2,4,1367.0,6880,17494
SYNC,2,8,778.6666666666666,6888,17504
SYNC,2,16,477.0,6896,17534
SYNC,2,32,321.0,6912,17562
SYNC,2,64,266.0,6944,17592
SYNC,2,128,203.0,7008,17662
SYNC,2,256,188.0,7136,17814
SYNC,4,1,3021.3333333333335,6920,17536
SYNC,4,2,1505.3333333333333,6920,17542
SYNC,4,4,860.0,6920,17542
SYNC,4,8,521.3333333333334,6928,17552
SYNC,4,16,328.0,6936,17582
SYNC,4,32,227.0,6952,17606
SYNC,4,64,180.0,6984,17646
SYNC,4,128,164.0,7048,17710
SYNC,4,256,149.0,7176,17854
SYNC,8,1,2044.3333333333333,7000,17632
SYNC,8,2,1002.6666666666666,7000,17638
SYNC,8,4,586.0,7000,17638
SYNC,8,8,383.0,7008,17648
SYNC,8,16,250.0,7016,17674
SYNC,8,32,203.0,7032,17708
SYNC,8,64,156.0,7064,17742
SYNC,8,128,141.0,7128,17806
SYNC,8,256,133.0,7256,17958
ASYNC,1,1,22187.666666666668,7312,17776
ASYNC,1,2,11424.666666666666,7312,17782
ASYNC,1,4,5823.0,7312,17778
ASYNC,1,8,3071.0,7312,17790
ASYNC,1,16,1625.0,7328,17832
ASYNC,1,32,966.3333333333334,7344,17862
ASYNC,1,64,578.0,7376,17896
ASYNC,1,128,403.6666666666667,7440,17956
ASYNC,1,256,352.0,7568,18126
ASYNC,2,1,18547.333333333332,7792,18030
ASYNC,2,2,9563.0,7792,18034
ASYNC,2,4,4831.0,7792,18030
ASYNC,2,8,2497.3333333333335,7792,18044
ASYNC,2,16,1377.6666666666667,7824,18098
ASYNC,2,32,747.3333333333334,7856,18132
ASYNC,2,64,492.0,7920,18162
ASYNC,2,128,321.0,8048,18232
ASYNC,2,256,239.33333333333334,8304,18408
ASYNC,4,1,16823.0,8744,18474
ASYNC,4,2,8604.333333333334,8744,18480
ASYNC,4,4,4325.666666666667,8744,18472
ASYNC,4,8,2258.0,8744,18490
ASYNC,4,16,1198.3333333333333,8808,18572
ASYNC,4,32,696.0,8872,18610
ASYNC,4,64,430.0,9000,18650
ASYNC,4,128,289.0,9256,18714
ASYNC,4,256,227.0,9768,18906
ASYNC,8,1,15976.666666666666,10648,19366
ASYNC,8,2,7929.666666666667,10648,19372
ASYNC,8,4,4070.6666666666665,10648,19356
ASYNC,8,8,2158.6666666666665,10648,19382
ASYNC,8,16,1119.6666666666667,10776,19506
ASYNC,8,32,619.6666666666666,10904,19566
ASYNC,8,64,391.0,11160,19600
ASYNC,8,128,273.0,11672,19686
ASYNC,8,256,211.0,12696,19934

83
samples/subsys/zbus/benchmark/benchmark_256KB.robot Normal file
View file

@ -0,0 +1,83 @@
*** Settings ***
Library Process
Library String
Library SerialLibrary
Library CSVLibrary
Suite Teardown Terminate All Processes kill=True
*** Variables ***
${csv_file} zbus_dyn_benchmark_256kb.csv
${board} hifive1_revb
${serial_port} /dev/ttyACM0
*** Tasks ***
Clear Old CSV File
Empty Csv File ${csv_file}
Zbus Benchmark
FOR ${async} IN "n" "y"
FOR ${consumers} IN 1 2 4 8
FOR ${msg_size} IN 1 2 4 8 16 32 64 128 256
Benchmark Report For message_size=${msg_size} one_to=${consumers} asynchronous=${async}
END
END
END
*** Keywords ***
Run Memory Report
[Arguments] ${type}
${result} Run Process west build -t ${type}_report shell=True
Should Be Equal As Integers ${result.rc} 0
${mem} Get Substring ${result.stdout} -20
${mem} Strip String ${mem}
${mem} Convert To Integer ${mem}
RETURN ${mem}
Measure Results
${total} Set Variable 0
Add Port ${serial_port} timeout=120 baudrate=115200
Set Encoding ascii
FOR ${count} IN RANGE 3
${result} Run Process west flash shell=True
Should Be Equal As Integers ${result.rc} 0
${val} Read Until expected=@ encoding=ascii
${val} Read Until encoding=ascii
${val} Strip String ${val}
${val} Convert To Integer ${val}
${total} Evaluate ${total}+${val}
END
${duration} Evaluate ${total}/3.0
RETURN ${duration}
[Teardown] Delete All Ports
Benchmark
[Arguments] ${message_size}=256 ${one_to}=1 ${asynchronous}=n
${result} Run Process
... west build -b ${board} -p always -- -DCONFIG_BM_MESSAGE_SIZE\=${message_size} -DCONFIG_BM_ONE_TO\=${one_to} -DCONFIG_BM_ASYNC\=${asynchronous}
... shell=True
Should Be Equal As Integers ${result.rc} 0
${duration} Measure Results
RETURN ${duration}
Benchmark Report For
[Arguments] ${message_size}=256 ${one_to}=1 ${asynchronous}=n
${duration} Benchmark message_size=${message_size} one_to=${one_to} asynchronous=${asynchronous}
${ram_amount} Run Memory Report ram
${rom_amount} Run Memory Report rom
IF ${asynchronous} == "y"
${async_str} Set Variable ASYNC
ELSE
${async_str} Set Variable SYNC
END
@{results} Create List
... ${async_str}
... ${one_to}
... ${message_size}
... ${duration}
... ${ram_amount}
... ${rom_amount}
Log To Console \n${results}
Append To Csv File ${csv_file} ${results}

6
samples/subsys/zbus/benchmark/prj.conf Normal file
View file

@ -0,0 +1,6 @@
CONFIG_LOG=y
CONFIG_LOG_MODE_MINIMAL=y
CONFIG_HEAP_MEM_POOL_SIZE=1024
CONFIG_ASSERT=n
CONFIG_ZBUS=y
CONFIG_ZBUS_LOG_LEVEL_INF=y

41
samples/subsys/zbus/benchmark/sample.yaml Normal file
View file

@ -0,0 +1,41 @@
sample:
name: Benchmark
tests:
sample.zbus.benchmark_async:
tags: zbus
min_ram: 16
filter: CONFIG_SYS_CLOCK_EXISTS
harness: console
harness_config:
type: multi_line
ordered: true
regex:
- "I: Benchmark 1 to 8: Dynamic memory, ASYNC transmission and message size 256"
- "I: Bytes sent = 262144, received = 262144"
- "I: Average data rate: (.*)B/s"
- "I: Duration: (.*)ms"
- "@(.*)"
extra_configs:
- CONFIG_BM_ONE_TO=8
- CONFIG_BM_MESSAGE_SIZE=256
- CONFIG_BM_ASYNC=y
platform_exclude: qemu_x86 qemu_x86_64 qemu_riscv32_smp native_posix native_posix_64 qemu_riscv32_smp qemu_cortex_a53_smp qemu_riscv64_smp qemu_leon3 qemu_xtensa qemu_cortex_a53 qemu_riscv32 qemu_malta qemu_malta_be qemu_arc_hs6x qemu_riscv64 m2gl025_miv hifive_unleashed
sample.zbus.benchmark_sync:
tags: zbus
min_ram: 16
filter: CONFIG_SYS_CLOCK_EXISTS
harness: console
harness_config:
type: multi_line
ordered: true
regex:
- "I: Benchmark 1 to 8: Dynamic memory, SYNC transmission and message size 256"
- "I: Bytes sent = 262144, received = 262144"
- "I: Average data rate: (.*)B/s"
- "I: Duration: (.*)ms"
- "@(.*)"
extra_configs:
- CONFIG_BM_ONE_TO=8
- CONFIG_BM_MESSAGE_SIZE=256
- CONFIG_BM_ASYNC=n
platform_exclude: qemu_x86 qemu_x86_64 qemu_riscv32_smp native_posix native_posix_64 qemu_riscv32_smp qemu_cortex_a53_smp qemu_riscv64_smp qemu_leon3 qemu_xtensa qemu_cortex_a53 qemu_riscv32 m2gl025_miv m2gl025_miv

216
samples/subsys/zbus/benchmark/src/benchmark.c Normal file
View file

@ -0,0 +1,216 @@
/*
* Copyright (c) 2022 Rodrigo Peixoto <rodrigopex@gmail.com>
* SPDX-License-Identifier: Apache-2.0
*/
#include "messages.h"
#include <stdint.h>
#include <zephyr/fatal.h>
#include <zephyr/kernel.h>
#include <zephyr/logging/log.h>
#include <zephyr/sys/util_macro.h>
#include <zephyr/zbus/zbus.h>
LOG_MODULE_DECLARE(zbus, CONFIG_ZBUS_LOG_LEVEL);
ZBUS_CHAN_DEFINE(bm_channel, /* Name */
struct external_data_msg, /* Message type */
NULL, /* Validator */
NULL, /* User data */
ZBUS_OBSERVERS(s1
#if (CONFIG_BM_ONE_TO >= 2LLU)
,
s2
#if (CONFIG_BM_ONE_TO > 2LLU)
,
s3, s4
#if (CONFIG_BM_ONE_TO > 4LLU)
,
s5, s6, s7, s8
#if (CONFIG_BM_ONE_TO > 8LLU)
,
s9, s10, s11, s12, s13, s14, s15, s16
#endif
#endif
#endif
#endif
), /* observers */
ZBUS_MSG_INIT(0) /* Initial value {0} */
);
#define BYTES_TO_BE_SENT (256LLU * 1024LLU)
static uint64_t count;
#if (CONFIG_BM_ASYNC == 1)
ZBUS_SUBSCRIBER_DEFINE(s1, 4);
#if (CONFIG_BM_ONE_TO >= 2LLU)
ZBUS_SUBSCRIBER_DEFINE(s2, 4);
#if (CONFIG_BM_ONE_TO > 2LLU)
ZBUS_SUBSCRIBER_DEFINE(s3, 4);
ZBUS_SUBSCRIBER_DEFINE(s4, 4);
#if (CONFIG_BM_ONE_TO > 4LLU)
ZBUS_SUBSCRIBER_DEFINE(s5, 4);
ZBUS_SUBSCRIBER_DEFINE(s6, 4);
ZBUS_SUBSCRIBER_DEFINE(s7, 4);
ZBUS_SUBSCRIBER_DEFINE(s8, 4);
#if (CONFIG_BM_ONE_TO > 8LLU)
ZBUS_SUBSCRIBER_DEFINE(s9, 4);
ZBUS_SUBSCRIBER_DEFINE(s10, 4);
ZBUS_SUBSCRIBER_DEFINE(s11, 4);
ZBUS_SUBSCRIBER_DEFINE(s12, 4);
ZBUS_SUBSCRIBER_DEFINE(s13, 4);
ZBUS_SUBSCRIBER_DEFINE(s14, 4);
ZBUS_SUBSCRIBER_DEFINE(s15, 4);
ZBUS_SUBSCRIBER_DEFINE(s16, 4);
#endif
#endif
#endif
#endif
#define S_TASK(name) \
void name##_task(void) \
{ \
struct external_data_msg *actual_message_data; \
const struct zbus_channel *chan; \
struct bm_msg msg_received; \
\
while (!zbus_sub_wait(&name, &chan, K_FOREVER)) { \
zbus_chan_claim(chan, K_NO_WAIT); \
\
actual_message_data = zbus_chan_msg(chan); \
__ASSERT_NO_MSG(actual_message_data->reference != NULL); \
\
memcpy(&msg_received, actual_message_data->reference, \
sizeof(struct bm_msg)); \
\
zbus_chan_finish(chan); \
\
count += CONFIG_BM_MESSAGE_SIZE; \
} \
} \
\
K_THREAD_DEFINE(name##_id, CONFIG_BM_MESSAGE_SIZE + 196, name##_task, NULL, NULL, NULL, 3, \
0, 0);
S_TASK(s1)
#if (CONFIG_BM_ONE_TO >= 2LLU)
S_TASK(s2)
#if (CONFIG_BM_ONE_TO > 2LLU)
S_TASK(s3)
S_TASK(s4)
#if (CONFIG_BM_ONE_TO > 4LLU)
S_TASK(s5)
S_TASK(s6)
S_TASK(s7)
S_TASK(s8)
#if (CONFIG_BM_ONE_TO > 8LLU)
S_TASK(s9)
S_TASK(s10)
S_TASK(s11)
S_TASK(s12)
S_TASK(s13)
S_TASK(s14)
S_TASK(s15)
S_TASK(s16)
#endif
#endif
#endif
#endif
#else /* SYNC */
static void s_cb(const struct zbus_channel *chan);
ZBUS_LISTENER_DEFINE(s1, s_cb);
#if (CONFIG_BM_ONE_TO >= 2LLU)
ZBUS_LISTENER_DEFINE(s2, s_cb);
#if (CONFIG_BM_ONE_TO > 2LLU)
ZBUS_LISTENER_DEFINE(s3, s_cb);
ZBUS_LISTENER_DEFINE(s4, s_cb);
#if (CONFIG_BM_ONE_TO > 4LLU)
ZBUS_LISTENER_DEFINE(s5, s_cb);
ZBUS_LISTENER_DEFINE(s6, s_cb);
ZBUS_LISTENER_DEFINE(s7, s_cb);
ZBUS_LISTENER_DEFINE(s8, s_cb);
#if (CONFIG_BM_ONE_TO > 8LLU)
ZBUS_LISTENER_DEFINE(s9, s_cb);
ZBUS_LISTENER_DEFINE(s10, s_cb);
ZBUS_LISTENER_DEFINE(s11, s_cb);
ZBUS_LISTENER_DEFINE(s12, s_cb);
ZBUS_LISTENER_DEFINE(s13, s_cb);
ZBUS_LISTENER_DEFINE(s14, s_cb);
ZBUS_LISTENER_DEFINE(s15, s_cb);
ZBUS_LISTENER_DEFINE(s16, s_cb);
#endif
#endif
#endif
#endif
static void s_cb(const struct zbus_channel *chan)
{
struct bm_msg msg_received;
const struct external_data_msg *actual_message_data = zbus_chan_const_msg(chan);
memcpy(&msg_received, actual_message_data->reference, sizeof(struct bm_msg));
count += CONFIG_BM_MESSAGE_SIZE;
}
#endif /* CONFIG_BM_ASYNC */
static void producer_thread(void)
{
LOG_INF("Benchmark 1 to %d: Dynamic memory, %sSYNC transmission and message size %u",
CONFIG_BM_ONE_TO, IS_ENABLED(CONFIG_BM_ASYNC) ? "A" : "", CONFIG_BM_MESSAGE_SIZE);
struct bm_msg msg;
struct external_data_msg *actual_message_data;
for (uint64_t i = (CONFIG_BM_MESSAGE_SIZE - 1); i > 0; --i) {
msg.bytes[i] = i;
}
zbus_chan_claim(&bm_channel, K_NO_WAIT);
actual_message_data = zbus_chan_msg(&bm_channel);
actual_message_data->reference = k_malloc(sizeof(struct bm_msg));
__ASSERT_NO_MSG(actual_message_data->reference != NULL);
actual_message_data->size = sizeof(struct bm_msg);
__ASSERT_NO_MSG(actual_message_data->size > 0);
zbus_chan_finish(&bm_channel);
uint32_t start = k_uptime_get_32();
for (uint64_t internal_count = BYTES_TO_BE_SENT / CONFIG_BM_ONE_TO; internal_count > 0;
internal_count -= CONFIG_BM_MESSAGE_SIZE) {
zbus_chan_claim(&bm_channel, K_NO_WAIT);
actual_message_data = zbus_chan_msg(&bm_channel);
memcpy(actual_message_data->reference, &msg, CONFIG_BM_MESSAGE_SIZE);
zbus_chan_finish(&bm_channel);
zbus_chan_notify(&bm_channel, K_MSEC(200));
}
uint32_t duration = (k_uptime_get_32() - start);
if (duration == 0) {
LOG_ERR("Something wrong. Duration is zero!\n");
k_oops();
}
uint64_t i = (BYTES_TO_BE_SENT * 1000) / duration;
uint64_t f = ((BYTES_TO_BE_SENT * 100000) / duration) % 100;
LOG_INF("Bytes sent = %lld, received = %llu", BYTES_TO_BE_SENT, count);
LOG_INF("Average data rate: %llu.%lluB/s", i, f);
LOG_INF("Duration: %ums", duration);
printk("\n@%u\n", duration);
}
K_THREAD_DEFINE(producer_thread_id, 1024, producer_thread, NULL, NULL, NULL, 5, 0, 0);

19
samples/subsys/zbus/benchmark/src/messages.h Normal file
View file

@ -0,0 +1,19 @@
/*
* Copyright (c) 2022 Rodrigo Peixoto <rodrigopex@gmail.com>
* SPDX-License-Identifier: Apache-2.0
*/
#ifndef _MESSAGES_H_
#define _MESSAGES_H_
#include <stddef.h>
#include <stdint.h>
struct external_data_msg {
void *reference;
size_t size;
};
struct bm_msg {
uint8_t bytes[CONFIG_BM_MESSAGE_SIZE];
};
#endif /* _MESSAGES_H_ */