Commit graph

4988 commits

Author SHA1 Message Date
Piotr Kosycarz
ca770dbeb7 scripts: pylib: twister: twisterlib: prevent empty gcda files
In case of problem with parsing hex data from coverage dump,
do not create empty gcda file.
Such empty file will break gcovr parsing.

Signed-off-by: Piotr Kosycarz <piotr.kosycarz@nordicsemi.no>
2024-09-19 15:14:33 +02:00
Kamil Piszczek
8c0bc54ab0 mgmt: mcumgr: transport: bluetooth: add all gatt permission levels
Added new Kconfigs for defining permission level of GATT
characteristics that are part of the Bluetooth SMP service in the
MCUmgr subsys.

Removed the CONFIG_MCUMGR_TRANSPORT_BT_AUTHEN as the new Kconfig
options are mutually exclusive and need to be groupa as the Kconfig
choice option.

Signed-off-by: Kamil Piszczek <Kamil.Piszczek@nordicsemi.no>
2024-09-19 15:13:19 +02:00
Dmitrii Golovanov
c5f2df49cc scripts: footprint: Add converter to twister_footprint.json
Add new script `pack_as_twister.py` to convert memory footprint
data prepared by `./footprint/scripts/track.py` into JSON files
compatible with Twister report schema. Next, the data can be
transformed and uploaded to ElasticSearch data storage the same
way as memory footprint (and other) reports executed by Twister.

Add to `plan.txt` an optional column with the corresponding
test suite names for 'footprints' as an example for test instance
name composing with `--test-name` command argumnent.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2024-09-19 03:28:52 -04:00
Grzegorz Swiderski
eadc3d08c2 twister: Fix inconsistency among DT compat filters
This small change concerns the following filter functions:

   1. `dt_compat_enabled(C)`:
      There's a node with compatible `C` and status "okay".

   2. `dt_enabled_alias_with_parent_compat(A, C)`:
      There's a node with alias `A` and status "okay", and its parent
      has compatible `C`.

   3. `dt_label_with_parent_compat_enabled(L, C)`:
      There's a node with label `L`, and its parent has compatible `C`
      and status "okay".

All three functions involve checking whether some node or its parent has
a given compatible, but the way this has been checked is inconsistent.
Function (1) has done it with this Python conditional:

   compat in node.compats

while (2) and (3) have used:

   parent.matching_compat == compat

The first check works well with nodes that have multiple compatibles,
and it is more aligned with the notion of "has_compat" as seen in the
devicetree macros for C, CMake, and Kconfig.

Arguably, `matching_compat` shouldn't have been used here, because it is
actually a property of a node's binding, moreso than of the node itself.
In practice, it's usually equal to the first compatible for which edtlib
has found a binding, which at first glance is just more constrained than
the `node.compats` check. However, there also exist obscure cases where
the `node.compats` are empty, while the `node.matching_compat` is not.

For now, the three functions can use a combined check, to improve
consistency and utility while avoiding breakage:

   node.matching_compat == compat or compat in node.compats

Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
2024-09-18 19:56:06 +01:00
Grzegorz Swiderski
89bf698d90 twister: Remove dt_compat_enabled_with_alias filter
It has been deprecated since Zephyr v2.6.0.

The filter that replaced it - `dt_enabled_alias_with_parent_compat` -
had shared code with it, which can now be inlined into `ast_expr()` to
match the parser's overall code structure.

Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
2024-09-18 19:56:06 +01:00
Johan Hedberg
db88324f26 scripts: check_compliance: Fix incorrect soc root lookup
The script was only looking at the Zephyr base repository and failing to
look for soc roots e.g. in other modules.

Signed-off-by: Johan Hedberg <johan.hedberg@silabs.com>
2024-09-18 15:29:32 +02:00
Torsten Rasmussen
fc741d5344 compliance: Revert "scripts: handle auto defined ZEPHYR_<name>_MODULE"
With the requirement in #78316 for Zephyr modules to always define
ZEPHYR_<name>_MODULE Kconfig setting then there is no longer a need
for this commit. Simplify check_compliance by reverting ths commit.

This reverts commit 35e28e6315.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2024-09-18 15:29:06 +02:00
Lukasz Mrugala
418b1e0e21 scripts: twister: Add TestCase status printing
When at verbosity 1, we print out the status of TestInstances.
This makes it harder to notice changes at TestCase level,
which require perusing the logs.

This adds TestCase status and reason printing
if verbosity level is 2 or more.
Reason printing is suppressed if the reason is empty or None.

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2024-09-17 15:04:04 -04:00
Reto Schneider
371a48ac16 edtlib: Remove unreachable code
This fixes the following error reported by
./scripts/ci/check_compliance.py:

> 1 checks failed
> ERROR   : Test Pylint failed:
> W0101:Unreachable code (unreachable)
> File:scripts/dts/python-devicetree/src/devicetree/edtlib.py
> Line:2271
> Column:16
>
> Complete results in compliance.xml

Signed-off-by: Reto Schneider <reto.schneider@husqvarnagroup.com>
2024-09-17 14:58:25 -04:00
Wilfried Chauveau
6194159608 scripts: twister: handlers: Enable BinaryHandler to run sysbuilt tests
Running sysbuilt tests fails because of missing "run" target.

This adds the default domain context to the command.

Signed-off-by: Wilfried Chauveau <wilfried.chauveau@arm.com>
2024-09-17 09:13:26 -05:00
Lukasz Mrugala
14a72b5158 scripts: Plug TwisterStatus type gaps
Some dict.get() calls did not use a TwisterStatus
as a default value, thus using a NoneType where
TwisterStatus should appear.

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2024-09-16 20:18:09 +02:00
Anas Nashif
02bca9f6f6 twister: abort on unrecognised platforms
identify misnamed platforms and abort.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-09-16 10:07:51 +02:00
Hake Huang
7f63faa4f5 twister: hwmap: add script parameter support
for pre/post/post_flash script, add timeout as param

Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
2024-09-16 10:03:53 +02:00
Benjamin Cabé
fd4f3ce246 scripts: compliance: add sphinx-lint linter
ReStructuredText can sometimes be tricky to get right, especially for
folks that might be more familiar with Markdown.

This adds a Sphinx/RST linter to the compliance check script to help
catch common issues that can easily go unnoticed and cause rendering
issues.

Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
2024-09-13 11:43:04 -05:00
Benjamin Cabé
df294e34e1 doc: sphinx-lint: fix bad usage of "default role"
Fixes bad usage of single backticks in lieu of double backticks for
rendering inline literals, or simple '*' for italics.

When appropriate, a better construct than double backticks has been
selected (ex. :file:, :kconfig:option:, :c:func:, ...), or proper :ref:
have been used if the original intention was to have a link.

Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
2024-09-13 11:42:51 -05:00
Lukasz Mrugala
f78949b18c scripts: twister: Process KeyboardInterrupt Hardening
If you interrupt process() operation, we want Twister
to exit as gracefully as it can. This avoids the
UnboundLocalError that could appear e.g.
when interrupting the operation via SIGINT.

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2024-09-13 13:42:05 +02:00
Guennadi Liakhovetski
dc5f1bfb3f west: fix for Python prior to 3.10
"type | None" syntax is only available with Python from version 3.10,
fix building with earlier 3.x Python versions.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
2024-09-12 14:51:03 +02:00
Anas Nashif
a053f1ae05 twister: add simics as supported simulator
Enable simics as a support simulation platform.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-09-12 14:49:21 +02:00
Alberto Escolar Piedras
9d5a09668b native_simulator: Get latest from upstream
Align with native_simulator's upstream main
b4b9791ac822ae300363dc3ebbc7b7ac200632ce

Which includes:
* b4b9791 NATIVE_SIMULATOR_IF macros: Provide version with explicit
          section names

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2024-09-12 14:47:57 +02:00
Pieter De Gendt
d2bae91803 scripts: pylib: twister: Pass testsuite name to cmake
Add a CMake argument with the current testsuite name.

Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
2024-09-12 10:04:05 +02:00
Wilfried Chauveau
a55bb955af scripts: twister: add type hint to twister_main.py
Twister main expects an instance of TwisterEnv. Let's the linter know
about this.

Signed-off-by: Wilfried Chauveau <wilfried.chauveau@arm.com>
2024-09-12 10:03:44 +02:00
Wilfried Chauveau
30dc833cab scripts: twister: Use args rather than injecting members
This changes how some arguments are set in the `Handler`s.
`options`, `generator_cmd` and `suite_name_check` are now passed as
arguments to the constructor rather than injected from an other module.

Signed-off-by: Wilfried Chauveau <wilfried.chauveau@arm.com>
2024-09-12 10:03:44 +02:00
Wilfried Chauveau
6d9e5df401 scripts: twister: TwisterEnv is always used with an options parameter
Adjusting the type hints and code accordingly.

Signed-off-by: Wilfried Chauveau <wilfried.chauveau@arm.com>
2024-09-12 10:03:44 +02:00
Wilfried Chauveau
5ad5af3713 scripts: twisterlib: Fix warning about deprecation of truthiness
Python 3.12 warns that

> Testing an element's truth value will raise an exception in future
> versions. Use specific 'len(elem)' or 'elem is not None' test instead.
>    if elem_ts := root.find('testsuite'):

Signed-off-by: Wilfried Chauveau <wilfried.chauveau@arm.com>
2024-09-12 10:03:44 +02:00
Wilfried Chauveau
deeec33934 scripts: twister: Solve deprecation notice when running tests
CONF_FILE, DTC_OVERLAY_FILE and OVERLAY_CONFIG are deprecated but still
used by the tests causing warnings when running them.

This adds a test_data specific to validate the emission of the warning,
and removes the offending args from the other test_data files.

Signed-off-by: Wilfried Chauveau <wilfried.chauveau@arm.com>
2024-09-12 10:03:44 +02:00
Wilfried Chauveau
28add0cd36 scripts: twister: check for deprecation notice for space-separated lists
Space separated lists are deprecated but this notice is not checked for.
extract_fields_from_arg_list also converts lists back to space-separated
lists causing a warning on get_scenario

Signed-off-by: Wilfried Chauveau <wilfried.chauveau@arm.com>
2024-09-12 10:03:44 +02:00
Tobias Pisani
73c8235881 west: runners: Implement the west rtt command for jlink
Moves the telnet client into runners/core.py as well, as this is now shared
between openocd and jlink.

Signed-off-by: Tobias Pisani <mail@topisani.dev>
2024-09-10 12:39:42 -04:00
Tobias Pisani
18f45b5f06 west: runners: Implement the west rtt command for openocd
This was non-trivial, as openocd is a bit weird to work with. Using only
commands passed with '-c' arguments, I couldn't get it to reliably resume
(or just not halt) the target when started. I tried using the 'sleep'
command, and various 'configure -event XX { resume }' events, but nothing
panned out, as it seems to always halt after all `-c` commands have been
run.

To avoid that, this waits for the TCL RPC port to be up, and sends a
resume command there. This works reliably.

Signed-off-by: Tobias Pisani <mail@topisani.dev>
2024-09-10 12:39:42 -04:00
Tobias Pisani
b3b8360f39 west: runners: Add west rtt command with pyocd implementation
This command runs separately from a debug server, instead of attaching
to a running server. This is both the easiest out of the box experience,
and also should be possible to implement consistently for most runners.

This commit includes an initial implementation for pyocd.

Signed-off-by: Tobias Pisani <mail@topisani.dev>
2024-09-10 12:39:42 -04:00
Maciej Perkowski
c743821f96 twister: fix dts path for module
Fixes twister with modules (TypeError for undefined dts_root)

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2024-09-10 11:42:51 +02:00
Dmitrii Golovanov
d9f567051e ci: elasticsearch: Upload script improvements
Multiple improvements of the `upload_test_results_es.py` script:

 * JSON objects flattening.

   This feature allows `twister.json` file preprocessing to simplify
   its Elasticsearch index structure for complex hierarhical objects,
   for example with memory footprint, or code coverage data.

   A new command line option `--flatten` is added to change testsuite data
   structure in regard of one of its list objects: either `testcases` or
   `recording`, so each item there becomes an independent data record
   inheriting all other testsuite properties, whereas the children
   object's properties are renamed with the parent object's name
   as a prefix: 'testcases_' or 'recording_' respectively.
   Only one testsuite property can be flattened this way per index upload.
   Other children objects will be treated accorging to the index structure.

   Related new command line options (with help text explanations):
    `--flatten-dict-name`,
    `--flatten-list-names`,
    `--flatten-separator`,
    `--transpose-separator`,
    `--escape-separator`

 * A new command line option `--transform` is added to allow regexp group
   parsing in string propertites extracting additional derived properties.

 * A new command line option `--exclude` is added to exclude testsuite
   properties not needed to store at Elasticsearch index.

 * Branch name `--run-branch` and Workflow ID `--run-workflow` command
   line options as additional key fields to allow data from different
   branches, workflows and triggering events in the same index.

 * A new command line option `--map-file` is added to apply
   an explicit index structure to the `twister.json` input data.

 * Add bulk operation timeout parameter for heavy/long uploads.

Other changes:
  * batch upload error handling and logging;
  * inline documentation improvements;
  * some corner case fixes on empty objects.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2024-09-09 15:24:26 +03:00
Jamie McCrae
9354e181d6 scripts: ci: check_compliance: Add MCUboot Kconfig
Adds a Kconfig used by MCUboot to the allow list

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2024-09-09 15:23:16 +03:00
Lukasz Mrugala
345eb8e94a scripts: twister: Elevate Status Error
Status errors previously logged an error, but didn't fail the running test.
This commit changes that
 and introduces a new StatusAttributeError to use there.

One test is modified so it follows proper status form.
One test for the new error has been added.

Status errors now will properly mark the Instance as ERROR
and not run TestCases as SKIP.
This necessitated some code layout changes in runner.py

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2024-09-09 09:28:37 +02:00
Pieter De Gendt
5a3c9c914a scripts: pylib: twister: Write version to testplan.json
The twister environment version is already set, write it to the
testplan.json file.

Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
2024-09-06 11:31:57 -04:00
Piotr Kosycarz
f0bfaa4c19 scripts: pylib: twister: fix missing newlines at handler.log
Keep new lines at handler.log.

Signed-off-by: Piotr Kosycarz <piotr.kosycarz@nordicsemi.no>
2024-09-04 19:07:19 -04:00
Attie Grande
e6dc930322 bindesc: Add support for the build version values
This patch adds bindesc support for the build version values for the
kernel and application - BUILD_VERSION and APP_BUILD_VERSION.

The kernel's BUILD_VERSION can be overridden at build time.

Signed-off-by: Attie Grande <attie.grande@argentum-systems.co.uk>
2024-09-04 07:03:47 -04:00
Grzegorz Chwierut
392304706b twister: Pass device flash timeout to pytest-harness.
Pass flashing timeout to pytest-harness and use them

Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
2024-09-03 10:41:45 +02:00
Vinicius Miguel
0492235d0d scripts: ci: Fix crash when vendor list is malformed
Also introduces friendly error message.

Signed-off-by: Vinicius Miguel <vmiguel1@gmail.com>
2024-09-02 12:03:03 +02:00
Anas Nashif
d145db01b4 twister: setup soc/arch roots based on module settings
Get data for various roots from modules and do not hardcode location
when retrieving list of boards in testplan.

Fixes #71761

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-08-30 11:46:41 -04:00
Johan Hedberg
5562bb9c3a scripts: zephyr_module: Add Kconfig symbol for blobs presence
For any module that defines blobs, add a new Kconfig symbol to indicate
whether the blobs have been fetched or not. Example output for the
hal_silabs module:

   # (no blobs present)
  $ scripts/zephyr_module.py --kconfig-out=/dev/stdout \
    -m ../modules/hal/silabs
  menu "hal_silabs (../modules/hal/silabs)"
  osource "/Users/johedber/src/zephyr/modules/hal/silabs/zephyr/Kconfig"
  config ZEPHYR_HAL_SILABS_MODULE
  	bool
  	default y
  config ZEPHYR_HAL_SILABS_MODULE_BLOBS
  	bool
  endmenu
  $ west blobs fetch hal_silabs
   # (blob fetching output)
  $ scripts/zephyr_module.py --kconfig-out=/dev/stdout \
    -m ../modules/hal/silabs
  menu "hal_silabs (../modules/hal/silabs)"
  osource "/Users/johedber/src/zephyr/modules/hal/silabs/zephyr/Kconfig"
  config ZEPHYR_HAL_SILABS_MODULE
  	bool
  	default y
  	select TAINT_BLOBS

  config ZEPHYR_HAL_SILABS_MODULE_BLOBS
  	bool
  	default y
  endmenu

The generated output for modules which do not define blobs is not affected.

Having this additional symbol for blobs lets us specify Kconfig
dependencies for features which require the blobs to be present.

Signed-off-by: Johan Hedberg <johan.hedberg@silabs.com>
2024-08-30 11:45:56 -04:00
Andrej Butok
3f5fc42fba runners: linkserver: Add .elf support to the linkserver flash command
As the .bin & .hex build output is optional
and it can be disabled by CONFIG_BUILD_OUTPUT_BIN/HEX,
add support for the mandatory .elf build output
to the linkserver runner flash command.

Signed-off-by: Andrej Butok <andrey.butok@nxp.com>
2024-08-29 11:37:54 +02:00
Pieter De Gendt
2c496c094b scripts: ci: Fix clang-format compliance for additions or removals
If a diff only has added or removed lines we need to match both '-' and '+'
characters in the hunk context.

Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
2024-08-28 16:35:20 -04:00
Anas Nashif
a5e3c0a792 ci: do not run twister on some script changes
Do not run twister on some script changes that do not require a twister
run.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-08-28 13:58:56 -04:00
Dmitrii Golovanov
87c1e2a40f twister: Group test plan reporting options
Re-arrange Twister command line options for test plan reporting
to a dedicated group with mutally-exclusive options to reflect
actual implementation and its dry-run execution mode.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2024-08-27 15:13:52 -04:00
Anas Nashif
4077249cc7 ci: rerun issue check on PR edit
Re-run issue check when a PR is updated, i.e. when someone adds
'Fixes...` to the PR body.

This is mostly for release branches and has no effect on main branch.

Also, add concurrency check in the workflow.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-08-27 11:33:55 -04:00
Torsten Rasmussen
35e28e6315 scripts: handle auto defined ZEPHYR_<name>_MODULE settings in compliance
Update check_compliance to handle auto defined ZEPHYR_<name>_MODULE
Kconfig symbols.

check_compliance runs three levels of Kconfig check.
A basic and a full, both which uses the generated Kconfig.modules
created according to Zephyr modules present.

A Kconfig check where no Zephyr modules are sourced. This check ensures
that Zephyr Kconfig tree doesn't refer to Kconfig symbols defined in
Zephyr module's local Kconfig trees.

However, there are a few auto generated symbols which are allowed,
such as: `ZEPHYR_<name>_MODULE` and `ZEPHYR_<name>_MODULE_BLOBS`.

Therefore, when testing no blobs, filter the generated Kconfig.modules
file, so that no sourcing of extra Kconfig files are performed but
lines defining `ZEPHYR_<name>_MODULE` and `ZEPHYR_<name>_MODULE_BLOBS`
are preserved.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2024-08-27 10:50:08 -04:00
Sven Ginka
63f8f7ff2a scripts: RISCV instruction SEPC
In RISCV "sepc" is actually the name of an instruction.
As this prevents building. We should remove the line.

Signed-off-by: Sven Ginka <s.ginka@sensry.de>
2024-08-27 10:48:58 -04:00
Pieter De Gendt
fda5ca34c2 scripts: ci: Fix clang-format notice at file beginning/end
The clang-format check outputs a git diff with surrounding context. It
naively removed the first and last 3 lines, but this does not work if
there are less lines.

Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
2024-08-27 12:45:21 +02:00
Andrej Butok
c1fe3150e9 runners: pyocd: Add support for .elf files to the pyocd flash command
As the .bin & .hex build output is optional
and it can be disabled by CONFIG_BUILD_OUTPUT_BIN/HEX,
add support for the mandatory .elf build output
to the pyocd runner flash command.

Signed-off-by: Andrej Butok <andrey.butok@nxp.com>
2024-08-27 12:44:55 +02:00
Anas Nashif
b2a9fca9c8 ci: twister: remove reference to opensearch
We have been using elasticsearch for a while now. Remove references and
old script.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-08-27 12:44:31 +02:00