Added integration_toolchains to allow building/testing with multiple
toolchains available in the environment.
This changes the output structure and adds another level in the path
under twister_out signifying the toolchain used. The toolchain used
(variant) is also part of the json output now.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
As we aim to enhance our test coverage, environment module
is a target-rich environment. Some errors in the original
module were discovered and tests regarding these cases were
commented out until they are fixed.
Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
Extended parsing of report.xml file - produced by pytest-twister-harness
plugin. Now testcases are properly extracted and added to twister
report. Added unit tests.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Add support test levels and the ability to assign a specific test to one
or more levels. Using command line options of twister it is then possible
to select a level and just execute the tests included in this level.
Additionally, a test configuration allows definiing level
dependencies and additional inclusion of tests into a specific level if
the test itself does not have this information already.
In the configuration file you can include complete components using
regular expressions and you can specify which test level to import from
the same file, making management of levels easier.
To help with testing outside of upstream CI infrastructure, additional
options are available in the configuration file, which can be hosted
locally. As of now, those options are available:
- Ability to ignore default platforms as defined in board definitions
(Those are mostly emulation platforms used to run tests in upstream
CI)
- Option to specify your own list of default platforms overriding what
upstream defines.
- Ability to override build_onl_all options used in some testscases.
This will treat tests or sample as any other just build for default
platforms you specify in the configuation file or on the command line.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Removed tests for dropped functionality.
Renamed classes and files to reflect new structure.
As we refactor the reporting infrastructure, tests will follow once we
have everything in place.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
We now get toolchain from cmake and can operate even if the toolchain
variant is not set in the envrionment, this test was verifying the old
behavior, so remove. There is not much twister related here to test and
the environment is not setup correctly to run all of this cmake code. We
are testing twister after all.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
