Commit graph

11 commits

Author SHA1 Message Date
Christian Clauss
8ce7a58be2 tests: Four typos in tests directory.
Found by codespell.

Signed-off-by: Christian Clauss <cclauss@me.com>
2025-02-25 23:34:53 +11:00
Damien George
6dbb7ab18f tests/README: Update TLS certificate generation instructions.
Fix the command that converts `ec_key.pem` to `ec_key.der`, and increase
the certificate validity to 10 years.

Signed-off-by: Damien George <damien@micropython.org>
2025-01-14 14:38:41 +11:00
Damien George
a2554f0957 tests/run-tests.py: Add support for tests to use unittest.
All the existing tests require a .exp file (either manually specified or
generated running the test first under CPython) that is used to check the
output of running the test under MicroPython.  The test passes if the
output matches the expected output exactly.

This has worked very well for a long time now.  But some of the newer
hardware tests (eg UART, SPI, PWM) don't really fit this model, for the
following main reasons:

- Some but not all parts of the test should be skipped on certain hardware
  targets.  With the expected-output approach, skipping tests is either all
  or nothing.

- It's often useful to output diagnostics as part of the test, which should
  not affect the result of the test (eg the diagnostics change from run to
  run, like timing values, or from target to target).

- Sometimes a test will do a complex check and then print False/True if it
  passed or not, which obscures the actual test result.

To improve upon this, this commit adds support to `run-tests.py` for a test
to use `unittest`.  It detects this by looking at the end of the output
after running the test, looking for the test summary printed by `unittest`
(or an error message saying `unittest` was not found).  If the test uses
`unittest` then it should not have a .exp file, and it's not run under
CPython.  A `unittest` based test passes or fails based on the summary
printed by `unittest`.

Note that (as long as `unittest` is installed on the target) the tests are
still fully independent and you can still run them without `run-tests.py`:
you just run it as usual, eg `mpremote run <test.py>`.  This is very useful
when creating and debugging tests.

Note also that the standard test suite testing Python semantics (eg
everything in `tests/basics/`) will probably never use unittest.  Only more
advanced tests will, and ones that are not runnable under CPython.

Signed-off-by: Damien George <damien@micropython.org>
2024-12-06 13:41:52 +11:00
Damien George
8978102f35 tests/run-tests.py: Change --target/--device options to --test-instance.
Previously to this commit, running the test suite on a bare-metal board
required specifying the target (really platform) and device, eg:

    $ ./run-tests.py --target pyboard --device /dev/ttyACM1

That's quite a lot to type, and you also need to know what the target
platform is, when a lot of the time you either don't care or it doesn't
matter.

This commit makes it easier to run the tests by replacing both of these
options with a single `--test-instance` (`-t` for short) option.  That
option specifies the executable/port/device to test.  Then the target
platform is automatically detected.

The `--test-instance` can be passed:
- "unix" (the default) to use the unix version of MicroPython
- "webassembly" to test the webassembly port
- anything else is considered a port/device to pass to Pyboard

There are also some shortcuts to specify a port/device, following
`mpremote`:
- a<n> is short for /dev/ttyACM<n>
- u<n> is short for /dev/ttyUSB<n>
- c<n> is short for COM<n>

For example:

    $ ./run-tests.py -t a1

Note that the default test instance is "unix" and so this commit does not
change the standard way to run tests on the unix port, by just doing
`./run-tests.py`.

As part of this change, the platform (and it's native architecture if it
supports importing native .mpy files) is show at the start of the test run.

Signed-off-by: Damien George <damien@micropython.org>
2024-11-04 12:47:47 +11:00
Andrew Leech
c2eebe0609 tests/README: Update instructions for key/cert pair usage on device.
Signed-off-by: Andrew Leech <andrew@alelec.net>
2024-10-09 14:37:59 +11:00
Phil Howard
dda9b9c6da all: Prune trailing whitespace.
Prune trailing whitespace across the whole project (almost), done
automatically with:

    grep -IUrl --color "[[:blank:]]$" --exclude-dir=.git --exclude=*.exp |\
        xargs sed -i 's/[[:space:]]*$//'

Exceptions:
- Skip third-party code in lib/ and drivers/cc3100/
- Skip generated code in bluetooth_init_cc2564C_1.5.c
- Preserve command output whitespace in docs, eg:
  docs/esp8266/tutorial/repl.rst

Signed-off-by: Phil Howard <phil@gadgetoid.com>
2024-03-07 16:25:17 +11:00
Carlosgg
f3d1495fd3 all: Update bindings, ports and tests for mbedtls v3.5.1.
Changes include:

- Some mbedtls source files renamed or deprecated.

- Our `mbedtls_config.h` files are renamed to `mbedtls_config_port.h`, so
  they don't clash with mbedtls's new default configuration file named
  `mbedtls_config.h`.

- MBEDTLS_TLS_DEFAULT_ALLOW_SHA1_IN_KEY_EXCHANGE is deprecated.

- MBEDTLS_HAVE_TIME now requires an `mbedtls_ms_time` function to be
  defined but it's only used for TLSv1.3 (currently not enabled in
  MicroPython so there is a lazy implementation, i.e. seconds * 1000).

- `tests/multi_net/ssl_data.py` is removed (due to deprecation of
  MBEDTLS_TLS_DEFAULT_ALLOW_SHA1_IN_KEY_EXCHANGE), there are the existing
  `ssl_cert_rsa.py` and `sslcontext_server_client.py` tests which do very
  similar, simple SSL data transfer.

- Tests now use an EC key by default (they are smaller and faster), and the
  RSA key has been regenerated due to the old PKCS encoding used by openssl
  rsa command, see
  https://stackoverflow.com/questions/40822328/openssl-rsa-key-pem-and-der-conversion-does-not-match
  (and `tests/README.md` has been updated accordingly).

Signed-off-by: Carlos Gil <carlosgilglez@gmail.com>
2024-01-30 11:08:46 +11:00
Damien George
0c81ffd31a tests/multi_net: Generate smaller certs with 2048-bit RSA.
Otherwise running the tests can take a long time when the server is a slow
target (RP2040 takes 23 seconds for a handshake when using 4096-bit RSA).

Also add instructions on how to generate elliptic curve key/certs.

Signed-off-by: Damien George <damien@micropython.org>
2024-01-05 11:34:01 +11:00
Carlosgg
f3f215e9bd extmod/modssl_mbedtls: Add SSLContext certificate methods.
This commit adds:

1) Methods to SSLContext class that match CPython signature:

	- `SSLContext.load_cert_chain(certfile, keyfile)`
	- `SSLContext.load_verify_locations(cafile=, cadata=)`
	- `SSLContext.get_ciphers()` --> ["CIPHERSUITE"]
	- `SSLContext.set_ciphers(["CIPHERSUITE"])`

2) `sslsocket.cipher()` to get current ciphersuite and protocol
   version.

3) `ssl.MBEDTLS_VERSION` string constant.

4) Certificate verification errors info instead of
   `MBEDTLS_ERR_X509_CERT_VERIFY_FAILED`.

5) Tests in `net_inet` and `multi_net` to test these new methods.

`SSLContext.load_cert_chain` method allows loading key and cert from disk
passing a filepath in `certfile` or `keyfile` options.

`SSLContext.load_verify_locations`'s `cafile` option enables the same
functionality for ca files.

Signed-off-by: Carlos Gil <carlosgilglez@gmail.com>
2023-12-12 16:25:07 +11:00
Angus Gratton
dd8a69b5f2 tests/README: Document ./run-internalbench.py.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-09-29 15:41:28 +10:00
Angus Gratton
ad308bc322 tests: Add an explanation of run-perfbench.py.
Also changes this file to a Markdown file.

Signed-off-by: Angus Gratton <gus@projectgus.com>
2022-06-28 14:21:41 +10:00