This commit implements the 'e' half-float format: 10-bit mantissa, 5-bit
exponent. It uses native _Float16 if supported by the compiler, otherwise
uses custom bitshifting encoding/decoding routines.
Signed-off-by: Matthias Urlichs <matthias@urlichs.de>
Signed-off-by: Damien George <damien@micropython.org>
So that the MicroPython-specific behaviour can be isolated, and the CPython
compatible test don't need a .exp file.
Signed-off-by: Damien George <damien@micropython.org>
Add `pop()`, `appendleft()`, and `extend()` methods, support iteration
and indexing, and initializing from an existing sequence.
Iteration and indexing (subscription) have independent configuration flags
to enable them. They are enabled by default at the same level that
collections.deque is enabled (the extra features level).
Also add tests for checking new behavior.
Signed-off-by: Damien George <damien@micropython.org>
This new DMA API corrects possible cache coherency issues on chips with
D-Cache, when working with buffers at arbitrary memory locations (i.e.
supplied by Python code).
The API is used by SPI to fix an issue with corrupt data when reading from
SPI using DMA in certain cases. A regression test is included (it depends
on external hardware connection).
Explanation:
1) It's necessary to invalidate D-Cache after a DMA RX operation completes
in case the CPU reads (or speculatively reads) from the DMA RX region
during the operation. This seems to have been the root cause of issue
#13471 (only when src==dest for this case).
2) More generally, it is also necessary to temporarily mark the first and
last cache lines of a DMA RX operation as "uncached", in case the DMA
buffer shares this cache line with unrelated data. The CPU could
otherwise write the other data at any time during the DMA operation (for
example from an interrupt handler), creating a dirty cache line that's
inconsistent with the DMA result.
Fixes issue #13471.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
This test cannot run on boards that have a native USB REPL, so rename it to
indicate that its "special". This makes it easier to run a subset of
tests, for example:
./run-multitests.py multi_bluetooth/ble*.py
./run-multitests.py multi_bluetooth/perf*.py
./run-multitests.py multi_bluetooth/stress*.py
Signed-off-by: Damien George <damien@micropython.org>
When the websocket closes currently, it does not send a proper
"close"-frame, but rather encodes the 0x8800-sequence inside a binary
packet, which is wrong. The close packet is a different kind of websocket
frame, according to https://www.rfc-editor.org/rfc/rfc6455.
This change resolves an error in Firefox when the websocket closes.
Signed-off-by: Felix Dörre <felix@dogcraft.de>
This is now easy to support, since the first machine-word of a native
function tells how to find the prelude, from which the function name can be
extracted in the same way as for bytecode.
Signed-off-by: Damien George <damien@micropython.org>
Similar to the previous commit but for MP_BLUETOOTH_IRQ_GATTC_READ_DONE:
the pending_value_handle needs to be reset before calling
mp_bluetooth_gattc_on_read_write_status(), which will call the Python IRQ
handler, which may in turn call back into BTstack to perform an action like
a write. In that case the pending_value_handle will need to be available
for the write/read/etc to proceed.
Fixes issue #13634.
Signed-off-by: Damien George <damien@micropython.org>
The pending_value_handle needs to be freed and reset before calling
mp_bluetooth_gattc_on_read_write_status(), which will call the Python IRQ
handler, which may in turn call back into BTstack to perform an action like
a write. In that case the pending_value_handle will need to be available
for the write/read/etc to proceed.
Fixes issue #13611.
Signed-off-by: Damien George <damien@micropython.org>
If a return is executed within the try block of a try-finally then the
return value is stored on the top of the Python stack during the execution
of the finally block. In this case the Python stack is one larger than it
normally would be in the finally block.
Prior to this commit, the compiler was not taking this case into account
and could have a Python stack overflow if the Python stack used by the
finally block was more than that used elsewhere in the function. In such
a scenario the last argument of the function would be clobbered by the
top-most temporary value used in the deepest Python expression/statement.
This commit fixes that case by making sure enough Python stack is allocated
to the function.
Fixes issue #13562.
Signed-off-by: Damien George <damien@micropython.org>
Prior to this commit it would skip every second cipher returned from
mbedtls.
The corresponding test is also updated and now passes on esp32, rp2, stm32
and unix.
Signed-off-by: Damien George <damien@micropython.org>
Adds support to asyncio.gather() for the case that one or more (or all)
sub-tasks finish and/or raise an exception before the gather starts.
Signed-off-by: Damien George <damien@micropython.org>
Similar to the previous commit but for MP_BLUETOOTH_IRQ_GATTC_READ_DONE:
the pending_value_handle needs to be reset before calling
mp_bluetooth_gattc_on_read_write_status(), which will call the Python IRQ
handler, which may in turn call back into BTstack to perform an action like
a write. In that case the pending_value_handle will need to be available
for the write/read/etc to proceed.
Fixes issue #13634.
Signed-off-by: Damien George <damien@micropython.org>
The pending_value_handle needs to be freed and reset before calling
mp_bluetooth_gattc_on_read_write_status(), which will call the Python IRQ
handler, which may in turn call back into BTstack to perform an action like
a write. In that case the pending_value_handle will need to be available
for the write/read/etc to proceed.
Fixes issue #13611.
Signed-off-by: Damien George <damien@micropython.org>
The current `ssl` module has quite a few differences to the CPython
implementation. This change moves the MicroPython variant to a new `tls`
module and provides a wrapper module for `ssl` (in micropython-lib).
Users who only rely on implemented comparible behavior can continue to use
`ssl`, while users that rely on non-compatible behavior should switch to
`tls`. Then we can make the facade in `ssl` more strictly adhere to
CPython.
Signed-off-by: Felix Dörre <felix@dogcraft.de>
By moving to GitHub actions, all MicroPython CI builds are now on GitHub
actions. This allows faster parallel builds and saves time by not building
when no relevant files changed.
This reveals a few failing tests, so those are temporarily disabled until
they can be fixed.
Signed-off-by: David Lechner <david@pybricks.com>
Signed-off-by: Damien George <damien@micropython.org>
If a return is executed within the try block of a try-finally then the
return value is stored on the top of the Python stack during the execution
of the finally block. In this case the Python stack is one larger than it
normally would be in the finally block.
Prior to this commit, the compiler was not taking this case into account
and could have a Python stack overflow if the Python stack used by the
finally block was more than that used elsewhere in the function. In such
a scenario the last argument of the function would be clobbered by the
top-most temporary value used in the deepest Python expression/statement.
This commit fixes that case by making sure enough Python stack is allocated
to the function.
Fixes issue #13562.
Signed-off-by: Damien George <damien@micropython.org>
The timing of the DMA transfer can vary a bit, so tweak the allowed values.
Also test the return value of `rp2.DMA.irq.flags()` to make sure the IRQ is
correctly signalled.
Signed-off-by: Damien George <damien@micropython.org>
Changes include:
- Some mbedtls source files renamed or deprecated.
- Our `mbedtls_config.h` files are renamed to `mbedtls_config_port.h`, so
they don't clash with mbedtls's new default configuration file named
`mbedtls_config.h`.
- MBEDTLS_TLS_DEFAULT_ALLOW_SHA1_IN_KEY_EXCHANGE is deprecated.
- MBEDTLS_HAVE_TIME now requires an `mbedtls_ms_time` function to be
defined but it's only used for TLSv1.3 (currently not enabled in
MicroPython so there is a lazy implementation, i.e. seconds * 1000).
- `tests/multi_net/ssl_data.py` is removed (due to deprecation of
MBEDTLS_TLS_DEFAULT_ALLOW_SHA1_IN_KEY_EXCHANGE), there are the existing
`ssl_cert_rsa.py` and `sslcontext_server_client.py` tests which do very
similar, simple SSL data transfer.
- Tests now use an EC key by default (they are smaller and faster), and the
RSA key has been regenerated due to the old PKCS encoding used by openssl
rsa command, see
https://stackoverflow.com/questions/40822328/openssl-rsa-key-pem-and-der-conversion-does-not-match
(and `tests/README.md` has been updated accordingly).
Signed-off-by: Carlos Gil <carlosgilglez@gmail.com>
This can perform arbitrary channel mixing between two images.
Alpha blend & maximum functions are demonstrated in the test.
However, it should make most of the usual photo editing blends
possible. (for dissolve, fill a mask bitmap with random values,
which may be expensive to do from circuitpython code; we can
specifically accelerate it if we need to)
Prior to this commit it would skip every second cipher returned from
mbedtls.
The corresponding test is also updated and now passes on esp32, rp2, stm32
and unix.
Signed-off-by: Damien George <damien@micropython.org>
These were added back in commit c4935f3049
because the tests required CPython 3.8, which was quite new at the time.
But CPython 3.8 was released over 4 years ago (October 2019) and the CI
test runners, and developers, have this (or a more recent) CPython version.
Removing the .exp files also helps keep MicroPython semantics the same as
CPython.
The asyncio_fair.py test it adjusted slightly to have more deterministic
timing and output.
Signed-off-by: Damien George <damien@micropython.org>
The get, set and del item methods didn't correctly lookup the value
from the parent native instance because the functions took the type
from the instance.
Fixes#8758
Adds support to asyncio.gather() for the case that one or more (or all)
sub-tasks finish and/or raise an exception before the gather starts.
Signed-off-by: Damien George <damien@micropython.org>
morph9 is a form of morph which performs 9 different convolutions,
like a version of mix where each coefficient is a (2n+1)x(2n+1) matrix.
Most use cases are covered by morph-then-mix, but some advanced operations
may be more efficient to implement via morph9.
Compare the full absolute path instead of relying on the path form
passed by the user.
For instance, this will make
python3 run-tests.py -d basics
python3 run-tests.py -d ./basics
python3 run-tests.py -d ../tests/basics
python3 run-tests.py -d /full/path/to/basics
all behave the same by correctly treating the bytes_compare3 and
builtin_help tests as special, whereas previously only the first
invocation would do that and hence result in these tests to fail
when called with a different path form.
Signed-off-by: stijn <stijn@ignitron.net>
This allows operations between channels in an image. It can be used for
the following use cases:
* Conversion to B&W or sepia
* Adding color casts
* Mixing or swapping arbitrary channels
* Inverting or scaling arbitrary channels
bitmapfilter.morph is taken from openmv's imlib.
It is substantially faster than blur/sharpen implemented in ulab,
by up to 10x. It also avoids making many allocations.
Codespell doesn't pick up "re-used" or "re-uses", and ignores the tests/
directory, so fix these manually.
Signed-off-by: Damien George <damien@micropython.org>
Implement the typical 're-run the failed tests' most test runners have, for
convenience. Accessible via the new --run-failures argument, and
implemented using a json file containing a list of the failed tests.
Signed-off-by: stijn <stijn@ignitron.net>
Otherwise running the tests can take a long time when the server is a slow
target (RP2040 takes 23 seconds for a handshake when using 4096-bit RSA).
Also add instructions on how to generate elliptic curve key/certs.
Signed-off-by: Damien George <damien@micropython.org>
This matches the behaviour of run-tests.py, which sets cwd to the directory
containing the test script, which helps to isolate the filesystem.
It means that the SSL tests no longer need to know the name of their
containing directory to find the certificate files, and helps to run these
tests on bare-metal.
Signed-off-by: Damien George <damien@micropython.org>
The existing thread_sleep1.py test only tests execution, not accuracy, of
time.sleep. Also the existing test only tests sleep(0) on targets like rp2
that can only create a single thread.
The new test in this commit checks for timing accuracy on the main thread
and one other thread when they run at the same time.
Signed-off-by: Damien George <damien@micropython.org>
The aim of this commit is to make it so that the existing thread tests can
be used to test the _thread module on the rp2 port. The rp2 port only
allows up to one thread to be created at a time, and does not have the GIL
enabled.
The following changes have been made:
- run-tests.py skips mutation tests on rp2, because there's no GIL.
- run-tests.py skips other tests on rp2 that require more than one thread.
- The tests stop trying to start a new thread after there is an OSError,
which indicates that the system cannot create more threads.
- Some of these tests also now run the test function on the main thread,
not just the spawned threads.
- In some tests the output printing is adjusted so it's the same regardless
of how many threads were spawned.
- Some time.sleep(1) are replaced with time.sleep(0) to make the tests run
a little faster (finish sooner when the work is done).
For the most part the tests are unchanged for existing platforms like esp32
and unix.
Signed-off-by: Damien George <damien@micropython.org>
The poll_obj_t instances have their pollfd field point into this
allocation. So if re-allocating results in a move, we need to update the
existing poll_obj_t's.
Update the test to cover this case.
Fixes issue #12887.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This function in standard Python is a building block for custom REPLs:
```python
from codeop import compile_command
print("Repl in (Circuit-)Python")
ns = {}
PS1="<<< "
PS2=",,, "
command = ""
while True:
line = input(PS2 if command else PS1)
if command:
command = command + "\n" + line
else:
command = line
try:
if (code := compile_command(command)):
command = ""
exec(code, ns)
except Exception as e:
command = ""
print(e)
```
This adds asyncio ssl support with SSLContext and the corresponding
tests in `tests/net_inet` and `tests/multi_net`.
Note that not doing the handshake on connect will delegate the handshake to
the following `mbedtls_ssl_read/write` calls. However if the handshake
fails when a client certificate is required and not presented by the peer,
it needs to be notified of this handshake error (otherwise it will hang
until timeout if any). Finally at MicroPython side raise the proper
mbedtls error code and message.
Signed-off-by: Carlos Gil <carlosgilglez@gmail.com>
Changes are:
- use ssl.SSLContext.wrap_socket instead of ssl.wrap_socket
- disable check_hostname and call load_default_certs() where appropriate,
to get CPython to run the tests correctly
- pass socket.AF_INET to getaddrinfo and socket.socket(), to force IPv4
- change tests to use github.com instead of google.com, because certificate
validation was failing with google.com
Signed-off-by: Damien George <damien@micropython.org>
And only enable this method when the relevant feature is available in
mbedtls. Otherwise, if mbedtls doesn't support getting the peer
certificate, this method always returns None and it's confusing why it does
that. It's better to remove the method altogether, so the error trying to
use it is more obvious.
Signed-off-by: Damien George <damien@micropython.org>
This commit adds:
1) Methods to SSLContext class that match CPython signature:
- `SSLContext.load_cert_chain(certfile, keyfile)`
- `SSLContext.load_verify_locations(cafile=, cadata=)`
- `SSLContext.get_ciphers()` --> ["CIPHERSUITE"]
- `SSLContext.set_ciphers(["CIPHERSUITE"])`
2) `sslsocket.cipher()` to get current ciphersuite and protocol
version.
3) `ssl.MBEDTLS_VERSION` string constant.
4) Certificate verification errors info instead of
`MBEDTLS_ERR_X509_CERT_VERIFY_FAILED`.
5) Tests in `net_inet` and `multi_net` to test these new methods.
`SSLContext.load_cert_chain` method allows loading key and cert from disk
passing a filepath in `certfile` or `keyfile` options.
`SSLContext.load_verify_locations`'s `cafile` option enables the same
functionality for ca files.
Signed-off-by: Carlos Gil <carlosgilglez@gmail.com>
Previously, negative amplitudes were clamped to zero.
Now, they are allowed to range from -ALMOST_ONE to +ALMOST_ONE.
This is useful in certain circumstances, such as using synthio
to create CV-like outputs that can be positive or negative, by
using the amplitude property of the note.
Otherwise passing in a non-integer can lead to an invalid memory access.
Thanks to Junwha Hong and Wonil Jang @S2Lab, UNIST for finding the issue.
Fixes issue #13007.
Signed-off-by: Damien George <damien@micropython.org>
Set the position of new line tokens as the end of the preceding line
instead of the beginning of the next line. This is done by first moving
the pointer to the end of the current line to skip any whitespace, record
the position for the token, then finaly skip any other line and whitespace.
The previous behavior was to skip every new line and whitespace, including
the indent of the next line, before recording the token position.
(Note that both lex->emit_dent and lex->nested_bracket_level equal 0 if
had_physical_newline == true, which allows simplifying the if-logic for
MP_TOKEN_NEWLINE.)
And update the cmd_parsetree.py test expected output, because the position
of the new-line token has changed.
Fixes issue #12792.
Signed-off-by: Mathieu Serandour <mathieu.serandour@numworks.fr>
This updates a small number of files that change with ruff-format's (vs
black's) rules.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This test depends on the order in which qstrs are stored in ROM, which
affects the order in which `dir()` will probe the object to see what it
supports. Because of the lazy-loading in asyncio/__init__.py, if it
tries to do e.g. `wait_for_ms` before `funcs` then it will import funcs,
making `funcs` later succeed. But in the other way around, `funcs` will
initially not be found.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
These tests are designed to measure changes in performance relating to:
- string interning / searching for existing strings
- map lookup
- string operations
- string hashing
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The unwritten API contract expected of a VFS.getcwd() by mp_vfs_getcwd()
is that its return value should be either "" or "/" when the CWD is at
the root of the VFS and otherwise start with a slash and not end with a
slash. This was not correctly implemented in VfsPosix for instances with
a non-empty root - the required leading slash, if any, was cut off
because the root length includes a trailing slash. This would result in
missing slashes in the middle of the return value of os.getcwd() or in
uninitialized garbage from beyond a string's null terminator when the
CWD was at the VFS root.
Signed-off-by: Christian Walther <cwalther@gmx.ch>
The unwritten API contract expected of a VFS by mp_vfs_lookup_path() is
that paths passed in are relative to the root of the VFS if they start
with '/' and relative to the current directory of the VFS otherwise.
This was not correctly implemented in VfsPosix for instances with a
non-empty root - all paths were interpreted relative to the root. Fix
that. Since VfsPosix tracks its CWD using the "external" CWD of the Unix
process, the correct handling for relative paths is to pass them through
unmodified.
Also, when concatenating absolute paths, fix an off-by-one resulting in
a harmless double slash (the root path already has a trailing slash).
Signed-off-by: Christian Walther <cwalther@gmx.ch>
These tests test an unrealistic situation and only pass by accident due
to a bug. The upcoming fix for the bug would make them fail.
The unrealistic situation is that VfsPosix methods are called with
relative paths while the current working directory is somewhere outside
of the root of the VFS. In the intended use of VFS objects via
os.mount() (as opposed to calling methods directly as the tests do),
this never happens, as mp_vfs_lookup_path() directs incoming calls to
the VFS that contains the CWD.
Make the testing situation realistic by changing the working directory
to the root of the VFS before calling methods on it, as the subsequent
relative path accesses expect.
Thanks to the preceding commit, the tests still pass, but still for the
wrong reason. The following commit "Fix relative paths on non-root VFS"
will make them pass for the correct reason.
Signed-off-by: Christian Walther <cwalther@gmx.ch>
A VfsPosix created with a relative root path would get confused when
chdir() was called on it and become unable to properly resolve absolute
paths, because changing directories effectively shifted its root. The
simplest fix for that would be to say "don't do that", but since the
unit tests themselves do it, fix it by making a relative path absolute
before storing it.
Signed-off-by: Christian Walther <cwalther@gmx.ch>
This wasn't correctly accounting for the bits-per-pixel and was returning a
bufinfo struct with the incorrect length. Instead, just forward directly
to the underlying buffer object.
Fixes issue #12563.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This ensures that the buffer is large enough for the specified width,
height, bits-per-pixel, and stride.
Also makes the legacy FrameBuffer1 constructor re-use the FrameBuffer
make_new to save some code size.
Fixes issue #12562.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This commit adds tests for bound method comparison and hashing to support
the changes in the previous commit.
Signed-off-by: Ned Konz <ned@productcreationstudio.com>
If a non-string buffer was passed to execfile, then it would be passed
as a non-null-terminated char* to mp_lexer_new_from_file.
This changes mp_lexer_new_from_file to take a qstr instead (as in almost
all cases a qstr will be created from this input anyway to set the
`__file__` attribute on the module).
This now makes execfile require a string (not generic buffer) argument,
which is probably a good fix to make anyway.
Fixes issue #12522.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
"Raise SomeException() from None" is a common Python idiom to suppress
chained exceptions and thus shouldn't trigger a warning on a version of
Python that doesn't support them in the first place.
The following code:
server = await asyncio.start_server(...)
async with server:
... code that raises ...
would lose the original exception because the server's task would not have
had a chance to be scheduled yet, and so awaiting the task in wait_closed
would raise the cancellation instead of the original exception.
Additionally, ensures that explicitly cancelling the parent task delivers
the cancellation correctly (previously was masked by the server loop), now
this only happens if the server was closed, not when the task was
cancelled.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
1. Remove the skip for detecting support for polling user-defined objects
as this is always possible now on all ports.
2. Don't print when the scheduled task runs as the ordering of this
relative to the other prints is dependent on other factors (e.g. if
using the native emitter).
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Update expired certificate, increase time validity period to five years and
fix command arguments typos in commentaries.
Signed-off-by: Carlos Gil <carlosgilglez@gmail.com>
