Commit graph

17390 commits

Author SHA1 Message Date
0b0802c483 objint_mpz: Fix pow3(1,2,0).
This finding is based on fuzzing micropython.
I manually minimized the test case it provided.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-25 06:31:00 -05:00
ab620f4084 py/mpprint: Fix printing pointers with upper bit set.
On a build like nanbox, `mp_uint_t` is wider than `u/intptr_t`.  Using a
signed type for fetching pointer values resulted in erroneous results: like
`<function f at 0xfffffffff7a60bc0>` instead of
`<function f at 0xf7a60bc0>`.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-25 11:01:42 +10:00
87b7a9d734 stm32: Add casts when printing small integers.
All these arguments are of type `mp_{u,}int_t`, but the actual value is
always a small integer.  Cast it so that it can format with the `%d/%u`
formatter.

Before, the compiler plugin produced an error in the PYBD_SF6 build, which
is a nanboxing build with 64-bit ints.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-25 11:01:16 +10:00
ee4f27affa py/objcell: Fix printing of cell ID/pointer.
On the nanbox build, `o->obj` is a 64-bit type but `%p` formats a 32-bit
type, leading to undefined behavior.

Print the cell's ID as a hex integer instead.

This location was found using an experimental gcc plugin for `mp_printf`
error checking.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-25 11:00:41 +10:00
4495610f8d shared/netutils: Cast the ticks value before printing.
Before, the compiler plugin produced an error in the PYBD_SF6 build, which
is a nanboxing build with 64-bit ints.

I made the decision here to cast the value even though some significant
bits might be lost after 49.7 days.  However, the format used is "% 8d",
which produces a consistent width output for small ticks values (up to
about 1.1 days).  I judged that it was more valuable to preserve the fixed
width display than to accurately represent long time periods.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-25 11:00:08 +10:00
2d93909ebe extmod/modlwip: Print timeout with correct format string.
As timeout is of type `mp_uint_t`, it must be printed with UINT_FMT.

Before, the compiler plugin produced an error in the PYBD_SF6 build, which
is a nanboxing build with 64-bit ints.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-25 10:59:51 +10:00
18a835e9b4 examples/usercmodule: Cast arguments for printf.
These locations were found using an experimental gcc plugin for `mp_printf`
error checking.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-25 10:59:28 +10:00
338ca3b68f unix/coverage: Remove unused printf arguments.
Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-25 10:59:17 +10:00
aa9152ae0c unix/coverage: Provide argmuents of expected integer types.
Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-25 10:59:04 +10:00
db7e935917 unix/coverage: Cast values to int for format printing.
During the coverage test, all the values encountered are within the range
of `%d`.

These locations were found using an experimental gcc plugin for `mp_printf`
error checking.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-25 10:58:24 +10:00
61006d8016 unix/coverage: Cast values to fit %x formatting code.
This fixes the following diagnostic produced by the plugin:

    error: argument 3: Format ‘%x’ requires a ‘int’ or
        ‘unsigned int’ (32 bits), not ‘long unsigned int’ [size 64]
        [-Werror=format=]

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-25 10:57:59 +10:00
a06857a11a unix/coverage: Cast type names to qstr explicitly.
Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-25 10:57:54 +10:00
a1a8eacdce unix/coverage: Avoid type checking an invalid string.
We still want this not to crash a runtime but the new static checker
wouldn't like it.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-25 10:57:39 +10:00
d0d111356f py: Fix mp_printf integer size mismatches.
The type of the argument must match the format string.  Add casts to ensure
that they do.

It's possible that casting from `size_t` to `unsigned` loses the correct
values by masking off upper bits, but it seems likely that the quantities
involved in practice are small enough that the `%u` formatter (32 bits on
most platforms, 16 on pic16bit) will in fact hold the correct value.

The alternative, casting to a wider type, adds code size.

These locations were found using an experimental gcc plugin for `mp_printf`
error checking, cross-building for x64 windows on Linux.

In one case there was already a cast, but it was written incorrectly and
did not have the intended effect.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-25 10:56:57 +10:00
519cba4d05 py: Cast type names to qstr explicitly.
The name field of type objects is of type `uint16_t` for efficiency, but
when the type is passed to `mp_printf` it must be cast explicitly to type
`qstr`.

These locations were found using an experimental gcc plugin for `mp_printf`
error checking, cross-building for x64 windows on Linux.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-25 10:56:02 +10:00
7493275918 py/mpconfig,ports: Define new HEX_FMT formatting macro.
Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-25 10:55:28 +10:00
0c8d35b322 ports: Eliminate define of {U,}INT_FMT where redundant.
The default definition in `py/mpconfig.h` for 32-bit architectures is
`%u/%d`, so these can be removed.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-25 10:54:27 +10:00
Angus Gratton
096ff8b9ee tests/micropython: Rename viper boundary tests that depend on big int.
These tests all depend on generating arbitrarily long (>64-bit) integers.

It would be possible to have these tests work in this case I think, as the
results are always masked to shorter values.  But quite fiddly.  So just
rename them so they are automatically skipped if the target doesn't have
big int support.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-07-24 15:50:34 +10:00
Angus Gratton
5d9ef6bfb6 py/objint_longlong: Fix left shift of negative values.
Previous comment was wrong, left shifting a negative value is UB in C. Use
the same approach as small int shifts (from runtime.c).

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-07-24 15:50:00 +10:00
Damien George
3185bb5827 py/obj: Add new type flag to indicate subscr accepts slice-on-stack.
The recently merged 5e9189d6d1 now allows
temporary slices to be allocated on the C stack, which is much better than
allocating them on the GC heap.

Unfortunately there are cases where the C-allocated slice can escape and be
retained as an object, which leads to crashes (because that object points
to the C stack which now has other values on it).

The fix here is to add a new `MP_TYPE_FLAG_SUBSCR_ALLOWS_STACK_SLICE`.
Native types should set this flag if their subscr method is guaranteed not
to hold on to a reference of the slice object.

Fixes issue #17733 (see also #17723).

Signed-off-by: Damien George <damien@micropython.org>
2025-07-24 14:55:34 +10:00
Damien George
45aa65b67d webassembly/objjsproxy: Fix binding of self to JavaScript methods.
Fixes a bug in the binding of self/this to JavaScript methods.

The new semantics match Pyodide's behaviour, at least for the included
tests.

Signed-off-by: Damien George <damien@micropython.org>
2025-07-24 14:38:08 +10:00
Damien George
9b61bb93f9 webassembly/proxy_c: Provide constants for fixed JsProxy refs.
Signed-off-by: Damien George <damien@micropython.org>
2025-07-24 14:37:30 +10:00
Damien George
f6e23fdef1 tools/ci.sh: Test building all natmod examples with all ARM-M archs.
And run both armv6m and armv7m under qemu.

Signed-off-by: Damien George <damien@micropython.org>
2025-07-24 14:01:45 +10:00
Damien George
e750ecff70 qemu/Makefile: Allow passing flags to test_natmod via RUN_TESTS_EXTRA.
Signed-off-by: Damien George <damien@micropython.org>
2025-07-24 14:01:45 +10:00
Damien George
bba15e0a0b examples/natmod: Use LINK_RUNTIME=1 when building for armv6m.
To get division helper functions, eg `__aeabi_uidiv`, `__aeabi_idiv` and
`__aeabi_uidivmod`.

Signed-off-by: Damien George <damien@micropython.org>
2025-07-24 14:01:45 +10:00
Phil Howard
cbc6aed8fa lib/pico-sdk: Fix Pico SDK fetching develop picotool.
SDK 2.1.1 shipped with PICOTOOL_FETCH_FROM_GIT configured to fetch the
"develop" branch. This broke downstream CI, which was trusting Pico
SDK to fetch the correct version.

RPi have added a "2.1.1-correct-picotool" tag which fixes this.

lib/pico-sdk: Bump to "2.1.1-correct-picotool" tag.

Signed-off-by: Phil Howard <github@gadgetoid.com>
2025-07-24 13:51:02 +10:00
Andrew Leech
bc77b27bad unix/mpthreadport: Ensure consistent type of PTHREAD_STACK_MIN.
It seems GCC 14 got stricter with warn/errs like -Wsign-compare and types a
"bare number" as a long int that can't be compared to a (unsigned) size_t.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
2025-07-24 13:29:36 +10:00
Damien George
7729e80fdd all: Go back to using default ruff quote style.
Commit dc2fcfcc55 seems to have accidentally
changed the ruff quote style to "preserve", instead of keeping it at the
default which is "double".

Put it back to the default and update relevant .py files with this rule.

Signed-off-by: Damien George <damien@micropython.org>
2025-07-24 12:48:18 +10:00
Damien George
6a4306a0df unix/mpconfigport: Include time.h to get definition of time_t.
Without this there's a build error on macOS (at least).  This was likely
due to a combination of 9b7d85227e and
df05caea6c.

Signed-off-by: Damien George <damien@micropython.org>
2025-07-24 11:42:53 +10:00
Yoctopuce dev
8b3439e26c unix/variants/longlong: Use REPR_C on this variant.
There is currently no build using REPR_C in the unix CI tests.  As
discussed in PR #16953, this is something that combines well with the
longlong build.

Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
2025-07-24 11:07:30 +10:00
Yoctopuce dev
d6876e2273 py/obj: Fix REPR_C bias toward zero.
Current implementation of REPR_C works by clearing the two lower bits of
the mantissa to zero.  As this happens after each floating point operation,
this tends to bias floating point numbers towards zero, causing decimals
like .9997 instead of rounded numbers.  This is visible in test cases
involving repeated computations, such as `tests/misc/rge_sm.py` for
instance.

The suggested fix fills in the missing bits by copying the previous two
bits.  Although this cannot recreate missing information, it fixes the bias
by inserting plausible values for the lost bits, at a relatively low cost.

Some float tests involving irrational numbers have to be softened in case
of REPR_C, as the 30 bits are not always enough to fulfill the expectations
of the original test, and the change may randomly affect the last digits.
Such cases have been made explicit by testing for REPR_C or by adding a
clear comment.

The perf_test fft code was also missing a call to round() before casting a
log_2 operation to int, which was causing a failure due to a last-decimal
change.

Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
2025-07-24 11:07:30 +10:00
59ee59901b extmod/vfs_posix: Add MICROPY_VFS_POSIX_WRITABLE option.
When this configuration flag is set, VfsPosix instances can be written.
Otherwise, they will always be created "read only".

This flag is useful when fuzzing micropython: Without VfsPosix, the fuzzing
input script cannot be read; but with writable VfsPosix, fuzzing scripts
can potentially perform undesired operations on the host filesystem.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-24 10:39:41 +10:00
3c603f7baf extmod/vfs_posix: Add additional readonly checks.
Otherwise operations such as unlink can be performed on a nominally
read-only VfsPosix.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-24 10:39:25 +10:00
d79000df70 tests/extmod: Add (failing) test for VfsPosix in readonly mode.
I noticed that operations such as unlink could be performed on a nominally
read-only VfsPosix.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-24 10:39:23 +10:00
Andrew Leech
6515cd05f1 extmod/vfs_lfsx: Allow overriding the LFS2 on-disk version format.
Back in LFS2 version 2.6 they updated the on-disk version from 2.0 to 2.1
which broke back compatibility (aka older versions could no long read new
version disk format), see
https://github.com/littlefs-project/littlefs/releases/tag/v2.6.0

Then in LFS2 v2.7 an optional `config->disk_version` was added to force the
library to use an older disk format instead, see:
https://github.com/littlefs-project/littlefs/releases/tag/v2.7.0

This commit simply exposes `config->disk_version` as a compile time option
if LFS2_MULTIVERSION is set, otherwise there is no change in behavior.
This is Useful for compatibility with older LFS versions.

Note: LFS2_MULTIVERSION needs to be defined at the make / CFLAGS level,
setting it in mpconfigboard.h doesn't work as it's not included in the
`lfs2.c` file in any way.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
2025-07-23 15:51:08 +10:00
Angus Gratton
28082d1d25 extmod/mbedtls: Undefine ARRAY_SIZE if defined by platform.
This is an annoying regression caused by including mpconfig.h in 36922df -
the mimxrt platform headers define ARRAY_SIZE and mbedtls also defines in
some source files, using a different parameter name which is a warning in
gcc.

Technically mimxrt SDK is to blame here, but as this isn't a named warning
in gcc the only way to work around it in the mimxrt port would be to
disable all warnings when building this particular mbedTLS source file.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-07-23 15:47:17 +10:00
Angus Gratton
89f9ee9d7c tests/multi_net: Update DTLS multi-net test.
The original version of this test had to exchange a 1 byte UDP packet
before the DTLS handshake. This is no longer needed due to MSG_PEEK
support.

The test also doesn't work with HelloVerify enabled, as the first
connection attempt always fails with an
MBEDTLS_ERR_SSL_HELLO_VERIFY_REQUIRED result. Anticipate this by listening
for the client twice on the server side.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-07-23 15:47:16 +10:00
Angus Gratton
9b7d85227e extmod/mbedtls: Implement recommended DTLS features, make optional.
- DTLS spec recommends HelloVerify and Anti Replay protection be enabled,
  and these are enabled in the default mbedTLS config. Implement them here.

- To help compensate for the possible increase in code size, add a
  MICROPY_PY_SSL_DTLS build config macro that's enabled for EXTRA and
  above by default.

This allows bare metal mbedTLS ports to use DTLS with HelloVerify support.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-07-23 15:47:16 +10:00
Angus Gratton
41e0ec96cb extmod/mbedtls: Implement DTLS HelloVerify cookie support.
This is already enabled in the ESP-IDF mbedTLS config, so provide an
implementation of the cookie store functions. This allows DTLS connections
between two esp32 boards.

The session cookie store is a very simple dictionary associated with the
SSLContext. To work, the server needs to reuse the same SSLContext (but
cookies are never cleaned up, so a server with a high number of clients
should recycle the context periodically.)

Server code still needs to handle the MBEDTLS_ERR_SSL_HELLO_VERIFY_REQUIRED
error by waiting for the next UDP packet from the client.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-07-23 15:47:16 +10:00
Damien George
c6423d5d8e tests/multi_bluetooth: Synchronise MTU exchange in BLE MTU tests.
With the recent update to ESP-IDF 5.4.2, there is a change in BLE event
behaviour which makes `tests/multi_bluetooth/ble_mtu.py` and
`tests/multi_bluetooth/ble_mtu_peripheral.py` now fail on ESP32 with IDF
5.4.2.

The change in behaviour is that MTU_EXCHANGE events can now occur before
CENTRAL_CONNECT/PERIPHERAL_CONNECT events.  That seems a bit strange,
because the MTU exchange occurs after the connection.  And looking at the
timing of the events there is exactly 100ms between them, ie MTU_EXCHANGE
fires and then exactly 100ms later CENTRAL_CONNECT/PERIPHERAL_CONNECT
fires.

It's unknown if this is a bug in (Espressif's) NimBLE, a subtle change in
scheduling with still valid behaviour, an intended change, a change allowed
under the BLE spec, or something else.

But in order to move forward with updating to IDF 5.4.2, the relevant tests
have been adjusted so they can pass.  The test just needs to wait a bit
between doing the connect and doing the MTU exchange, so the other side
sees the original/correct ordering of events.  This wait is done using the
multitest synchronisation primitives (broadcast and wait).

Signed-off-by: Damien George <damien@micropython.org>
2025-07-23 12:53:28 +10:00
Damien George
10ef3e4ac2 esp32: Update to use ESP-IDF v5.4.2.
This is a patch release of the IDF.  Comparing with 5.4.1, firmware size is
up by about 1.5k on ESP32 and 9k on ESP32-S3.  But IRAM usage (of the IDF)
is down by about 500 byte on ESP32 and DRAM usage is down by about 20k on
ESP32 and 10k on ESP32-S3.

Testing on ESP32, ESP32-S2, ESP32-S3 and ESP32-C3 shows no regressions,
except in BLE MTU ordering (the MTU exchange event occuring before the
connect event).

Signed-off-by: Damien George <damien@micropython.org>
2025-07-23 12:52:49 +10:00
Damien George
1ab1f857b3 tests/extmod_hardware/machine_uart_irq_break.py: Remove send_uart.
This is no longer needed, the esp32 port can now pass this test using just
a single UART.

Signed-off-by: Damien George <damien@micropython.org>
2025-07-23 12:51:10 +10:00
Damien George
79b2d4ff22 esp32/machine_uart: Improve sendbreak so it doesn't reconfig the UART.
Currently, `UART.sendbreak()` on esp32 will reconfigure the UART to a
slower baudrate and send out a null byte, to synthesise a break condition.
That's not great because it changes the baudrate of the RX path as well,
which could miss incoming bytes while sending the break.

This commit changes the sendbreak implementation to just reconfigure the TX
pin as GPIO in output mode, and hold the pin low for the required duration.

Signed-off-by: Damien George <damien@micropython.org>
2025-07-23 12:50:29 +10:00
Damien George
941b7e35ac tests/run-tests.py: Use TEST_TIMEOUT as timeout for bare-metal tests.
This parameter is already used for PC-based tests (eg unix and webassembly
ports), and it makes sense for it to be used for bare-metal ports as well.
That way the timeout is configurable for all targets.

Because this increases the default timeout from 10s to 30s, this fixes some
long-running tests that would previously fail due to a timeout such as
`thread/stress_aes.py` on ESP32.

Signed-off-by: Damien George <damien@micropython.org>
2025-07-23 12:42:44 +10:00
Damien George
cc774c3daf tools/pyboard.py: Add timeout argument to Pyboard.exec_/exec.
Signed-off-by: Damien George <damien@micropython.org>
2025-07-23 12:42:44 +10:00
Damien George
b7e734bfb7 tests/net_inet: Update micropython.org certificate for SSL tests.
Signed-off-by: Damien George <damien@micropython.org>
2025-07-23 12:40:45 +10:00
Damien George
8a457b8cf9 tools/ci.sh: Change averaging to 1 for run-perfbench.py test.
The `run-perfbench.py` test is run as part of CI, but the actual
performance results are not used.  Rather, the test is just testing that
all the performance tests run correctly.

So there's no need to run with the default averaging of 8 (which runs each
test 8 times and takes the average time for the performance result) which
can take a lot of time for slower builds, eg unix sanitize, settrace and
stackless builds.

This commit changes the averaging to just 1.

Signed-off-by: Damien George <damien@micropython.org>
2025-07-23 12:39:42 +10:00
Damien George
677a0e0124 docs/reference/speed_python: Document schedule/GIL limitation of native.
Signed-off-by: Damien George <damien@micropython.org>
2025-07-23 11:37:16 +10:00
Damien George
b070765427 tests/thread: Allow thread tests to pass with the native emitter.
The native emitter will not release/bounce the GIL when running code, so
if it runs tight loops then no other threads get a chance to run (if the
GIL is enabled).  So for the thread tests, explicitly include a call to
`time.sleep(0)` (or equivalent) to bounce the GIL and give other threads a
chance to run.

For some tests (eg `thread_coop.py`) the whole point of the test is to test
that the GIL is correctly bounced.  So for those cases force the use of the
bytecode emitter for the busy functions.

Signed-off-by: Damien George <damien@micropython.org>
2025-07-23 11:37:00 +10:00
Damien George
b15065b95e github/workflows: Add new CI job to test unix port with GIL enabled.
Signed-off-by: Damien George <damien@micropython.org>
2025-07-23 11:35:38 +10:00