Commit graph

17516 commits

Author SHA1 Message Date
90e366eefa parse: Don't allow creation of invalid UTF8 strings or identifiers.
.. even when compiling non UTF-8 files or byte strings.

Closes: #17855
Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-08-17 08:59:52 -05:00
a614243deb py: Reduce code size from utf8_check.
All sites immediately threw a UnicodeError, so roll that into
the new function utf8_require.

unicode.c was designed not to require runtime.h, so move the
checking function into objstr.c.

Reduce the number of #if sites by making a do-nothing variant
that is used instead when !STR_UNICODE or !STR_UNICODE_CHECK.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-08-17 08:59:52 -05:00
90012e7d6a tests: Add test of invalid unicode strings.
.. from non UTF-8 inputs. In this case, MicroPython raises
UnicodeError while CPython uses SyntaxError. By catching either
exception, the test does not require an .exp file.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-08-17 08:59:52 -05:00
dependabot[bot]
1588c455c4 github/workflows: Bump actions/checkout from 4 to 5.
Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 5.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-08-16 18:36:57 +10:00
Maureen Helm
afa7265ffa zephyr: Upgrade to Zephyr v4.2.0.
Updates the Zephyr port build instructions and CI to use the latest
Zephyr release tag.

Tested on max32690fthr and frdm_k64f.

Signed-off-by: Maureen Helm <maureen.helm@analog.com>
2025-08-16 15:10:30 +10:00
Thomas Watson
bba3542018 extmod/modlwip: Remove unused include and functions.
The lwIP includes are now port-specific.

The `sys_arch_{,un}protect` functions are not used according to line 34
of `extmod/lwip-include/lwipopts_common.h`.

Neither have been touched in nearly a decade.

Signed-off-by: Thomas Watson <twatson52@icloud.com>
2025-08-16 14:52:52 +10:00
Damien George
b7cfafc1ee alif/alif.mk: Add MPY_CROSS_FLAGS setting.
The HP and HE CPUs have double-precision hardware floating point, so can
use the armv7emdp architecture.

This allows frozen code to use native/viper/asm_thumb decorators.

Fixes issue #17896.

Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 12:45:33 +10:00
Damien George
f0f5abb7a3 alif/mpconfigport: Enable cryptolib and hashlib.md5/sha1.
They are enabled when SSL/mbedTLS is included in the firmware.  These new
features cost around +1400 bytes of code size.

Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 12:44:39 +10:00
Damien George
f1462448d0 alif/modtime: Implement the rest of the time module.
Adds: `time.time()`, `time.time_ns()`, `time.localtime()`, `time.mktime()`
and `time.gmtime()`.

Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 12:35:24 +10:00
Damien George
326730d8b2 alif/mbedtls: Implement the mbedTLS time function.
Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 12:35:23 +10:00
Damien George
46b366d7b2 alif/fatfs_port: Implement get_fattime.
Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 12:35:21 +10:00
Damien George
0feb4f5ea4 alif/mphalport: Implement mp_hal_time_ns.
Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 12:35:18 +10:00
Damien George
e58848a21e alif/machine_rtc: Implement RTC.datetime to get and set the RTC.
The LPRTC peripheral is a 32-bit counter with a 16-bit prescaler.  It's
configured here to count at 1Hz (to get maximum date range) and then the
prescaler value is used to get 30 microsecond resolution.  That's
essentially a 32+15=47-bit counter.

Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 12:34:28 +10:00
Damien George
b3cd1a355e extmod/modtime: Move tuple creation to common localtime implementation.
This factors code out of the ports and into the common `time.localtime`
implementation in `extmod/modtime.c`.  That helps to reduce code
duplication, prevent errors in implementation, and reduce code size on
some ports (mimxrt and stm32 at least).

Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 12:09:13 +10:00
Damien George
40cc4e4f74 py/objtype: Make mp_obj_new_type a static function.
It's only used once, in the same file it's defined, and making it static
reduces code size.

Along with this, the associated example code comment in `ports/unix/main.c`
has been removed.

Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 11:45:11 +10:00
Damien George
3efbd726eb py/parse: Remove explicit checks for invalid folding operations.
They are instead checked by `binary_op_maybe()`, which catches exceptions
for invalid int/float operations.

This is a follow-up to 69ead7d98e

Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 11:38:20 +10:00
Phil Howard
141f7d0c35 py/mkrules.cmake: Clean genhdr and frozen_mpy dirs.
CMake builds relied upon the parent Makefile removing the entire
build directory to successfully clean build artifacts.

py/mkrules.cmake: Add ADDITIONAL_CLEAN_FILES properties to ensure a
		  "make clean" from within the build directory removes
                  the genhdr and frozen_mpy directories.

Signed-off-by: Phil Howard <github@gadgetoid.com>
2025-08-15 01:24:53 +10:00
0615d13963 py/objringio: Detect incorrect constructor calls.
ringbuffer.size must be at least 2, and is a 16-bit quantity.

This fixes several cases including the one the fuzzer discovered, which
would lead to a fatal signal when accessing the object.

Fixes issue #17847.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-08-15 01:21:25 +10:00
803da9645f extmod/modframebuf: Save code size in setpixel.
There's a slight code size increase paid for by using setpixel_checked for
the last pixel of a line, instead of repeating the checks inline.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-08-15 01:17:00 +10:00
e15219800e extmod/modframebuf: Fix crash in scroll() for large inputs.
If mp_int_t is wider than int, then the tests such as `xend < 0` can fail
even when the amount of scrolling requested is out of range.  This resulted
in a segmentation fault when attempting an out-of-bounds access to the
framebuffer.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-08-15 01:16:24 +10:00
Damien George
2bba507148 tests: Require SSL certificate file to be available for test to run.
Previously, any test needing an SSL certificate file would automatically
skip if the file could not be found.  But that makes it too easy to
accidentally skip tests.

Instead, change it so that the test fails if the certificate file doesn't
exist.  That matches, for example, the fact that the test fails if
networking (LAN, WiFi) is not active.

Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 01:03:00 +10:00
Damien George
a279c64046 tests: Add .native.exp output files for tests that differ with native.
Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 00:36:27 +10:00
Damien George
3c72c3a1e6 tests/micropython/opt_level_lineno.py: Force test func to use bytecode.
So that the test can run the same on all targets when used with the native
emitter.

Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 00:36:27 +10:00
Damien George
95d1794afd tests/misc/print_exception.py: Use "raise e" instead of no-arg "raise".
This allows the test to run with the native emitter.

The test semantics remain the same.

Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 00:36:27 +10:00
Damien George
0f5f6484a2 tests/run-tests.py: Add support for .native.exp expected output files.
There are currently a few tests that are excluded when using the native
emitter because they test printing of exception tracebacks, which includes
line numbers.  And the native emitter doesn't store line numbers, so gets
these tests wrong.

But we'd still like to run these tests using the native emitter, because
they test useful things even if the line number info is not in the
traceback (eg that threads which crash print out their exception).

This commit adds support for native-specific .exp files, which are of the
form `<test>.py.native.exp`.  If such an .exp file exists then it take
precedence over any normal `<test>.py.exp` file.

(Actually, the implementation here is general enough that it also supports
`<test>.py.bytecode.exp` as well, if bytecode ever needs a specific exp
file.)

Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 00:36:27 +10:00
Damien George
0cb2c69b3f tests/misc/rge_sm.py: Remove unused code from the test.
This cleans up the test to remove all unused code, making it smaller,
a bit faster to deploy to a target to run, and also use less RAM on the
target (which may help it run on targets that are just slightly out of
memory running it).

Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 00:30:41 +10:00
Damien George
c16fe5b5ed webassembly: Enable C-stack checking.
This gets the recursive stress-tests working on this port.

For relatively small Python functions the maximum recursive depth is about
150 nested calls.

Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 00:23:16 +10:00
Damien George
f9b6d8e608 github/workflows: Run webassembly and zephyr workflows if tests/ change.
Because these ports run tests as part of CI, and need to be run if any of
the tests change, to check those changes.

Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 00:23:16 +10:00
Damien George
54e6cfc6e3 tests/basics: Skip tests of io module individually using SKIP.
Instead of using a feature check.  This is more consistent with how other
optional modules are skipped.

Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 00:23:16 +10:00
Damien George
e2744ce679 tests/run-tests.py: Autodetect if the target has unicode support.
The unicode tests are now run on all targets that enable unicode.  And
other unicode tests (namely `extmod/json_loads.py`) are now properly
skipped if the target doesn't have unicode support.

Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 00:23:16 +10:00
Damien George
1db71f9e55 tests/run-tests.py: Generalise addition of port specific test directory.
Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 00:23:16 +10:00
Damien George
6565827955 tests/run-tests.py: Always include stress/ tests directory in tests.
Ports that now run the stress tests, that didn't prior to this commit are:
cc3200, esp8266, minimal, nrf, renesas-ra, samd, qemu, webassembly.

Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 00:23:16 +10:00
Damien George
f493075d88 tests/run-tests.py: Automatically include float tests when possible.
This simplifies the code by removing the explicit addition of the "float/"
test directory for certain targets.  It also means the tests won't be added
incorrectly, eg on a unix build without float.

Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 00:23:16 +10:00
Angus Gratton
744270ac1b py/misc: Add explicit dependency on py/mpconfig.h.
Macros in misc.h depend on values defined by including mpconfig.h.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-08-13 10:13:08 +10:00
4614ee9e68 py/binary: Add MICROPY_PY_STRUCT_UNSAFE_TYPECODES.
This adds a compile-time flag to disable some "unsafe" and non-standard
typecodes in struct, array and related modules.

This is useful to turn off when fuzzing, as improper use of these typecodes
can crash MicroPython.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-08-12 14:34:19 +10:00
Alessandro Gatti
0ee3f99da2 py/asmrv32: Make lt/le comparisons emitter shorter.
This commit simplifies the emitter code in charge of generating opcodes
performing less-than and less-than-or-equal comparisons.

By rewriting the SLT/SLTU opcode generator (handling less-than
comparisons) and de-inlining the less-than comparison generator call in
the less-than-or-equal generator, the output binary is ~80 bytes smaller
(measurements taken from the QEMU port).

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-08-11 22:31:13 +10:00
Alessandro Gatti
b1d5c656de tests/micropython: Remove big ints dependence for viper boundary tests.
This commit provides an implementation for viper boundary tests that can
work even without big int support.

Since it uses a fixed-size buffer to hold values to work with, this
should work on any platform as long as its integers are at least 32 bits
wide, regardless its configuration on how big integers can get.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-08-11 16:46:15 +10:00
Jared Hancock
f10707febb extmod/modlwip: Support family specification in getaddrinfo.
`socket.getaddrinfo()` supports the specification of an address family;
however, the LwIP implementation does not use it. This change allows the
application to specify the address family request in DNS resolution. If
no family is specified, it falls back to the default preference
configured with `network.ipconfig()`.

Signed-off-by: Jared Hancock <jared.hancock@centeredsolutions.com>
2025-08-11 16:22:49 +10:00
Jared Hancock
14ccdeb4d7 extmod/modre: Add support for start- and endpos.
Pattern objects have two additional parameters for the ::search and ::match
methods to define the starting and ending position of the subject within
the string to be searched.

This allows for searching a sub-string without creating a slice.  However,
one caveat of using the start-pos rather than a slice is that the start
anchor (`^`) remains anchored to the beginning of the text.

Signed-off-by: Jared Hancock <jared@greezybacon.me>
2025-08-11 14:11:56 +10:00
Daniël van de Giessen
485dac783b tools/codeformat.py: Print filename + linenumber when dedenting fails.
Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
2025-08-11 13:35:34 +10:00
Daniël van de Giessen
6e450dba7e tools/codeformat.py: Iterate lines instead of modifying list in-place.
Co-authored-by: David Lechner <david@pybricks.com>
Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
2025-08-11 13:35:27 +10:00
Jos Verlinde
20e1ae0733 tools/mpremote: Fix encoding error in PyboardCommand.
This is a fix for utf-8 decoding errors that are thrown when non-utf-8
content is received.  For instance during a reboot of an ESP8266 module.

The fix is to handle conversion errors by replacing illegal characters.
Note that these illegal characters most often occur during an MCU reboot
sequence when the MCU is using baudrates different from 115200.

Signed-off-by: Jos Verlinde <Jos_Verlinde@hotmail.com>
2025-08-11 13:28:56 +10:00
Daniël van de Giessen
8c47ff7153 esp32/network_ppp: Correctly clean up PPP PCB after close.
If PPP is still connected, freeing the PCB will fail and thus instead we
should trigger a disconnect and wait for the lwIP callback to actually
free the PCB.

When PPP is not connected we should check if the freeing failed, warn
the user if so, and only mark the connection as inactive if not.

When all this happens during garbage collection the best case is that
the PPP connection is already dead, which means the callback will be
called immediately and cleanup will happen correctly. The worst case is
that the connection is still alive, thus we are unable to free the PCB
(lwIP won't let us) and it remains referenced in the netif_list, meaning
a use-after-free happens later when lwIP traverses that linked list.

This change does not fully prevent that, but it *does* improve how the
PPP.active(False) method on the ESP32 port behaves: It no longer
immediately tries to free (and fails), but instead triggers a disconnect
and lets the cleanup happen correctly through the status callback.

Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
2025-08-11 12:38:37 +10:00
Daniël van de Giessen
adcfdf625b esp32/network_ppp: Use non-thread-safe API inside status callback.
The status callback runs on the lwIP tcpip_thread, and thus must use the
non-thread-safe API because the thread-safe API would cause a deadlock.

Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
2025-08-11 12:38:37 +10:00
Daniël van de Giessen
361c615f3e esp32/network_ppp: Use thread-safe API for PPPoS input.
A small follow-up to 3b1e22c669, in which
the entire PPP implementation was reworked to more closely resemble the
extmod version. One of the differences between extmod and the ESP32 port
is that the ESP32 port uses the thread-safe API, but in that changeset
the PPP input function was accidentally changed to use the non-safe API.

Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
2025-08-11 12:38:37 +10:00
Daniël van de Giessen
1273751a3b esp32: Support building against IDFv5.5.
Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
2025-08-11 12:23:02 +10:00
Damien George
1401fdb8b3 tests/run-tests.py: Run tests-with-regex-output as normal tests.
Some tests (currently given by the `special_tests` list) have output which
must be mached via a regex, because it can change from run to run (eg the
address of an object is printed).  These tests are currently classified as
`is_special` in the test runner, which means they get special treatment.
In particular they don't set the emitter as specified by `args.emit`.  That
means these tests do not run via .mpy or using the native emitter, even if
those options are given on the command line.

This commit fixes that by considering `is_special` as different to
`tests_with_regex_output`.  The former is used for things like target
feature detection (which are not really tests) and when extra command line
options need to be passed to the unix micropython executable.  The latter
(now called `tests_with_regex_output`) are specifically for tests that have
output to be matched via regex.

The `thread_exc2.py` test now needs to be excluded when running using the
native emitter, because the native emitter doesn't print traceback info.
And the `sys_settrace_cov.py` test needs to be excluded because set-trace
output is different with the native emitter.

Signed-off-by: Damien George <damien@micropython.org>
2025-08-10 11:40:54 +10:00
Damien George
c35427cbdb tests/run-tests.py: Move tests to skip with native emitter to a list.
This makes `run-tests.py` a little more organised, by putting all the
tests-to-skip-when-using-the-native-emitter in a dedicated list.

This should make it easier to maintain the list, and understand why a test
is there.

Signed-off-by: Damien George <damien@micropython.org>
2025-08-10 11:25:38 +10:00
Damien George
7e8705fe2b all: Bump version to 1.27.0-preview.
Signed-off-by: Damien George <damien@micropython.org>
2025-08-10 08:42:11 +10:00
Damien George
4ce2dd2cda all: Bump version to 1.26.0.
Signed-off-by: Damien George <damien@micropython.org>
2025-08-09 23:31:31 +10:00