Commit graph

434 commits

Author SHA1 Message Date
880dd0a1cd vfs: Check that open() resulted in a file-like object.
That is, an object whose type defines the protocol slot.

Note that due to protocol confusion, a variant of the original
crasher that returned e.g., a machine.Pin instance could still
lead to a crash. (#17852)

Closes: #17841

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-08-08 08:53:14 -05:00
TianShuang Ke
ca9916968c esp32: Add support for ESP32-C2 (aka ESP8684).
Includes:
esp32/esp32c2: Adapt to target chip ESP32C2.
esp32/esp32c2: Fix heap size is too small to enable Bluetooth.

Signed-off-by: TianShuangKe <qinyun575@gmail.com>
Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-07-30 10:51:29 +10:00
Damien George
7729e80fdd all: Go back to using default ruff quote style.
Commit dc2fcfcc55 seems to have accidentally
changed the ruff quote style to "preserve", instead of keeping it at the
default which is "double".

Put it back to the default and update relevant .py files with this rule.

Signed-off-by: Damien George <damien@micropython.org>
2025-07-24 12:48:18 +10:00
d79000df70 tests/extmod: Add (failing) test for VfsPosix in readonly mode.
I noticed that operations such as unlink could be performed on a nominally
read-only VfsPosix.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-24 10:39:23 +10:00
Angus Gratton
9b7d85227e extmod/mbedtls: Implement recommended DTLS features, make optional.
- DTLS spec recommends HelloVerify and Anti Replay protection be enabled,
  and these are enabled in the default mbedTLS config. Implement them here.

- To help compensate for the possible increase in code size, add a
  MICROPY_PY_SSL_DTLS build config macro that's enabled for EXTRA and
  above by default.

This allows bare metal mbedTLS ports to use DTLS with HelloVerify support.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-07-23 15:47:16 +10:00
Damien George
97fd18a7e2 tests/extmod/select_poll_eintr.py: Pre-allocate global variables.
This is a workaround for the case where threading is enabled without a GIL.
In such a configuration, creating a new global variable is not atomic and
threads have race conditions resizing/accessing the global dict.

Signed-off-by: Damien George <damien@micropython.org>
2025-07-22 23:22:05 +10:00
Angus Gratton
516aa02104 py/objint_longlong: Add arithmetic overflow checks.
Long long big integer support now raises an exception on overflow rather
than returning an undefined result.

Also adds an error when shifting by a negative value.

The new arithmetic checks are added in the misc.h header.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-07-18 00:12:05 +10:00
Angus Gratton
6d93b150b8 tests/extmod/json_loads_int_64.py: Add test cases for LONGINT parse.
These tests cover the use of mp_obj_new_int_from_str_len when
mp_parse_num_integer overflows the SMALLINT limit, and also the case where
the value may not be null terminated.

Placed in a separate test file so that extmod/json test doesn't rely on
bigint support.

Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-07-18 00:10:48 +10:00
Angus Gratton
2d8d64059f tests: Add specific tests for "long long" 64-bit bigints.
These will run on all ports which support them, but importantly
they'll also run on ports that don't support arbitrary precision
but do support 64-bit long ints.

Includes some test workarounds to account for things which will overflow
once "long long" big integers overflow (added in follow-up commit):

- uctypes_array_load_store test was failing already, now won't parse.
- all the ffi_int tests contain 64-bit unsigned values, that won't parse
  as long long.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-07-18 00:10:44 +10:00
Yanfeng Liu
274306860b tests/extmod: Close UDP sockets at end of test.
This adds call to release UDP port in a timely manner, so they can be
reused in subsequent tests.  Otherwise, one could face issue like #17623.

Signed-off-by: Yanfeng Liu <yfliu2008@qq.com>
2025-07-15 13:57:21 +10:00
Damien George
908f938c44 tests/extmod/asyncio_iterator_event.py: Use format instead of f-string.
Some targets don't have f-strings enabled, so try not to use them in tests.
Rather, use `str.format`, which is more portable.

Signed-off-by: Damien George <damien@micropython.org>
2025-07-12 22:37:56 +10:00
Yoctopuce dev
df05caea6c shared/timeutils: Standardize supported date range on all platforms.
This is code makes sure that time functions work properly on a
reasonable date range, on all platforms, regardless of the epoch.
The suggested minimum range is 1970 to 2099.

In order to reduce code footprint, code to support far away dates
is only enabled specified by the port.

New types are defined to identify timestamps.

The implementation with the smallest code footprint is when
support timerange is limited to 1970-2099 and Epoch is 1970.
This makes it possible to use 32 bit unsigned integers for
all timestamps.

On ARM4F, adding support for dates up to year 3000 adds
460 bytes of code. Supporting dates back to 1600 adds
another 44 bytes of code.

Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
2025-07-09 11:54:21 +10:00
Damien George
168e2c8f66 tests/extmod/select_poll_eintr.py: Skip test if target can't bind.
Eg on PYBV10 with THREAD variant, the firmware has both the `_thread` and
`socket` modules but no NIC.

Signed-off-by: Damien George <damien@micropython.org>
2025-07-08 13:19:00 +10:00
Damien George
bb484b6d81 tests/extmod/machine_uart_tx.py: Support STM32WB boards.
Signed-off-by: Damien George <damien@micropython.org>
2025-07-08 13:17:57 +10:00
5ade8b7058 tests/extmod: Add platform_basic.py for basic coverage test of platform.
Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-06-16 23:37:17 +10:00
Damien George
e4d556b149 tests/extmod/random_extra_float.py: Skip when funcs not available.
This test was factored out from `random_extra.py` back in commit
6572029dc0, and the skip logic copied from
that file.  But the skip logic needs to test that the `random` and
`uniform` functions exist, not `randint`.

This commit fixes that skip logic.

Signed-off-by: Damien George <damien@micropython.org>
2025-06-12 21:45:40 +10:00
Damien George
b15348415e extmod/modframebuf: Add support for blit'ing read-only data.
Currently the `FrameBuffer.blit(buf, x, y)` method requires the `buf`
argument to be another `FrameBuffer`, which is quite restrictive because it
doesn't allow blit'ing read-only memory/data.

This commit extends `blit()` to allow the `buf` argument to be a tuple or
list of the form:

    (buffer, width, height, format[, stride])

where `buffer` can be anything with the buffer protocol and may be
read-only, eg `bytes`.

Also, the palette argument to `blit()` may be of the same form.

The form of this tuple/list was chosen to be the same as the signature of
the `FrameBuffer` constructor (that saves quite a bit of code size doing it
that way).

Signed-off-by: Damien George <damien@micropython.org>
2025-06-04 02:40:45 +10:00
Alessandro Gatti
9ef16b466b extmod/modjson: Detect unterminated composite entities.
This commit makes the JSON parser raise an exception when handling
objects or arrays whose declaration is incomplete, as in missing the
closing marker (brace or bracket) and if the missing marker would have
been the last non-whitespace character in the incoming string.

Since CPython's JSON parser would raise an exception in such a case,
unlike MicroPython's, this commit aligns MicroPython's behaviour with
CPython.

This commit fixes issue #17141.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-05-19 02:09:40 +02:00
Damien George
44bcfe53de tests/extmod/vfs_lfs_error.py: Test value of all OSError's errno.
To make sure they have the correct value.

Signed-off-by: Damien George <damien@micropython.org>
2025-05-15 13:09:49 +10:00
Damien George
487c94c253 tests/extmod/vfs_rom.py: Clear sys.path before running test.
Otherwise if the target has certain files/directories (such as "test") in
its filesystem then these interfere with the unit tests.

Signed-off-by: Damien George <damien@micropython.org>
2025-05-12 13:33:42 +10:00
Daniël van de Giessen
9287a1e6ea lib/littlefs: Update LittleFS to v2.10.2.
Going above the root directory (/../foo) now gives an error.  This is an
intentional change made by LittleFS.  It required a update of the testsuite
and is a (minor) compatibility break.

Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
2025-05-07 17:01:16 +10:00
Yoctopuce dev
bdb7e036d2 extmod/asyncio: Fix early exit of asyncio scheduler.
This commit fixes three open issues related to the asyncio scheduler
exiting prematurely when the main task queue is empty, in cases where
CPython would not exit (for example, because the main task is not done
because it's on a different queue).

In the first case, the scheduler exits because running a task via
`run_until_complete` did not schedule any dependent tasks.

In the other two cases, the scheduler exits because the tasks are queued in
an event queue.

Tests have been added which reproduce the original issues.  These test
cases document the unauthorized use of `Event.set()` from a soft IRQ, and
are skipped in unsupported environments (webassembly and native emitter).

Fixes issues #16759, #16569 and #16318.

Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
2025-05-07 14:56:47 +10:00
Angus Gratton
79abdad9e9 tests/extmod: Rename ssl tests that only use the tls module.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-05-02 17:24:58 +10:00
Angus Gratton
70ed315193 py/malloc: Add mutex for tracked allocations.
Fixes thread safety issue that could cause memory corruption on ports
with (MICROPY_PY_THREAD && !MICROPY_PY_THREAD_GIL) - currently only rp2 and
unix have this configuration.

Adds unit test for TLS sockets that exercises this code path.  I wasn't
able to make this fail on rp2, the race condition window is pretty narrow
and may not have a direct impact on a quiet system.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-05-02 17:24:16 +10:00
Alessandro Gatti
ce7f65f967 tests/extmod/vfs_posix.py: Fix test on Android.
This commit makes a slight change to the vfs_posix test suite to let it
pass on Android.

On Android, non-root processes can perform most filesystem operations
only on a restricted set of directories.  The vfs_posix test suite
attempted to enumerate the filesystem root directory, and said directory
happens to be restricted for non-root processes.  This would raise
an EACCES OSError and terminate the test with a unexpected failure.

To fix this, rather than enumerating the filesystem root directory the
enumeration target is the internal shared storage area root - which
doesn't have enumeration restrictions for non-root processes.  The path
is hardcoded because it is guaranteed to be there on pretty much any
recent-ish device for now (it stayed the same for more than a decade for
compatibility reasons).  The proper way would be to query the storage
subsystem via a JNI round-trip call, but this introduces too much
complexity for something that is unlikely to break going forward.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-04-24 22:11:42 +10:00
Damien George
898c04ae0e tests/extmod/vfs_mountinfo.py: Don't import unused errno module.
Signed-off-by: Damien George <damien@micropython.org>
2025-04-24 22:06:11 +10:00
Damien George
a081b2e151 tests/extmod/vfs_lfs_ilistdir_del.py: Skip test if not enough memory.
Signed-off-by: Damien George <damien@micropython.org>
2025-04-24 22:06:11 +10:00
Damien George
c83e907d9d tests/extmod: Skip binascii tests when hexlify/unhexlify don't exist.
These functions are only available when `MICROPY_PY_BUILTINS_BYTES_HEX` is
enabled.

Signed-off-by: Damien George <damien@micropython.org>
2025-04-24 22:06:11 +10:00
Damien George
037f2dad72 tests: Update UART and SPI tests to work on Alif boards.
Signed-off-by: Damien George <damien@micropython.org>
2025-04-09 00:22:33 +10:00
Damien George
e4051a1ca6 extmod/vfs_rom: Implement minimal VfsRom.getcwd() method.
This is needed if you chdir to a ROMFS and want to query your current
directory.

Prior to this change, using `os.getcwd()` when in a ROMFS would raise:

    AttributeError: 'VfsRom' object has no attribute 'getcwd'

Signed-off-by: Damien George <damien@micropython.org>
2025-03-27 17:04:12 +11:00
Anson Mansfield
9fcc25b9d7 tests/extmod/vfs_mountinfo.py: Add test for no-args mount output.
Signed-off-by: Anson Mansfield <amansfield@mantaro.com>
2025-03-27 16:37:04 +11:00
Damien George
fa42487e45 extmod/moddeflate: Keep DeflateIO state consistent on window alloc fail.
Allocation of a large compression window may fail, and in that case keep
the `DeflateIO` state consistent so its other methods (such as `close()`)
still work.  Consistency is kept by only updating the `self->write` member
if the window allocation succeeds.

Thanks to @jimmo for finding the bug.

Signed-off-by: Damien George <damien@micropython.org>
2025-03-27 11:58:59 +11:00
Damien George
14ba32bb20 extmod/vfs_rom: Add bounds checking for all filesystem accesses.
Testing with ROMFS shows that it is relatively easy to end up with a
corrupt filesystem on the device -- eg due to the ROMFS deploy process
stopping half way through -- which could lead to hard crashes.  Notably,
there can be boot loops trying to mount a corrupt filesystem, crashes when
importing modules like `os` that first scan the filesystem for `os.py`, and
crashing when deploying a new ROMFS in certain cases because the old one is
removed while still mounted.

The main problem is that `mp_decode_uint()` has an loop that keeps going as
long as it reads 0xff byte values, which can happen in the case of erased
and unwritten flash.

This commit adds full bounds checking in the new `mp_decode_uint_checked()`
function, and that makes all ROMFS filesystem accesses robust.

Signed-off-by: Damien George <damien@micropython.org>
2025-02-26 16:11:19 +11:00
Keenan Johnson
321b30ca56 extmod/modtls_mbedtls: Wire in support for DTLS.
This commit enables support for DTLS, i.e. TLS over datagram transport
protocols like UDP.  While support for DTLS is absent in CPython, it is
worth supporting it in MicroPython because it is the basis of the
ubiquitous CoAP protocol, used in many IoT projects.

To select DTLS, a new set of "protocols" are added to SSLContext:
- ssl.PROTOCOL_DTLS_CLIENT
- ssl.PROTOCOL_DTLS_SERVER

If one of these is set, the library assumes that the underlying socket is a
datagram-like socket (i.e. UDP or similar).

Our own timer callbacks are implemented because the out of the box
implementation relies on `gettimeofday()`.

This new DTLS feature is enabled on all ports that use mbedTLS.

This commit is an update to a previous PR #10062.

Addresses issue #5270 which requested DTLS support.

Signed-off-by: Keenan Johnson <keenan.johnson@gmail.com>
2025-02-14 12:55:25 +11:00
Damien George
30acb16ad3 extmod/vfs_rom: Remove ability to create VfsRom from an address.
It's not necessary to support this, which allows an arbitrary memory
address to be specified and potentially allows invalid memory accesses.

Requiring an object with the buffer protocol is safer, and also means that
the length of the region is always specified.

Signed-off-by: Damien George <damien@micropython.org>
2025-02-11 17:56:13 +11:00
Damien George
c3a18d74eb extmod/modmarshal: Add new marshal module.
This commit implements a small subset of the CPython `marshal` module.  It
implements `marshal.dumps()` and `marshal.loads()`, but only supports
(un)marshalling code objects at this stage.  The semantics match CPython,
except that the actual marshalled bytes is not compatible with CPython's
marshalled bytes.

The module is enabled at the everything level (only on the unix coverage
build at this stage).

Signed-off-by: Damien George <damien@micropython.org>
2025-02-11 16:54:20 +11:00
Angus Gratton
195bf05115 tests: Add a test for SSL socket memory leaks.
Test is for an issue reported on the micropython-lib Discord as
effecting the rp2 port umqtt.simple interface when reconnecting with TLS,
however it's a more generic problem.

Currently this test fails on RPI_PICO_W and ESP32_GENERIC_C3 (and no doubt
others). Fixes are in the subsequent commits.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-02-03 15:02:02 +11:00
Alessandro Gatti
e73cf71a24 tests/extmod/re_sub.py: Fix test execution on Python 3.13.
This commit fixes a test failure for `extmod/re_sub.py` where the code,
whilst being correct, would not make the test pass due to a newer
Python version than expected.

On Python 3.13, running `tests/extmod/re_sub.py` would yield a
deprecation warning about `re.sub` not providing the match count as a
keyword parameter.  This warning would be embedded in the expected test
result and thus the test would always fail.

Co-authored-by: stijn <stijn@ignitron.net>
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-01-15 07:46:09 +11:00
Damien George
fca8ea6c39 tests/extmod/vfs_rom.py: Import errno for test.
It's needed by the test.  This previously passed because the compiler
(actually parser) optimises away errno constants.

Signed-off-by: Damien George <damien@micropython.org>
2025-01-06 17:55:11 +11:00
Damien George
c73204128e tests/extmod: Add VfsRom test.
Provides full coverage of the VfsRom driver.

Signed-off-by: Damien George <damien@micropython.org>
2024-12-23 13:04:55 +11:00
Damien George
d10cda66dc tests/extmod: Add test for uctypes.addressof function.
Signed-off-by: Damien George <damien@micropython.org>
2024-12-18 11:01:23 +11:00
Damien George
2c80d36998 tests/extmod: Convert machine1.py test to use unittest.
Signed-off-by: Damien George <damien@micropython.org>
2024-12-06 13:48:23 +11:00
Angus Gratton
eb80b04944 tests/extmod: Workaround CPython warning in asyncio_new_event_loop test.
This started failing in CI on the mingw build, after CPython updated to
3.12.7. The test prints two warnings during interpreter shutdown of
"Task was destroyed but it is pending!".

This didn't happen on other CPython builds, and I think that's because of
finalizer order in CPython interpreter shutdown but not certain (the loop
finalizer calls loop.close() if not already closed).

Adding explicit calls to loop.close() causes the warning to be printed on
every run with CPython 3.12.7 on Linux.

Next, added the workaround exception handler to swallow this exception
as MicroPython doesn't produce an equivalent.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2024-11-28 23:12:34 +11:00
Corran Webster
e70048cf59 extmod/modframebuf: Fix 0 radius bug in FrameBuffer.ellipse.
This fixes a bug in FrameBuffer.ellipse where it goes into an infinite loop
if both radii are 0.

This fixes the bug with a simple pre-check to see if both radii are 0, and
in that case sets a single pixel at the center. This is consistent with the
behaviour of the method when called with just one of the radii set to 0,
where it will draw a horizontal or vertical line of 1 pixel width.

The pixel is set with setpixel_checked so it should handle out-of-bounds
drawing correctly.

This fix also includes three new tests: one for the default behaviour, one
for drawing out-of-bounds, and one for when the sector mask is 0.

Fixes issue #16053.

Signed-off-by: Corran Webster <cwebster@unital.dev>
2024-11-28 22:56:44 +11:00
Damien George
09ea901317 tests/extmod: Add test to compare time_ns with time.
They should be close together.

Signed-off-by: Damien George <damien@micropython.org>
2024-10-24 23:24:09 +11:00
Damien George
1ec0c9b886 tests/extmod: Use time_ns instead of time in lfs mtime test.
Because VfsLfs2 uses time_ns to create timestamps for files, and for the
test to give consistent results it also needs to use this same function.

Signed-off-by: Damien George <damien@micropython.org>
2024-10-24 23:24:09 +11:00
Damien George
ca6aed7649 tests/extmod: Fix access of RTC class in machine.RTC test.
This previously passed on some targets that automatically import the
`machine` module in `boot.py`.

Signed-off-by: Damien George <damien@micropython.org>
2024-10-24 16:29:43 +11:00
Damien George
6d98280fda tests/extmod: Add a simple test for machine.RTC.
Tests at least that the datetime can be set and get correctly.

Signed-off-by: Damien George <damien@micropython.org>
2024-10-24 11:43:03 +11:00
Damien George
3f54e5dff2 tests/extmod: Support esp32,mimxrt,stm32,samd ports in UART TX test.
Getting this test running on stm32- and mimxrt-based boards requires adding
a small delay after constructing the UART so that the initial idle frame
has time to be transmitted before the test starts.

Also, the timing margin needs to account for an additional 1-bit worth of
time on some MCUs.

Thanks to @robert-hh for the esp32, mimxrt and samd settings.

Signed-off-by: Damien George <damien@micropython.org>
2024-10-22 22:21:17 +11:00
Damien George
0646177997 tests/extmod: Make invalid-blockdev test work consistently on all ports.
Some ports (eg stm32) configure the FAT driver differently (eg with
multi-partition support) and that leads to a slightly different sequence of
block reads, compared to other configurations (eg rp2).

Comment out the printing in `readblocks()` so the tests are deterministic
(the printing is still useful for debugging).

Signed-off-by: Damien George <damien@micropython.org>
2024-10-22 14:32:51 +11:00