Commit graph

1435 commits

Author SHA1 Message Date
Damien George
89e6c58c80 extmod/modvfs: Add vfs.rom_ioctl function and its ioctl constants.
This is a generic interface to allow querying and modifying the read-only
memory area of a device, if it has such an area.

Signed-off-by: Damien George <damien@micropython.org>
2025-03-06 12:52:35 +11:00
Damien George
14ba32bb20 extmod/vfs_rom: Add bounds checking for all filesystem accesses.
Testing with ROMFS shows that it is relatively easy to end up with a
corrupt filesystem on the device -- eg due to the ROMFS deploy process
stopping half way through -- which could lead to hard crashes.  Notably,
there can be boot loops trying to mount a corrupt filesystem, crashes when
importing modules like `os` that first scan the filesystem for `os.py`, and
crashing when deploying a new ROMFS in certain cases because the old one is
removed while still mounted.

The main problem is that `mp_decode_uint()` has an loop that keeps going as
long as it reads 0xff byte values, which can happen in the case of erased
and unwritten flash.

This commit adds full bounds checking in the new `mp_decode_uint_checked()`
function, and that makes all ROMFS filesystem accesses robust.

Signed-off-by: Damien George <damien@micropython.org>
2025-02-26 16:11:19 +11:00
Christian Clauss
5e206fdeb5 all: Upgrade codespell to v2.4.1.
This commit upgrades from codespell==2.2.6 to the current codespell==2.4.1,
adding emac to the ignore-words-list.

Signed-off-by: Christian Clauss <cclauss@me.com>
2025-02-25 16:11:33 +11:00
Keenan Johnson
321b30ca56 extmod/modtls_mbedtls: Wire in support for DTLS.
This commit enables support for DTLS, i.e. TLS over datagram transport
protocols like UDP.  While support for DTLS is absent in CPython, it is
worth supporting it in MicroPython because it is the basis of the
ubiquitous CoAP protocol, used in many IoT projects.

To select DTLS, a new set of "protocols" are added to SSLContext:
- ssl.PROTOCOL_DTLS_CLIENT
- ssl.PROTOCOL_DTLS_SERVER

If one of these is set, the library assumes that the underlying socket is a
datagram-like socket (i.e. UDP or similar).

Our own timer callbacks are implemented because the out of the box
implementation relies on `gettimeofday()`.

This new DTLS feature is enabled on all ports that use mbedTLS.

This commit is an update to a previous PR #10062.

Addresses issue #5270 which requested DTLS support.

Signed-off-by: Keenan Johnson <keenan.johnson@gmail.com>
2025-02-14 12:55:25 +11:00
Thomas Watson
aef6705a32 extmod/lwip-include: Increase number of lwIP timers when mDNS enabled.
Despite the code comments claiming one is sufficient, the mDNS application
is capable of using up to twelve timers.  Three per IP protocol are started
at once in `mdns_start_multicast_timeouts_ipvX`, then another two per
protocol can be started in `mdns_handle_question`.  Further timers can be
started for two additional callbacks.

Having certain timers, such as `MDNS_MULTICAST_TIMEOUT`, fail to start due
to none being free will break mDNS forever as the app will never realize
it's safe to transmit a packet.  Therefore, this commit goes somewhat
overkill and allocates the maximal amount of timers; it's uncertain if all
can run simultaneously, or how many callback timers are needed.

Each timer struct is 16 bytes on standard 32 bit builds.  Plus, say, 8
bytes of allocater overhead, that's 288 more bytes of RAM used which
shouldn't be too horrible.  Users who don't need mDNS can manually disable
it to recover the RAM if necessary.

This fixes mDNS on W5500_EVB_PICO (among other boards).  Before, mDNS would
work for a bit after connection until the host's cache expired a minute or
two later.  Then the board would never respond to further queries.  With
this patch, all works well.

Signed-off-by: Thomas Watson <twatson52@icloud.com>
2025-02-14 12:32:00 +11:00
Damien George
30acb16ad3 extmod/vfs_rom: Remove ability to create VfsRom from an address.
It's not necessary to support this, which allows an arbitrary memory
address to be specified and potentially allows invalid memory accesses.

Requiring an object with the buffer protocol is safer, and also means that
the length of the region is always specified.

Signed-off-by: Damien George <damien@micropython.org>
2025-02-11 17:56:13 +11:00
Damien George
c3a18d74eb extmod/modmarshal: Add new marshal module.
This commit implements a small subset of the CPython `marshal` module.  It
implements `marshal.dumps()` and `marshal.loads()`, but only supports
(un)marshalling code objects at this stage.  The semantics match CPython,
except that the actual marshalled bytes is not compatible with CPython's
marshalled bytes.

The module is enabled at the everything level (only on the unix coverage
build at this stage).

Signed-off-by: Damien George <damien@micropython.org>
2025-02-11 16:54:20 +11:00
Angus Gratton
97f444bfa0 extmod/mbedtls: Try GC before failing to setup socket on esp32, unix.
On mbedTLS ports with non-baremetal configs (mostly esp32, technically also
unix port), mbedTLS memory is allocated from the libc heap. This means an
old SSL socket may be holding large SSL buffers and preventing a new SSL
socket from being allocated.

As a workaround, trigger a GC pass and retry before failing outright.

This was originally implemented as a global mbedTLS calloc function, but
there is complexity around the possibility of C user modules calling into
mbedTLS without holding the GIL. It would be interesting to try making a
generic version for any malloc which fails, but this would require checking
for a Python thread and probably making the GIL recursive.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-02-03 15:02:02 +11:00
Damien George
abb13b1e1e extmod/lwip-include: Factor common lwIP config into lwipopts_common.h.
This lwIP configuration file has options that are common to all ports, and
the ports are updated to use this file.  This change is a no-op, the lwIP
configuration remains the same for the four ports using this common file.

This reduces code duplication, keeps the ports in sync, and makes it easier
to update the configuration for all ports at once.

Signed-off-by: Damien George <damien@micropython.org>
2025-01-29 12:36:27 +11:00
Jared Hancock
f29bd5a65e extmod/modlwip: Fix incorrect peer address for IPv6.
For IPv6 connections, the peer address was previously defined as only the
first four bytes of the IP address. For IPv6 addresses, this resulted in an
incorrect IPv4 address. For instance, receiving a packet via `::recvfrom`
from `'fe80::87:e7ff:fe48:629a'` is returned as having a peer address of
`'254.128.0.0'`

Signed-off-by: Jared Hancock <jared.hancock@centeredsolutions.com>
2025-01-29 10:49:34 +11:00
Glenn Strauss
f2cd1a3db6 lib/mbedtls: Update to mbedtls v3.6.2.
Signed-off-by: Glenn Strauss <gstrauss@gluelogic.com>
2025-01-17 10:10:57 +11:00
Yoctopuce
61995b53c3 extmod/moddeflate: Add missing size_t cast.
To prevent compiler warnings.

Signed-off-by: Yoctopuce <dev@yoctopuce.com>
2025-01-02 13:33:39 +11:00
Yoctopuce
b323b42770 extmod/modsocket: Add missing static in private function definitions.
Signed-off-by: Yoctopuce <dev@yoctopuce.com>
2025-01-02 13:33:39 +11:00
Damien George
5e9dd4b6a2 extmod/vfs_reader: Add support for opening a memory-mappable file.
If the file can be memory mapped (because it responds to the buffer
protocol) then return a memory-reader that directly references the ROM data
of the file.

Signed-off-by: Damien George <damien@micropython.org>
2024-12-23 13:04:54 +11:00
Damien George
50637ff239 extmod/vfs_rom: Add VfsRom filesystem object.
This commit defines a new ROMFS filesystem for storing read-only files that
can be memory mapped, and a new VfsRom driver.  Files opened from this
filesystem support the buffer protocol.  This allows naturally getting the
memory-mapped address of the file using:
- memoryview(file)
- uctypes.addressof(file)

Furthermore, if these files are .mpy files then their content can be
referenced in-place when importing.  Such imports take up a lot less RAM
than importing from a normal filesystem.  This is essentially dynamically
frozen .mpy files, building on the revamped v6 .mpy file format.

Signed-off-by: Damien George <damien@micropython.org>
2024-12-23 13:04:54 +11:00
Damien George
8b6bd43eab extmod/vfs: Guard mutating fs functions with MICROPY_VFS_WRITABLE.
Enabled by default.  Useful for ports that need the VFS but don't have any
writable filesystems.

Signed-off-by: Damien George <damien@micropython.org>
2024-12-20 21:58:50 +11:00
Damien George
8970ede7cc extmod/moductypes: Fix large return values of addressof and INT_MAYBE.
So they don't return a negative number for an address (prior to this fix
they would return negative addresses for values that were larger than the
maximum small-int value).

Signed-off-by: Damien George <damien@micropython.org>
2024-12-18 11:00:08 +11:00
Alessandro Gatti
28b5244666 extmod/modplatform: Distinguish RISC-V 64 from RISC-V 32.
This commit lets the platform module report a more accurate architecture
name when running on a RISC-V 64 bits platform.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2024-12-10 12:10:36 +11:00
iabdalkader
a118cf9ab0 extmod/extmod.mk: Fix libmetal build prefix.
libmetal source files already have the build directory prefix, because
they're auto-generated inside the build directory. When they're added
to `SRC_THIRDPARTY_C`, another build directory prefix is added resulting
in the object files being generated in a nested build directory.
This patch strips the build directory prefix before adding libmetal's
source files.

Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2024-12-10 10:25:02 +11:00
Alessandro Gatti
3de3821abf extmod/modplatform: Add Android to the recognised platforms list.
This commit adds code to distinguish between regular Linux and Android,
also adding a specific entry for the platform libc.  The reported libc
is marked as "bionic" and its version matches the Android platform API
version (there are no definitions for a specific bionic version).

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2024-11-30 12:42:51 +11:00
Alessandro Gatti
09cf01d7c1 extmod/modplatform: Add Clang to the known compilers list.
This commit adds support to distinguish between GCC and Clang to report
the appropriate compiler version.  Usually Clang also disguises itself
as GCC for compatibility reasons, but these changes look for
Clang-specific definitions first to avoid that problem.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2024-11-30 12:42:51 +11:00
Alessandro Gatti
579e840de6 extmod/modplatform: Distinguish AArch64 from AArch32.
This commit adds a new platform architecture name for Arm CPUs running
in 64 bits mode ("aarch64").  The 32 bits name is left as "arm" to
maintain compatibility with existing code.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2024-11-30 12:42:51 +11:00
Corran Webster
e70048cf59 extmod/modframebuf: Fix 0 radius bug in FrameBuffer.ellipse.
This fixes a bug in FrameBuffer.ellipse where it goes into an infinite loop
if both radii are 0.

This fixes the bug with a simple pre-check to see if both radii are 0, and
in that case sets a single pixel at the center. This is consistent with the
behaviour of the method when called with just one of the radii set to 0,
where it will draw a horizontal or vertical line of 1 pixel width.

The pixel is set with setpixel_checked so it should handle out-of-bounds
drawing correctly.

This fix also includes three new tests: one for the default behaviour, one
for drawing out-of-bounds, and one for when the sector mask is 0.

Fixes issue #16053.

Signed-off-by: Corran Webster <cwebster@unital.dev>
2024-11-28 22:56:44 +11:00
Angus Gratton
181800eebd extmod/network_cyw43: Allow configuring active AP interface.
Configuring the AP for cyw43 writes to some buffers that are only sent to
the modem when the interface is brought up. This means you can't configure
the AP after calling active(True), the new settings seem to be accepted but
the radio doesn't change.

This is different to the WLAN behaviour on other ports. The esp8266 port
requires calling active(True) on the AP before configuring, even.

Fix this by bouncing the AP interface after a config change, if it's
active. Configuring with active(False) still works the same as before.

Adds a static variable to track interface active state, rather than relying
on the LWIP interface state. This is because the interface state is updated
by a driver callback and there's a race: if code calls active(True) and
then config(a=b) then the driver doesn't know it's active yet and the
changes aren't correctly applied.

It is possible this pattern will cause the AP to come up briefly with the
default "PICOabcd" SSID before being reconfigured, however (due to the
aforementioned race condition) it seems like this may not happen at all
before the new config is applied.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2024-11-20 14:42:26 +11:00
Damien George
af743eaf59 extmod/network_cyw43: Fix uninitialised variable in status('stations').
The `num_stas` was uninitialised and if it happened to take the value 0
then no results were returned.  It now has the correct maximum value.

Signed-off-by: Damien George <damien@micropython.org>
2024-11-20 14:23:20 +11:00
Angus Gratton
f562aa1291 extmod/network_cyw43: Fix isconnected() result on AP interface.
This function is documented to return True if any stations are connected to
the AP. Without this fix it returns True whenever the driver has brought
the AP interface up.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2024-11-20 14:21:57 +11:00
Damien George
5dc9eda195 extmod/vfs_blockdev: Support bool return from Python read/write blocks.
Commit f4ab9d9247 inadvertently broke some
Python block devices, for example esp32 and stm32 SDCard classes.  Those
classes return a bool from their `readblocks` and `writeblocks` methods
instead of an integer errno code.  With that change, both `False` and
`True` return values are now be interpreted as non-zero and hence the block
device call fails.

The fix in this commit is to allow a bool and explicitly convert `True` to
0 and `False` to `-MP_EIO`.

Signed-off-by: Damien George <damien@micropython.org>
2024-11-13 14:21:29 +11:00
Daniël van de Giessen
77406b4240 extmod/network_ppp: Allow stream=None to suspend PPP.
This allows the stream to be set to `None`, which essentially stops all PPP
communication without disconnecting the session.

This allows replacing the stream on-the-fly to suspend it, for example to
send AT commands to a modem without completely disconnecting and
re-establishing the PPP connection:

    uart = ppp.config('stream')
    ppp.config(stream=None)
    uart.write(b'+++')
    # do some AT commands
    uart.write(b'ATO\r\n')
    ppp.config(stream=uart)

Any attempted communication by PPP while the stream is not connected will
register as simple packet loss to the LwIP stack because we return 0 for
any write calls, and protocols like TCP will then automatically handle
retrying.

Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
2024-11-13 13:11:32 +11:00
Daniël van de Giessen
161e2bd37d extmod/network_ppp: Add stream config parameter.
This makes the stream that the PPP object wraps, which is normally only set
once via the constructor, accessible and configurable via the
`ppp.config()` method.

Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
2024-11-13 13:10:58 +11:00
Damien George
eab2869990 extmod/modlwip: Don't allow writing to a TCP socket that is connecting.
This follows the behaviour of unix MicroPython (POSIX sockets) and the
esp32 port.

Signed-off-by: Damien George <damien@micropython.org>
2024-11-13 11:44:09 +11:00
Damien George
d278ba5cc5 extmod/nimble: Remove asserts of ediv_rand_present and adjust comments.
Recent versions of NimBLE (since release 1.6.0) removed this variable; see
7cc8c08d67.

We never used it except in an assert, so remove those asserts to make the
code compatible with newer NimBLE versions (eg for the esp32 port).

Signed-off-by: Damien George <damien@micropython.org>
2024-11-05 11:27:06 +11:00
Damien George
919756cea4 extmod/modlwip: Fix IGMP address type when IPv6 is enabled.
This was missed in 628abf8f25.  The the bug
was that, when IPv6 is enabled, the `sizeof(ip_addr_t)` is much larger than
IPv4 size, which is what's needed for IGMP addressing.

Fixes issue #16100.

Signed-off-by: Damien George <damien@micropython.org>
2024-11-04 13:12:18 +11:00
iabdalkader
68f1c20145 extmod/modtls_mbedtls: Support alternate sign callbacks in Python.
This commit enables the implementation of alternative mbedTLS cryptography
functions, such as ECDSA sign and verify, in pure Python.  Alternative
functions are implemented in Python callbacks, that get invoked from
wrapper functions when needed.  The callback can return None to fall back
to the default mbedTLS function.

A common use case for this feature is with secure elements that have
drivers implemented in Python.  Currently, only the ECDSA alternate sign
function wrapper is implemented.

Tested signing with a private EC key stored on an NXP SE05x secure element.

Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2024-10-25 01:16:06 +11:00
iabdalkader
2644f577f1 extmod/modtls_mbedtls: Add a thread-global ptr for current SSL context.
This is necessary for mbedTLS callbacks that do not carry any user state,
so those callbacks can be customised per SSL context.

Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2024-10-25 01:12:36 +11:00
Jared Hancock
078ead24f3 extmod/network_wiznet5k: Reset mDNS when interface is brought up.
The LwIP interface is removed in wiznet5k_deinit() which is called as part
of the init sequence.  Therefore, if using mDNS, then the interface will
need to be re-added when bringing the interface up.

Additionally, this allows to set the hostname from MicroPython code prior
to bringing the interface up and mDNS responding to the (new) hostname.
This allows the hostname to be configured and saved on the flash or be
based on dynamic information such as the MAC or unique_id().

Signed-off-by: Jared Hancock <jared.hancock@centeredsolutions.com>
2024-10-23 16:29:28 +11:00
Corran Webster
d1574de3b6 extmod/modframebuf: Fix FrameBuffer size check for stride corner-cases.
This is a fix for issue #15944, and handles corner cases in the FrameBuffer
code when using stride values where the last line's stride may extend past
the end of the underlying buffer.  This commit includes extra tests for
these corner cases.

For example a GS8 format FrameBuffer with a width of 8, height of 2 and
stride of 10 should be able to fit into a buffer of size 18 (10 bytes for
the first horizontal line, and 8 bytes for the second -- the full 10 bytes
are not needed).

Similarly a 1 by 9 FrameBuffer in MONO_VLSB format with a stride of 10
should be able to fit into a buffer of length 11 (10 bytes for the first
8 lines, and then one byte for the 9th line.

Being able to do this is particularly important when cropping the corner of
an existing FrameBuffer, either to copy a sprite or to clip drawing.

Signed-off-by: Corran Webster <cwebster@unital.dev>
2024-10-22 13:05:31 +11:00
stijn
6835743dcc extmod/vfs_posix_file: Skip flush of tty handles in msvc debug builds.
In MSVC debug builds with debug error reporting set to showing a dialog (to
allow attaching the debugger), any application which imports the logging
module and leaves the default handlers would result in this dialog because
logging.shutdown is called at exit and that flushes the default handler
which has stderr as its stream.

This commit fixes that by not fsync'ing stdin/out/err.

Also adds a comment related to checking whether a file is stdin/out/err,
which is difficult to fix properly.

Signed-off-by: stijn <stijn@ignitron.net>
2024-10-09 15:03:07 +11:00
Angus Gratton
f4ab9d9247 extmod/vfs_blockdev: Implement common helper for read and write.
- Code size saving as all of these functions are very similar.
- Resolves the "TODO" of the plain read and write functions not propagating
  errors. An error in the underlying block device now causes VFatFs to
  return EIO, for example.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2024-09-26 22:08:48 +10:00
Angus Gratton
4f6d4b2b49 extmod/vfs_blockdev: Check block device function positive results.
A positive result here can result in eventual memory corruption
as littlefs expects the result of a cache read/write function to be
0 or a negative integer for an error.

Closes #13046

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2024-09-26 22:08:48 +10:00
iabdalkader
3294606e23 extmod/libmetal: Fix libmetal rules for mkdir dependencies.
Dependency on auto-generated libmetal should be an order only prerequisite.

Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2024-09-06 17:14:03 +10:00
cajt
65244d291a extmod/modlwip: Fix compile error for lwIP with SLIP support.
Fixes a compile error if STM32 port is compiled with:

    make BOARD=(..) MICROPY_PY_LWIP=1 MICROPY_PY_LWIP_SLIP=1

`sio_send()` and `sio_tryread()` now use `mp_get_stream`.

Signed-off-by: Carl Treudler <cjt@users.sf.net>
2024-09-06 17:11:47 +10:00
Damien George
bc952d37fe extmod/network_ppp_lwip: Add network.PPP via lwIP.
This commit adds a new `network.PPP` interface which works on any port that
has bare-metal lwIP, eg rp2, stm32, mimxrt.

It has been tested on stm32.  A board needs to enable
`MICROPY_PY_NETWORK_PPP_LWIP` and then it can use it as follows:

    import network

    ppp = network.PPP(uart)
    ppp.connect()

    while not ppp.isconnected():
        pass

    # use `socket` module as usual, etc

    ppp.disconnect()

Usually the application must first configure the cellular/etc UART link to
get it connected and in to PPP mode first (eg ATD*99#), before handing over
control to `network.PPP`.

The PPP interface automatically configures the UART IRQ callback to call
PPP.poll() on incoming data.

Signed-off-by: Damien George <damien@micropython.org>
2024-08-29 23:46:13 +10:00
Damien George
664dd7b54a extmod: Update make and cmake scripts to work with latest lwIP.
Signed-off-by: Damien George <damien@micropython.org>
2024-08-29 23:46:13 +10:00
Jared Hancock
e901ff8557 extmod/network_wiznet5k: Add support for IPv6.
This adds support for the WIZNET5K nic to use IPv6 with the LWIP stack.
Additionally, if LWIP_IPV6 is disabled, the device is configured to drop
all IPv6 packets to reduce load on the MCU.

Signed-off-by: Jared Hancock <jared@greezybacon.me>
2024-08-26 13:02:44 +10:00
Angus Gratton
b82c9ca706 extmod/modtls_mbedtls: Optimise the DER certificate parsing fix.
Small code size and binary size optimisation for the fix merged in
4d6d84983f.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2024-08-26 12:48:12 +10:00
Angus Gratton
86f2c285eb py: Add new cstack API for stack checking, with limit margin macro.
Currently the stack limit margin is hard-coded in each port's call to
`mp_stack_set_limit()`, but on threaded ports it's fiddlier and can lead to
bugs (such as incorrect thread stack margin on esp32).

This commit provides a new API to initialise the C Stack in one function
call, with a config macro to set the margin.  Where possible the new call
is inlined to reduce code size in thread-free ports.

Intended replacement for `MP_TASK_STACK_LIMIT_MARGIN` on esp32.

The previous `stackctrl.h` API is still present and unmodified apart from a
deprecation comment.  However it's not available when the
`MICROPY_PREVIEW_VERSION_2` macro is set.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2024-08-14 12:55:45 +10:00
Peter Züger
4d6d84983f extmod/modtls_mbedtls: Fix DER parsing and calculation of key/cert len.
`mbedtls_pk_parse_key()` expects `key_len` to include the NULL terminator
for PEM data but not for DER encoded data.  This also applies to
`mbedtls_x509_crt_parse()` and `cert_len`.

Since all PEM data contains "-----BEGIN" this is used to check if the data
is PEM (as per mbedtls code).

This can be done for both v2 and v3 of mbedtls since the fundamental
behaviour/expectation did not change.  What changed is that in v3 the
PKCS#8 DER parser now checks that the passed key buffer is fully utilized
and no bytes are remaining (all other DER formats still do not check this).

Signed-off-by: Peter Züger <zueger.peter@icloud.com>
2024-08-13 17:32:32 +10:00
iabdalkader
1743a7d721 extmod/modopenamp: Use mp_event_* functions for poll/wait.
These are the new helper functions to use for polling/waiting.

Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2024-08-08 18:02:21 +10:00
iabdalkader
bc7e39d549 extmod/modopenamp: Fix Endpoint callback required arg.
The callback arg is not actually required.

Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2024-08-08 17:59:50 +10:00
iabdalkader
7f49897ada extmod/modopenamp: Add support for building Open-AMP on device side.
Tested with two VMs each running on a different core.

Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2024-08-08 17:59:32 +10:00
iabdalkader
4350cbcb48 extmod/modopenamp_remoteproc: Fix entry point address int overflow.
Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2024-08-08 17:59:17 +10:00
iabdalkader
1216f2c313 extmod/libmetal: Remove source file listed twice in sources.
This causes multiple definition of symbols on some builds.

Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2024-08-08 17:59:06 +10:00
Junwha
2994354634 extmod/vfs: Fix buffer overflow of string comparison in umount.
The comparison between the given unmount string and existing mount strings
were made by the given string, which leads to buffer overflow.

Fixes issue #13006.

Signed-off-by: Junwha <qbit@unist.ac.kr>
2024-07-23 12:13:49 +10:00
Terence Stenvold
390390ec37 extmod/vfs_fat: Set default volume label on mkfs if it's defined.
Using mkfs doesn't set a volume label for FAT filesystems.  This commit
will set the volume label if `MICROPY_HW_FLASH_FS_LABEL` is defined.
2024-07-23 11:39:17 +10:00
stijn
444d7bacbe extmod/moductypes: Validate the descriptor tuple.
Fixes various null dereferencing, out-of-bounds memory accesses and
`assert(0)` failures in the case of an invalid `uctypes` descriptor.

By design `uctypes` can crash because it accesses arbitrary memory, but at
least describing the descriptor layout should be forced to be correct and
not crash.

Fixes issue #12702.

Signed-off-by: stijn <stijn@ignitron.net>
2024-07-22 14:40:45 +10:00
Michael Vornovitsky
6db91dfefb extmod/modbtree: Add checks for already-closed database.
Fixes use-after-free when accessing the database after it is closed with
`btree_close`.  `btree_close` always succeeds when called with an
already-closed database.

The new test checks that operations that access the underlying database
(get, set, flush, seq) fail with a `ValueError` when the btree is already
closed.  It also checks that closing and printing the btree succeed when
the btree is already closed.

Fixes issue #12543.

Signed-off-by: Michael Vornovitsky <michaelvornovitskiy@outlook.com>
2024-07-22 10:42:29 +10:00
Damien George
8159dcc276 extmod/modos: Include os.sep entry if MICROPY_VFS is enabled.
This simplifies configuration by removing the `MICROPY_PY_OS_SEP` option
and instead including `os.sep` if `MICROPY_VFS` is enabled.  That matches
the configuration of all existing ports that enabled `os.sep` (they also
had `MICROPY_VFS` enabled), and brings consistency to other ports.

Fixes issue #15116.

Signed-off-by: Damien George <damien@micropython.org>
2024-07-20 12:32:31 +10:00
Damien George
5f3ecc29f8 extmod/modmachine: Use sys.exit as implementation of machine.soft_reset.
It does the same thing, raising `SystemExit`.

Signed-off-by: Damien George <damien@micropython.org>
2024-07-20 12:18:07 +10:00
Damien George
69c25ea865 shared/runtime/pyexec: Make a raised SystemExit always do a forced exit.
The current situation with SystemExit and soft reset is the following:
- `sys.exit()` follows CPython and just raises `SystemExit`.
- On the unix port, raising `SystemExit` quits the application/MicroPython,
  whether at the REPL or in code (this follows CPython behaviour).
- On bare-metal ports, raising `SystemExit` at the REPL does nothing,
  raising it in code will stop the code and drop into the REPL.
- `machine.soft_reset()` raises `SystemExit` but with a special flag set,
  and bare-metal targets check this flag when it propagates to the
  top-level and do a soft reset when they receive it.

The original idea here was that a bare-metal target can't "quit" like the
unix port can, and so dropping to the REPL was considered the same as
"quit".  But this bare-metal behaviour is arguably inconsistent with unix,
and "quit" should mean terminate everything, including REPL access.

This commit changes the behaviour to the following, which is more
consistent:
- Raising `SystemExit` on a bare-metal port will do a soft reset (unless
  the exception is caught by the application).
- `machine.soft_reset()` is now equivalent to `sys.exit()`.
- unix port behaviour remains unchanged.

Tested running the test suite on an stm32 board and everything still
passes, in particular tests that skip by raising `SystemExit` still
correctly skip.

Signed-off-by: Damien George <damien@micropython.org>
2024-07-20 12:13:14 +10:00
Owen
4fdad8eabe extmod/modre: Rename re_exec to re_exec_helper to avoid clash on BSD.
The `re_exec` symbol is the name of a FreeBSD regex function, so needs to
be renamed to avoid a clash when building on FreeBSD.  (This clash was
fixed once before but then accidentally reintroduced by the u-module
renaming in 7f5d5c72718af773db751269c6ae14037b9c0727.)

Fixes issue #15430.

clarify as helper function
2024-07-15 10:57:44 +10:00
robert-hh
ee1036023e extmod/machine_spi: Support firstbit=LSB for machine.SoftSPI.
Being able to send data out in LSB format can be useful, and having support
in the low-level driver is much better than requiring Python code to
reorder the bits before sending them / after receiving them.  In particular
if the hardware does not support the LSB format (eg RP2040) then one needs
to use the SoftSPI in LSB mode.

For this change a default definition of `MICROPY_PY_MACHINE_SPI_MSB/_LSB`
was added to `py/mpconfig.h`, making them available to all ports.  The
identical defines in `esp32/mpconfigport.h` were deleted.

Resolves issues #5340, #11404.

Signed-off-by: robert-hh <robert@hammelrath.com>
2024-07-12 22:59:31 +10:00
iabdalkader
20b00ca501 extmod/network_nina: Fix the AP security mode constants.
The only AP security mode supported is actually WPA/WPA2 not WEP. The
firmware command `0x19` starts the AP using `WIFI_AUTH_WPA_WPA2_PSK`
mode.

There are no functional changes in this commit, it just fixes the constant
names and removes the useless sanity checks for WEP.

Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2024-07-12 01:03:46 +10:00
robert-hh
2be45dd682 extmod/modmachine: Allow more than one argument to machine.freq().
The limit is set by a `MICROPY_PY_MACHINE_FREQ_NUM_ARGS_MAX` define, which
defaults to 1 and is set for stm32 to 4.

For stm32 this fixes a regression introduced in commit
e1ec6af654 where the maximum number of
arguments was changed from 4 to 1.

Signed-off-by: robert-hh <robert@hammelrath.com>
2024-07-11 23:38:07 +10:00
Jon Foster
92484d8822 all: Use new mp_obj_new_str_from_cstr() function.
Use new function mp_obj_new_str_from_cstr() where appropriate.  It
simplifies the code, and makes it smaller too.

Signed-off-by: Jon Foster <jon@jon-foster.co.uk>
2024-07-04 15:55:03 +10:00
Sylvain Zimmer
838794ebcc extmod/mbedtls: Enable GCM and ECDHE-RSA in common mbedtls config.
Enable support for cipher suites like
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, as suggested in
https://github.com/micropython/micropython/issues/14204#issuecomment-2024366349
and https://github.com/micropython/micropython/issues/10485#issuecomment-1396426824

Tests have been run on the top 500 domains from moz.com.  Without this
patch, 155 out of 500 fail to connect because of TLS issues.  This patch
fixes them all.  And it seems all existing mbedtls flags are needed to get
good coverage of those top 500 domains.

The `ssl_poll.py` test has the cipher bits increased from 512 to 1024 in
its test key/cert so that it can work with ECDHE-RSA which is now the
chosen cipher.

Signed-off-by: Sylvain Zimmer <sylvain@sylvainzimmer.com>
Signed-off-by: Damien George <damien@micropython.org>
2024-07-02 13:19:17 +10:00
Felix Dörre
57008a1e69 extmod/machine_usb_device: Add USBDevice.remote_wakeup method.
This simply exposes the TinyUSB function.

Signed-off-by: Felix Dörre <felix@dogcraft.de>
2024-07-02 10:23:26 +10:00
Damien George
74f52374e4 extmod/extmod.mk: Disable maybe-uninitialized warnings in libm_dbl.
These warnings are emitted by arm-none-eabi-gcc 14.1.0 with -O2 enabled.

Signed-off-by: Damien George <damien@micropython.org>
2024-06-27 11:17:59 +10:00
Damien George
8ac9c8f392 extmod/modasyncio: Add support for a callback on TaskQueue push.
Allows passing in a callback to `TaskQueue()` that is called when something
is pushed on to the queue.

Signed-off-by: Damien George <damien@micropython.org>
2024-06-20 00:11:05 +10:00
Alessandro Gatti
411d66586c extmod/modplatform: Add picolibc to the recognised libcs list.
This adds picolibc to the list of the recognised libc options.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2024-06-17 13:12:54 +10:00
Damien George
5903ee561c extmod/modlwip: Consolidate socket.accept timeout logic.
This makes the code a bit simpler to understand for the three cases of
timeout behaviour (-1, 0, non-zero), and eliminates a dependency on the
(slow) `mp_hal_delay_ms(100)` call.

Signed-off-by: Damien George <damien@micropython.org>
2024-06-08 09:02:01 +10:00
Damien George
df0d7e9429 extmod/modlwip: Make socket.connect raise ETIMEDOUT on non-zero timeout.
If the socket timeout is 0 then a failed socket.connect() raises
EINPROGRESS (which is what the lwIP bindings already did), but if the
socket timeout is non-zero then a failed socket.connect() should raise
ETIMEDOUT.  The latter is fixed in this commit.

A test is added for these timeout cases.

Signed-off-by: Damien George <damien@micropython.org>
2024-06-08 09:02:01 +10:00
robert-hh
a71471be66 extmod/network_lwip: Allow using the CIDR notation for addr4.
There was a little omisssion in the code.

Signed-off-by: robert-hh <robert@hammelrath.com>
2024-06-04 12:45:01 +10:00
robert-hh
7e7cc2b427 extmod/network_ninaw10: Implement the ipconfig methods for ninaw10.
This implements network.ipconfig() and network.WLAN.ipconfig() when the
ninaw10 driver is used for WLAN.

Due to a omission in the ninaw10 driver stack, setting the DNS address has
no effect.  But the interface is kept here just in case it's fixed
eventually.

dhcp4 and has_dhcp4 are dummy arguments.  Ninaw10 seems to always use DHCP.

Signed-off-by: robert-hh <robert@hammelrath.com>
2024-06-04 12:45:01 +10:00
Damien George
c0a25a69eb extmod/modos: Only sync FAT filesystems using disk_ioctl.
Eventually this needs to be made a generic call to the underlying VFS.  But
for now this prevents `disk_ioctl()` crashing on non-FAT filesystems.

Signed-off-by: Damien George <damien@micropython.org>
2024-05-27 12:44:02 +10:00
Jared Hancock
b1e9602702 extmod/modlwip: Use Nagle algorithm and add support for TCP_NODELAY.
This adds support to use the Nagle algorithm implemented already in lwIP to
determine when TCP data should be sent.

As currently written, MicroPython will only create packets if there is <25%
remaining in the send buffer.  Using it, sending a small message of ~50
bytes will not trigger output of the message on the network.  So it will
remained queued until the TCP interval timer expires, which can be up to
500ms.

Using Nagle's algorithm, the first write, no matter how small, will
generate a packet on the network.  And sending lots of data still makes
efficient use of the link.

In addition to this, an application designer may choose to always create
packets for every write by setting the TCP_NODELAY socket option.  That's
also implemented in this commit.
2024-05-23 22:25:06 +10:00
Jared Hancock
8762fe8b4c extmod/network_wiznet5k: Properly enable socket buffers for W5100(S).
The W5100 and W5100S only have 4 available sockets and 16kB of socket
buffer.  Allocating 16kB to either the receive or transmit buffer of a
single socket is not allowed, so the current setup does not change the
allocation for socket 0 from the reset default.  ctlwizchip is returning -1
to indicate the error, but the response isn't being inspected and probably
doesn't need to be.

Signed-off-by: Jared Hancock <jared@greezybacon.me>
2024-05-07 17:42:34 +10:00
Damien George
967ad38ac7 extmod/modasyncio: Make mp_asyncio_context variable public.
So it can be accessed by a port if needed, for example to see if asyncio
has been imported.

Signed-off-by: Damien George <damien@micropython.org>
2024-04-24 16:23:59 +10:00
iabdalkader
8936d3af46 extmod/modopenamp: Use metal logging functions exclusively.
Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2024-03-29 17:59:04 +11:00
iabdalkader
aa0f3ebe93 extmod/modopenamp: Set a default log handler for ports.
Use the existing metal log handling mechanism instead of overriding the
metal_log, which causes build issues when logging is enabled.

Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2024-03-29 17:58:18 +11:00
Damien George
c3e37d1fac extmod/modtls_axtls: Add verify_mode and CERT_NONE constant.
Supporting `verify_mode` and `CERT_NONE` is required for the new `ssl.py`
code, as well as `requests` to work.

Signed-off-by: Damien George <damien@micropython.org>
2024-03-28 17:36:13 +11:00
Damien George
8b0efde927 examples/natmod/framebuf: Enable FrameBuffer.poly method.
Signed-off-by: Damien George <damien@micropython.org>
2024-03-28 16:18:09 +11:00
iabdalkader
7753045a8f extmod: Add interface and security constants at WLAN class level.
Other constants such as `machine.Pin.OUT` are defined on the class that
uses them, rather than at the module level.  This commit makes that the
case for WLAN network interfaces, adding IF_xxx and SEC_xxx constants.

The SEC_xxx constants are named as such to match the `security` keyword
that they are used with.  And the IF_xxx constants have IF as a prefix so
they are grouped together as names.

This scheme of putting constants on the class means that only the available
features (eg security configurations) are made available as constants.  It
also means that different network interfaces (eg WLAN vs LAN) can keep
their own specific constants within their class, such as PHY_xxx for LAN.

Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2024-03-28 12:52:28 +11:00
Jared Hancock
086d4d127d extmod/network_wiznet5k: Properly enable interrupt signal on w5100s.
According to the datasheet, the IEN bit to enable the interrupt is in the
MR2 register, not the MR register.

This is just cleanup as the interrupt appears to be enabled by default
after resetting the chip.

Tested on W5100S_EVB_PICO.
2024-03-25 13:25:19 +11:00
Felix Dörre
1c6012b0b5 extmod/modnetwork: Implement IPv6 API to set and get NIC configuration.
This commit implements a new <AbstractNIC>.ipconfig() function for the NIC
classes that use lwIP, to set and retrieve network configuration for a NIC.
Currently this method supports:
- ipconfig("addr4"): obtain a tuple (addr, netmask) of the currently
  configured ipv4 address
- ipconfig("addr6"): obtain a list of tuples (addr, state,
  prefered_lifetime, valid_lifetime) of all currently active ipv6
  addresses; this includes static, slaac and link-local addresses
- ipconfig("has_dhcp4"): whether ipv4 dhcp has supplied an address
- ipconfig("has_autoconf6"): if there is a valid, non-static ipv6 address
- ipconfig(addr4="1.2.3.4/24"): to set the ipv4 address and netmask
- ipconfig(addr6="2a01::2"): to set a static ipv6 address; note that this
  does not configure an interface route, as this does not seem supported by
  lwIP
- ipconfig(autoconf6=True): to enable ipv6 network configuration with slaac
- ipconfig(gw4="1.2.3.1"): to set the ipv4 gateway
- ipconfig(dhcp4=True): enable ipv4 dhcp; this sets ipv4 address, netmask,
  gateway and a dns server
- ipconfig(dhcp4=False): stops dhcp, releases the ip, and clears the
  configured ipv4 address.
- ipconfig(dhcp6=True): enable stateless dhcpv6 to obtain a dns server

There is also a new global configuration function network.ipconfig() that
supports the following:
- network.ipconfig(dns="2a01::2"): set the primary dns server (can be a
  ipv4 or ipv6 address)
- network.ipconfig(prefer=6): to prefer ipv6 addresses to be returned as
  dns responses (falling back to ipv4 if the host does not have an ipv6
  address); note that this does not flush the dns cache, so if a host is
  already in the dns cache with its v4 address, subsequent lookups will
  return that address even if prefer=6 is set

This interface replaces NIC.ifconfig() completely, and ifconfig() should be
marked as deprecated and removed in a future version.

Signed-off-by: Felix Dörre <felix@dogcraft.de>
2024-03-19 16:42:10 +11:00
Damien George
cd8eea2ae9 all: Update extmod, ports, examples to build with new berkeley-db lib.
This provides a MicroPython-specific berkeley-db configuration in
extmod/berkeley-db/berkeley_db_config_port.h, and cleans up the include
path for this library.

Fixes issue #13092.

Signed-off-by: Damien George <damien@micropython.org>
2024-03-16 12:19:12 +11:00
Felix Dörre
01c31ea804 extmod/os_dupterm: Handle exception properly when it occurs in parallel.
When an exception is handled and the stream is closed, but while this
happens, another exception occurs or dupterm is deactivated for another
reason, the initial deactivation crashes, because its dupterm is removed.

Co-authored-by: Damien George <damien@micropython.org>

Signed-off-by: Felix Dörre <felix@dogcraft.de>
2024-03-16 10:59:03 +11:00
iabdalkader
81aba8253a extmod/modopenamp_remoteproc: Add new OpenAMP RemoteProc class.
RemoteProc provides an API to load firmware and control remote processors.

Note: port-specific operations must be implemented to support this class.

Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2024-03-15 18:11:28 +11:00
iabdalkader
14dae42fe7 extmod/modopenamp: Add new OpenAMP module.
This module implements OpenAMP's basic initialization and shared resources
support, and provides support for OpenAMP's RPMsg component, by providing
an `endpoint` type (a logical connection on top of RPMsg channel) which can
be used to communicate with the remote core.

Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2024-03-15 18:11:28 +11:00
iabdalkader
61ee59ad89 extmod/libmetal: Add MicroPython platform for libmetal.
Add a MicroPython platform for libmetal, based on the generic platform.
The MicroPython platform uses common mp_hal_xxx functions and allows ports
to customize default configurations for libmetal.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2024-03-15 18:11:28 +11:00
Angus Gratton
9d0d262be0 extmod/machine_usb_device: Add support for Python USB devices.
This new machine-module driver provides a "USBDevice" singleton object and
a shim TinyUSB "runtime" driver that delegates the descriptors and all of
the TinyUSB callbacks to Python functions.  This allows writing arbitrary
USB devices in pure Python.  It's also possible to have a base built-in
USB device implemented in C (eg CDC, or CDC+MSC) and a Python USB device
added on top of that.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2024-03-15 14:22:11 +11:00
Daniël van de Giessen
58a596f4a9 extmod/nimble: Check for active before setting address mode.
`BLE().config(addr_mode=...)` is not safe to call if the NimBLE stack is
not yet active (because it tries to acquire mutexes which should be
initialized first).

Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
2024-03-15 12:59:58 +11:00
Damien George
bfc3dde2c9 extmod/modmachine: Add MICROPY_PY_MACHINE_RESET configuration option.
Disabled by default, but enabled on all boards that previously had
`MICROPY_PY_MACHINE_BARE_METAL_FUNCS` enabled.

Signed-off-by: Damien George <damien@micropython.org>
2024-03-15 12:04:37 +11:00
Damien George
dd134e4836 extmod/modmachine: Add MICROPY_PY_MACHINE_MEMX configuration option.
Enabled by default.

Signed-off-by: Damien George <damien@micropython.org>
2024-03-15 12:04:34 +11:00
Damien George
23ccbcf230 extmod/modmachine: Add MICROPY_PY_MACHINE_SIGNAL configuration option.
Enabled by default.

Signed-off-by: Damien George <damien@micropython.org>
2024-03-15 12:04:25 +11:00
iabdalkader
c231c89651 extmod/network_ninaw10: Fix error messages.
Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2024-03-14 18:26:41 +01:00
iabdalkader
2b6f81f2b9 extmod/network_ninaw10: Set the proper security mode if none provided.
If no security mode is provided, use WPA for station and WEP for AP.  Note
only WEP is supported in AP mode.

Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2024-03-14 17:40:25 +11:00
iabdalkader
8b4a21cd64 extmod/network_ninaw10: Activate the NIC on demand.
Activate the NIC on calls to connect() or config() if it's not already
active. This change makes the NINA NIC more in line with CYW43 and other
NICs, which allow configuring the NIC before or after it is activated.

Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2024-03-14 17:40:18 +11:00
MikeTeachman
0b145fdc2d esp32/machine_i2s: Integrate new I2S IDF driver.
The legacy I2S "shim" is removed and replaced by the new I2S driver.  The
new driver fixes a bug where mono audio plays only in one channel.

Application code size is reduced by 2672 bytes with this change.  Tested on
ESP32, ESP32+spiram, ESP32-S3 using example code from
https://github.com/miketeachman/micropython-i2s-examples

Signed-off-by: Mike Teachman <mike.teachman@gmail.com>
2024-03-08 13:31:02 +11:00
Phil Howard
dda9b9c6da all: Prune trailing whitespace.
Prune trailing whitespace across the whole project (almost), done
automatically with:

    grep -IUrl --color "[[:blank:]]$" --exclude-dir=.git --exclude=*.exp |\
        xargs sed -i 's/[[:space:]]*$//'

Exceptions:
- Skip third-party code in lib/ and drivers/cc3100/
- Skip generated code in bluetooth_init_cc2564C_1.5.c
- Preserve command output whitespace in docs, eg:
  docs/esp8266/tutorial/repl.rst

Signed-off-by: Phil Howard <phil@gadgetoid.com>
2024-03-07 16:25:17 +11:00
Angus Gratton
decf8e6a8b all: Remove the "STATIC" macro and just use "static" instead.
The STATIC macro was introduced a very long time ago in commit
d5df6cd44a.  The original reason for this was
to have the option to define it to nothing so that all static functions
become global functions and therefore visible to certain debug tools, so
one could do function size comparison and other things.

This STATIC feature is rarely (if ever) used.  And with the use of LTO and
heavy inline optimisation, analysing the size of individual functions when
they are not static is not a good representation of the size of code when
fully optimised.

So the macro does not have much use and it's simpler to just remove it.
Then you know exactly what it's doing.  For example, newcomers don't have
to learn what the STATIC macro is and why it exists.  Reading the code is
also less "loud" with a lowercase static.

One other minor point in favour of removing it, is that it stops bugs with
`STATIC inline`, which should always be `static inline`.

Methodology for this commit was:

1) git ls-files | egrep '\.[ch]$' | \
   xargs sed -Ei "s/(^| )STATIC($| )/\1static\2/"

2) Do some manual cleanup in the diff by searching for the word STATIC in
   comments and changing those back.

3) "git-grep STATIC docs/", manually fixed those cases.

4) "rg -t python STATIC", manually fixed codegen lines that used STATIC.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2024-03-07 14:20:42 +11:00