Updates the Zephyr port build instructions and CI to use the latest
Zephyr release tag.
Tested on max32690fthr and frdm_k64f.
Signed-off-by: Maureen Helm <maureen.helm@analog.com>
This is a fix for utf-8 decoding errors that are thrown when non-utf-8
content is received. For instance during a reboot of an ESP8266 module.
The fix is to handle conversion errors by replacing illegal characters.
Note that these illegal characters most often occur during an MCU reboot
sequence when the MCU is using baudrates different from 115200.
Signed-off-by: Jos Verlinde <Jos_Verlinde@hotmail.com>
Detection of ESP-XX devices was based on just the names of the USB driver
name, and did not account for the switch of the newer ESP-xx devices to
USB-CDC. On Windows this caused unwanted/unneeded resets as the DTR/RTS
signals are also used for automatic device reset over USB-CDC. See
https://github.com/micropython/micropython/issues/9659#issuecomment-3124704572
This commit uses the Espressif registered VID 0x303A to detect USB-CDC
ports, to enable the same DTR/RTS settings as used on the UART-USB
connection.
Also improved the robustness of the code using `getattr()`.
Signed-off-by: Jos Verlinde <Jos_Verlinde@hotmail.com>
Not all errors defined in stdlib errno are available on Windows.
Specifically, errno.ENOTBLK is not.
Fixes issue #17773.
Signed-off-by: Jos Verlinde <Jos_Verlinde@hotmail.com>
Commit dc2fcfcc55 seems to have accidentally
changed the ruff quote style to "preserve", instead of keeping it at the
default which is "double".
Put it back to the default and update relevant .py files with this rule.
Signed-off-by: Damien George <damien@micropython.org>
This is a patch release of the IDF. Comparing with 5.4.1, firmware size is
up by about 1.5k on ESP32 and 9k on ESP32-S3. But IRAM usage (of the IDF)
is down by about 500 byte on ESP32 and DRAM usage is down by about 20k on
ESP32 and 10k on ESP32-S3.
Testing on ESP32, ESP32-S2, ESP32-S3 and ESP32-C3 shows no regressions,
except in BLE MTU ordering (the MTU exchange event occuring before the
connect event).
Signed-off-by: Damien George <damien@micropython.org>
The `run-perfbench.py` test is run as part of CI, but the actual
performance results are not used. Rather, the test is just testing that
all the performance tests run correctly.
So there's no need to run with the default averaging of 8 (which runs each
test 8 times and takes the average time for the performance result) which
can take a lot of time for slower builds, eg unix sanitize, settrace and
stackless builds.
This commit changes the averaging to just 1.
Signed-off-by: Damien George <damien@micropython.org>
This test passes sometimes and fails other times. Eventually that should
be fixed, but for now just skip this test.
Signed-off-by: Damien George <damien@micropython.org>
This test passes sometimes and fails other times. Eventually that should
be fixed, but for now just skip this test.
Signed-off-by: Damien George <damien@micropython.org>
Stackless mode makes `tests/thread/stress_aes.py` slow, around 75 seconds
for this CI job (probably due to contention among the many threads for the
GC lock, to allocate frames for function calls). So increase the timeout
to allow this test to pass.
Signed-off-by: Damien George <damien@micropython.org>
The qemu emulation introduces enough overhead that the
`tests/thread/stress_aes.py` test overruns the default timeout. So
increase it to allow this test to pass.
Signed-off-by: Damien George <damien@micropython.org>
This commit fixes building the "btree" example natmod on RV32 when
Picolibc is being used and uses thread-local storage for storing the
errno variable.
The fix is surprisingly simple: Picolibc allows overriding the function
that will provide a pointer to the "errno" variable, and the btree
natmod integration code already has all of this machinery set up as part
of its library integration. Redirecting Picolibc to the already
existing pointer provider function via a compile-time definition is
enough to let the module compile and pass QEMU tests.
This workaround will work on any Picolibc versions (Arm, RV32, Xtensa,
etc.) even if TLS support was not enabled to begin with, and will
effectively do nothing if the toolchain used will rely on Newlib to
provide standard C library functions.
Given that the btree module now builds and passes the relevant natmod
tests, said module is now part of the QEMU port's natmod testing
procedure, and CI now will build the btree module for RV32 as part to
its checks.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
There have been recent build failures in build_renesas_ra_board. It
appears to be the case that a security update for this package was recently
issued by Ubuntu for CVE-2025-4565 and the buggy version is no longer on
package servers. However, it is still referred to by the cached apt
metadata in the GitHub runners.
Add `apt-get update` to fix this, and audit for other sites in `ci.sh`
where it might also be necessary.
Signed-off-by: Jeff Epler <jepler@gmail.com>
Targets without the `errno` module enabled will not render `OSError`s
with the name of the error. Instead they just print the numeric error
code.
Add support for such targets by explicitly recognising certain error codes.
Signed-off-by: Damien George <damien@micropython.org>
This removes the need for an explicit `sys_settrace_features.py.exp` file.
This means that people testing locally will also need to install Python
3.11 in some way, such as with pyenv or uv, and use it during
`make VARIANT=coverage test`, or they will get failures.
When using Python from GitHub actions/setup-python, pip3 can't be wrapped
by sudo, because this invokes the operating system python instead.
Signed-off-by: Jeff Epler <jepler@gmail.com>
The additional overhead of the settrace profiler means that the
`aes_stress.py` test was running too slowly on GitHub CI. Double the
timeout to 60 seconds.
Signed-off-by: Jeff Epler <jepler@gmail.com>
Changes in this commit:
- Handle SerialException on Windows when device disconnects.
- Print clean 'device disconnected' message instead of stack trace.
- Fix terminal formatting issues on Linux after disconnect.
- Return disconnected state after console cleanup to avoid terminal issues.
This ensures proper disconnect messages on both platforms without showing
confusing error traces to users.
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
This check, runtime-enabled by default in gcc 13 (and existing at least
since gcc 12, but runtime-disabled) changes the stack layout in ways that
are not compatible with assumptions spread across the core code (nlr, gc,
and stack checking).
Signed-off-by: Jeff Epler <jepler@gmail.com>
The existing `mpremote df` command is not very good, because it needs to
assume that all directories in the root directory are mount points, and
also doesn't correctly stat filesystems when the current directory is not
the root. This leads to wrong results
With the introduction of `vfs.mount()` to return a list of mounted
filesystems and their path, a much better df can be implemented, as done in
this commit.
The new df will also fall back to using the old approach of listing the
root directory if the no-arg `vfs.mount()` query is not supported.
Signed-off-by: Damien George <damien@micropython.org>
Add support for R_ARM_ABS32 relocations in native .mpy files. These can be
rewritten in the same way that data relocations are.
Fixes issue #14430.
Signed-off-by: Damien George <damien@micropython.org>
This commit fixes CI test runs for the `nanbox` target, which were
broken by the unconditional native emitter code output changes in the
test runner.
The `nanbox` configuration does not enable native emitters of any kind,
and with a full test run that includes executing emitted native code
things would break when doing CI runs.
This is worked around by introducing a common subset of tests that do
not involve the native emitter, and a more comprehensive set of tests
that include both non-emitter and emitter tests. The `nanbox` CI test
run will stop at the first subset, whilst other configurations will run
that and execute further tests.
Function names have been kept the same for steps that involve native
code, with the `nanbox` subset having another one. This should not
trigger any breakage in existing CI configurations or external scripts.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
`tools/boardgen.py` is used by the `make-pins.py` scripts in many ports to
generate the pin definitions for the machine module.
In #17391 it was found that this is currently generating the C structs for
board pin definitions with inconsistent ordering (across different build
runs), which makes it sometimes impossible to get a consistent binary file
even for no change in source files.
This commit fixes that by sorting the board pin names alphabetically.
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
This commit takes the QEMU/Arm CI build and test step and splits it into
three separate steps (bigendian, sabrelite, thumb), to allow them to run
in parallel.
Currently the QEMU/Arm CI build step would take up to 16 minutes, often
being the last step blocking a full test run. With this commit, when
the steps run in parallel the time it takes to complete the QEMU/Arm
build and test procedure is cut in half - taking between 8 to 9 minutes
depending on the CI runner load.
The existing `ci_build_and_test_arm` function has been removed, in
favour of having three separate functions - one per configuration. They
are called `ci_build_and_test_arm_bigendian`,
`ci_build_and_test_arm_sabrelite`, and `ci_build_and_test_arm_thumb`.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit fixes a small yet harmless issue that occurs when invoking
`ci_native_mpy_modules_build` on a persistent environment, as only X64
MPY files would be removed by the cleaning process.
Now the correct architecture is passed at all times when cleaning before
building a natmod for a particular architecture, forcing a full build of
all files to better simulate the CI environment (where there's no state
persisted between runs for this step).
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit lets the CI pipeline build all natmods for the Xtensa
target, now that ROM symbols can be used in the linking process.
The restriction was put in place due to build failures on certain
natmods for Xtensa, as ROM symbols would not be used, causing undefined
symbol errors at build time.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit lets mpy_ld.py resolve symbols not only from the object
files involved in the linking process, or from compiler-supplied static
libraries, but also from a list of symbols referenced by an absolute
address (usually provided by the system's ROM).
This is needed for ESP8266 targets as some C stdlib functions are
provided by the MCU's own ROM code to reduce the final code footprint,
and therefore those functions' implementation was removed from the
compiler's support libraries. This means that unless `LINK_RUNTIME` is
set (which lets tooling look at more libraries to resolve symbols) the
build process will fail as tooling is unaware of the ROM symbols'
existence. With this change, fixed-address symbols can be exposed to
the symbol resolution step when performing natmod linking.
If there are symbols coming in from a fixed-address symbols list and
internal code or external libraries, the fixed-address symbol address
will take precedence in all cases.
Although this is - in theory - also working for the whole range of ESP32
MCUs, testing is currently limited to Xtensa processors and the example
natmods' makefiles only make use of this commit's changes for the
ESP8266 target.
Natmod builds can set the MPY_EXTERN_SYM_FILE variable pointing to a
linkerscript file containing a series of symbols (weak or strong) at a
fixed address; these symbols will then be used by the MicroPython
linker when packaging the natmod. If a different natmod build method is
used (eg. custom CMake scripts), `tools/mpy_ld.py` can now accept a
command line parameter called `--externs` (or its short variant `-e`)
that contains the path of a linkerscript file with the fixed-address
symbols to use when performing the linking process.
The linkerscript file parser can handle a very limited subset of
binutils's linkerscript syntax, namely just block comments, strong
symbols, and weak symbols. Each symbol must be in its own line for the
parser to succeed, empty lines or comment blocks are skipped. For an
example of what this parser was meant to handle, you can look at
`ports/esp8266/boards/eagle.rom.addr.v6.ld` and follow its format.
The natmod developer documentation is also updated to reflect the new
command line argument accepted by `mpy_ld.py` and the use cases for the
changes introduced by this commit.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
If the USB serial device locks up, then writes to that device can hang
forever. That can make the test runner `tests/run-tests.py` lock up, among
other problems.
This commit introduces a 5 second write-timeout, and catches any OSError's
raised during `enter_raw_repl()`. Now, if a USB serial device locks up,
`enter_raw_repl()` will eventually raise an exception.
Signed-off-by: Damien George <damien@micropython.org>
This applies the mpremote commit 03fe9c55ea
to pyboard.py.
The `timeout_overall` is used in `enter_raw_repl()`. It prevents waiting
forever for a serial device that does not respond to the Ctrl-C/Ctrl-D/etc
commands and is constantly outputting data.
Signed-off-by: Damien George <damien@micropython.org>
This applies the mpremote commit 0d46e45a1f
to pyboard.py.
If the target does not return any data then `read_until()` will block
indefinitely. Fix this by making the initial read part of the general read
look, which always checks `inWaiting() > 0` before reading from the serial
device.
Also added the UART timeout to the constructor. This is not currently used
but may be used as an additional safeguard.
Signed-off-by: Damien George <damien@micropython.org>
Add `mpremote fs tree` command to show a tree of the device's files. It:
- Shows a treeview from current path or specified path.
- Uses the graph chars ("├── ", "└── ") (not configurable).
- Has the options:
-v/--verbose adds the serial device name to the top of the tree
-s/--size add a size to the files
-h/--human add a human readable size to the files
Signed-off-by: Jos Verlinde <jos_verlinde@hotmail.com>
This version of the IDF uses about 1KB more IRAM and 1KB more DRAM on most
boards, but 6.5KB more DRAM usage on the S3. It seems that's due to a lot
of small increases in many components.
Signed-off-by: Ihor Nehrutsa <Ihor.Nehrutsa@gmail.com>
Previously, the navigation ended up messy when the (long) description of
the item became used as a 2nd level header, meaning that it was placed in
the navigation. Check for this when generating cpydiff so that new cases
don't sneak in unnoticed.
Signed-off-by: Jeff Epler <jepler@gmail.com>
The heading character for the difference title was always "~", but items
had been added which had just a single heading level. This made the
generated table of contents confused about heading levels, because heading
levels are not fixed in rst, but are inferred from the order they appear in
the document.
Signed-off-by: Jeff Epler <jepler@gmail.com>
In the syntax_space cpydiff, all the warnings were shown after the other
output. This is because the output always showed all of stdout first and
all of stdout second.
By running Python in unbuffered mode and using `stderr=STDOUT`, the two
streams are interleaved in exactly the order they're printed, so the
SyntaxWarnings are interleaved with the other output.
By using the `encoding=` argument of Popen, the need to explicitly convert
to utf-8 is avoided. The encoding of the input also becomes utf-8 in this
case, which all the test cases are (well, they're all ASCII, I think). As
in `run-tests.py`, setting PYTHONIOENCODING ensures the Python
interpreter's input and output are in utf-8, which is not always the case,
especially on Windows systems.
I spot-checked the generated doc pages and they all seemed to make sense
still.
Signed-off-by: Jeff Epler <jepler@gmail.com>
