This commit cleans up the Viper code generation blocks for
register-indexed load and store operations.
An attempt is made to simplify the code in the common code generator
code block, by moving architecture-specific code to the appropriate
native generation backends whenever possible. This should make that
specific bit of code in the Viper generator clearer and easier to
maintain in the long term.
To achieve this, six generic assembler meta-opcodes have been
introduced, named `ASM_{LOAD,STORE}{8,16,32}_REG_REG_REG`. A
platform-independent implementation for those operations is provided, so
backends that cannot emit a shorter sequence for the requested operation
or are fine with the platform-independent implementation can just not
provide said meta-opcodes.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit lets the Viper code generator use optimised code sequence
for register-indexed load and store operations when generating Thumb
code.
Register-indexed load and store operations for Thumb now can take at
most two machine opcodes for halfword and word values, and just a single
machine opcode for byte values. The original implementation could
generate up to four opcodes in the worst case (dealing with word
values).
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit lets the Viper code generator use optimised code sequences
for register-indexed load and store operations when generating Arm code.
The existing code defaulted to generic multi-operations code sequences
for Arm code on most cases. Now optimised implementations are provided
for register-indexed loads and stores of all data sizes, taking at most
two machine opcodes for each operation.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This groups non-decimal values by fours, such as bbb_bbbb_bbbb. It also
supports `{:_d}` to use underscore for decimal numbers (grouped in threes).
Use of incorrect ":,b" is not diagnosed.
Thanks to @dpgeorge for the suggestion to reduce code size.
Signed-off-by: Jeff Epler <jepler@gmail.com>
This commit renames the NORETURN macro, indicating to the compiler
that a function does not return, into MP_NORETURN to maintain the same
naming convention of other similar macros.
To maintain compaitiblity with existing code NORETURN is aliased to
MP_NORETURN, but it is also deprecated for MicroPython v2.
This changeset was created using a similar process to
decf8e6a8b ("all: Remove the "STATIC"
macro and just use "static" instead."), with no documentation or python
scripts to change to reflect the new macro name.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Avoids the new Wunterminated-string-literal when compiled with gcc 15.1.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
IOBase is quite an important building block of other parts of the system,
such as `mpremote mount` and running .mpy and native tests.
This feature costs +244 bytes of firmware size on ARM Thumb2 architectures,
which is worth the cost for the extra features it enables.
The change here means that `io.IOBase` is now enabled on all nrf boards,
(previously it was only nRF52840 and nRF9160) and also B_L072Z_LRWAN1
(there is no change to other ports or boards).
Signed-off-by: Damien George <damien@micropython.org>
This ensures the check in MP_NLR_JUMP_HEAD works as expected and
nlr_jump_fail gets called so we get a bit better error message.
Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
Fixes thread safety issue that could cause memory corruption on ports
with (MICROPY_PY_THREAD && !MICROPY_PY_THREAD_GIL) - currently only rp2 and
unix have this configuration.
Adds unit test for TLS sockets that exercises this code path. I wasn't
able to make this fail on rp2, the race condition window is pretty narrow
and may not have a direct impact on a quiet system.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
This requires explicitly naming and initializing all members so add that
where needed and possible. For MP_DEFINE_NLR_JUMP_CALLBACK_FUNCTION_1
this would require initializing the .callback member, but that's a bit
of a waste since the macro is always followed by a call to
nlr_push_jump_callback() to initialize exactly that member, so rewrite
the macro without initializers.
Signed-off-by: stijn <stijn@ignitron.net>
This fix handles attrtuple as well, eg. os.uname(). A test case has been
added in basics/attrtuple2.py.
Fixes issue #16969.
Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
The "index fixing" behavior of get_fast_slice_indexes are not desired here;
the underlying behavior of mp_obj_slice_indexes actually is.
Fixes issue #17016.
Signed-off-by: Jeff Epler <jepler@gmail.com>
Otherwise, when compiling on 16-bit systems (where `mp_uint_t` is 16 bits
wide) the compiler warns about "left shift count >= width of type", from
the static inline functions that have RV32_ENCODE_TYPE_xxx macros which
do a lot of bit shifting.
Signed-off-by: Damien George <damien@micropython.org>
Rather than having Make calling CMake to generate a list of submodules and
then run a Make target (which is complex and prone to masking other
errors), implement the submodule update logic in CMake itself.
Internal CMake-side changes are that GIT_SUBMODULES is now a CMake list,
and the trigger variable name is changed from ECHO_SUBMODULES to
UPDATE_SUBMODULES.
The run is otherwise 100% a normal CMake run now, so most of the other
special casing can be removed.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
This is a pretty fundamental built-in and having CPython-compatible
behaviour is beneficial. The code size increase is not much, and
ports/boards can still disable it if needed to save space.
Addresses issue #5384.
Signed-off-by: Damien George <damien@micropython.org>
This commit introduces an additional symbol resolution mechanism to the
natmod linking process. This allows the build scripts to look for required
symbols into selected libraries that are provided by the compiler
installation (libgcc and libm at the moment).
For example, using soft-float code in natmods, whilst technically possible,
was not an easy process and required some additional work to pull it off.
With this addition all the manual (and error-prone) operations have been
automated and folded into `tools/mpy_ld.py`.
Both newlib and picolibc toolchains are supported, albeit the latter may
require a bit of extra configuration depending on the environment the build
process runs on. Picolibc's soft-float functions aren't in libm - in fact
the shipped libm is nothing but a stub - but they are inside libc. This is
usually not a problem as these changes cater for that configuration quirk,
but on certain compilers the include paths used to find libraries in may
not be updated to take Picolibc's library directory into account. The bare
metal RISC-V compiler shipped with the CI OS image (GCC 10.2.0 on Ubuntu
22.04LTS) happens to exhibit this very problem.
To work around that for CI builds, the Picolibc libraries' path is
hardcoded in the Makefile directives used by the linker, but this can be
changed by setting the PICOLIBC_ROOT environment library when building
natmods.
Signed-off-by: Volodymyr Shymanskyy <vshymanskyi@gmail.com>
Co-authored-by: Alessandro Gatti <a.gatti@frob.it>
This function will attempt to create a `VfsRom` instance and mount it at
location "/rom" in the filesystem.
Signed-off-by: Damien George <damien@micropython.org>
This is a generic interface to allow querying and modifying the read-only
memory area of a device, if it has such an area.
Signed-off-by: Damien George <damien@micropython.org>
For a given MicroPython firmware/executable it can be sometimes important
to know how it was built, which variant/board configuration it came from.
This commit adds a new field `sys.implementation._build` that can help
identify the configuration that MicroPython was built with.
For now it's either:
* <VARIANT> for unix, webassembly and windows ports
* <BOARD>-<VARIANT> for microcontroller ports (the variant is optional)
In the future additional elements may be added to this string, separated by
a hyphen.
Resolves issue #16498.
Signed-off-by: Damien George <damien@micropython.org>
This change allows tuples to be passed as the prefix/suffix argument to the
`str.startswith()` and `str.endswith()` methods. The methods will return
`True` if the string starts/ends with any of the prefixes/suffixes in the
tuple.
Also adds full support for the `start` and `end` arguments to both methods
for compatibility with CPython.
Tests have been updated for the new behaviour.
Signed-off-by: Glenn Moloney <glenn.moloney@gmail.com>
The mantissa parsing code uses a floating point variable to accumulate
digits. Using an `mp_float_uint_t` variable instead and casting to
`mp_float_t` at the very end reduces code size. In some cases, it also
improves the rounding behaviour as extra digits are taken into account
by the int-to-float conversion code.
An extra test case handles the special case where mantissa overflow occurs
while processing deferred trailing zeros.
Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
This commit changes the Xtensa inline assembly parser to use a slightly
simpler (and probably a tiny bit more efficient) way to look up register
names when decoding instruction parameters.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit introduces a few changes aimed at reducing the amount of
space taken by the inline assembler once compiled:
* The register string table uses 2 bytes for each qstr rather than the
usual 4
* The opcode table uses 2 bytes for each qstr rather than the usual 4
* Opcode masks are not embedded in each opcode entry but looked up via
an additional smaller table, reducing the number of bytes taken by
an opcode's masks from 12 to 2 (with a fixed overhead of 24 bytes for
the the masks themselves stored elsewhere)
* Some error messages had a trailing period, now removed
* Error messages have been parameterised when possible, and the overall
text length is smaller.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit fixes a compilation warning (turned error) about a
potentially uninitialised variable being used. The warning can be
ignored as the variable in question is always written to, but the code
has been changed to silence that warning.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit upgrades from codespell==2.2.6 to the current codespell==2.4.1,
adding emac to the ignore-words-list.
Signed-off-by: Christian Clauss <cclauss@me.com>
This works similarly to the existing support in "bare metal" make ports,
with the caveat that CMake will only set this value on a clean build and
will reuse the previous value otherwise.
This is slightly different to the CMake built-in support for CFLAGS,
as this variable is used when evaluating source files for qstr
generation, etc.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
This commit implements a small subset of the CPython `marshal` module. It
implements `marshal.dumps()` and `marshal.loads()`, but only supports
(un)marshalling code objects at this stage. The semantics match CPython,
except that the actual marshalled bytes is not compatible with CPython's
marshalled bytes.
The module is enabled at the everything level (only on the unix coverage
build at this stage).
Signed-off-by: Damien George <damien@micropython.org>
This allows retrieving the code object of a function using
`function.__code__`, and then reconstructing a function from a code object
using `FunctionType(code_object)`.
This feature is controlled by `MICROPY_PY_FUNCTION_ATTRS_CODE` and is
enabled at the full-features level.
Signed-off-by: Damien George <damien@micropython.org>
The `mp_obj_code_t` and `mp_type_code` code object was defined internally
in both `py/builtinevex.c` and `py/profile.c`, with completely different
implementations (the former very minimal, the latter quite complete).
This commit factors these implementations into a new, separate source file,
and allows the code object to have four different modes, selected at
compile-time:
- MICROPY_PY_BUILTINS_CODE_NONE: code object not included in the build.
- MICROPY_PY_BUILTINS_CODE_MINIMUM: very simple code object that just holds
a reference to the function that it represents. This level is used when
MICROPY_PY_BUILTINS_COMPILE is enabled.
- MICROPY_PY_BUILTINS_CODE_BASIC: simple code object that holds a reference
to the proto-function and its constants.
- MICROPY_PY_BUILTINS_CODE_FULL: almost complete implementation of the code
object. This level is used when MICROPY_PY_SYS_SETTRACE is enabled.
Signed-off-by: Damien George <damien@micropython.org>
This commit fixes two Xtensa sequences in order to terminate early when
loading and storing word values via an immediate index.
This was meant to be part of 55ca3fd675
but whilst it was part of the code being tested, it didn't end up in the
commit.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit marks as const the condition code tables used when figuring
out which opcode sequence must be emitted depending on the requested
comparison type.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
When a port automatically compiles `mpy-cross`, if `USER_C_MODULES` is
provided by the user on the command line then it is also applied to the
`mpy-cross` build. That can lead to build errors if the path is relative
and not found when building `mpy-cross`.
Fix that by explicitly resetting `USER_C_MODULES` when invoking the
`mpy-cross` build.
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
- Renamed gc_sweep to gc_sweep_free_blocks.
- Call gc_sweep_run_finalisers from top level.
- Reordered the gc static functions to be in approximate
runtime sequence (with forward declarations) rather than
in declaration order.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Do this by tracking being inside gc collection with a
separate flag, GC_COLLECT_FLAG. In gc_free(),
ignore this flag when determining if the heap is locked.
* For finalisers calling gc_free() when heap is otherwise unlocked,
this allows memory to be immediately freed (potentially
avoiding a MemoryError).
* Hard IRQs still can't call gc_free(), as heap will be locked via
gc_lock().
* If finalisers are disabled then all of this code can be compiled
out to save some code size.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Currently a finalizer may run and access memory which has already been
freed. (This happens mostly during gc_sweep_all() but could happen during
any garbage collection pass.)
Includes some speed improvement tweaks to skip empty FTB blocks. These help
compensate for the inherent slowdown of having to walk the heap twice.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Follow up to 13b13d1fdd, based on some
testing on godbolt, the manual code optimisation seems unnecessary for code
size, at least on gcc x86_64 and ARM, and it's definitely not good for
clarity.
Signed-off-by: Jeff Epler <jepler@gmail.com>
The way CMake gathers the submodule list, it can quietly be empty
if the previous step fails. This makes it an explicit error.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
This commit introduces the ability to emit optimised code paths on
Xtensa for load/store operations indexed via an immediate offset.
If an immediate offset for a load/store operation is within a certain
range that allows it to be embedded into an available opcode then said
opcode is emitted instead of the generic code sequence.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit improves the RV32 code sequence that is emitted if a
function needs to set up an exception handler as its prologue.
The old code would clear a temporary register and then copy that value
to places that needed to be initialised with zero values. On RV32
there's a dedicated register that's hardwired to be equal to zero, which
allows us to bypass the extra register clear and use the zero register
to initialise values.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit improves the emitted code sequences for address generation
in the Viper subsystem when loading/storing 16 and 32 bit values via a
register offset.
The Xtensa opcodes ADDX2 and ADDX4 are used to avoid performing the
extra shifts to align the final operation offset. Those opcodes are
available on both xtensa and xtensawin MicroPython architectures.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This includes making int("01") parse in base 10 like standard Python.
When a base of 0 is specified it means auto-detect based on the prefix, and
literals begining with 0 (except when the literal is all 0's) like "01" are
then invalid and now throw an exception.
The new error message is different from CPython. It says e.g.,
`SyntaxError: invalid syntax for integer with base 0: '09'`
Additional test cases were added to cover the changed & added code.
Co-authored-by: Damien George <damien@micropython.org>
Signed-off-by: Jeff Epler <jepler@gmail.com>
The esp32 IDF toolchain can give a "may be used uninitialized" warning, at
least for ESP32-S3 with gcc 14.2.0. Silence that warning by initializing
the variable with NULL.
Co-authored-by: Daniel van de Giessen <daniel@dvdgiessen.nl>
Signed-off-by: IhorNehrutsa <Ihor.Nehrutsa@gmail.com>
This commit fixes code generation for loading halfwords using an offset
greater than 255.
The old code blindly encoded the offset into a `LDRH Rd, [Rn, #imm]`
opcode, but only the lowest 8 bits would be put into the opcode itself.
This commit instead generates a two-opcodes sequence, a constant load into
R8, and then `LDRH Rd, [Rn, R8]`.
This fixes `tests/extmod/vfs_rom.py` for the qemu/SABRELITE board.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit fixes code generation for loading a local's address if its
index is greater than 63.
The old code blindly encoded the offset into an `ADD Rd, Rn, #imm` opcode,
but only the lowest 8 bits would be put into the opcode itself. This
commit instead generates a two-opcodes sequence, a constant load into R8,
and then an `ADD Rd, Rn, R8` opcode.
This fixes `tests/float/math_domain.py` for the qemu/SABRELITE board.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Prior to this fix, the assembler generated `LDRH Rd, [Rn, #imm]!`, so the
second `LDRH` from the same origin would load from the wrong base.
Co-authored-by: Alessandro Gatti <a.gatti@frob.it>
Signed-off-by: Damien George <damien@micropython.org>
This commit adds support for writing inline assembler functions when
targeting a RV32IMC processor.
Given that this takes up a bit of rodata space due to its large
instruction decoding table and its extensive error messages, it is
enabled by default only on offline targets such as mpy-cross and the
qemu port.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This makes the existing popcount(uint32_t) implementation found in the
RV32 emitter available to the rest of the codebase. This version of
popcount will use intrinsic or builtin implementations if they are
available, falling back to a generic implementation if that is not the
case.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This adds an optimisation for loading .mpy files from a reader that points
to ROM. In such a case qstr, str and bytes data, along with bytecode, are
all referenced in-place in ROM.
Signed-off-by: Damien George <damien@micropython.org>
This commit defines a new ROMFS filesystem for storing read-only files that
can be memory mapped, and a new VfsRom driver. Files opened from this
filesystem support the buffer protocol. This allows naturally getting the
memory-mapped address of the file using:
- memoryview(file)
- uctypes.addressof(file)
Furthermore, if these files are .mpy files then their content can be
referenced in-place when importing. Such imports take up a lot less RAM
than importing from a normal filesystem. This is essentially dynamically
frozen .mpy files, building on the revamped v6 .mpy file format.
Signed-off-by: Damien George <damien@micropython.org>
This commit adds the compiled native module file to the list of files to
remove when `make clean` is issued in a native module source directory.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit adds support for RV32IMC native modules, as in embedding native
code into a self-contained MPY module and and make its exported functions
available to the MicroPython environment.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
MicroPython relies on a number of submodules for third party and chip
vendor libraries. Users need to check these out before building their
desired ports and Github Actions CI here needs to clone them all multiple
times for every build. Many of these are getting significantly larger over
time, slowing down usage and consuming more disk space.
Newer versions of git have features to avoid pulling all historic / blob
data which can have a significant impact of total data use. This commit
uses a standard feature of git to do a partial clone, with automatic
fallback to previous behavior on error.
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
This commit fixes a warning occurring on Clang when calling
`__builtin___clear_cache` with non-void pointers for its start and end
memory area locations. The code now uses a char pointer for the end
location, and it still builds without warnings on GCC.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Use an explicit cast to suppress the implicit conversion which started
popping up in recent compiler versions (and wasn't there yet in 07bf3179).
Signed-off-by: stijn <stijn@ignitron.net>
Allows verbose build to work the same on esp32 port as other ports.
To minimise copy/paste, split the BUILD_VERBOSE section of mkenv.mk
out to its own verbose.mk and include this in the port Makefile.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Recent MSVC versions have changed the definition of NAN to a non-constant
expression! This is a bug, C standard says it should be a constant.
Good explanation and workaround at: https://stackoverflow.com/a/79199887
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
In `deque_subscr()`, if `index_val` equals `self->alloc`, the index
correction `index_val -= self->alloc` does not execute, leading to an
out-of-bounds access in `self->items[index_val]`.
The fix in this commit ensures that the index correction is applied
whenever `index_val >= self->alloc`, preventing access beyond the allocated
buffer size.
Signed-off-by: Jan Sturm <jansturm92@googlemail.com>
The PIC16 port didn't catch up with the other ports, so it required a bit
of work to make it build with the latest version of XC16.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This is necessary for mbedTLS callbacks that do not carry any user state,
so those callbacks can be customised per SSL context.
Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
When descriptors are enabled, lookup of the `__get__`, `__set__` and
`__delete__` descriptor methods should not be delegated to `__getattr__`.
That follows CPython behaviour.
Signed-off-by: Damien George <damien@micropython.org>
Check a target exists before accessing properties. Otherwise
usermod_gather_sources would recurse into garbage property names and break.
Signed-off-by: Phil Howard <phil@gadgetoid.com>
Allowing passing keyword arguments to a native base's __init__, i.e.
`make_new` in the C code. Previously only positional arguments were
allowed.
The main trade-off in this commit is that every call to the native base's
`make_new` is now going to be preceded by a call to
`mp_map_init_fixed_table` even though most of what that does is unused and
instead it merely serves as a way to pass the number of keyword arguments.
Fixes issue #15465.
Signed-off-by: stijn <stijn@ignitron.net>
The ESP32 port contains a workaround to avoid having a certain function
in `py/parse.c` being generated incorrectly. The compiler in question
is not part of any currently supported version of ESP-IDF anymore, and the
problem inside the compiler (well, assembler in this case) has been
corrected a few years ago.
This commit removes all traces of that workaround from the source tree.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit fixes the addition of a stray separator before the number
when printing an MPZ-backed integer and the first group is three digits
long.
This fixes#8984.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit adds a new `RingIO` type which exposes the internal ring-buffer
code for general use in Python programs. It has the stream interface
making it similar to `StringIO` and `BytesIO`, except `RingIO` has a fixed
buffer size and is automatically safe when reads and writes are in
different threads or an IRQ.
This new type is enabled at the "extra features" ROM level.
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Otherwise it's very difficult to reason about thread safety in a
scheduler callback, as it can run at any time on any thread - including
racing against any bytecode operation on any thread.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
When MicroPython is used as a submodule and built from the containing
project, e.g. for the embed port, `make submodules` fails because it goes
looking for the sub-sub-module paths in the outer repository instead of in
the micropython repository. Fix this by invoking git inside the micropython
submodule.
Signed-off-by: Christian Walther <cwalther@gmx.ch>
The RV32 emitter used an additional temporary register, as certain code
sequences required extra storage. This commit removes its usage in all
but one case, using REG_TEMP2 instead.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
The RV32 emitter sometimes generated short load opcodes even when it
was not supposed to. This commit fixes an off-by-one error in its
offset eligibility range calculation and corrects one case of offset
calculation, operating on the raw label index number rather than its
effective offset in the stack (C.LW assumes all loads are
word-aligned).
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
The RV32 emitter always scheduled short jumps even outside the emit
compiler pass. Running the full test suite through the native emitter
instead of just the tests that depend on the emitter at runtime (as in,
`micropython/native_*` and `micropython/viper_* tests`) uncovered more
places where the invalid behaviour was still present.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Skip whitespace characters between pairs of hex numbers.
This makes `bytes.fromhex()` compatible with cpython.
Includes simple test in `tests/basic/builtin_str_hex.py`.
Signed-off-by: Glenn Moloney <glenn.moloney@gmail.com>
Currently the stack limit margin is hard-coded in each port's call to
`mp_stack_set_limit()`, but on threaded ports it's fiddlier and can lead to
bugs (such as incorrect thread stack margin on esp32).
This commit provides a new API to initialise the C Stack in one function
call, with a config macro to set the margin. Where possible the new call
is inlined to reduce code size in thread-free ports.
Intended replacement for `MP_TASK_STACK_LIMIT_MARGIN` on esp32.
The previous `stackctrl.h` API is still present and unmodified apart from a
deprecation comment. However it's not available when the
`MICROPY_PREVIEW_VERSION_2` macro is set.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
These were changed in v1.11 (2019). Prepare to remove the compatibility
macros as part of V2 changes.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
This is needed for a workaround on esp32 port (in child commit),
which produces incorrect results otherwise.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
The `emit_load_reg_with_object()` helper function will clobber `REG_TEMP0`.
This is currently OK on architectures where `REG_RET` and `REG_TEMP0` are
the same (all architectures except RV32), because all callers of
`emit_load_reg_with_object()` use either `REG_RET` or `REG_TEMP0` as the
destination register. But on RV32 these registers are different and so
when `REG_RET` is the destination, `REG_TEMP0` is clobbered, leading to
incorrectly generated machine code.
This commit fixes the issue simply by using `REG_TEMP0` as the destination
register for all uses of `emit_load_reg_with_object()`, and adds a comment
to make sure the caller of this function is careful.
Signed-off-by: Damien George <damien@micropython.org>
Introduce SRC_USERMOD_LIB_ASM to allow users to include assembly files as
part of their user modules. It could be used to include optimized
functions or outputs of other programming languages.
Signed-off-by: George Hopkins <george-hopkins@null.net>
Use explicit casts to suppress warnings about implicit conversions, add a
workaround for constant expression conditional, and make functions static
inline (as is done in the rest of the codebase) to suppress 'warning C4505:
unreferenced function with internal linkage has been removed'.
(Follow up to fix commit 908ab1ceca)
Signed-off-by: stijn <stijn@ignitron.net>
This fixes various null dereferencing and out-of-bounds access because
super_attr assumes the held obj is effectively an object of the held type,
which is now verified.
Fixes issue #12830.
Signed-off-by: stijn <stijn@ignitron.net>
When subclassing a native type, calling native members in `__init__` before
`super().__init__()` has been called could cause a crash. In this
situation, `self` in `mp_convert_member_lookup` is the
`native_base_init_wrapper_obj`. The check added in this commit ensures
that an `AttributeError` is raised before this happens, which is consistent
with other failed lookups.
Also fix a typo in a related comment.
Signed-off-by: Laurens Valk <laurens@pybricks.com>
These are old, unused, and most of them no longer compile. The `gc_test()`
function is superseded by the test suite.
Signed-off-by: Damien George <damien@micropython.org>
Also put this function inside the `MICROPY_PY_BUILTINS_SLICE` guard,
because it's only usable when that option is enabled.
Signed-off-by: Damien George <damien@micropython.org>
These TODOs don't need to be done:
- Calling functions with keyword arguments is less common than without
them, so adding an extra byte overhead to all calls regardless of whether
they use keywords or not would overall increase generated bytecode size.
- Restricting `range` objects to machine-sized ints has been adequate for
a long time now, so no need to change that and make it more complicated
and slower.
- Printing spaces in tab completion does not need to be optimised.
Signed-off-by: Damien George <damien@micropython.org>
Being able to send data out in LSB format can be useful, and having support
in the low-level driver is much better than requiring Python code to
reorder the bits before sending them / after receiving them. In particular
if the hardware does not support the LSB format (eg RP2040) then one needs
to use the SoftSPI in LSB mode.
For this change a default definition of `MICROPY_PY_MACHINE_SPI_MSB/_LSB`
was added to `py/mpconfig.h`, making them available to all ports. The
identical defines in `esp32/mpconfigport.h` were deleted.
Resolves issues #5340, #11404.
Signed-off-by: robert-hh <robert@hammelrath.com>
The limit is set by a `MICROPY_PY_MACHINE_FREQ_NUM_ARGS_MAX` define, which
defaults to 1 and is set for stm32 to 4.
For stm32 this fixes a regression introduced in commit
e1ec6af654 where the maximum number of
arguments was changed from 4 to 1.
Signed-off-by: robert-hh <robert@hammelrath.com>
Use new function mp_obj_new_str_from_cstr() where appropriate. It
simplifies the code, and makes it smaller too.
Signed-off-by: Jon Foster <jon@jon-foster.co.uk>
There were lots of places where this pattern was duplicated, to convert a
standard C string to a MicroPython string:
x = mp_obj_new_str(s, strlen(s));
This commit provides a simpler method that removes this code duplication:
x = mp_obj_new_str_from_cstr(s);
This gives clearer, and probably smaller, code.
Signed-off-by: Jon Foster <jon@jon-foster.co.uk>
As per discussion in #15347, non-standard binary literals have been
removed in favour of their hexadecimal counterparts.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Before this change, long/mpz ints propagated into all future calculations,
even if their value could fit in a small-int object. With this change, the
result of a big-int binary op will now be converted to a small-int object
if the value fits in a small-int.
For example, a relatively common operation like `x = a * b // c` where
a,b,c all small ints would always result in a long/mpz int, even if it
didn't need to, and then this would impact all future calculations with
x.
This adds +24 bytes on PYBV11 but avoids heap allocations and potential
surprises (e.g. `big-big` is now a small `0`, and can safely be accessed
with MP_OBJ_SMALL_INT_VALUE).
Performance tests are unchanged on PYBV10, except for `bm_pidigits.py`
which makes heavy use of big-ints and gains about 8% in speed.
Unix coverage tests have been updated to cover mpz code that is now
unreachable by normal Python code (removing the unreachable code would lead
to some surprising gaps in the internal C functions and the functionality
may be needed in the future, so it is kept because it has minimal
overhead).
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This commit changes a few code sequences to use more compressed opcodes
where possible. The sequences in question are the ones that show up the
most in the test suite and require the least amount of code changes, namely
short offset loads from memory to RET/ARG registers, indirect calls through
the function table, register-based jumps, locals' offset calculation,
reg-is-null jumps, and register comparisons.
There are no speed losses or gains from these changes, but there is an
average 15-20% generated code size reduction.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
If `array.append()` fails with an exception due to heap exhaustion, the
next attempt to grow the buffer will cause a buffer overflow because the
free slot count is increased before performing the allocation, and will
stay as if the allocation succeeded.
Signed-off-by: Yoctopuce <dev@yoctopuce.com>
Fixes and improvements to `int.to_bytes()` are:
- No longer overflows if byte size is 0 (closes#13041).
- Raises OverflowError in any case where number won't fit into byte length
(now matches CPython, previously MicroPython would return a truncated
bytes object).
- Document that `micropython int.to_bytes()` doesn't implement the optional
signed kwarg, but will behave as if `signed=True` when the integer is
negative (this is the current behaviour). Add tests for this also.
Requires changes for small ints, MPZ large ints, and "long long" large
ints.
Adds a new set of unit tests for ints between 32 and 64 bits to increase
coverage of "long long" large ints, which are otherwise untested.
Tested on unix port (64 bit small ints, MPZ long ints) and Zephyr STM32WB
board (32 bit small ints, long long large ints).
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
The code generating the entry to the finally handler of an async-with
statement was simply wrong for the case of the native emitter. Among other
things the layout of the stack was incorrect.
This is fixed by this commit. The setup of the async-with finally handler
is now put in a dedicated emit function, for both the bytecode and native
emitters to implement in their own way (the bytecode emitter is unchanged,
just factored to a function).
With this fix all of the async-with tests now work when using the native
emitter.
Signed-off-by: Damien George <damien@micropython.org>
A value thrown/injected into a native generator needs to be stored in a
dedicated variable outside `nlr_buf_t`, following the `inject_exc` variable
in `py/vm.c`.
Signed-off-by: Damien George <damien@micropython.org>
This emitter prints out pseudo-machine instructions, instead of the usual
output of the native emitter. It can be enabled on any port via
`MICROPY_EMIT_NATIVE_DEBUG` (make sure other native emitters are disabled)
but the easiest way to use it is with mpy-cross:
$ mpy-cross -march=debug file.py
Signed-off-by: Damien George <damien@micropython.org>
Selected load/store code sequences have been optimised for RV32IMC when the
chance to use fewer and smaller opcodes was possible.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
MPY files can now hold generated RV32IMC native code. This can be
accomplished by passing the `-march=rv32imc` flag to mpy-cross.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This adds a native code generation backend for RISC-V RV32I CPUs, currently
limited to the I, M, and C instruction sets.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This fixes a minor issue in the changes made by
7dff38fdc1: the type flags for deque were
meant to be conditionalized based on MICROPY_PY_COLLECTIONS_DEQUE_ITER, but
the computed conditionalized value wasn't used.
Signed-off-by: Dan Halbert <halbert@halwitz.org>
Allows passing in a callback to `TaskQueue()` that is called when something
is pushed on to the queue.
Signed-off-by: Damien George <damien@micropython.org>
Support for raw str/bytes already exists, and extending that to raw
f-strings is easy. It also reduces code size because it eliminates an
error message.
Signed-off-by: Damien George <damien@micropython.org>
This is quite a simple and small change to support concatenation of
adjacent f-strings, and improve compatibility with CPython.
Signed-off-by: Damien George <damien@micropython.org>
Also define `mp_type_bytearray`. These all help to write native modules.
Signed-off-by: Brian Pugh <bnp117@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
Although the original motivation given for the workaround[1] is correct,
nlr.o and nlrthumb.o are linked with a small enough distance that the
problem does not occur, and the workaround isn't necessary. The distance
between the b instruction and its target (nlr_push_tail) is just 64
bytes[2], well within the ±2046 byte range addressable by an
unconditional branch instruction in Thumb mode.
The workaround induces a relocation in the text section (textrel), which
isn't supported everywhere, notably not on musl-libc[3], where it causes
a crash on start-up. With the workaround removed, micropython works on an
ARMv5T Linux system built with musl-libc.
This commit changes nlrthumb.c to use a direct jump by default, but
leaves the long jump workaround as an option for those cases where it's
actually needed.
[1]: commit dd376a239d
Author: Damien George <damien.p.george@gmail.com>
Date: Fri Sep 1 15:25:29 2017 +1000
py/nlrthumb: Get working again on standard Thumb arch (ie not Thumb2).
"b" on Thumb might not be long enough for the jump to nlr_push_tail so
it must be done indirectly.
[2]: Excerpt from objdump -d micropython:
000095c4 <nlr_push_tail>:
95c4: b510 push {r4, lr}
95c6: 0004 movs r4, r0
95c8: f02d fd42 bl 37050 <mp_thread_get_state>
95cc: 6943 ldr r3, [r0, #20]
95ce: 6023 str r3, [r4, #0]
95d0: 6144 str r4, [r0, #20]
95d2: 2000 movs r0, #0
95d4: bd10 pop {r4, pc}
000095d6 <nlr_pop>:
95d6: b510 push {r4, lr}
95d8: f02d fd3a bl 37050 <mp_thread_get_state>
95dc: 6943 ldr r3, [r0, #20]
95de: 681b ldr r3, [r3, #0]
95e0: 6143 str r3, [r0, #20]
95e2: bd10 pop {r4, pc}
000095e4 <nlr_push>:
95e4: 60c4 str r4, [r0, #12]
95e6: 6105 str r5, [r0, #16]
95e8: 6146 str r6, [r0, #20]
95ea: 6187 str r7, [r0, #24]
95ec: 4641 mov r1, r8
95ee: 61c1 str r1, [r0, #28]
95f0: 4649 mov r1, r9
95f2: 6201 str r1, [r0, #32]
95f4: 4651 mov r1, sl
95f6: 6241 str r1, [r0, #36] @ 0x24
95f8: 4659 mov r1, fp
95fa: 6281 str r1, [r0, #40] @ 0x28
95fc: 4669 mov r1, sp
95fe: 62c1 str r1, [r0, #44] @ 0x2c
9600: 4671 mov r1, lr
9602: 6081 str r1, [r0, #8]
9604: e7de b.n 95c4 <nlr_push_tail>
[3]: https://www.openwall.com/lists/musl/2020/09/25/4
Signed-off-by: J. Neuschäfer <j.ne@posteo.net>
Two cases, one assigning to a slice.
Closes https://github.com/micropython/micropython/issues/13283
Second is extending a slice from itself, similar logic.
In both cases the problem occurs when m_renew causes realloc to move the
buffer, leaving a dangling pointer behind.
There are more complex and hard to fix cases when either argument is a
memoryview into the buffer, currently resizing to a new address breaks
memoryviews into that object.
Reproducing this bug and confirming the fix was done by running the unix
port under valgrind with GC-aware extensions.
Note in default configurations with GIL this bug exists but has no impact
(the free buffer won't be reused while the function is still executing, and
is no longer referenced after it returns).
Signed-off-by: Angus Gratton <angus@redyak.com.au>
This commit swaps the order of the `flags` and `name` struct initialisers
for `mp_obj_type_t`, to fix an incompatibility with C++. The original
order of the initialiser didn't match the definition of the type, and
although that's still legal C, it's not legal C++.
Signed-off-by: Vonasmic <kasarkal123@gmail.com>
This is required because the .mpy native ABI was changed by the
introduction of `mp_proto_fun_t`, see commits:
- 416465d81e
- 5e3006f117
- e2ff00e811
And three `mp_binary` functions were added to `mp_fun_table` in
commit d2276f0d41.
Signed-off-by: Damien George <damien@micropython.org>
These are needed to read/write array.array objects, which is useful in
native code to provide fast extensions that work with big arrays of data.
Signed-off-by: Damien George <damien@micropython.org>
Newer versions of gcc (14 and up) have more sophisticated dead-code
detection, and the asm clobbers list needs to contain "memory" to inform
the compiler that the asm code actually does something.
Tested that adding this "memory" line does not change the generated code on
ARM Thumb2, x86-64 and Xtensa targets (using gcc 13.2).
Fixes issue #14115.
Signed-off-by: Damien George <damien@micropython.org>
Enabled by MICROPY_COMPILE_ALLOW_TOP_LEVEL_AWAIT. When enabled, this means
that scope such as module-level functions and REPL statements can yield.
The outer C code must then handle this yielded generator.
Signed-off-by: Damien George <damien@micropython.org>
This fixes a bug where a random Python object may become
un-garbage-collectable until an enclosing Python file (compiled on device)
finishes executing.
Details:
The mp_parse_tree_t structure is stored on the stack in top-level functions
such as parse_compile_execute() in pyexec.c (and others).
Although it quickly falls out of scope in these functions, it is usually
still in the current stack frame when the compiled code executes. (Compiler
dependent, but usually it's one stack push per function.)
This means if any Python object happens to allocate at the same address as
the (freed) root parse tree chunk, it's un-garbage-collectable as there's a
(dangling) pointer up the stack referencing this same address.
As reported by @GitHubsSilverBullet here:
https://github.com/orgs/micropython/discussions/14116#discussioncomment-8837214
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
This commit implements the 'e' half-float format: 10-bit mantissa, 5-bit
exponent. It uses native _Float16 if supported by the compiler, otherwise
uses custom bitshifting encoding/decoding routines.
Signed-off-by: Matthias Urlichs <matthias@urlichs.de>
Signed-off-by: Damien George <damien@micropython.org>
