Commit graph

5853 commits

Author SHA1 Message Date
46bfbad1bb
Add locale.getlocale()
This returns the localization of the running CircuitPython, such as
en_US, fr, etc.

Additional changes are needed in build infrastructure since the
string "en_US" should not appear to be translated in weblate, ever;
instead the value comes from the translation metadata.

Closes: #8602
2023-11-14 21:20:03 -06:00
Scott Shawcroft
8ff0682937
Merge pull request #8605 from tannewt/fix_esp_heap_alloc
Fix split heap on ESP
2023-11-14 14:52:28 -08:00
Scott Shawcroft
1f2355742b
Fix split heap on ESP
It was setting the DMA capability requirement which excludes PSRAM.

Fixes #8597. Fixes #8573.
2023-11-14 12:00:59 -08:00
Scott Shawcroft
4bdd3e6150
Merge pull request #8589 from jepler/issue8588
Rework ci_fetch_deps and use it from makefiles too
2023-11-14 10:04:37 -08:00
fb840159fb
Remove some stuff about reserved psram & supervisor allocations
this is obsoleted by the new split heap code 🎉
2023-11-13 12:55:26 -06:00
6db9f2e73f
Rework ci_fetch_deps and use it from makefiles too
* teach ci_fetch_deps about --filter=blob:none
 * change logic ensuring tags in frozen/ are fetched
 * since check=True was all the time, remove unused kwarg
 * add fetch-board-submodules

Closes: #8588
2023-11-12 12:32:34 -06:00
Andrew Leech
4cf741062b extmod/vfs_reader: Add file ioctl to set read buffer size.
Can be used to speed up importing a file from a vfs based filesystem.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
2023-11-09 11:20:31 +11:00
Dan Halbert
191a5a31c7
add CIRCUITPY-CHANGE annotations 2023-11-08 09:08:10 -05:00
stijn
958c6d917d windows: Use the MicroPython logo as application icon.
Add a .ico file with common icon image size, created from
vector-logo-2.png, and embed it into the resulting executable.

Signed-off-by: stijn <stijn@ignitron.net>
2023-11-07 17:22:52 +11:00
Jim Mussared
4212799fd8 py/qstr: Special case qstr_find_strn for empty string.
This handles the case where an empty bytes/bytearray/str could pass in
NULL as the str argument (with length zero). This would result in UB in
strncmp. Even though our bare-metal implementation of strncmp handles
this, best to avoid it for when we're using system strncmp.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-11-07 16:01:50 +11:00
Jim Mussared
b6a9778484 py/misc: Change sizeof to offsetof for variable-length alloc.
This fixes the case where e.g.

    struct foo_t {
      mp_obj_t x;
      uint16_t y;
      char buf[];
    };

will have `sizeof(struct foo_t)==8`, but `offsetof(struct foo_t, buf)==6`.

When computing the size to allocate for `m_new_obj_var` we need to use
offsetof to avoid over-allocating. This is important especially when it
might cause it to spill over into another GC block.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-11-03 16:03:18 +11:00
Mathieu Serandour
c85db05244 py/lexer: Change token position for new lines.
Set the position of new line tokens as the end of the preceding line
instead of the beginning of the next line.  This is done by first moving
the pointer to the end of the current line to skip any whitespace, record
the position for the token, then finaly skip any other line and whitespace.

The previous behavior was to skip every new line and whitespace, including
the indent of the next line, before recording the token position.

(Note that both lex->emit_dent and lex->nested_bracket_level equal 0 if
had_physical_newline == true, which allows simplifying the if-logic for
MP_TOKEN_NEWLINE.)

And update the cmd_parsetree.py test expected output, because the position
of the new-line token has changed.

Fixes issue #12792.

Signed-off-by: Mathieu Serandour <mathieu.serandour@numworks.fr>
2023-11-03 15:56:10 +11:00
Damien George
9a4d4db3a1 py/runtime: Remove declaration of function from inside function.
Signed-off-by: Damien George <damien@micropython.org>
2023-11-03 15:21:51 +11:00
Jim Mussared
2eda513870 py/mkrules.mk: Add rule for compiling auto-generated source files.
This prevents each port Makefile from having to add an explicit rule for
`build-BOARD/pins_BOARD.c`.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-11-03 13:57:47 +11:00
Scott Shawcroft
8137e2d6d2
Switch all ports to auto-growing split heap
This simplifies allocating outside of the VM because the VM doesn't
take up all remaining memory by default.

On ESP we delegate to the IDF for allocations. For all other ports,
we use TLSF to manage an outer "port" heap. The IDF uses TLSF
internally and we use their fork for the other ports.

This also removes the dynamic C stack sizing. It wasn't often used
and is not possible with a fixed outer heap.

Fixes #8512. Fixes #7334.
2023-11-01 15:24:16 -07:00
Alessandro Gatti
95ce61d0ad esp32: Use better build settings for ESP32-C3.
ESP32-C3 is not Xtensa-based, so build settings are now tailored a bit
better following that fact.  ESP-IDF 5.x already adds architecture-specific
modules by itself so there is no need to specify either the `xtensa` or the
`riscv` module in the build settings.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2023-11-01 11:38:09 +11:00
2296c85e29
fix another error seen only on clean builds 2023-10-30 16:25:25 +01:00
774f6ac6ab
Switch to using MP_ERROR_TEXT instead of translate, globally 2023-10-30 09:49:06 +01:00
Jim Mussared
64c79a5423 py/qstr: Add support for sorted qstr pools.
This provides a significant performance boost for qstr_find_strn, which is
called a lot during parsing and loading of .mpy files, as well as interning
of string objects (which happens in most string methods that return new
strings).

Also adds comments to explain the "static" qstrs.  These are part of the
.mpy ABI and avoid needing to duplicate string data for QSTRs known to
already be in the firmware.  The static pool isn't currently sorted, but in
the future we could either split the static pool into the sorted regions,
or in the next .mpy version just sort them.

Based on initial work done by @amirgon in #6896.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-10-30 11:10:02 +11:00
Dan Halbert
32b6ac79d5
Merge pull request #8519 from jepler/compressed-message-type
Rename compressed_string_t to mp_rom_error_text_t to match upstream
2023-10-27 10:53:44 -04:00
Alessandro Gatti
b6c369a396 py/asm{arm,thumb,x64,x86,xtensa}: Remove unused macros.
`ASM_MOV_REG_IMM_FIX_U16` and `ASM_MOV_REG_IMM_FIX_WORD` are no longer
used anywhere in the code.

See discussion in #12771.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2023-10-27 15:41:02 +11:00
Jim Mussared
3bf70f16e9 py/mkrules.mk: Add MICROPY_PREVIEW_VERSION_2.
This provides a way to enable features and changes slated for MicroPython
2.x, by running `make MICROPY_PREVIEW_VERSION_2=1`. Also supported for
the cmake ports (except Zephyr).

This is an alternative to having a 2.x development branch (or equivalently,
keeping a 1.x release branch). Any feature or change that needs to be
"hidden" until 2.x can use this flag (either in the Makefile or the
preprocessor).

A good example is changing function arguments or other public API features,
in particular to aid in improving consistency between ports.

When `MICROPY_PREVIEW_VERSION_2` is enabled, the REPL banner is amended to
say "MicroPython (with v2.0 preview) vX.Y.Z", and sys.implementation gets a
new field `_v2` set to `True`.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-10-27 15:28:46 +11:00
Scott Shawcroft
cf08ef0b80
Enable warnings in full builds only 2023-10-26 13:43:04 -07:00
Dan Halbert
3f1b9af025 py/mpprint.c: fix formatting 2023-10-26 15:30:42 -04:00
c695f8b20f
remove some unneeded diffs 2023-10-26 19:39:43 +02:00
a069dc92e0
restore cprintf/vcprintf prototypes 2023-10-26 19:39:26 +02:00
Scott Shawcroft
f439f02492
Add warnings and warn about displayio changes
Follow up to #8493
2023-10-25 15:45:45 -07:00
de541cf155
Fix pointer-ness, const-ness of compressed messages
micropython puts the pointer-ness into the typedef; we can put the
const-ness there too.

this reduces the delta to micropython; for instance, emitinlinextensa
and emitinlinethumb now match upstream.
2023-10-25 21:40:11 +02:00
54a5878ee0
WIP 2023-10-25 21:24:52 +02:00
Scott Shawcroft
be30c12a74
Pass subobject into native subscr
This allows PixelBuf to call transmit after setting a value.

Fixes #8488
2023-10-25 11:44:32 -07:00
acf350a1b7
In py/ use MP_ERROR_TEXT instead of translate 2023-10-25 08:19:31 +02:00
55874b6470
Rename compressed_string_t to mp_rom_error_text_t to match upstream 2023-10-25 08:14:13 +02:00
Scott Shawcroft
63079c75b0
Re-add passing native methods the subclass instance
Fixes #8488
2023-10-24 16:20:51 -07:00
Scott Shawcroft
e62db5adcd
Fix native property setting from subclass 2023-10-24 16:20:51 -07:00
Scott Shawcroft
e1df598199
Split displayio hardware support from core
These are moved:
* Display -> busdisplay.BusDisplay
* FourWire -> fourwire.FourWire
* EPaperDisplay -> epaperdisplay.EPaperDisplay
* I2CDisplay -> i2cdisplaybus.I2CDisplayBus

`paralleldisplay` is now `paralleldisplaybus` (and registered as
`paralleldisplay` too).

Bus related helpers are split out of display_core into bus_core.
It is in still displayio since it is a dependency of both
busdisplay and epaperdisplay.

Fixes #7667
2023-10-24 15:43:34 -07:00
Scott Shawcroft
168c40e940
Merge pull request #8508 from dhalbert/v1.21-merge
V1.21 merge
2023-10-24 15:36:06 -07:00
Scott Shawcroft
ecaf9e6b14
Fix native emitter compile 2023-10-24 14:48:02 -07:00
Scott Shawcroft
d8148559c2
Enable native subpackage support 2023-10-24 13:49:18 -07:00
Scott Shawcroft
d14bb575cb
Fix mp_frozen_names 2023-10-24 13:02:09 -07:00
Dan Halbert
2c795acf1e py/compile.c: add missing line for native labels in await 2023-10-24 15:39:26 -04:00
Bob Abeles
6725be4259 And formatting, again 2023-10-23 20:43:57 -07:00
Bob Abeles
3f4332be80 Another CI formatting issue 2023-10-23 20:40:51 -07:00
Bob Abeles
892d89e8b7 Fix CI detected line-length issue 2023-10-23 20:35:31 -07:00
Bob Abeles
2e996587fe Improve make translation data performance. 2023-10-23 20:08:15 -07:00
Dan Halbert
6cd5150ac1 fix gc_free() to build in mpy-cross 2023-10-23 20:44:03 -04:00
Dan Halbert
3f0b807e74 pre-commit C formatting fix 2023-10-23 20:37:42 -04:00
Dan Halbert
8f254035dd pre-commit fixes 2023-10-23 20:14:40 -04:00
Scott Shawcroft
f13ea9a49f
Fix async tests by adding back __await__ use. Remove u* lookup 2023-10-23 16:13:11 -07:00
Damien George
bb4be837c3 py/makeqstrdefs.py: Print a nicer error when preprocessing stage fails.
Signed-off-by: Damien George <damien@micropython.org>
2023-10-23 10:39:39 +11:00
Dan Halbert
8017a1ad30 ports/unix VARIANT=coverage fixes 2023-10-20 16:51:04 -04:00
7ab5252cdd
Add CIRCUITPY_MESSAGE_COMPRESSION_LEVEL
to trade compile speed & flash size

Initially enable the faster mode on rp2040 and espressif, where there's
usually plenty of flash available (these advanced techniques save hundreds
to thousands of bytes, which is important on a lot of old samd21 boards
but is a drop in the lake of a 4MB flash chip)
2023-10-20 19:18:18 +01:00
Dan Halbert
fdfc4421b1 MICROPY_NONSTANDARD_TYPECODES default; check MICROPY_PY_ERRNO in objexcept.c 2023-10-20 13:35:38 -04:00
Dan Halbert
6eb0607a53 fix build when MICROPY_PY_ERRNO is 0 2023-10-20 10:43:22 -04:00
Dan Halbert
4b42a6f4a0 restore old uzlib; remove remaining U and u prefixes 2023-10-19 21:29:57 -04:00
Dan Halbert
367e13c69f change CIRCUITPY change markers to CIRCUITPY-CHANGE 2023-10-19 16:42:36 -04:00
Dan Halbert
c0a4abc03c Fix merge bugs; remove shared/tinyusb/* 2023-10-19 16:02:42 -04:00
Dan Halbert
f2ebe6839c Initial MicroPython v1.21.0 merge; not compiled yet 2023-10-18 17:49:14 -04:00
Jim Mussared
3883f29485 py/modthread: Initialise nlr_jump_callback_top on threads.
The main thread gets this because the thread state is in bss, but
subsequent threads need this field to be initialised.

Also added a note to mpstate.h to help avoid missing this in the future.

Fixes issue #12695.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-10-18 09:18:23 +11:00
Damien George
6967ff3c58 py/persistentcode: Bump .mpy sub-version.
This is required because the previous commit changed the .mpy native ABI.

Signed-off-by: Damien George <damien@micropython.org>
2023-10-16 11:25:31 +11:00
Damien George
9b63421fb3 py/dynruntime: Add mp_get_buffer.
Signed-off-by: Damien George <damien@micropython.org>
2023-10-16 11:23:11 +11:00
Damien George
9c7ea9b14a py/obj: Generalise mp_get_buffer so it can raise if a flag is set.
This allows mp_get_buffer_raise() to be changed to a simple inline function
that in the majority of cases costs the same (in code size) to call as the
original mp_get_buffer_raise(), because the flags argument is a constant.

Signed-off-by: Damien George <damien@micropython.org>
2023-10-16 11:22:55 +11:00
Damien George
516385c4cc py/objboundmeth: Optimise check for types in binary_op.
Signed-off-by: Damien George <damien@micropython.org>
2023-10-13 15:29:09 +11:00
Daniël van de Giessen
4f5e165d0b py/objboundmeth: Support comparing and hashing bound methods.
This behaviour matches CPython.  It's useful to be able to store bound
method objects in dicts/sets, and compare for equality, eg when storing
them in a list and using list.remove().

Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
2023-10-13 15:11:49 +11:00
Dan Halbert
2f391e5679 restore SUPEROPT_GC, SUPEROPT_VM 2023-10-12 15:25:28 -04:00
Dan Halbert
a13185bb34 fix inclusion of ulab sources 2023-10-12 11:03:00 -04:00
Jim Mussared
5015779a6f py/builtinevex: Handle invalid filenames for execfile.
If a non-string buffer was passed to execfile, then it would be passed
as a non-null-terminated char* to mp_lexer_new_from_file.

This changes mp_lexer_new_from_file to take a qstr instead (as in almost
all cases a qstr will be created from this input anyway to set the
`__file__` attribute on the module).

This now makes execfile require a string (not generic buffer) argument,
which is probably a good fix to make anyway.

Fixes issue #12522.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-10-12 15:17:59 +11:00
Scott Shawcroft
9633c4e78f
Merge remote-tracking branch 'adafruit/main' into v1.20-merge 2023-10-11 11:21:57 -07:00
Scott Shawcroft
6e278900bd
Only include mpconfigport.h once 2023-10-11 10:04:03 -07:00
Scott Shawcroft
9be3999e78
Add back makefile print variable 2023-10-11 10:04:02 -07:00
Scott Shawcroft
2910dea6fd
Almost fix extra_coverage test 2023-10-11 10:04:02 -07:00
Scott Shawcroft
22a44c6003
Try and fix mpy-cross variants 2023-10-11 10:03:46 -07:00
Dan Halbert
7869706551 force FROZEN_MANIFEST build; add back make print-% 2023-10-10 22:10:17 -04:00
Matthias Urlichs
3fb1bb131f py/vm: Don't emit warning when using "raise ... from None".
"Raise SomeException() from None" is a common Python idiom to suppress
chained exceptions and thus shouldn't trigger a warning on a version of
Python that doesn't support them in the first place.
2023-10-09 09:46:02 +11:00
Jim Mussared
69e34b6b6b all: Switch to new preview build versioning scheme.
See https://github.com/micropython/micropython/issues/12127 for details.

Previously at the point when a release is made, we update mpconfig.h
and set a git tag. i.e. the version increments at the release.

Now the version increments immediately after the release. The workflow is:
1. Final commit in the cycle updates mpconfig.h to set (X, Y, 0, 0) (i.e.
   clear the pre-release state).
2. This commit is tagged "vX.Y.0".
3. First commit for the new cycle updates mpconfig.h to set (X, Y+1, 0, 1)
   (i.e. increment the minor version, set the pre-release state).
4. This commit is tagged "vX.Y+1.0-preview".

The idea is that a nightly build is actually a "preview" of the _next_
release. i.e. any documentation describing the current release may not
actually match the nightly build. So we use "preview" as our semver
pre-release identifier.

Changes in this commit:
 - Add MICROPY_VERSION_PRERELEASE to mpconfig.h to allow indicating that
   this is not a release version.
 - Remove unused MICROPY_VERSION integer.
 - Append "-preview" to MICROPY_VERSION_STRING when the pre-release state
   is set.
 - Update py/makeversionhdr.py to no longer generate MICROPY_GIT_HASH.
 - Remove the one place MICROPY_GIT_HASH was used (it can use
   MICROPY_GIT_TAG instead).
 - Update py/makeversionhdr.py to also understand
   MICROPY_VERSION_PRERELEASE in mpconfig.h.
 - Update py/makeversionhdr.py to convert the git-describe output into
   semver-compatible "X.Y.Z-preview.N.gHASH".
 - Update autobuild.sh to generate filenames using the new scheme.
 - Update remove_old_firmware.py to match new scheme.
 - Update mpremote's pyproject.toml to handle the "-preview" suffix in the
   tag. setuptools_scm maps to this "rc0" to match PEP440.
 - Fix docs heading where it incorrectly said "vvX.Y.Z" for release docs.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-10-06 12:10:14 +11:00
Damien George
e00a144008 all: Bump version to 1.21.0.
Signed-off-by: Damien George <damien@micropython.org>
2023-10-06 10:32:07 +11:00
Scott Shawcroft
bf3d84195f
Fix decompressing using qstrs after running the VM
The qstr state still pointed to qstr pools in the released MP heap.
2023-10-05 14:56:11 -07:00
Scott Shawcroft
49511b0746
Bring back ordereddict.move_to_end() 2023-10-05 12:48:45 -07:00
Scott Shawcroft
94c7082e9c
Fix super init. CP changed it to support kwargs 2023-10-05 12:44:30 -07:00
Scott Shawcroft
18c03a74dd
Fix a few tests
* Re-enable a couple FATFS configurations we added.
* Remove MICROPY_PY_IO_FILEIO.
* Remove uasyncio from standard unix build.
* Re-add our unicode printing improvements.
2023-10-05 10:59:08 -07:00
Dan Halbert
7e0e6fcdca Metro M4 now compiles 2023-10-03 15:03:59 -04:00
Damien George
cf490a7091 all: Fix various spelling mistakes found by codespell 2.2.6.
Signed-off-by: Damien George <damien@micropython.org>
2023-10-03 11:24:50 +11:00
1b9ecabf8b
Fix constructing empty namedtuple
this change from micropython was not taken with the merge
2023-10-02 09:08:03 -05:00
18a5a897f1
makeqstrdef: restore CIRCUITPY behavior
this fixes the qrio test, among others
2023-10-02 08:50:47 -05:00
1ccce65311
file no longer used 2023-10-02 08:48:19 -05:00
6d59e55599
fix making mpy-cross automatically 2023-10-02 07:28:19 -05:00
Dan Halbert
2fc5a934a1 add back generic subscript iterator, gc_never_free 2023-09-29 23:01:02 -04:00
Dan Halbert
68a2927385 MP_REGISTER_ROOT_POINTER for port-specific root pointers 2023-09-29 15:46:42 -04:00
Dan Halbert
1c388ab315 finish converting to MP_REGISTER_ROOT_POINTER() 2023-09-29 10:49:34 -04:00
Jim Mussared
276bfa3146 py/lexer: Add missing initialisation for fstring_args_idx.
This was missed in 692d36d779. Probably
never noticed because everything enables `MICROPY_GC_CONSERVATIVE_CLEAR`,
but found via ASAN thanks to @gwangmu & @chibinz.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-09-29 13:58:26 +10:00
Ihor Nehrutsa
d83c1a43d4 py: Change ifdef DEBUG_PRINT to if DEBUG_PRINT.
Signed-off-by: Ihor Nehrutsa <Ihor.Nehrutsa@gmail.com>
2023-09-29 13:04:38 +10:00
Angus Gratton
2fcd28f713 py/mkrules.mk: Don't strip binary if STRIP variable is unset.
This provides a way to build a non-DEBUG host binary that still has symbols
and debug information.

Document this for the unix port, and update a comment in the unix port
Makefile.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-09-29 13:01:12 +10:00
Dan Halbert
76ff01452b Trinket M0 comes up; still very much wip 2023-09-28 16:22:10 -04:00
MicroDev
27fd60d739
implement suggested changes
- update the docs
- split out common `watchdog_reset`
- revert to using `None` instead of `WatchDogMode.NONE`
2023-09-24 15:23:38 +00:00
cd462c51b3
re-fix stream function definition 2023-09-22 14:26:36 -05:00
2c3d81148e
re-add CP-specific C files 2023-09-22 14:26:25 -05:00
c53cb3ef25
re-add CP-specific exception related types 2023-09-22 14:26:13 -05:00
79309c1208
re-add CP-specific exception types 2023-09-22 14:25:50 -05:00
4bbf42c4c3
fix duplicate array_locals_dict_table 2023-09-22 14:25:36 -05:00
7c2fbff965
fix use of locals_dict 2023-09-22 14:25:23 -05:00
99479a69f8
fix definition of slice type with optional "make_new" 2023-09-22 13:56:36 -05:00
e48984872b
allow definition of namedtuples in core 2023-09-22 13:56:16 -05:00
a181172336
remove redundant declaratoin 2023-09-22 13:55:24 -05:00
0265ae07f8
remove unused functions 2023-09-22 13:55:15 -05:00
51314fb7de
fix availability of mp_obj_array_locals_dict 2023-09-22 13:55:08 -05:00
2c70537aba
fix memoryview cast&hex 2023-09-22 13:54:20 -05:00
085cf0e3e4
make mp_stream_flush available 2023-09-22 13:52:04 -05:00
556ec9d300
fixes for mp_type_get_protocol_slot 2023-09-22 13:51:57 -05:00
566688b028
fix mpy magic numbers 2023-09-22 13:50:35 -05:00
bb0f8149df
fix circuitpy memoryview.cast 2023-09-22 13:44:48 -05:00
0d796249e8
remove unused function 2023-09-22 13:43:47 -05:00
5367c9f7ef
fix "native exception getter" 2023-09-22 13:42:19 -05:00
f9f75d8202
MAKE_ENUM_TYPE for new style type defs 2023-09-22 13:39:24 -05:00
Scott Shawcroft
7bad82a219
C6 compiles, runs but wifi crashes 2023-09-22 10:02:35 -07:00
6b5c7b6ce6
fix const vs static generatorexit again 2023-09-22 10:45:49 -05:00
9dda69cf9e
add missing declarations that got lost in the merge 2023-09-22 10:40:11 -05:00
f77e0b8992
fix comment marker 2023-09-22 10:39:50 -05:00
60fa26022b
build fixes for persistent code 2023-09-22 10:39:41 -05:00
5c0f8f7786
fixes for circuitpython nativeglue 2023-09-22 10:39:28 -05:00
a285a33076
Restore our defines for exception chaining 2023-09-22 10:39:12 -05:00
9104654930
makeqstrdata: ensure _lt and _gt qstrs are sorted early
this fixes a build error because their numbers have to be <256
2023-09-22 10:38:52 -05:00
Dan Halbert
10b95a1998 add translate.h and linker.h includes back to runtime.h; remove linker.h refs; remove top-level lib/cyw43-driver 2023-09-20 22:00:09 -04:00
a00f2d8940
array extend/append need extern linkage & right names 2023-09-20 11:46:51 -05:00
7a386545ca
re-add translation make rules 2023-09-20 11:46:35 -05:00
a760794449
re-add objtraceback.o 2023-09-20 11:46:22 -05:00
f83a235c85
re-add mp_obj_exception_get_native 2023-09-20 11:46:14 -05:00
76ef77a620
fix exception chaining, preprocessor error 2023-09-20 11:26:57 -05:00
e068ff7234
fix compile errors, fix mp_raise prototypes 2023-09-20 11:26:41 -05:00
1ab884ab0f
fix compile errors 2023-09-20 11:26:26 -05:00
c87703512a
fix conditional nesting 2023-09-20 11:26:17 -05:00
37c8b0df9c
fix argument name 2023-09-20 11:26:03 -05:00
6b8ee2ca6f
remove old python style checking for "special accessors"
this appears to be properly handled by other (new?) code, but
tests will tell.
2023-09-20 11:25:58 -05:00
bb6d530c1d
slots & macro moved to header 2023-09-20 11:25:09 -05:00
7806044e5e
string construction changed 2023-09-20 11:25:01 -05:00
9cedfc4cbd
implemented as macro now 2023-09-20 11:24:53 -05:00
ee51968d9d
can't be static, is called from builtins 2023-09-20 11:24:45 -05:00
4ed939f26e
fix compile error 2023-09-20 11:24:26 -05:00
680b94d463
fix compile erors 2023-09-20 11:24:22 -05:00
be62395c47
fix indentation 2023-09-20 11:24:15 -05:00
f9fb567a07
fix CP vs MP differences in exception-throwing functions 2023-09-20 11:24:07 -05:00
37a881f4d3
Get the type object for the constructor call 2023-09-20 11:23:49 -05:00
082b299a0c
spelling 2023-09-20 11:23:39 -05:00
8e00c69cdf
spelling 2023-09-20 11:22:34 -05:00
cd3ca6b6a6
fix array object name 2023-09-20 11:22:25 -05:00
fb2eb6a7d9
add mp_obj_is_tuple_compatible 2023-09-20 11:22:01 -05:00
e4e58ac223
add mp_obj_get_type_qstr 2023-09-20 11:21:56 -05:00
a9d6661d5b
add and fix exception prototypes 2023-09-20 11:21:50 -05:00
b499b7fdc2
add missing type objects 2023-09-20 11:21:34 -05:00
57c3dfd579
fix missing define of NONSTANDARD_TYPECODES 2023-09-20 11:21:10 -05:00
649ea62f57
fix typo 2023-09-20 11:21:02 -05:00
73612ac119
Fix availability of MP_ERROR_TEXT in misc.h 2023-09-20 11:20:56 -05:00
aceb573c88
fix makeversionhdr 2023-09-20 11:20:45 -05:00
dc277f0aff
fix HEAP_PTR macro 2023-09-20 11:20:39 -05:00
36371fe757
fix compiling gc.c 2023-09-20 11:20:29 -05:00
0c64275cc6
Add header for PLACE_IN_ITCM 2023-09-20 11:19:57 -05:00
Angus Gratton
fa68523968 py/nlrx64: Mark nlr_push() as naked function when possible.
Supported from GCC 8 and up, and Compiler Explorer suggests it works as
expected with Clang since 3.6 (2014).

- Fixes situation where building embedded MicroPython with -O0 and
  MICROPY_NLR_X64 crashes at runtime (due to nlr_push pushing the
  frame pointer register EBP). Closes #12421.

- Allows removing the macOS tweak to undo pushing EBP onto the stack
  in the generated function prelude.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-09-20 11:48:36 +10:00
Dan Halbert
4d175ab41a convert to MP_DEFINED_CONST_OBJ_TYPE() 2023-09-19 21:09:29 -04:00
Dan Halbert
2c0fa0f7dc initial merge from v1.20.0; just satisifying conflicts 2023-09-19 11:10:12 -04:00
a8b81ce943
Enhance unexpected keyword argument detection
Before, incorrect use when calling a core function would just say
"extra keyword arguments given"; now, it will name the argument:

```python
>>> Synthesizer(bad_kwarg="boo")
TypeError: unexpected keyword argument 'bad_kwarg'
```
2023-09-15 10:39:14 -05:00
Angus Gratton
3e8aed9fcc py/gc: Add "max new split" value in result of gc.mem_free().
Follow-up to 519c24dd48 when MICROPY_GC_SPLIT_HEAP_AUTO is enabled, based
on discussion at
https://github.com/orgs/micropython/discussions/12316#discussioncomment-6858007

gc.mem_free() is always a heuristic, but this makes it a more useful
heuristic for common use cases.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-09-15 12:19:13 +10:00
4b41fdb586
Fast(ish) special purpose bitbang spi over i2c
with the i2c bus operating at 400kHz this achieves a 4.8kHz SPI clock
rate which could be worse.

It accepts the same style of init sequence as displayio.

tested by scoping the pins on the espressif lcd dev kit with a dummy init sequence:
```python
dotclockframebuffer.ioexpander_send_init_sequence(
    bus=bus,
    i2c_address=expander_addr,
    gpio_address=1,
    gpio_data_len=1,
    gpio_data=0xff,
    cs_bit=1,
    mosi_bit=3,
    clk_bit=2,
    init_sequence=init_sequence)
```
2023-09-14 14:42:48 -05:00
Damien George
397697aa83 py/persistentcode: Always close reader even if an exception is raised.
Fixes issue #3874.

Signed-off-by: Damien George <damien@micropython.org>
2023-09-14 13:03:31 +10:00
Damien George
5e122b11ea py/parse: Always free lexer even if an exception is raised.
Fixes issue #3843.

Signed-off-by: Damien George <damien@micropython.org>
2023-09-14 13:03:31 +10:00
Damien George
c9089e71a1 py/runtime: Add helpers to call a general function on nlr jump callback.
Signed-off-by: Damien George <damien@micropython.org>
2023-09-14 13:03:31 +10:00
Scott Shawcroft
0928a95bb2
Merge pull request #8351 from jepler/dotclockframebuffer
Dotclockframebuffer
2023-09-07 13:21:29 -07:00
IhorNehrutsa
6e79b55dc8 py/malloc: Fix DEBUG_print() args in m_realloc_maybe.
Signed-off-by: Ihor Nehrutsa <IhorNehrutsa@gmail.com>
2023-09-04 10:26:09 +10:00
David Lechner
ffb43b2dd3 py/modthread: Return thread id from start_new_thread().
In CPython, `_thread.start_new_thread()` returns an ID that is the same ID
that is returned by `_thread.get_ident()`.  The current MicroPython
implementation of `_thread.start_new_thread()` always returns `None`.

This modifies the required functions to return a value. The native thread
id is returned since this can be used for interop with other functions, for
example, `pthread_kill()` on *nix. `_thread.get_ident()` is also modified
to return the native thread id so that the values match and avoids the need
for a separate `native_id` attribute.

Fixes issue #12153.

Signed-off-by: David Lechner <david@pybricks.com>
2023-09-03 18:49:18 +10:00
Jim Mussared
a64f2fdca0 py/dynruntime.h: Implement MP_OBJ_NEW_QSTR.
Because mpy_ld.py doesn't know the target object representation, it emits
instances of `MP_OBJ_NEW_QSTR(MP_QSTR_Foo)` as const string objects, rather
than qstrs. However this doesn't work for map keys (e.g. for a locals dict)
because the map has all_keys_are_qstrs flag is set (and also auto-complete
requires the map keys to be qstrs).

Instead, emit them as regular qstrs, and make a functioning MP_OBJ_NEW_QSTR
function available (via `native_to_obj`, also used for e.g. making
integers).

Remove the code from mpy_ld.py to emit qstrs as constant strings, but leave
behind the scaffold to emit constant objects in case we want to do use this
in the future.

Strictly this should be a .mpy sub-version bump, even though the function
table isn't changing, it does lead to a change in behavior for a new .mpy
running against old MicroPython. `mp_native_to_obj` will incorrectly return
the qstr value directly as an `mp_obj_t`, leading to unexpected results.
But given that it's broken at the moment, it seems unlikely that anyone is
relying on this, so it's not work the other downsides of a sub-version bump
(i.e. breaking pure-Python modules that use @native). The opposite case of
running an old .mpy on new MicroPython is unchanged, and remains broken in
exactly the same way.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-09-01 16:14:22 +10:00
Daniël van de Giessen
d1f288c041 py/modstruct: Support pad bytes in struct format.
This adds support for the x format code in struct.pack and struct.unpack.

The primary use case for this is ignoring bytes while unpacking.  When
interfacing with existing systems, it may often happen that you either have
fields in a struct that aren't properly specified or you simply don't care
about them.  Being able to easily skip them is useful.

Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
2023-09-01 14:50:49 +10:00
mcskatkat
e0a1480600 py/objstr: Fix str % {} edge case.
Eliminate `TypeError` when format string contains no named conversions.
This matches CPython behavior.

Signed-off-by: mcskatkat <mc_skatkat@hotmail.com>
2023-09-01 14:31:57 +10:00
ed9cacf41d
Add DotClockFramebuffer 2023-08-30 10:30:15 -05:00
Jim Mussared
f3eccb154c py/mpconfig: Enable SSL finalizers if finalizers are enabled.
The rp2 port was enabling SSL and had finalizers enabled via the "extra
features" level, but missed explicitly enabling `MICROPY_PY_SSL_FINALISER`
(like esp32, stm32, and mimxrt did).

This commit makes `MICROPY_PY_SSL_FINALISER` default to enabled if
finalizers are enabled, and removes the explicit setting of this for
esp32, stm32, mimxrt (because they all use the "extra features" level).

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-30 12:38:13 +10:00
Jim Mussared
b2b5bcce28 py/profile: Remove the requirement to disable MICROPY_COMP_CONST.
The only reason that const had to be disabled was to make the test output
match CPython when const was involved.  Instead, this commit fixes the test
to handle the lines where const is used.

Also:
- remove the special handling for MICROPY_PERSISTENT_CODE_SAVE in
  unix/mpconfigport.h, and make this automatic.
- move the check for MICROPY_PERSISTENT_CODE_SAVE to where it's used (like
  we do for other similar checks) and add a comment explaining it.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-30 10:58:04 +10:00
Jim Mussared
bc1af5ab7f cc3200/Makefile: Build firmware.zip.
This allows the cc3200 port to be build with the standard autobuild script
rather than the custom build-cc3200-latest.sh (which is now removed).

This also fixes the path inside the zip file (by using the `-j` flag to
zip).

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-24 15:44:20 +10:00
4d8b354c13
Use qstrs to improve compression
any qstr of >= 3 chars long is a candidate for use; the input byte value
(1) is used to indicate a fixed-length qstr number follows.

Before accounting for the code size changes, this saves ~100 bytes
on trinket m0, but it may end up being a wash due to added code.

A cutoff of length>2 is slightly better for ja and ru and worse for others.
This has to do with the relative frequency of latin characters vs non-latin
characters, is my guess.

lang    after   before  change (- = more free space)
en_US   3132    2996    -134
fr      2136    2060    -76
ja      1916    1724    -232
ru      2196    2000    -196
2023-08-23 15:16:53 -05:00
Dan Halbert
88c22d5052 remove last uses of 'u' prefix 2023-08-22 12:57:47 -04:00
0cab3359df
ignore some cast-align diagnostics 2023-08-20 14:26:48 -05:00
1399d0f25b
guard mp_type_native_gen_wrap declaration with MICROPY_EMIT_NATIVE 2023-08-20 14:26:27 -05:00
f9e99bd3c3
Fix picow build 2023-08-20 11:30:36 -05:00
d5d566dc65
Don't assume __ICACHE_PRESENT is defined
This seems to only be defined in cmsis files for M7 family MCUs, so
it's not for e.g., makerdiary_m60_keyboard which enables loading of native
code. Lower MCUs don't ever have icache, so the default is "off".
2023-08-20 10:15:56 -05:00
d277294a14
Alias m_new_ll to m_new
this saves changing uses of it in ports/ and if we re-add ll later, it'll
make it easier to find the places that needed long-lived allocation.
2023-08-20 10:15:55 -05:00
7e38b79e21
Add mp_type_native_coro_wrap 2023-08-20 10:15:55 -05:00
b0bf36ef3b
Fix building with !MICROPY_PY_ASYNC_AWAIT 2023-08-19 17:17:50 -05:00
a94301122a
Make natmods work again.
And put back our magic number, because our bytecode format differs
from upstream

drop btree & framebuf natmods, they had additional problems I didn't
want to fix right now.
2023-08-19 16:59:13 -05:00
Dan Halbert
bfccb77ec1 asyncio test fixes and asyncio library updates 2023-08-18 13:16:16 -04:00
8d5e844b00
Implement traceback length limit 2023-08-15 13:10:31 -05:00
df1dd6ad3c
make 'import uerrno' work again in unix port
this fixes tests that 'import uerrno'.
2023-08-15 11:50:13 -05:00
Jim Mussared
9573d31071 all: Remove query-variants make target.
This is difficult to implement on cmake-based ports, and having the list
of variants in mpconfigboard.{cmake,mk} duplicates information that's
already in board.json.

This removes the existing query-variants make target from stm32 & rp2
and the definition of BOARD_VARIANTS from the various board files.

Also renames the cmake variable to MICROPY_BOARD_VARIANT to match other
variables such as MICROPY_BOARD. The make variable stays as
BOARD_VARIANT.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-15 17:37:44 +10:00
Jim Mussared
ad123ed013 esp32/Makefile: Implement make submodules to match other ports.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-15 17:37:44 +10:00
Angus Gratton
519c24dd48 py/gc: Add new MICROPY_GC_SPLIT_HEAP_AUTO "auto grow heap" mode.
When set, the split heap is automatically extended with new areas on
demand, and shrunk if a heap area becomes empty during a GC pass or soft
reset.

To save code size the size allocation for a new heap block (including
metadata) is estimated at 103% of the failed allocation, rather than
working from the more complex algorithm in gc_try_add_heap(). This appears
to work well except in the extreme limit case when almost all RAM is
exhausted (~last few hundred bytes). However in this case some allocation
is likely to fail soon anyhow.

Currently there is no API to manually add a block of a given size to the
heap, although that could easily be added if necessary.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-08-15 10:48:02 +10:00
Angus Gratton
d325ee4509 py/gc: Apply some code formatting cleanup.
This commit:
- Breaks up some long lines for readability.
- Fixes a potential macro argument expansion issue.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-08-15 10:41:02 +10:00
Dan Halbert
3d48e87e60 jepler: fix 18 tests (redo commit to be correct) 2023-08-14 18:34:32 -04:00
Dan Halbert
b73f05fe64 turn off viper tests; thanks jepler 2023-08-14 16:43:07 -04:00
Dan Halbert
8cf5b6e707 fixes to unix port support; thanks @jepler 2023-08-14 09:51:52 -04:00
Dan Halbert
d582407b06 pre-commit fixes 2023-08-14 00:59:22 -04:00
Dan Halbert
2171e67c1b merge latest adafruit/main 2023-08-13 19:43:54 -04:00
Dan Halbert
27c7807693 silabs build needs work; other builds are working 2023-08-13 19:05:16 -04:00
Dan Halbert
850877a85f wip; fix generators; typos 2023-08-13 00:30:59 -04:00
Dan Halbert
433059023d fix -j1 builds, particularly CPX 2023-08-12 16:49:03 -04:00
Dan Halbert
48e404ab90 wip; fix frozen; still need to fix -j1 for frozen 2023-08-12 15:44:27 -04:00
Dan Halbert
1b19ed4c39 change compilation verbosity options; fix mpy-cross 2023-08-12 13:38:17 -04:00
Dan Halbert
8a89a3d425 sort TRANSLATION()'s 2023-08-11 13:36:03 -04:00
Dan Halbert
10b94796c0 wip; fix more m_malloc and gc_alloc 2023-08-11 00:30:34 -04:00
Dan Halbert
fe0e2f13bc wip; fix qstr processing 2023-08-10 20:06:32 -04:00
Dan Halbert
0dca530b70 wip; Trinket M0 compiles 2023-08-09 15:20:10 -04:00
Dan Halbert
2ff8667e75 wip; remove long-lived functionality; remove PR#2614
Trinket M0 compile has no compilation errors, but has link errors to fix.
2023-08-08 20:41:17 -04:00
Dan Halbert
0d2c3c3f08 wip: continuing compilation fixes; mp_obj_alloc everywhere 2023-08-07 20:45:57 -04:00
Damien George
ef71028f77 extmod/modselect: Add optimisation to use system poll when possible.
A previous commit removed the unix-specific select module implementation
and made unix use the common one.

This commit adds an optimisation so that the system poll function is used
when polling objects that have a file descriptor.  With this optimisation
enabled, if code registers both file-descriptor-based objects, and non-
file-descriptor-based objects with select.poll() then the following occurs:

- the system poll is called for all file-descriptor-based objects with a
  timeout of 1ms

- then the bare-metal polling implementation is used for remaining objects,
  which calls into their ioctl method (which can be in C or Python)

In the case where all objects have file descriptors, the system poll is
called with the full timeout requested by the caller.  That makes it as
efficient as possible in the case everything has a file descriptor.

Benefits of this approach:

- all ports use the same select module implementation

- the unix port now supports polling of all objects and matches bare metal
  implementations

- it's still efficient for existing cases where only files and sockets are
  polled (on unix)

- the bare metal implementation does not change

- polling of SSL objects will now work on unix by calling in to the ioctl
  method on SSL objects (this is required for asyncio ssl support)

Note that extmod/vfs_posix_file.c has poll disable when the optimisation is
enabled, because the code is not reachable when the optimisation is used.

Signed-off-by: Damien George <damien@micropython.org>
2023-08-07 12:11:40 +10:00
46db0366b8
OrderedDict.move_to_end: alternate implementation of #8234
this implementation is hoped to be smaller. (feather_m4_express/fr fits
unlike the other PR; approximate savings ~600 bytes)

Minor difference to standard Python: A `dict` object has a
`move_to_end` method. However, calling this method always results in
TypeError.

Implementing it this way means that the method table can still be shared
between OrderedDict and builtin dict.

Closes #4408.
2023-08-04 09:26:26 -05:00
Damien Tournoud
2dcd745434 py/gc: Speed up incremental GC cycles by tracking the last used block.
In applications that use little memory and run GC regularly, the cost of
the sweep phase quickly becomes prohibitives as the amount of RAM
increases.

On an ESP32-S3 with 2 MB of external SPIRAM, for example, a trivial GC
cycle takes a minimum of 40ms, virtually all of it in the sweep phase.

Similarly, on the UNIX port with 1 GB of heap, a trivial GC takes 47 ms,
again virtually all of it in the sweep phase.

This commit speeds up the sweep phase in the case most of the heap is empty
by keeping track of the ID of the highest block we allocated in an area
since the last GC.

The performance benchmark run on PYBV10 shows between +0 and +2%
improvement across the existing performance tests.  These tests don't
really stress the GC, so they were also run with gc.threshold(30000) and
gc.threshold(10000).  For the 30000 case, performance improved by up to
+10% with this commit.  For the 10000 case, performance improved by at
least +10% on 6 tests, and up to +25%.

Signed-off-by: Damien George <damien@micropython.org>
2023-08-04 17:25:16 +10:00
Dan Halbert
465d1c673e wip 2023-08-03 23:29:00 -04:00
Dan Halbert
0d4bc8c163 initial v1.19.1 merge; not compiled yet 2023-08-01 13:50:05 -04:00
272a2dcdb0
Merge pull request #8218 from jepler/esp32-camera
Update the camera board to use ESP32-S3 microcontroller
2023-07-31 14:48:50 -05:00
cd181eb5b4
expand on the 0/1 requirement 2023-07-31 10:44:06 -05:00
e8307a4c02
enable-if-any/all deserve some commentary. 2023-07-31 10:21:53 -05:00
Dan Halbert
f466a4373e
Merge pull request #8222 from tannewt/fat_window_align
Align fatfs window buffer for tinyusb
2023-07-27 20:35:23 -04:00
Scott Shawcroft
e45a61f57e
Align fatfs window buffer for tinyusb
Also, update tinyusb.
2023-07-27 09:15:58 -07:00
Dan Halbert
5cc42d5041
Merge pull request #8126 from jepler/update-ulab-6.3.2
update ulab to 6.3.2
2023-07-26 11:54:42 -04:00
c026704fd2
gifio does require displayio 2023-07-26 08:54:02 -05:00
d83b8db678
restore fontio define 2023-07-25 21:34:57 -05:00
9a18524c30
simplify circuitpy_mpconfig with enable-if-any, -all
and make corresponding simplifications in shared-bindings-matrix,
but directly using the final defines from CFLAGS instead of the
status quo.

The net changes are to disable audiocore & audiomixer on some espressif
devices that have no audio output at all. Other than that, the
shared-bindings-matrix seems to be identical.
2023-07-25 19:56:08 -05:00
fd60cccb06
Re-enable ulab at the expense of some other modules
.. and gate audiomixer based on audiocore (should be no net change)

.. and get rid of now-redundant enables in the espressif mpconfigport
2023-07-25 16:01:11 -05:00
8579766ede
Optimize ulab for size on samd51/same51 2023-07-25 09:35:49 -05:00
Jim Mussared
975a687447 py/mpconfig: Add MICROPY_PY_PLATFORM, enabled at extra features level.
Previously this was explicitly enabled on esp32/stm32/renesas/mimxrt/samd,
but didn't get a default feature level because it wasn't in py/mpconfig.h.

With this commit it's now enabled at the "extra features" level, which adds
rp2, unix-standard, windows, esp8266, webassembly, and some nrf boards.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-24 23:53:20 +10:00
Damien George
8ef5622b9b py/runtime: Always initialise sched_state in mp_init.
When MICROPY_SCHEDULER_STATIC_NODES is enabled, the logic is unchanged.

When MICROPY_SCHEDULER_STATIC_NODES is disable, sched_state is now always
initialised to MP_SCHED_IDLE when calling mp_init().  For example, the use
of mp_sched_vm_abort(), if it aborts a running scheduled function, can lead
to the scheduler starting off in a locked state when the runtime is
restarted, and then it stays locked.  This commit fixes that case by
resetting sched_state.

Signed-off-by: Damien George <damien@micropython.org>
2023-07-24 15:04:27 +10:00
Jim Mussared
3533924c36 extmod/moddeflate: Add deflate module providing the DeflateIO class.
This provides similar functionality to the former zlib.DecompIO and
especially CPython's gzip.GzipFile for both compression and decompression.

This class can be used directly, and also can be used from Python to
implement (via io.BytesIO) zlib.decompress and zlib.compress, as well as
gzip.GzipFile.

Enable/disable this on all ports/boards that zlib was previously configured
for.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 19:32:40 +10:00
Jim Mussared
198311c780 py/stream: Add mp_stream___exit___obj that calls mp_stream_close.
There are enough places that implement __exit__ by forwarding directly to
mp_stream_close that this saves code size.

For the cases where __exit__ is a no-op, additionally make their
MP_STREAM_CLOSE ioctl handled as a no-op.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 18:49:03 +10:00
Jim Mussared
add1200343 all: Remove the zlib module.
This will be replaced with a new deflate module providing the same
functionality, with an optional frozen Python wrapper providing a
replacement zlib module.

binascii.crc32 is temporarily disabled.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 18:48:29 +10:00
Scott Shawcroft
525dad71c3
Add RuntimeError_varg and fix imx capitalization 2023-07-20 16:00:43 -07:00
Scott Shawcroft
d7fa7380b8
Move some find_qstr wrappers to tcm next to it 2023-07-20 11:20:32 -07:00
KB Sriram
03e93f8e6c py/objdict: Fix fromkeys to return the right type.
Fixes https://github.com/adafruit/circuitpython/issues/8173

It looks like a small fix, and mostly independent of upstream
plans around https://github.com/micropython/micropython/pull/6173

I also filed an issue upstream
https://github.com/micropython/micropython/issues/12011
2023-07-14 22:03:06 -07:00
Jim Mussared
671b35ceae py/builtinimport: Fix built-in imports when external import is disabled.
Follow-up to 24c02c4eb5 for when
MICROPY_ENABLE_EXTERNAL_IMPORT=0.  It now needs to try both extensible and
non-extensible modules.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-13 14:56:33 +10:00
Damien George
606ec9bfb1 py/compile: Fix async for's stack handling of iterator expression.
Prior to this fix, async for assumed the iterator expression was a simple
identifier, and used that identifier as a local to store the intermediate
iterator object.  This is incorrect behaviour.

This commit fixes the issue by keeping the iterator object on the stack as
an anonymous local variable.

Fixes issue #11511.

Signed-off-by: Damien George <damien@micropython.org>
2023-07-13 13:50:50 +10:00
Scott Shawcroft
2686beab36
Basic USB host support and keyboard workflow
Connects up read, write and ctrl_transfer to TinyUSB. USB Host
support is available on iMX RT and RP2040.

Fixes #6527 (imx) and fixes #5986 (rp2).
2023-07-10 14:22:21 -07:00
Dan Halbert
4c55dc2445
Merge pull request #8096 from jepler/lto-parallel
Speed LTO builds by using multiple threads
2023-06-20 13:32:47 -04:00
cae02f1cdf
Speed LTO builds by using multiple threads
On my i5-1235U laptop this speeds LTO "partition=balanced" builds
substantially, because each "partition" can be run on a separate
CPU thread. I used "pygamer" as my test build with a parallelism of
`-j4`, and took the best elapsed time reported over 4 builds.

The improvement was from 34.6s to 24.0s (-30%).

A link-only build (rm build-pygamer/firmware.elf; make -j...) improved
from1 17.4s to 5.1s (-70%)

The size of the resulting firmware is unchanged.

Boards that are nearly full use "-flto-partition=one" to improve code
size optimization. When LTO partition is "one", this feature doesn't help
but it doesn't seem to negatively affect anything either (tested
building trinket_m0)
2023-06-20 11:01:46 -05:00
Jim Mussared
2fbc08c462 extmod/asyncio: Rename uasyncio to asyncio.
The asyncio module now has much better CPython compatibility and
deserves to be just called "asyncio".

This will avoid people having to write `from uasyncio import asyncio`.

Renames all files, and updates port manifests to use the new path. Also
renames the built-in _uasyncio to _asyncio.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-19 17:33:03 +10:00
Damien George
bf9ca0bb33 py/mkrules.mk: Allow $(AFLAGS) to set flags to $(AS).
Signed-off-by: Damien George <damien@micropython.org>
2023-06-15 11:02:15 +10:00
Jared Hancock
b3cd41dd4b py/lexer: Allow conversion specifiers in f-strings (e.g. !r).
PEP-498 allows for conversion specifiers like !r and !s to convert the
expression declared in braces to be passed through repr() and str()
respectively.

This updates the logic that detects the end of the expression to also stop
when it sees "![rs]" that is either at the end of the f-string or before
the ":" indicating the start of the format specifier. The "![rs]" is now
retained in the format string, whereas previously it stayed on the end
of the expression leading to a syntax error.

Previously: `f"{x!y:z}"` --> `"{:z}".format(x!y)`
Now: `f"{x!y:z}"` --> `"{!y:z}".format(x)`

Note that "!a" is not supported by `str.format` as MicroPython has no
`ascii()`, but now this will raise the correct error.

Updated cpydiff and added tests.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-14 19:11:04 +10:00
Damien George
5ce1a03a78 py/makemoduledefs.py: Automatically declare delegation attr functions.
So that the delegation functions don't need to be put somewhere global,
like in mpconfigport.h.  That would otherwise make it hard for extension
modules to use delegation.

Signed-off-by: Damien George <damien@micropython.org>
2023-06-14 19:03:46 +10:00
Damien George
44295c9daa py/makemoduledefs.py: Fix declaring multiple module delegations.
Signed-off-by: Damien George <damien@micropython.org>
2023-06-14 19:03:29 +10:00
David Lechner
b02a5fa10a py/nlraarch64: Fix dangerous use of input register.
Starting with 2757acf6, the `top` variable in `nlr_jump()` in
`nlraarch64.c` was assigned to register `x19` by the compiler.  However,
the assembly code writes over that register with

    ldp x19, x20, [%0,  #32]

since `%0` is now `x19`. This causes the next line

    ldp lr,  x9,  [%0,  #16]

to load the wrong values.

To fix the issue, we move the value of the `top` variable from an unknown
register to a known register at the beginning of the asm code then only use
known/hard-coded registers after that.

Fixes issue #11754.

Signed-off-by: David Lechner <david@pybricks.com>
2023-06-14 17:43:44 +10:00
David Lechner
8cf9898dd3 py/parsenum: Fix typo in #endif comment.
This fixes a `#endif` comment to exactly match the `#if`.

Signed-off-by: David Lechner <david@pybricks.com>
2023-06-14 17:32:01 +10:00
666fb94ca3
Merge pull request #8056 from jepler/matrixportal-s3
Add Adafruit MatrixPortal S3 board def & update protomatter
2023-06-13 19:34:08 -05:00
d4913b04e0 rgbmatrix: update protomatter
and re-organize so that esp32 s2/s3 don't do as much at reset

.. it's not necessary (because most data is in esp-idf managed memory)
and doing this saves me from having to debug why reconstruct isn't working
properly on that platform.

This needs to be tested on other platforms again before being merged!
2023-06-08 10:43:32 -05:00
Damien George
f01d5fb657 py/mkrules.mk: Automatically configure frozen options when manifest set.
Following how mkrules.cmake works.  This makes it easy for a port to enable
frozen code, by defining FROZEN_MANIFEST in its Makefile.

Signed-off-by: Damien George <damien@micropython.org>
2023-06-08 23:12:56 +10:00
Jim Mussared
a1fbb1980c extmod/modtimeq: Remove timeq module.
This is a MicroPython-specific module that existed to support the old
version of uasyncio.  It's undocumented and not enabled on all ports and
takes up code size unnecessarily.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:28 +10:00
Jim Mussared
5e50975a6d py/modsys: Allow sys.path to be assigned to.
Previously sys.path could be modified by append/pop or slice assignment.

This allows `sys.path = [...]`, which can be simpler in many cases, but
also improves CPython compatibility.

It also allows sys.path to be set to a tuple which means that you can
clear sys.path (e.g. temporarily) with no allocations.

This also makes sys.path (and sys.argv for consistency) able to be disabled
via mpconfig. The unix port (and upytesthelper) require them, so they
explicitly verify that they're enabled.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00
Jim Mussared
7d2ee8aed0 py/mpconfig: Enable module delegation if sys needs it.
Otherwise you can get into the confusing state where e.g. sys.ps1 is
enabled in config (via `MICROPY_PY_SYS_PS1_PS2`) but still doesn't actually
get enabled.

Also verify that the required delegation options are enabled in modsys.c.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00
Jim Mussared
e6926d6021 py/objmodule: Workaround for MSVC with no module delegation.
When compiling mpy-cross, there is no `sys` module, and so there will
be no entries in the `mp_builtin_module_delegation_table`.

MSVC doesn't like this, so instead pretend as if the feature isn't
enabled at all.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00
Jim Mussared
13c817e61c py/objmodule: Add a table of built-in modules with delegation.
This replaces the previous QSTR_null entry in the globals dict which could
leak out to Python (e.g. via iteration of mod.__dict__) and could lead to
crashes.

It results in smaller code size at the expense of turning a lookup into a
loop, but the list it is looping over likely only contains one or two
elements.

To allow a module to register its custom attr function it can use the new
`MP_REGISTER_MODULE_DELEGATION` macro.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00