Picolibc has both recursive and non-recursive mutex uses. The bulk of the
library locking uses the global libc lock, which is a recursive mutex as
that greatly simplifies the implementation.
The only use of non-recursive mutexes is in the stdio code when dealing
with file system I/O via fopen.
Using mutexes for both APIs is valid; the assumption picolibc makes is that
the non-recursive mutexes are somehow cheaper or faster and should be
preferred. However, in Zephyr, recursive mutexes are the default and the
non-recursive locks for picolibc were implemented using semaphores.
Switch the non-recursive picolibc locks to just invoking the existing
recursive functions using mutexes. This avoids pulling in another lock
implementation, saving a bit of space.
This also lets the kernel.memory_protection.mem_map test work on
qemu_x86_tiny where the amount of memory available is 320kB and that is
nearly filled by this test case, leaving too little space for allocating
pages in the k_mem_map_unmap test.
Signed-off-by: Keith Packard <keithp@keithp.com>
Now that the gcc compiler driver uses the -T flag instead of -Wl,-T, we can
remove the hack here that kept the picolibc specs file from inserting the
picolibc linker script.
Signed-off-by: Keith Packard <keithp@keithp.com>
The spin loop in _exit() needs a Z_SPIN_DELAY() for the
POSIX architecture, so it does not hang the whole
executable on that infinite loop but only the thread
that exit'ed.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
When using the Zephyr SDK toolchain, prefer the pre-built version of
picolibc over using the picolibc module. This will reduce the time it takes
to build applications.
Signed-off-by: Keith Packard <keithp@keithp.com>
Enable the common malloc implementation when using picolibc. Support
existing Picolibc configurations by respecting existing PICOLIBC_HEAP_SIZE
settings.
When PICOLIBC_HEAP_SIZE is set to a value other than -2, then
*always* set COMMON_LIBC_MALLOC_ARENA_SIZE to that value.
Signed-off-by: Keith Packard <keithp@keithp.com>
Move the abort implementation into common so its shared among the
libc. As part of this start using the common abort on newlib.
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
When using the toolchain C library, that must be added to the link command
after all other libraries and modules in the system to resolve undefined
symbols.
Signed-off-by: Keith Packard <keithp@keithp.com>
The init infrastructure, found in `init.h`, is currently used by:
- `SYS_INIT`: to call functions before `main`
- `DEVICE_*`: to initialize devices
They are all sorted according to an initialization level + a priority.
`SYS_INIT` calls are really orthogonal to devices, however, the required
function signature requires a `const struct device *dev` as a first
argument. The only reason for that is because the same init machinery is
used by devices, so we have something like:
```c
struct init_entry {
int (*init)(const struct device *dev);
/* only set by DEVICE_*, otherwise NULL */
const struct device *dev;
}
```
As a result, we end up with such weird/ugly pattern:
```c
static int my_init(const struct device *dev)
{
/* always NULL! add ARG_UNUSED to avoid compiler warning */
ARG_UNUSED(dev);
...
}
```
This is really a result of poor internals isolation. This patch proposes
a to make init entries more flexible so that they can accept sytem
initialization calls like this:
```c
static int my_init(void)
{
...
}
```
This is achieved using a union:
```c
union init_function {
/* for SYS_INIT, used when init_entry.dev == NULL */
int (*sys)(void);
/* for DEVICE*, used when init_entry.dev != NULL */
int (*dev)(const struct device *dev);
};
struct init_entry {
/* stores init function (either for SYS_INIT or DEVICE*)
union init_function init_fn;
/* stores device pointer for DEVICE*, NULL for SYS_INIT. Allows
* to know which union entry to call.
*/
const struct device *dev;
}
```
This solution **does not increase ROM usage**, and allows to offer clean
public APIs for both SYS_INIT and DEVICE*. Note that however, init
machinery keeps a coupling with devices.
**NOTE**: This is a breaking change! All `SYS_INIT` functions will need
to be converted to the new signature. See the script offered in the
following commit.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
init: convert SYS_INIT functions to the new signature
Conversion scripted using scripts/utils/migrate_sys_init.py.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
manifest: update projects for SYS_INIT changes
Update modules with updated SYS_INIT calls:
- hal_ti
- lvgl
- sof
- TraceRecorderSource
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
tests: devicetree: devices: adjust test
Adjust test according to the recently introduced SYS_INIT
infrastructure.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
tests: kernel: threads: adjust SYS_INIT call
Adjust to the new signature: int (*init_fn)(void);
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
libstdc++ is supported with Picolibc only when the toolchain version of
Picolibc is use -- libstdc++ must be built using a specific Picolibc build
and libstdc++ is included with the toolchain.
Ideally, we'd allow the use of the Picolibc module whenever we weren't
using the GNU libstdc++, including when using the minimal libc++. However,
the obvious dependency settings create a loop:
config PICOLIBC
depends on PICOLIBC_SUPPORTED
config PICOLIBC_SUPPORTED
depends on !(GLIBCXX_LIBCPP && "$(ZEPHYR_TOOCHAIN_VARIANT" = "zephyr")
config GLIBCXX_LIBCPP
depends on NEWLIB_LIBC || PICOLIBC
To break this loop, we replace GLIBCXX_LIBCPP in the second block with
CPP:
config PICOLIBC_SUPPORTED
depends on !(CPP && "$(ZEPHYR_TOOCHAIN_VARIANT" = "zephyr")
This means that picolibc cannot be used with any C++ apps when using the
Zephyr SDK, even when not using the GNU libstdc++. However, Zephyr SDK 0.16
will come with an additional Kconfig file that includes:
config PICOLIBC_SUPPORTED
def_bool y
depends on "$(ZEPHYR_TOOLCHAIN_VARIANT)" = "zephyr"
This will override the Kconfig bits included in Zephyr and allow use of the
Picolibc module with C++ code, including using the minimal libc++ bits.
Signed-off-by: Keith Packard <keithp@keithp.com>
The `sys/stat.h` header has never been a part of ISO C so move it
to `zephyr/include/posix/sys/`.
To ensure a smooth migration, leave a stub header in
`lib/libc/minimal/include/sys/` that prints a deprecation warning
suggesting developers either include `<zephyr/posix/sys/stat.h>`
or use `CONFIG_POSIX_API=y`.
Signed-off-by: Chris Friedt <cfriedt@meta.com>
This commit updates the Picolibc configurations to remove any
unnecessary defaults and dependencies and conform to the de-facto
standard convention across the Zephyr repository.
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
While reviewing the dependency between Picolibc POSIX APIs and Zephyr, I
found that the picolibc libc-hooks code copied a bunch of functions from
the newlib version which weren't needed. This required replacing a few
calls to the (now removed) '_write' hook with printk instead.
Signed-off-by: Keith Packard <keithp@keithp.com>
This adds the necessary modifier to the stdin/stdout/stderr
variables in picolib, and putting into the z_libc_partition.
This allows userspace applications to utilize these variables
for console I/O.
Fixes#51343
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Compliance check complains about static not being the first
modifier. So move them so there are no more complains
in the future.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The picolibc heap size configuration (`CONFIG_PICOLIBC_HEAP_SIZE`) is
used to set the statically allocated malloc heap size when userspace is
enabled.
The current default heap size of 1048576 bytes (MMU) and 65536 bytes
(MPU) is too large for most platforms that Zephyr supports and may
result in the picolibc tests being filtered out due to the increased
memory footprint of the compiled image (i.e. SRAM overflow).
This commit updates the default picolibc heap size to a more reasonable
16384 bytes for MMU platforms and 1024 bytes for MPU platforms.
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
As of today <zephyr/zephyr.h> is 100% equivalent to <zephyr/kernel.h>.
This patch proposes to then include <zephyr/kernel.h> instead of
<zephyr/zephyr.h> since it is more clear that you are including the
Kernel APIs and (probably) nothing else. <zephyr/zephyr.h> sounds like a
catch-all header that may be confusing. Most applications need to
include a bunch of other things to compile, e.g. driver headers or
subsystem headers like BT, logging, etc.
The idea of a catch-all header in Zephyr is probably not feasible
anyway. Reason is that Zephyr is not a library, like it could be for
example `libpython`. Zephyr provides many utilities nowadays: a kernel,
drivers, subsystems, etc and things will likely grow. A catch-all header
would be massive, difficult to keep up-to-date. It is also likely that
an application will only build a small subset. Note that subsystem-level
headers may use a catch-all approach to make things easier, though.
NOTE: This patch is **NOT** removing the header, just removing its usage
in-tree. I'd advocate for its deprecation (add a #warning on it), but I
understand many people will have concerns.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
When the heap is of a fixed size and there isn't a special malloc partition
in use, place the heap in uninitialized memory so that the application
doesn't spend time at startup erasing it. Picolibc malloc always clears
memory before returning it to applications, so this change will not be
visible to applications.
Signed-off-by: Keith Packard <keithp@keithp.com>
Any project with Kconfig option CONFIG_LEGACY_INCLUDE_PATH set to n
couldn't be built because some files were missing zephyr/ prefix in
includes
Re-run the migrate_includes.py script to fix all legacy include paths
Signed-off-by: Tomislav Milkovic <milkovic@byte-lab.com>
This commit changes the invocation of the picolibc malloc heap
initialisation function such that it is executed during the POST_KERNEL
phase instead of the APPLICATION phase.
This is necessary in order to ensure that the application
initialisation functions (i.e. the functions called during the
APPLICATIION phase) can make use of the libc heap.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Picolibc inherited its malloc arena configuration from newlib instead of
from minimal libc. This ended up making it a bit too fragile to run the
full set of zephyr tests. In particular:
* Z_MALLOC_PARTITION_EXISTS would get set when not used
* Setting an arena size depended on a bunch of other values, including
whether the system had an MMU or MPU, and whether the MPU required
power-of-two alignment or not.
This patch cleans things up so that there is a single heap size specifier,
PICOLIBC_HEAP_SIZE.
* If PICOLIBC_HEAP_SIZE is positive, this sets the size of the heap. On
MMU systems, picolibc will only use the remaining memory if that's
smaller.
* If PICOLIBC_HEAP_SIZE is zero, then there is no heap available and
malloc will always fail. This also disables Z_MALLOC_PARTITION_EXISTS.
* If PICOLIBC_HEAP_SIZE is negative, then picolibc uses all remaining
memory for the malloc heap.
The defaults are designed to allow tests to work without requiring
additional settings.
* For MMU enabled systems, the default value is 1048576. It would be nice
to have this use 'all available memory', but that's difficult to manage
as the API which returns free memory (k_mem_free_get) doesn't take into
account the amount of free virtual address space.
* For MPU enabled systems which require power-of-two aligned MPU regions,
the default value is 64kB.
* For other systems, the default value is -1, indicating that all
available memory be used for the malloc arena.
Signed-off-by: Keith Packard <keithp@keithp.com>
Picolibc already provides the functionality offered by cbprintf, so
there's no reason to use the larger and less functional version included
in zephyr.
Signed-off-by: Keith Packard <keithp@keithp.com>
For targets without thread local storage, we need to use the builtin
per-thread errno support provided by Zephyr as the multi-thread errno
support provided in picolibc relies on TLS.
Signed-off-by: Keith Packard <keithp@keithp.com>
Picolibc is a fork of newlib designed and tested on embedded systems. It
offers a smaller memory footprint (both ROM and RAM), and native TLS
support, which uses the Zephyr TLS support.
By default, the full printf version is included in the executable, which
includes exact floating point and long long input and output. A
configuration option has been added to switch to the integer-only
version (which also omits long long support).
Here are some size comparisons using qemu-cortex-m3 and this application
(parameters passed to printf to avoid GCC optimizing it into puts):
void main(void)
{
printf("Hello World! %s %d\n", CONFIG_BOARD, 12);
}
FLASH SRAM
minimal 8696 3952
picolibc int 7600 3960
picolibc float 12304 3960
newlib-nano int 11696 4128
newlib-nano float 30516 4496
newlib 34800 6112
---
v2:
Include picolibc-tls.ld
v3:
Document usage in guides/c_library.rst and
getting_started/toolchain_other_x_compilers.rst
v4:
Lost the lib/libc/picolibc directory somehow!
v5:
Add PICOLIBC_ALIGNED_HEAP_SIZE configuration option.
Delete PICOLIBC_SEMIHOST option support code
v6:
Don't allocate static RAM for TLS values; TLS
values only need to be allocated for each thread.
v7:
Use arm coprocessor for TLS pointer storage where supported for
compatibility with the -mtp=cp15 compiler option (or when the
target cpu type selects this option)
Add a bunch of tests
Round TLS segment up to stack alignment so that overall stack
remains correctly aligned
Add aarch64 support
Rebase to upstream head
v8:
Share NEWLIB, NEWLIB_NANO and PICOLIBC library configuration
variables in a single LIBC_PARTITIONS variable instead of
having separate PICOLIBC_PART and NEWLIB_PART variables.
v9:
Update docs to reference pending sdk-ng support for picolibc
v10:
Support memory protection by creating a partition for
picolibc shared data and any pre-defined picolibc heap.
v11:
Fix formatting in arch/arm/core/aarch64/switch.S
v12:
Remove TLS support from this patch now that TLS is upstream
Require THREAD_LOCAL_STORAGE when using PICOLIBC for architectures
that support it.
v13:
Merge errno changes as they're only needed for picolibc.
Adapt cmake changes suggested by Torsten Tejlmand Rasmussen
v14:
Update to picolibc 1.7 and newer (new stdin/stdout/stderr ABI)
v15:
Respond to comments from dcpleung:
* switch kernel/errno to use CONFIG_LIBC_ERRNO instead of
CONFIG_PICOLIBC
* Add comment to test/lib/sprintf as to why the %n test
was disabled for picolibc.
v16:
Switch picolibc to a module built with Zephyr. This eliminates
toolchain dependencies and allows compiler settings for Zephyr
to also be applied to picolibc.
v17:
Provide Zephyr-specific 'abort' implementation.
Support systems with MMU
v18:
Allow use of toolchain picolibc version.
v19:
Use zephyr/ for zephyr headers
v20:
Add locking
Use explicit commit for picolibc module
v21:
Create PICOLIBC_SUPPORTED config param. Set on arc, arm, arm64,
mips and riscv architectures.
Signed-off-by: Keith Packard <keithp@keithp.com>
