Add a new call for initializing cache on architectures that need that.
Avoid using SYS_INIT for this and instead call the hook in a fixed place
and run if implemented.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This adds a kconfig to enable making the interrupts
non-preemptible by other interrupts. Enabling this will set
the INTLEVEL to the max non-debug level before clearing
the EXCM bit.
Signed-off-by: Christopher J. Champagne <christopher.j.champagne@intel.com>
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
nocache ram is usually used by DMA to transfer data between
peripherals and ram. Some chips use isolated nocache ram,
which does not necessarily have to be in RAMABLE-REGION.
By specifying the zephyr,nocache-ram options, users can specify
the region where nocache-ram is located. If the user does not
specify it, it defaults to RAMABLE-REGION.
Signed-off-by: Weiwei Guo <guoweiwei@syriusrobotics.com>
Otherwise we can't escape from DEPRECATED being selected, and so getting
build warnings. It doesn't make sense that the option replacing the
deprecated one is used to automatically enable it.
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
If adding/removing to the domain of the current running
thread, we need to update the hardware MPU regions or else
the addition or removal would not be reflected to current
running thread.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds a spinlock to make sure writing to hardware MPU
regions is atomic, and cannot be interrupted until all
regions are written to hardware.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Fill the memory of all CPU's IRQ stack with 0xAA on init, so
that `z_stack_space_get` can calculate the remaining space
correctly.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Adds support for all relocation type produced by GCC
on AARCH64 platform using partial linking (-r flag) or
shared link (-fpic and -shared flag).
Signed-off-by: Adam Wojasinski <awojasinski@baylibre.com>
If PSCI is enabled, it will leverage psci reset API to achieve system
reboot, otherwise a weak dump reboot API is provided, and platform
can override these APIs if platform specified implementation provided.
Signed-off-by: Jiafei Pan <Jiafei.Pan@nxp.com>
Convention is to call k_mem_page_fault() with IRQs enabled if they were
enabled when the fault occurred.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This has two purposes: maintain the accessed and dirty page states, or
call the generic demand paging fault handler otherwise.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
A page table entry used for demand paging is always from the last level
page table and never completely zeroed. Anything else is considered
ARCH_DATA_PAGE_NOT_MAPPED.
Loaded pages use a PTE_PAGE_DESC table entry type. Paged-out pages use a
PTE_INVALID_DESC table entry type and the physical address field is
reused to hold the backing store location token value.
ARCH_DATA_PAGE_ACCESSED corresponds to the AF flag. It is initially
unset and manually set to catch when pages are being accessed and
eventually do something about it.
ARCH_DATA_PAGE_DIRTY corresponds to the lack of the AP_RO flag.
Similarly to the AF flag, pages are initially made read-only to catch
writes.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Let's consider free entries as being completely zeroed. Future patches
for demand paging support will populate entries and still mark them as
"invalid" which should not be considered free. Current code always clear
entries to be freed so no issues there.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
With this commit, it is now allowed to register any ISR and arg
combination for the same IRQ, except the case when the exact same
ISR-arg combination is already registered.
The previous assert logic had a restriction where the same ISR could not
be registered multiple times with different arguments.
Signed-off-by: Martin Åberg <martin.aberg@gaisler.com>
Add missing braces to comply with MISRA C:2012 Rule 15.6 and
also following Zephyr's style guideline.
Signed-off-by: Pisit Sawangvonganan <pisit@ndrsolution.com>
The callee-saved-registers can be helpful to debug the state of
a core upon an exception, however, currently there's no way to
access that information in user-implemented
`k_sys_fatal_error_handler()`, even though the csf is already stored
in the stack.
This patch conditionally add a `csf` member in the `arch_esf` when
`CONFIG_EXTRA_EXCEPTION_INFO=y`*, which the `_isr_wrapper` would update
when a fatal error occurs before invoking `z_riscv_fatal_error_csf()`.
Functions such as `k_sys_fatal_error_handler()` would then be able
to access the callee-saved-registers at the time of exception via
`esf->csf`.
* For SoCs that select `RISCV_SOC_HAS_ISR_STACKING`, the
`SOC_ISR_STACKING_ESF_DECLARE` has to include the `csf` member,
otherwise the build would fail.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
On 32-bit x86, it was supposed to print the first argument to
the function during stack trace. However, it only works when
code optimizations are totally disabled (i.e. -O0). As such,
printing the args is not meaningful to aid with debugging.
So change it to simply print the function address, the same
as x86 64-bit.
Also, since unwind_stack() has exactly one caller, make it
ALWAYS_INLINE to skip a function call.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Get rid of the switch statement and use an string array
for the cause instead. This saves about ~600 bytes.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Rename `cause_str` to `z_riscv_mcause_str` and make it non-static,
so that it can be used in user-implemented `k_sys_fatal_error_handler`.
Also, this function should return a constant string, so add `const`
to it.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Relocate the logging of mcause & mtval from `_Fault` to
`z_riscv_fatal_error_csf` so that they are always printed
upon exception.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Use generic hook infrastrucutre instead of custom Kconfig and hooks for
ARC.
Replace soc_early_asm_init_percpu() with platform_reset()
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Use generic hook infrastrucutre instead of custom Kconfig and hooks for
ARM.
Replace z_arm_platform_init() with platform_reset().
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Introduce soc and board hooks to replace arch specific code
and replace usages of SYS_INIT for platform initialization.
include/zephyr/platform/hooks.h introduces the hooks to be implemented
by boards and SoCs.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The bits allocated for each aggregator level only need to be enough to
encode CONFIG_MAX_IRQ_PER_AGGREGATOR, instead of the combined number of
IRQs from all aggregators in that level.
Add additional check for L3 interrupts as well, if it is enabled.
Updated the assert in `z_get_sw_isr_table_idx()` to be more verbose.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Add bootargs support to zefi. This implements
get_bootargs() when both efi and bootargs are
selected in config.
Signed-off-by: Jakub Michalski <jmichalski@internships.antmicro.com>
Signed-off-by: Filip Kokosinski <fkokosinski@antmicro.com>
Zefi was only including include/zephyr but not the generated include
directory that contains autoconf.h. This commit fixes that.
Signed-off-by: Jakub Michalski <jmichalski@internships.antmicro.com>
Signed-off-by: Filip Kokosinski <fkokosinski@antmicro.com>
Add bootargs support for multiboot. This
implements get_bootargs() when multiboot and
bootargs are selected in config.
Signed-off-by: Jakub Michalski <jmichalski@internships.antmicro.com>
Signed-off-by: Filip Kokosinski <fkokosinski@antmicro.com>
Add Kconfig option to specify Xen interface version to use. This will
make it easier to switch between different versions of Xen.
Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
Add the possibility to disable fault handling in spurious
interrupt handler on RISCs and replacce it with an infinite loop.
Signed-off-by: Magdalena Pastula <magdalena.pastula@nordicsemi.no>
The change of alignment check in #76045 could be wrong and
isn't unnecessary to fix the stack traces output, undo it for
now.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
This implements arch_thread_priv_stack_space_get() so this can
be used to figure out how much privileged stack space is used.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds the bits to initialize the privileged stack when
a thread is transitioning to user mode. This prevents
information leaking if the stack is reused, and also aids
in calculating stack space usage during system calls.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This implements arch_thread_priv_stack_space_get() so this can
be used to figure out how much privileged stack space is used.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds the bits to initialize the privileged stack for
each thread during thread initialization. This prevents
information leaking if the thread stack is reused, and
also aids in calculating stack space usage during system
calls.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This implements arch_thread_priv_stack_space_get() so this can
be used to figure out how much privileged stack space is used.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds the bits to initialize the privileged stack for
each thread during thread initialization. This prevents
information leaking if the thread stack is reused, and
also aids in calculating stack space usage during system
calls.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Only set the privileged stack pointer for thread stacks, but
nullify the pointer for kernel-only stacks, as these stacks
do not have the reserved space. The psp pointer may point to
arbitrary memory in this case if stack is not big enough.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds a new arch_thread_priv_stack_space_get() interface for
each architecture to report privileged stack space usage. Each
architecture will need to implement this function as each arch
has their own way of defining privileged stacks.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Before commit baa70d8d36 ("arch/arm64/mmu: fix page table reference
counting part 2") it was possible to perform a partial unmap of a block
mapping. Restore that ability and provide a test case to validate it.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This provides memory mappings with the ability to be initialized in their
paged-out state and be paged in on demand. This is especially nice for
anonymous memory mappings as they no longer have to allocate all memory
at mem_map time. This also allows for file mappings to be implemented by
simply providing backing store location tokens.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Having `CONFIG_EXCEPTION_STACK_TRACE_SYMTAB` to select the
`CONFIG_SYMTAB` or to explicitly not print the symbol name
during exception stack unwind seems unnecessary, as the extra
code to print the symbol name is negligible when compared with
the symbol table, so just remove it.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Currently it supports `esf` based unwinding only.
Then, update the exception stack unwinding to use
`arch_stack_walk()`, and update the Kconfigs & testcase
accordingly.
Also, `EXCEPTION_STACK_TRACE_MAX_FRAMES` is unused and
made redundant after this change, so remove it.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Currently it supports `esf` based unwinding only.
Then, update the exception stack unwinding to use
`arch_stack_walk()`, and update the Kconfigs & testcase
accordingly.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
This commit introduces a new ARCH_STACKWALK Kconfig which
determines if the `arch_stack_walk()` is available should the
arch supports it.
Starting from RISCV, this will be able to converge the exception
stack trace implementation & stack walking features. Existing
exception stack trace implementation will be updated later.
Eventually we will end up with the following:
1. If an arch implements `arch_stack_walk()`
`ARCH_HAS_STACKWALK` should be selected.
2. If the above is enabled, `ARCH_SUPPORTS_STACKWALK` indicates
if the dependencies are met for arch to enable stack walking.
This Kconfig replaces `<arch>_EXCEPTION_STACK_TRACE`
2. If the above is enabled, then, `ARCH_STACKWALK` determines
if `arch_stack_walk()` should be compiled.
3. `EXCEPTION_STACK_TRACE` should build on top of the
`ARCH_STACKWALK`, stack traces will be printed when it
is enabled.
4. `ARCH_STACKWALK_MAX_FRAMES` will be removed as it is
replaced by `ARCH_STACKWALK_MAX_FRAMES`
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
This will update the posix thread names to match
the zephyr thread names.
This will simplify debugging as the debugger will
recognize the thread names.
Signed-off-by: Rubin Gerritsen <rubin.gerritsen@nordicsemi.no>
Newer Xtensa toolchain needs to include xtensa-types.h so that
macros in tie.h can be used without compilation errors.
The exact verison needing this is unknown but first encountered
in RJ-2023.2. Tested with older toolchain and that did not
cause any compilation errors so just include xtensa-types.h
if xt-clang is used. Haven't seen newer toolchains being
generated with xcc, so skip that for now.
Note that Zephyr SDK and the public HAL in Zephyr do not provide
this file.
Signed-off-by: Anthony Giardina <anthony.giardina@intel.com>
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The r0 register holds the system_off function pointer. As r0 is a scratch
register, the pointer needs to moved to a preserved register before
branching to a (custom) marker function.
Furthermore, in accordance to rule 6.2.1.2 of aapcs32, the stack pointer
needs to align on 8 bytes. Hence r0 is pushed to the stack in addition to
the lr register, before calling the public interface of checking the
s2ram marker.
Signed-off-by: Hessel van der Molen <hvandermolen@dexels.com>
Relocate stack unwind backends from `arch/` to perf's
`backends/` folder, just like logging/shell/..
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
This commit fixes potential unpredictable behavior, caused by using
the ^ form of ldmia instruction, while exiting an exception in SMP
mode on Cortex-A/R.
Change:
Use "pop" instead of "ldmia" to restore user mode registers while
exiting from an exception via `z_arm_cortex_ar_exit_exc`.
Reason for change:
Processor mode is always set to system (MODE_SYS) before calling
`z_arm_cortex_ar_exit_exc` and hence, the user mode register can be
accessed directly without the ^ form of the instruction. Also, LDMIA
instruction is UNPREDICTABLE in SYStem mode.
Signed-off-by: Sudan Landge <sudan.landge@arm.com>
This commit fixes the unpredictable behavior, caused by using the
^ form of stmdb instruction, while entering an exception in SMP mode
on Cortex-A/R.
Change:
Use "push" instead of "stmdb" to store user mode registers on
stack while entering an exception in SYStem mode.
Reason for change:
As reported in discussion/#75339, processor is already in SYS mode
after entering `z_arm_cortex_ar_enter_exc()` in an exception and
using stmdb is UNPREDICTABLE in system mode. Also, the user mode
register can be accessed directly without the ^ form of the
instruction. The solution suggested to fix this is to use
`stmdb sp!, {r0-r3, r12, lr}` which can save the user registers,
update the SP and avoid an extra instruction.
We use "push {}" instruction instead since it is the preferred
mnemonic over `stmdb`.
Signed-off-by: Sudan Landge <sudan.landge@arm.com>
Port of similar change in arm64 that eliminates exclusive load/store
instructions, which may not work when MMU/MPU/cache are disabled.
Based on: 7904c6f0f3
Signed-off-by: Stan Skowronek <stan@corellium.com>
xtensa_mmu_init() is called really early in the boot process
where the _kernel struct has not yet been initialized, and
thus we cannot use it to determine if the current CPU is
the boot CPU. In some cases, this may skip the call to
initialize the page tables which leaves us with incorrect
page table entries. Fix it by using a static variable to
determine whether the page tables have been initialized so
we only do it once per boot.
Fixes#76909
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Implement stack trace function for x86_32 arch, that get required
thread register values and unwind stack with it.
Signed-off-by: Mikhail Kushnerov <m.kushnerov@yadro.com>
Implement stack trace function for x86_64 arch, that get required
thread register values and unwind stack with it.
Originally-by: Yonatan Goldschmidt <yon.goldschmidt@gmail.com>
Signed-off-by: Mikhail Kushnerov <m.kushnerov@yadro.com>
Implement stack trace function for riscv arch, that get required
thread register values and unwind stack with it.
Signed-off-by: Mikhail Kushnerov <m.kushnerov@yadro.com>
is_dblexc is constant for targets without MMU. There is no need
to check it when building without MMU.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
When the native simulator use was introduced,
the POSIX architecture and SOC were split in 2 versions:
One was the old version, which remained used by native_posix and the
other NATIVE_APPLICATION based targets.
The new version was a shim on top of the native simulator threading
and CPU start/stop emulation.
This was done to ensure no regressions were introduced in the old
targets while the native simulator was tested and matured.
The old SOC code was removed a small while after, and all
NATIVE_APPLICATION targets moved to use the shim version on top of the
native simulator.
Now we remove also the old arch code, so native_posix and its
NATIVE_APPLICATION kin, also use the native simulator NCT component
instead.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
custom arch_cpu_idle and arch_cpu_atomic_idle implementation was done
differently on different architectures. riscv implemented those as weak
symbols, xtensa used a kconfig and all other architectures did not
really care, but this was a global kconfig that should apply to all
architectures.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Clear the UNALIGN_TRP bit in the CCR register, if the config
CONFIG_TRAP_UNALIGNED_ACCESS is not set.
Despite the fact that the reset value of UNALIGN_TRP is 0, always clear
the bit. It is useful in double image systems. The new image can't rely
on settings left by the previous image.
Signed-off-by: Dawid Niedzwiecki <dawidn@google.com>
arch_kernel_init() was misused for all architecture initialization code
that is done in prep_c and prior to cstart on other architectures.
arch_kernel_init() is late in the init process and comes after EARLY
init level, making xtensa have a very special boot path.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
xtensa is the only architecutre doing thing differently and introduces
inconsistency in the init process and dependencies as we attemp to
cleanup init levels and remove misused of SYS_INIT.
Introduce prep_c for this architecture and align with other
architectures.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Account for the scenario when we are doing `esf`-based
unwinding from a function which doesn't have any callee.
In this case the `ra` is not saved on the stack and the
second function from the top of the frame could be missing.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Add new config, ARCH_SUPPORTS_COREDUMP_THREADS, and
only enable it for ARM CORTEX M where the gdb server
can support it.
Signed-off-by: Mark Holden <mholden@meta.com>
According to the riscv's `arch.h`:
+------------+ <- thread.arch.priv_stack_start
| Guard | } Z_RISCV_STACK_GUARD_SIZE
+------------+
| Priv Stack | } CONFIG_PRIVILEGED_STACK_SIZE
+------------+ <- thread.arch.priv_stack_start +
CONFIG_PRIVILEGED_STACK_SIZE +
Z_RISCV_STACK_GUARD_SIZE
The start of the privilege stack should be:
`thread.arch.priv_stack_start + Z_RISCV_STACK_GUARD_SIZE`
Instead of
`thread.arch.priv_stack_start - CONFIG_PRIVILEGED_STACK_SIZE`
For the `end`, use the same equation of `top_of_priv_stack` in
the `arch_user_mode_enter()`
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
When RISCV_ALWAYS_SWITCH_THROUGH_ECALL is enabled, do_swap() enables PMP
checking in is_kernel_syscall.
If the PMP stack guard is triggered and do_swap() is called from the
fault handler, a PMP error occurs because the stack usage violates the
previous PMP setting.
Remove the stack guard setting during a stack overflow handler to allow
enabling PMP checking safely in fault handler.
Signed-off-by: Jimmy Zheng <jimmyzhe@andestech.com>
When RISCV_ALWAYS_SWITCH_THROUGH_ECALL is enabled, do_swap() enables PMP
checking in is_kernel_syscall.
If a user thread violates memory protection and do_swap() is called from
the fault handler, a PMP error occurs because the thread is in privileged
mode but still using the old user mode PMP setting.
Update the PMP setting to privileged mode for fault handler.
This also enables the stack guard for user thread's privileged stack in
fault handler.
Signed-off-by: Jimmy Zheng <jimmyzhe@andestech.com>
When 'arch_switch()' switches though Ecall, 'exception_depth' is
incorrectly added to the next thread because the current thread is updated
before arch_switch().
Add 'exception_depth' back to the previous thread when Ecall is called from
'arch_switch()'.
Signed-off-by: Jimmy Zheng <jimmyzhe@andestech.com>
Before this, stack protection would be effective only after switching to
the first thread.
Even before the first thread is created, the kernel init code uses the
IRQ stack to set things up. Let's make sure this is safeguarded as well.
This also fixes the incompatibility between CONFIG_RISCV_PMP and
CONFIG_RISCV_ALWAYS_SWITCH_THROUGH_ECALL, the later needing an exception
call to switch to the first thread and exception code assuming stack
guard is already set up in the PMP.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Calls to other function may clobber ip & lr too so these register need to
be added to the clobberlist.
r3 is not actually used in z_arm_switch_to_main_no_multithreading so it is
also removed from the clobber list.
Signed-off-by: Wilfried Chauveau <wilfried.chauveau@arm.com>
Sets the property `PROPERTY_OUTPUT_FORMAT` to `elf32-bigarm` when
`CONFIG_BIG_ENDIAN` is set to `y`.
Signed-off-by: Sigmund Klåpbakken <sigmundklaa@outlook.com>
When this bit is not set, it defaults to 0 (little endian). This
causes issues for big-endian devices, as data will be accessed using
little endian.
Signed-off-by: Sigmund Klåpbakken <sigmundklaa@outlook.com>
Update the description of the `INCLUDE_RESET_VECTOR` Kconfig so
that it is more clear to the user what it does.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
ZSR_DEPC_SAVE is being used to determine whether we are faulting
inside double exception if this is not zero. It is possible that
the boot ROM or custom startup code leaves this non-zero, which
would result in a fake triple fault. So clear it at boot. Note
that the zeroing is done in MMU init code as these triple
faults are not actual hardware ones but only semantics, and will
occur once MMU is enabled.
Fixes#75194
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Add initial support for the Cortex-M85 Core which is an implementation
of the Armv8.1-M mainline architecture.
The support is based on the Cortex-M55 support that already exists in
Zephyr.
Signed-off-by: Duy Nguyen <duy.nguyen.xa@renesas.com>
Commit f7e11649fd ("arch/arm64/mmu: fix page table reference
counting") missed another case where the freeing of a whole table
"branch" didn't take into account the fact that some sub-tables might
be shared and therefore must be cleared only if the reference count is
down to 1.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
None of the moved definitions are meant to be used by any code outside
of arch/arm64/core/mmu.c. Move them away from global scope to the
private header where more such definitions already live.
This is especially relevant as the previous commit fixed some of those
definitions which then caused conflicts with some external SDK that
carries a copy of those original buggy definitions.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Inverting a mask whose type has only 32 bits doesn't produce the
expected result. Fix those to be 64-bit values.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Zephyr's thread stack size is not fixed, in most cases we would
need the `thread` argument to obtain the `stack_info`, unless
we are unwinding the irq stack, since that is fixed.
Otherwise we can only safely print the current `mepc` register,
unwinding the esf without the stack info of a thread can
result in undefined behavior.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Pass the current thread to `walk_stackframe()`, so that we do
not need to hardcode `_current` in `in_fatal_stack_bound()`,
which will allow it to reuse the `in_stack_bound()`
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
The generic stack pointer checker in the architecture code is
enough so we can remove the platform specific one. Besides,
xtensa_dc233c_stack_ptr_is_sane() does not do much checking
either.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This checks for stack boundaries during backtrace to make sure
we are not stepping into invalid memory.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Check that the stack frame pointer is valid before dumping
any registers while handling exceptions. If the pointer is
invalid, anything it points to will probably be also be
invalid. Accessing them may result in another access
violation.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
It is observed that during logging in fatal exception,
it prints "FATAL EXCEPTION(null)". Exact reason is unknown
as debugging through GDB would make it all work again. So
separating it into two log statements to avoid this situation.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When CONFIG_X86_EXCEPTION_STACK_TRACE is enabled, also forcibly
enable CONFIG_THREAD_STACK_INFO. Without the thread stack info,
it is possible the stack unwinding would go out of the thread
stack and into unknown memory, resulting in hard fault.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
There actually is no triple faults on Xtensa. Once PS.EXCM is
set, it keeps going through double exception vector for any
new exceptions. However, our exception code needs to unmask
PS.EXCM to enable register window operations. So after that,
any new exceptions will go through the kernel or user vectors
depending on PS.UM. If there is continuous faults, it may
keep ping-ponging between double and kernel/user exception
vectors that may never get resolved. Since we stash DEPC
during double exception, and the stashed one is only cleared
once the double exception has been processed, we can use
the stashed DEPC value to detect if the next exception could
be considered a triple fault. If such a case exists, simply
jump to an infinite loop, or quit the simulator, or invoke
debugger.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
arch_user_string_nlen() did not exactly work correctly as any
invalid pointers being passed are de-referenced naively, which
results in DTLB misses (MMU) or access errors (MPU). However,
arch_user_string_nlen() should always return to the caller
with appropriate error code set, and should never result in
thread termination. Since we are usually going through syscalls
when arch_user_string_nlen() is called, for MMU, the DTLB miss
goes through double exception. Since the pointer is invalid,
there is a high chance there is not even a L2 page table
associated with that bad address. So the DTLB miss cannot be
handled and it just keeps looping in double exception until
there is another exception type where we get to the C handler.
However, the stack frame is no longer the frame associated
with the call to arch_user_string_nlen(), and the call return
address would be incorrect. Forcing this incorrect address as
the next PC would result in some other exceptions, e.g.
illegal instruction, which would go to the C handler again.
This time it will go to the end of handler and would result
in thread termination. For MPU systems, access errors would
simply result in thread terminal in the C handler. Because of
these reasons, change the arch_user_string_nlen() to check if
the memory region can be accessed under kernel mode first
before feeding it to strnlen().
Also remove the exception fixup arrays as there is nothing
there anymore.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds a new function xtensa_mem_kernel_has_access() to
determine if a memory region can be accessed by kernel threads.
This allows checking for valid mapped memory before accessing
them to avoid relying on page faults to detect invalid access.
Also fixed an issue with arch_buffer_validate() on MPU where
it may return okay even if the incoming memory region has no
corresponding entry in the MPU table.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
arch_buffer_validate() is only to verify that user threads have
access to the memory region. It should not be used to verify
if kernel thread has access (which they should anyway). So
change the logic.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Introduce CONFIG_XTENSA_BREAK_ON_UNRECOVERABLE_EXCEPTIONS to
use BREAK instruction for unrecoverable exceptions. This
definitely requires debugger to be attached to the hardware
or simulator to catch that.
Also move the infinite loop to NOT result in an infinite
interrupt storm as the debug interrupt will be triggered over
and over again. Same for the simcall exit as it does not
need to be called repetitively.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
If there are any TLB misses during interrupt handling,
the user, kernel and double exception vector will be triggered
for the miss and the DEPC and EXCCAUSE overwritten as the TLB
missse are be handled in the assembly code and execution
returned to the original vector code. Because of this, both
DEPC and EXCCAUSE being read in the C handler are not the ones
that triggered the original exception (for example, level-1
interrupt). So stash both DEPC and EXCCAUSE such that
the original cause of exception is visible in the C handler.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Xtensa MPU code does not handle double exception in C. So there
is no need to clear DEPC on C handler exit.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
We have frame pointer struct and BSA struct to extract
the exception cause (exccause). There is no need to
resort to custom assembly to do that. Besides, given
that the BSA is different between different Xtensa cores,
there is no guarantee it is at the same place as what
the assembly assumes. So just do that without assembly.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Kconfig, .ld and comments fixing
Fixed address of UART1, WDT and RTC timer disabled by default
Signed-off-by: Raffael Rostagno <raffael.rostagno@espressif.com>
Assembly implementation for z_early_memset() and z_early_memcpy().
Otherwise the compiler will happily replace our C code with a direct
call to memset/memcpy which kind of defeats the purpose.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
We need those simple alternatives to be used during early boot when the
MMU is not yet enabled. However they don't have to be the slowest they
can be. Those functions are mainly used to clear .bss sections and copy
.data to final destination when doing XIP, etc. Therefore it is very
likely for provided pointers to be 64-bit aligned. Let's optimize for
that case.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
As we start to use data memory barriers in SMP scheduler code
explicitly (not only internaly in the atomics implementation)
let's enable barriers for ARC HS.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Check if an address is in the thread stack only when
`CONFIG_THREAD_STACK_INFO` is enabled, since otherwise the
`stack_info` will not be available.
This fixes compilation error when `CONFIG_THREAD_STACK_INFO`
is explicitly disabled.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Many boards/SoCs in-tree do this:
if !XIP
config FLASH_SIZE
default 0
config FLASH_BASE_ADDRESS
default 0
endif
And many other boards are missing this configuration (e.g. stm32 series).
Making this the default helps get non-XIP just working
Signed-off-by: Grant Ramsay <gramsay@enphaseenergy.com>
Commit f7e11649fd ("arch/arm64/mmu: fix page table reference
counting") missed a case where the freeing of a table wasn't propagated
properly to all domains. To fix this, the page freeing logic was removed
from set_mapping() and a del_mapping() was created instead, to be usedby
both by remove_map() and globalize_table().
A test covering this case should definitely be created but that'll come
later. Proper operation was verified through manual debug log
inspection for now.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
`isr.S` depends on `CONFIG_GEN_SW_ISR_TABLE`.
Do not build it if SW ISR table is not present.
Signed-off-by: Marcin Szymczyk <marcin.szymczyk@nordicsemi.no>
Now that the unwind starts from mepc already, the symbol
name at the mepc reg is kinda redundant, so just remove it.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Created the `arch_stack_walk()` function out from the original
`z_riscv_unwind_stack()`, it's been updated to support
unwinding any thread.
Updated the stack_unwind test case accordingly.
Increased the delay in `test_fatal_on_smp`, to wait
for the the fatal thread to be terminated, as stacktrace can
take a bit more time.
Doubled the kernel/smp testcase timeout from 60 (default) to
120s, as some of the tests can take a little bit more than 60s
to finish.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
An architecture can indicate that it has an implementation for
the `arch_stack_walk()` function by selecting
`ARCH_HAS_STACKWALK`.
Set the default value of `EXCEPTION_STACK_TRACE_MAX_FRAMES` to
`ARCH_STACKWALK_MAX_FRAMES` if the latter is available.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
This code was never formally tested before... and without the preceding
commit it obviously didn't work either.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Existing code confused table usage and table reference counts together.
This obviously doesn't work. A table with one reference to it and one
populated PTE is not the same as a table with 2 references to it and
no PTe in use.
So split the two concepts and adjust the code accordingly. A page needs
to have its PTE usage count drop to zero before the last reference is
released. When both counts are 0 then the page is free.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This is part of a series of move memory management related
stuff out of Z_ namespace into its own namespace.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Also any demand paging and page frame related bits are
renamed.
This is part of a series to move memory management related
stuff out of the Z_ namespace into its own namespace.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This is part of a series to move memory management related
stuff from Z_ namespace into its own namespace.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This is part of a series to move memory management functions
away from the z_ namespace and into its own namespace.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This is part of a series to move memory management functions
away from the z_ namespace and into its own namespace. Also
make documentation available via doxygen.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This is part of a series to move memory management functions
away from the z_ namespace and into its own namespace. Also
make documentation available via doxygen.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This renames z_phys_map() and z_phys_unmap() to
k_mem_map_phys_bare() and k_mem_unmap_phys_bare()
respectively. This is part of the series to move memory
management functions away from the z_ namespace.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
isr_wrapper has been converted to C but this inline comment was not
updated. This fixes the out-of-sync comment.
Signed-off-by: Wilfried Chauveau <wilfried.chauveau@arm.com>
Move this to a call in the init process. arch_* calls are no services
and should be called consistently during initialization.
Place it between PRE_KERNEL_1 and PRE_KERNEL_2 as some drivers
initialized in PRE_KERNEL_2 might depend on SMP being setup.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add a prompt to ROMSTART_RELOCATION_ROM configs to allow
projects config file selection.
This simplifies the usage in samples/app for board porting.
Signed-off-by: Jérémy LOCHE - MAKEEN Energy <jlh@makeenenergy.com>
Removed an unnecessary cast to void * from a function that already
had the correct signature.
This makes for more portable code as casting between code and data
pointers are frowned upon by the C standard.
Signed-off-by: Lars-Ove Karlsson <lars-ove.karlsson@iar.com>
Memory region defined in devicetree can have attributes
that are not intended to be parsed by MPU library,
but might be valid for other components.
Signed-off-by: Nikodem Kastelik <nikodem.kastelik@nordicsemi.no>
Platforms that support IPIs allow them to be broadcast via the
new arch_sched_broadcast_ipi() routine (replacing arch_sched_ipi()).
Those that also allow IPIs to be directed to specific CPUs may
use arch_sched_directed_ipi() to do so.
As the kernel has the capability to track which CPUs may need an IPI
(see CONFIG_IPI_OPTIMIZE), this commit updates the signalling of
tracked IPIs to use the directed version if supported; otherwise
they continue to use the broadcast version.
Platforms that allow directed IPIs may see a significant reduction
in the number of IPI related ISRs when CONFIG_IPI_OPTIMIZE is
enabled and the number of CPUs increases. These platforms can be
identified by the Kconfig option CONFIG_ARCH_HAS_DIRECTED_IPIS.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
With power managment is enabled, depending on the SoC power state
used when idle, the MMU may lose context and may need to be re-initialized.
When re-initializing the MMU, we must not re-create the page table
because it may overwrite changes done during the execution, but we still
need to set the asid and page table for the current context.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Created `GEN_OFFSET_STRUCT` & `GEN_NAMED_OFFSET_STRUCT` that
works for `struct`, and remove the use of `z_arch_esf_t`
completely.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
