In order to be functional and avoid linker errors, use a
default setting for BACKING_STORE_CHOICE that is something other
than "this is not implemented".
Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
With assert condition set to true, assertion never happens.
Change __ASSERT(true) to __ASSERT(false).
Signed-off-by: Dmitry Lukyantsev <dmitrylu@meta.com>
This adds a new kconfig for eviction algorithm which needs page
tracking. When enabled, k_mem_paging_eviction_add()/_remove()
and k_mem_paging_eviction_accessed() must be implemented.
If an algorithm does not do page tracking, there is no need to
implement these functions, and no need for the kernel MMU code
to call into empty functions. This should save a few function
calls and some CPU cycles.
Note that arm64 unconditionally calls those functions so
forces CONFIG_EVICTION_TRACKING to be enabled there.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Obviously, everyone knows that there are 8 bits per byte, so
there isn't a lot of magic happening, per se, but it's also
helpful to clearly denote where the magic number 8 is referring
to the number of bits in a byte.
Occasionally, 8 will refer to a field size or offset in a
structure, MMR, or word. Occasionally, the number 8 will refer
to the number of bytes in a 64-bit value (which should probably
be replaced with `sizeof(uint64_t)`).
For converting bits to bytes, or vice-versa, let's use
`BITS_PER_BYTE` for clarity (or other appropriate `BITS_PER_*`
macros).
Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
This provides memory mappings with the ability to be initialized in their
paged-out state and be paged in on demand. This is especially nice for
anonymous memory mappings as they no longer have to allocate all memory
at mem_map time. This also allows for file mappings to be implemented by
simply providing backing store location tokens.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
If only 2 page frames are queued and code executing in one frame is
making an access to memory in the second frame then the access will trap
and k_mem_paging_eviction_accessed() will be called to move that frame
to the end of the queue ... marking the new head frame unaccessible.
But that newly unaccessible frame contains the code that has yet to be
resumed to perform its memory access. Since it is now unaccessible, a
trap is triggered, the frame is moved to the end of the queue and the
new head frame (the one we trapped for initially) is marked unaccessible.
Execution is resumed with the memory access which is unaccessible again
and the cycle repeats infinitely.
Fix this by not marking the new head unaccessible if there is only one
queued frame left in the queue.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The first page frame index was reserved for head and tail ^pointers.
However there are cases where the first frame is actually made
evictable and would trigger the assertion guarding against that.
Fix this by applying an offset to actual frame indexes.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This is a O(1) LRU eviction algorithm. A bit more complex but way more
scalable than the NRU algorithm.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Let eviction algorithms be notified when a given page frame:
- should be considered as possible candidate
- should no longer be considered as candidate
- has just been marked as "accessed"
The NRU algorithm is unchanged so it implements those as empty stubs.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Currently, the NRU algorithm always picks the lowest (and very often the
same) clean unaccessed page over and over when e.g. doing large anonymous
memory mappings. Spread the eviction selection more uniformly by by
starting the search right after the last victim instead of always
restarting from 0.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
First, do align the buffer. The slab code puts pointers in there and it
does not like it if those are not properly aligned.
And return the actual error code from k_mem_slab_alloc() even if errors
shouldn't happen (it did happen to me because of the above ... with
assertions disabled).
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Also any demand paging and page frame related bits are
renamed.
This is part of a series to move memory management related
stuff out of the Z_ namespace into its own namespace.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This is part of a series to move memory management related
stuff from Z_ namespace into its own namespace.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Introduce z_page_frame_set() and z_page_frame_clear() to manipulate
flags. Obtain the virtual address using the existing
z_page_frame_to_virt(). This will make changes to the page frame
structure easier.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This moves including of demand_paging.h out of kernel/mm.h,
so that users of demand paging APIs must include the header
explicitly. Since the main user is kernel itself, we can be
more discipline about header inclusion.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Modify the signature of the k_mem_slab_free() function with a new one,
replacing the old void **mem with void *mem as a parameter.
The following function:
void k_mem_slab_free(struct k_mem_slab *slab, void **mem);
has the wrong signature. mem is only used as a regular pointer, so there
is no need to use a double-pointer. The correct signature should be:
void k_mem_slab_free(struct k_mem_slab *slab, void *mem);
The issue with the current signature, although functional, is that it is
extremely confusing. I myself, a veteran Zephyr developer, was confused
by this parameter when looking at it recently.
All in-tree uses of the function have been adapted.
Fixes#61888.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
irq_lock() returns an unsigned integer key.
Generated by spatch using semantic patch
scripts/coccinelle/irq_lock.cocci
Signed-off-by: Johann Fischer <johann.fischer@nordicsemi.no>
In order to bring consistency in-tree, migrate all subsystems code to
the new prefix <zephyr/...>. Note that the conversion has been scripted,
refer to zephyrproject-rtos#45388 for more details.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
In k_mem_paging_eviction_select(), the returned dirty bit value
may not be actually associated with the page selected, but
rather the last page examined. So fix this.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds a flash-based backing store for qemu_x86_tiny board for
testing demand paging. This allows us to test code execution where
.text section is not in physical memory at boot.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
These functions are those that need be implemented by backing
store outside kernel. Promote them from z_* so these can be
included in documentation.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
These functions and data structures are those that need
to be implemented by eviction algorithm and application
outside kernel. Promote them from z_* so these can be
included in documentation.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
If we evict enough pages to completely fill the backing store,
through APIs like k_mem_map(), z_page_frame_evict(), or
z_mem_page_out(), this will produce a crash the next time we
try to handle a page fault.
The backing store now always reserves a free storage location
for actual page faults.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Backing stores and eviction algorithms will be included here.
Exactly one must be chosen, with a default option to leave
the implementation to the application.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
If we evict enough pages to completely fill the backing store,
through APIs like k_mem_map(), z_page_frame_evict(), or
z_mem_page_out(), this will produce a crash the next time we
try to handle a page fault.
The backing store now always reserves a free storage location
for actual page faults.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Backing stores and eviction algorithms will be included here.
Exactly one must be chosen, with a default option to leave
the implementation to the application.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
