We briefly discussed the Kernel Address Sanitizer (KASAN) in a previous blog post, so today we’ll be diving into the guts of how it can be used.
KASAN is dynamic memory safety error detector to find out-of-bound and use-after-free bugs. There are currently three modes for KASAN. Arranged in decreasing order of overheads they are:
- Generic KASAN, large memory overheads, mainly used for debugging.
- Software tag-based KASAN, lower memory overheads, making it viable for real workloads and dogfood testing (using your own software in the real world).
- Hardware tag-based KASAN, low overheads making it possible to be used in production, as well as a memory bug detector or as a security mitigation.
Config options
KASAN is enabled with the CONFIG_KASAN config option when building the kernel. Then, to choose between the three modes, either set CONFIG_KASAN_GENERIC (generic KASAN), CONFIG_KASAN_SW_TAGS (software tag-based KASAN), or CONFIG_KASAN_HW_TAGS (hardware tag-based KASAN).
For software modes, there are also the CONFIG_KASAN_OUTLINE and CONFIG_KASAN_INLINE options. Outline means that memory checks are made by callbacks, whereas inline means that memory checks are made inlined. The difference is that outlining produces a smaller binary, at the cost of run-time overheads from calling functions, whereas inline produces a large binary, but runs 1.1-2x faster.
Additionally, if you want KASAN to report when affected slab objects were allocated or freed, the CONFIG_STACKTRACE option should also be enabled. To do the same with affected physical pages, the CONFIG_PAGE_OWNER option should be enabled, and the kernel should be booted with page_owner=on.
Additional boot parameters
The behavior of KASAN can be customized by various command line parameters. panic_on_warn causes KASAN to panic the kernel on encountering a bug. kasan_multi_shot allows KASAN to print a report for every invalid access it encounters, instead of only the first (note that for this to work, it disables panic_on_warn for KASAN reports).
Since hardware tag-based KASAN is used in production, it can be configured to disable various features as well. More details here.
Additional details on interpreting bug reports
On top of our previous look at KASAN bug reports, there’s another aspect of the report that I think deserves a closer look: the memory state around the buggy address.
Memory state around the buggy address:
ffff8801f44ec200: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb
ffff8801f44ec280: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc
>ffff8801f44ec300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03
^
ffff8801f44ec380: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb
ffff8801f44ec400: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc
From this memory state, we’re able to see that the the bad address only had 3 valid bits. This means we likely encountered an out-of-bounds error.
Another thing to note is that KASAN bug titles are best-effort, meaning that KASAN makes its best guess on the bug type but it could be wrong.
Generic KASAN was discussed in our earlier blog post, so we’ll take a look at software and hardware tag-based KASAN. Note that the quarantine for freed objects that we discussed previously only applies to generic KASAN.
Memory tagging
Memory tagging (aka memory coloring) is a mechanism to track illegal memory operations such as buffer overflows, and use-after-free errors.
Here’s the basic algorithm. Whenever memory is allocated, a tag is chosen and associated with the memory chunk. This tag is returned with the pointer to the memory chunk in the upper bits of the pointer. Every future load and store instruction then checks the pointer for the associated memory tag, and raises an exception if there’s a mismatch.
In the 64-bit ARM architecture, registers are 64-bits wide, but the top 16 bits must either be 0x0000 or 0xFFFF. This allows the ARMv8 instruction set to implement a Top-Byte Ignore (TBI) feature that can be used for memory tagging: by ignoring the top 8 bits of virtual addresses, pointers can now encapsulate memory tags.
Software tag-based KASAN
Now that we understand memory tagging, we can understand tag-based KASAN. Currently only implemented for the arm64 architecture, software tag-based KASAN applies software memory tagging to check access validity of pointers.
When memory is allocated, a random memory tag is generated and stored both in shadow memory and in the top byte of the returned pointer. Following which, as with generic KASAN, the compiler inserts checks before each memory access to ensure that the two memory tags match. There are also some special tags: 0xFF tags means that the pointer shouldn’t be matched; and 0xFE is reserved to tag freed memory regions.
Hardware tag-based KASAN
Hardware tag-based KASAN is similar to the software mode, but instead of relying on compiler instrumentation and shadow memory, hardware memory tagging support is used.
Along with TBI, hardware tag-based KASAN also relies on the Memory Tagging Extension (MTE) which allows the storage of a tag in just bits 59-56 of a virtual address, i.e. the lower nibble of the top byte.
There’s much more to KASAN that we left out in this post, for example, how to selectively disable KASAN (such as in software for memory allocators), or how to test that KASAN is working. For the brave souls interested in even more details about KASAN, the official Linux Kernel Documentation is a great resource.
References
The Linux Kernel Documentation on KASAN
Pingback: Vignettes of a Linux Kernel Mentee – Desmond Cheong