Efficient access to local storage for BPF programs

When a BPF program is used to filter or redirect packets in the networking subsystem, the program will often want to associate data with each packet as it moves through the kernel. The kernel's local BPF storage API, which associates extra data with some kernel objects, provides a way to do that. (See also the BPF map types that end in STORAGE.) Amery Hung and Jakub Sitnicki led two sessions at the 2026 Linux Storage, Filesystem, Memory-Management, and BPF Summit about how to make accesses to local storage data more efficient. Hung spoke about general performance problems related to locking, while Sitnicki examined the use of local storage in the networking subsystem in particular.

There is a feature that Hung has been working on for the past eight months, using local storage to enable faster communication between the kernel and user space. This talk was not about that feature, because in the course of working on it, Hung noticed that the BPF self tests sometimes failed due to contention on the lock that protects local storage from deadlocks. Such contention can cause reading from or writing to local storage to fail, which is an obvious source of problems.

Local storage is protected by two locks: a normal lock used to protect against concurrent accesses, and a per-CPU lock that exists to prevent deadlock. The latter lock is needed because a BPF program could potentially be attached to tracepoints that are triggered when the local storage is accessed. Without a per-CPU lock to make sure that the same CPU doesn't try to acquire the main lock twice, it would be possible for a BPF program attached to one of those tracepoints to attempt to access the local storage itself, and deadlock the kernel. In order to prevent that, the kfuncs that BPF uses to access local storage were made fallible, and the per-CPU lock ensures they return an error if two BPF programs try to access the local storage at the same time.

That is more conservative than necessary, however, because it means that two unrelated BPF programs could block each other's accesses even though there is no possibility of deadlock. To solve this problem, Hung would like to replace the main lock with a resilient queued spinlock (rqspinlock), a kind of lock that Kumar Kartikeya Dwivedi and Alexei Starovoitov developed to report deadlocks at run time. Since the main lock would prevent deadlocks, the per-CPU lock could be eliminated. With care, this approach could be used to fail the BPF operation only when there is an actual risk of a deadlock occurring.

But it is not as simple as just changing the type of lock used, Hung said. That is sufficient for most code paths, but there are some infallible operations that need to take the same lock, and rqspinlock can't handle that. For example, freeing a kernel data structure that has local storage should not fail just because the lock is contended. It's hard to make things fail gracefully.

Hung tried replacing some of the relevant allocations and deallocations with kmalloc_nolock(), but that did not work well either. The problem is that there are two allocators that can be used for local storage: the BPF allocator and the slab allocator. Using one or the other alone isn't acceptable for performance reasons; the BPF developers went through a lot of pain to decouple things so that either can be used.

At the end of the day, Hung gave up on solving the problem gracefully, and introduced an infallible freeing operation for local storage that leaks 176 bytes in the worst case. With a mostly working approach, the next step was to benchmark the change. A microbenchmark that created local storage allocations and then freed them in a loop saw a 1.5% increase in throughput after removing the per-CPU lock on x86_64. Arm systems didn't have a measurable performance difference. Microbenchmarks don't tell the whole story, however, since real-world performance is sensitive to the conditions of the rest of the system.

The change also unlocks some other potential performance improvements, including switching some uses of the slab allocator to be able to use sheaves, which gives a 5.1% speedup on another microbenchmark. One audience member asked whether Hung had any idea why there would be such a big performance difference between the two allocators. Hung wasn't sure — cache usage on x86 increased, that just didn't cause a performance regression.

The next steps are somewhat unclear; there remains some additional complexity around managing the lifetime of local storage objects, Hung said. Some kinds of kernel objects don't always fully initialize their local storage, which can result in kernel panics. Starovoitov had previously suggested using kernel object pointers as keys in a more general-purpose map, as an alternative to local storage, which was a potentially interesting solution.

Hung developed another benchmark comparing this approach (using hash tables 50,000 buckets in size) to the existing local storage implementation, and saw a speedup, but the BPF developers in attendance were skeptical. One audience member remarked that in their experience, smaller hash tables are faster and bigger hash tables are slower; a proper benchmark should test multiple sizes. Hung pointed out that rhashtable uses a dynamic number of buckets, and wondered how that would effect things.

Sitnicki said that on the systems he works with, local storage is used with around one million concurrent connections, a number that was met with some surprise. Hung agreed that the benchmarks probably needed to be extended. Another part of that is that different metrics matter for different kinds of kernel objects; local storage for sk_buff structures should be optimized for creation speed, etc., Hung said.

In the future, socket-local storage can also be optimized further by removing the need for dynamic allocations at all. The verifier may be able to do "helper fixups" to make it even faster, a claim on which Hung did not elaborate. At that point, the session devolved into an discussion of how, exactly, local storage performance ought to be benchmarked for different uses.

Metadata storage for packets

The next day, Sitnicki led a follow-up session dealing with storing metadata in sk_buff structures (SKBs) — a topic that had been discussed in 2025 as well. It is "not local storage, exactly, but the topic should feel familiar." BPF programs often need to store labels, timestamps, or other small amounts of data on packets, he said. His first and second attempts at enabling this were to expose the existing storage for express data path (XDP) and SKB metadata to BPF programs; that wasn't workable because that metadata can only be accessed from XDP and traffic control (TC) contexts, and not from the upper layers of the networking stack.

Letting that metadata persist would require the networking subsystem to commit to a layout for it, which the networking maintainers do not wish to do. John Fastabend questioned why the networking folks would be so adamant about that. Was the problem that they wanted the metadata to be the same across different hardware, he asked. Sitnicki didn't think that was the problem — one can allocate space for metadata from XDP and later consume it in TC, regardless of hardware.

Sitnicki has considered a few alternatives; for example, an SKB extension could be used to hold a pointer to local storage. That would cause three allocations per packet, though, which is not really acceptable. Another alternative would be to have an SKB extension specifically for BPF metadata. The extension could even be located in the same allocation as the SKB. This would mean one allocation per packet, and fewer indirections to access the data. On the other hand, it requires buy-in from the networking maintainers, results in bigger SKBs, and that memory cost must be paid for even if it isn't used. Sitnicki intended to run that option by the networking maintainers at the Netdev conference in July.

Starovoitov suggested having a boot flag to set the size of the metadata; that way people who don't enable the boot flag don't see any memory overhead, and the people who do enable the boot flag presumably have some idea what they're doing. Andrii Nakryiko asked why Sitnicki wanted to locate the metadata next to the SKB in the first place, especially since accessing it would still cause a cache miss in practice. He suggested adding a separate hashmap for metadata and using that. Sitnicki agreed that that was a second possible alternative.

Daniel Borkmann asked Sitnicki to elaborate on the difference between making the existing SKB metadata accessible at higher levels of the networking stack, and adding BPF-specific metadata in an SKB extension. The two approaches would effectively both put metadata alongside the SKB. Sitnicki agreed that was the case, and that he would be fine with either alternative. The difference is that the approach using an SKB extension would have a pointer indirection, even if it's just one allocation. That's more flexible, because for packets that don't need metadata, it could be left out, which would save memory.

Using an SKB extension does introduce some new challenges, however. For one thing, it affects how SKBs should be cloned, since the extension needs to be copied as well — which is not how SKB extensions work currently. The existing SKB extensions are only copied when activating a new extension, Sitnicki said. Also, some SKB extensions are hard to compile out if they are not needed; IPSec support always requires having space for an SKB extension used by IPSec hardware offload, for example.

Starovoitov wasn't sure why all of the SKB extensions needed to live in a single allocation alongside the SKB. Sitnicki explained that that is just how SKB extensions are implemented. Any approach that tries to add a separate mechanism just for BPF is going to be refused by the networking maintainers, because they have been trying to make SKB extensions "not suck" for a long time. They want any improvements in this area to work with the existing code.

There is another problem to consider, though. Even if some memory is set aside for BPF metadata, how can the memory be divvied up between BPF programs that all want to store things there? One option would be to just configure each BPF program that an administrator intends to run to have its own range. Another option would be to allocate space in the metadata dynamically, so that BPF programs can declare how much metadata they need and then the kernel can transparently provide access to it. The second one is a nicer user experience, but more complicated to implement. There was some discussion about the pros and cons of each approach that failed to reach a consensus.

An alternative, Sitnicki said, would be to use a pointer to the SKB as a key in an existing BPF hashmap. That sounds like a good solution, but picking the correct size for the hashmap is a tricky problem. Maybe rhashtables, which are resizeable, are the right answer, but that needs to be tested under real-world load. Using a hashmap also moves the responsibility for cleaning up the metadata when the SKB is freed onto the BPF program. Hooking into a tracepoint to detect when an SKB is freed does have some overhead, which needs to be considered as part of the cost of the approach.

Starovoitov questioned why the BPF program that acts as the final consumer for the metadata couldn't do the cleanup, avoiding the overhead of a tracepoint. SKBs can disappear at many different points in the networking stack, Sitnicki explained. If a packet with metadata is dropped, that metadata needs to be cleaned up somehow. Starovoitov suggested putting a single bit in the SKB to indicate whether it has metadata, and then call the tracepoint conditionally, so that there is only a performance overhead for packets that have metadata and that are dropped before they can be cleaned up.

Nakryiko and Starovoitov encouraged Sitnicki to nail down the specific proportion of packets that need metadata, the size of the metadata, and the overall number of packets needed for his use case before deciding on a design. All of these things can affect what the best solution will look like, and it's better to be specific rather than attempting to design a general mechanism that may not even gain any other users. At that point, the discussion broke down into speculation about the impacts of different designs, and the session ran out of time.