LWN.net Weekly Edition for June 4, 2026

Welcome to the LWN.net Weekly Edition for June 4, 2026

This edition contains the following feature content:

• A trademark dispute over MeshCore: an individual tries to register the trademark for MeshCore over objections from project contributors.
• Reconsidering x32 — again: another discussion the unloved but hard-to-remove x32 ABI.
• Open-source security is not a solo activity: improving the health of important open-source projects.
• Trying to make sense of package-manager metadata: a maze of messy metadata expressions, none alike.
• More coverage from the 2026 Linux Storage, Filesystem, Memory Management, and BPF Summit:
  • Policies for merging new filesystems: establishing criteria and policies for new kernel filesystems.
  • Separating memory descriptors from struct page: planning out the next steps in the big memory-descriptor transition.
  • Representing the true signatures of kernel functions: progress on the problem of allowing functions with signatures changed by optimization to be correctly traced.
  • Caching for extended attributes: creating some common infrastructure for an extended-attribute cache, rather than just developing another for FUSE.
  • BPF in the agentic era: Alexei Starovoitov talks about LLMs and the future of BPF development.

• A loadable crypto module for FIPS certification: an effort to allow a certified crypto module to be reused with multiple kernels and avoid recertification delays.

This week's edition also includes these inner pages:

• Brief items: Brief news items from throughout the community.
• Announcements: Newsletters, conferences, security updates, patches, and more.

Please enjoy this week's edition, and, as always, thank you for supporting LWN.net.

Comments (none posted)

A trademark dispute over MeshCore

MeshCore is a relatively new project, started in January 2025, that aims to build a scalable mesh network using low-power long-distance radios. While many other projects of the same general nature have been tried before, MeshCore grew quickly because of its more efficient message routing and enthusiastic community. In early 2026, an early proponent of the project made a sudden shift that left the rest of the community stunned and embroiled in a trademark dispute.

MeshCore is a MIT-licensed portable C++ library that has been adapted to a range of long-range radio (LoRa) devices. The project also provides a web-based flasher to load pre-made firmware onto supported devices, Home Assistant integration, and bindings for other languages. Unlike Meshtastic (another LoRa mesh-networking project that LWN covered in 2025), it uses an actual distributed routing protocol, rather than relying on a gossip protocol. Unlike Reticulum, it aims to be simple and usable on low-power embedded devices. There are more than 40,000 users all across the world identified by the MeshCore node map, many of whom use the project for reliable radio communication while hiking, to collect data from remote sensors, or simply because they're interested in radio mesh networks.

Many of those enthusiasts found their way to the project via Andy Kirby's YouTube channel. He is (or was) a MeshCore enthusiast who produced engaging videos about the project, encouraging people to try it and explaining what it can do. He also ran the project's web site and a Discord server popular among users of the project, acting as a community organizer and promoter. (The project got a separate, official Discord server in the summer of 2025.) While not a developer, Kirby has certainly contributed to the rapid growth of the project.

In March, he released a proprietary companion app for MeshCore, called MeshOS, that helps with managing devices running MeshCore firmware from a smartphone. It quickly became clear that Kirby was using large language models (LLMs) to work on the app, without disclosing this: he released six new versions with new features in just 15 days. While the MeshCore community is strongly against incorporating LLM-written work (according to a poll the core contributors to the project performed on April 11), it was also upset about the lack of disclosure. MeshCore did accept one LLM-assisted pull request, which included a disclosure of that fact, in early March.

MeshCore's existing app (named, with great creativity, "MeshCore") is developed by Liam Cottle, who is a human. The application is free to use, although it does not appear to be open source. Other open-source software that can communicate with MeshCore devices over Bluetooth exists as well, and the project publishes documentation on how to connect to a device running its firmware. Kirby's application, which provides many of the same features as the official app, requires purchasing a license in order to operate it.

Around the same time, the MeshCore web site that Kirby maintained was updated to make it look as though MeshOS was required to use MeshCore, redirecting links from the official MeshCore application to refer to MeshOS. Options to donate to the MeshCore project or buy a pre-flashed device were removed from the web site's store page, leaving licenses (including ones for MeshOS) the only products one could buy there. Then, on March 29, a law firm working on his behalf filed a trademark application with the UK's Intellectual Property office asserting that Kirby owns the "MeshCore" trademark. That application is still pending at time of writing.

Kirby had not discussed these plans with the core developers ahead of time. The project's developers swiftly created a new web site with a different design to set itself apart from Kirby's web site, and shared a blog post explaining their side of the story. Shortly thereafter, Kirby's web site was updated with a similar design, despite requests from the core team members that he not do that. Today, the two web sites look similar, and it would be fairly easy for a user who was not in the know to become confused about which one is official — although the project's GitHub organization does link to the new web site.

The UK Trade Marks Act of 1994 (as amended) that governs the registration of trademarks in the UK does prohibit, among other things, trademarks likely to deceive the public. It's possible that, given Kirby's split with the rest of the MeshCore community, the UK government will choose to deny his application. But it's far from certain, and is sure to prove a tedious legal process either way. When I reached out to Kirby, he said:

The intention behind the application is to provide clarity and consistency around the platform and the supported ecosystem as it grows.

... which does not really address why he would file an application over the objections of the people contributing code to the project.

While unfortunate, this story is nothing new. There are always people trying to figure out ways to use open-source projects for personal gain. In this case it is not the open-source software being used, but rather the community that has built itself up around that software. The open-source community has had to deal with trademark disputes before, including blatant attempts to make a profit off of an existing community. This has led to complicated systems to protect trademarks, and public conflicts when those systems are challenged.

What may be a bit different in this case is the ease with which a single individual, not supported by a for-profit company, could make a professional-looking web site and companion app to compete with the community versions. It certainly didn't help that Kirby was previously the maintainer of the project's web site, but even had that not been the case he could have registered a similar domain. Previous trademark disputes over open-source software have largely involved direct forks of software, entire companies with legal teams, or attempts to cash in on a tangential association. In the future, we may see more attempts to challenge open-source projects' trademarks more directly, if only because that is now within the reach of individuals.

Comments (6 posted)

Reconsidering x32 — again

The x32 ABI was meant to be the best of both worlds, providing the expanded registers and instruction set of the x86-64 architecture while preserving the lower memory use of 32-bit systems. The Linux kernel has supported x32 since the 3.4 release in 2012. The initial excitement around x32 did not last, though, and kernel developers are considering removing that support — and not for the first time. Even the most unloved features tend to have a few users, though, making removal hard.

The 64-bit x86 CPU architecture brought a number of long-desired features, including more registers, better system-call support and, of course, the ability to support larger virtual address spaces. There is a cost to that last feature, though; the size of addresses (and, thus, pointers) doubled from four to eight bytes. That change inevitably increases the amount of memory used by a program and, importantly, the amount of cache required to hold the pointed-to values. Since cache utilization has a huge effect on the performance of many programs, that extra cache footprint hurts.

The idea behind x32 is to provide all of the x86-64 features, but to use 32-bit addresses and data types, bringing the memory and cache footprint back down. Supporting this ABI required work in the kernel, but also at the toolchain level; the result was expected to be the fastest x86 ABI available. Evidence for that speedup was scarce when x32 kernel support was under development in 2011 but, by 2013, some realistic benchmark results were being posted, and some distributors were beginning to work on support.

Despite its apparent advantages, though, x32 never really took off. One can speculate on the reasons why that happened. Perhaps processor speed and cache sizes increased enough to mitigate the cost of 64-bit pointers. Most applications are not so performance sensitive that it is worth the trouble to support another ABI. The applications that broke with 64-bit pointers were quickly fixed. Memory use has grown to the point that it is challenging to shoehorn even a basic "hello world" app into the 2-3GB of virtual address space that 32-bit pointers afford. Other possible explanations surely exist as well.

Also noteworthy is that many distributions never supported this ABI, making it unavailable to a lot of users. Having to build one's own kernel (and possibly toolchain as well) can be a strong impediment to experimenting with an alternative ABI. Debian and Gentoo did add x32 support; Ubuntu also supported x32, but dropped it in 2023, citing a lack of demand and concerns about the additional attack surface created by enabling that ABI (which has had vulnerabilities in the past). Modern Debian kernels retain x32 support, but it is only enabled when the syscall.x32=y boot option is provided.

In 2018, Andy Lutomirski proposed removing support for the x32 architecture, saying that it was largely unused and the necessary system-call interface added a lot of complexity to the kernel. An extensive discussion ensued. Linus Torvalds remarked that the main use of x32 was "extreme benchmarking", and said he would not be opposed to removing support if there were no complaints.

But, naturally, there were complaints. Thorsten Glaser was reluctant to lose his x32 Debian desktop. Richard Purdie pointed out that the Yocto Project supported x32 and occasionally got bug reports. Rich Felker said that GCC performance benefits from an x32 build and that the ABI is useful for catching portability problems. Others expressed an interest in preserving the ABI as well. In the end, while the groundswell of support for x32 was not huge (and John Paul Adrian Glaubitz, the maintainer of the Debian x32 port, even stated his non-opposition to the change), the idea of removing it was dropped, and that support remains in current kernels.

On May 23, though, Sebastian Andrzej Siewior reopened the discussion around removing it. There is, he said, "practically no real use for x32" and that the performance benefits it offers are not compelling enough to cause users to adopt it. He suggested removing the configuration symbol that must be set to build x32 support into the kernel. If enough time passes without complaint, then all of the code implementing that support could be removed as well.

The reaction this time around has been more subdued — so far, at least. H. Peter Anvin did not oppose phasing out the x32 ABI, noting that it "happened too late to actually be able to become useful". He mostly took issue with Siewior's suggestion that the x32-specific system-call numbers could be reused, which is not a huge concern in any case, since system-call numbers are not in short supply. The x32 ABI still does not appear to be entirely unused, though. Sam James said that Gentoo still has some x32 users and Neal Gompa pointed out that PLD Linux has an actively supported port: "So it isn't broadly used indeed, but it's used enough that it's probably worth being slightly more careful about deciding to remove it". Glaubitz said that the Debian port is still maintained; "even Rust works there".

Those comments notwithstanding, the response to Siewior's proposal has seemingly been one of overwhelming indifference. The people most aware of x32 support in the kernel are the ones who have to maintain it, and there has been a distinct lack of developers offering to help with that task. The returns from maintaining this ABI would appear to be diminishing. Even so, kernel developers are generally reluctant to remove support for features that are actively in use. So, even though almost nobody wants it (and many actively want it gone), x32 support may yet live on in the kernel.

Comments (33 posted)

Open-source security is not a solo activity

Over time, many open-source maintainers face the same problem: they lack the time to do all of the work that their project needs, and no one else is stepping up to provide adequate help. Maintainers, though, are often reluctant to throw in the towel. The result is suboptimal all around; the maintainer is stressed out, project quality suffers, and users face security risks that they may not be fully aware of. At the 2026 Open Source Summit North America, Robin Bender Ginn spoke about this problem, when it might be time for maintainers to pass the torch, and the responsibilities of users.

Ginn is the executive director of the OpenJS Foundation, which promotes adoption and development of JavaScript technologies. As a result, her talk was in large part focused on maintainers of JavaScript-related projects. She began with the scale of that ecosystem; more than 98% of web sites include JavaScript code, she said, and there were more than two billion downloads of Node.js in the last year.

That scale means that there is a lot of pressure on maintainers. They may see the use of their projects grow from thousands of users to millions of users to "oh my gosh, I'm like maintaining global infrastructure". When that happens, burnout can follow. That leads to the question, Ginn said, "when does continuing to maintain that project increase the [security] risk instead of reduce it?" It might be more harmful than helpful to continue to maintain a project.

Signals of decline

There are two types of signals to look for when it comes to the security of open-source projects, she said: technical and human. The technical signals include things like CVEs taking longer to be fixed, the release cadence slips, or the continuous-integration (CI) pipeline breaks without any response. The human signals, Ginn said, may be harder to see; a maintainer may stop replying, pull requests go unreviewed, or the tone shifts in some of the reviews. "People get a little grumpy." From the outside, it might appear that maintainers are simply disengaged. From the inside, "it's often like a sign of survival because, you know, human capacity is finite".

If a package has not been patched or hasn't received commits in 12 months, for example, "that is a good indication you should migrate now". If a project has a single maintainer, no governance documentation, and no SECURITY.md file stating its security policy, "you may want to engage and sponsor" it, she suggested. If a project has a single maintainer, "particularly in the JavaScript world", it doesn't mean the project isn't healthy. But it is a good idea to support those folks, she said.

Many of the libraries that are relied upon today, she said, exist because JavaScript was missing important features such as async/await and standardized browser APIs. It is possible to deprecate some of those libraries because the platform has caught up, Ginn said. For example, Ecma International's TC39 committee on ECMAScript (JavaScript) standards work has brought a number of features to the language specification. Having features built into JavaScript, rather than implemented as separate libraries that are pulled in as dependencies, means a smaller attack surface, she said. Native implementations tend to be patched more quickly, and there is less polyfill code.

Investment, not heroics

Next, Ginn moved to talking about the importance of funding security work for open-source projects. It is difficult to find security folks to work on projects, and security improvements do not come from heroics: "They come from structure and they come from funding."

As an example, she was surprised to find that the Node.js project did not have a security working group four years ago. It became the first project to receive funding from Alpha-Omega, and also received money from the German government through the Sovereign Tech Agency to provide budget for dedicated security engineering. The result is that Node.js now has "a dedicated security group that's thriving". The project has a threat model, predictable CVE processes, secure defaults, and a clear end-of-life process for releases. "They went from 26 steps for a security release to, like, one or two". Nothing about the code changed, Ginn said, but it reduced stress on the maintainers.

Ginn said that end users also have a responsibility in keeping a project healthy. "When you depend on a healthy project like Node, your role isn't just to consume it, it's to help sustain it." She encouraged people to ensure that they are staying current with releases to help prevent vulnerabilities, and to keep up-to-date with news from the projects such as the fact that Node.js is changing its release schedule. Ginn also asked for responsible disclosure through official security channels, rather than reporting security vulnerabilities in public tickets.

"Healthy projects do not stay healthy by accident", she said, and called for people to participate in security working groups and help fund projects and maintainers they depend on.

Reducing scope

Another path to better security is reducing the scope of a project. Ginn brought up the jQuery JavaScript library; she said that about 78% of the top one million web sites use jQuery, and that its content-delivery network serves 78 billion requests per month.

Its popularity is due to the fact that it solved many problems that, early on, JavaScript did not. Since then, the jQuery maintainers decided to start deprecating parts of the project that had been replicated in the JavaScript platform. For example, the maintainers deprecated the jQuery Mobile project in 2021.

From the outside, Ginn said, deprecation may look like abandonment—but it is not. "Many times it's just the maintainers doing the responsible thing. It's making the project smaller, safer, and more sustainable." How the larger ecosystem reacts to deprecation matters, she said. Healthy communities follow maintainers' communications, including their blogs, social-media presence, and so forth. She advised that those using open-source projects "contribute before complaining", such as by helping with documentation, or assisting with migration.

Stability through governance

Sometimes a project does not need to reduce scope, it just needs better structure. Ginn used the Express project as an example. It is a web framework for Node.js that has more than 52 billion downloads per year. It was managed by two maintainers, until one stopped and the other became overburdened. The pressure became overwhelming, she said, and updates stopped.

Eventually, in 2024, the community stepped up to provide a plan called Express Forward to move the project out of maintenance mode and replace the single-maintainer model. It started with the "boring" work of release automation, cleaning up the issue tracker, and so forth.

Another project, the Lodash utility library for JavaScript, also had a single maintainer. John-David Dalton had a long streak of committing code every day for years, until he had "a really difficult period in his life" that caused a lot of stress and led to burnout. "This is, again, what he does in his free time, not his full-time job." This is an uncomfortable truth about open-source software, she said, "one person's life events can slow down [development of] software used by millions of applications" and create a supply-chain risk.

Ultimately, the OpenJS Foundation stepped in to help, and the project received funding from the Sovereign Tech Agency. The community formed a technical committee and worked to simplify the project. "They treated it as feature complete and focused on making it sustainable." That included deprecating APIs that now had native JavaScript equivalents, reducing support for obsolete runtimes, and making the library smaller.

Ginn noted that a single maintainer burning out can affect millions of applications. The responsibility for security updates falls on too few people. When someone burns out the question becomes who can realistically carry the responsibility of a project moving forward?

She talked a bit about the Linux Foundation's LFX Insights project, which analyzes open-source projects and attempts to measure the health of their communities. "So now your security teams can also look at the code, and then they can look at the community health".

Security incidents show how fragile the ecosystem is, she said. Ginn pointed to the Leftpad incident, Log4j vulnerability, and phishing attacks that are plaguing the npm ecosystem that is used to deploy packaged software for Node.js. These are not just coding problems, they are sustainability problems:

In open source, we like to celebrate the hero maintainer, the person who keeps that project alive forever and forever and forever. But, you know, the expectation of maintaining it forever is actually a security risk because deprecation actually isn't abandonment. Sustainable ecosystems require funding, governance, shared responsibility, and a lot of help from friends in the community. So letting go, when done responsibly with notice, migration paths, [and] community support, we don't consider it failure. We consider it leadership because the most secure projects are the ones that know when to merge, when to maintain, and when to move on.

It should not all fall on the shoulders of maintainers, though. Governments are relying more on open-source projects, she said, and she hoped that more of them would follow Germany's lead in funding projects. Companies that rely on open source should be assigning people to work on it; she named a few companies, such as Bloomberg, Netflix, and Target that are dedicating people's time to work on open source. "We recommend that you take an audit of the top 20 open-source projects you may use and just pick one and engage" by helping to review bugs, triage security issues, or test pull requests. "Anything that we can do to help those maintainers would be amazing".

Questions

Ginn had a bit of time left for questions; the first attendee who spoke up wanted to know about the effects of agentic AI on projects, and the flood of pull requests that it was enabling. How was that going to affect things? Was it going to improve things or burn more people out?

"All of the above", Ginn said. Right now it is super painful. The Node.js project had just instituted a policy that any pull request should be less than 5,000 lines of code, and large pull requests must be done by a core contributor. "So hopefully that will mitigate it some". Alpha-Omega had funded a new security engineer for the foundation, but that had barely made a dent in what they were facing, she said.

Another attendee wondered if a community forks a deprecated project, does the original developer have a responsibility to help with it, or is it solely up to the new maintainers? Ginn said that it was the latter and mentioned that some companies are forking projects and then selling support as a business model.

Ginn's talk provided some interesting case studies in stabilizing important open-source projects. Of course, the presence of a foundation and help finding funding were key ingredients in the success of those efforts. Those are, unfortunately, resources that are not readily available to many maintainers outside of ecosystems with support infrastructure, like that of the OpenJS Foundation.

[Thanks to the Linux Foundation, LWN's travel sponsor, for funding my travel to Minneapolis to attend the Open Source Summit.]

Comments (3 posted)

Trying to make sense of package-manager metadata

Package managers for operating systems and programming languages have been around for decades. Each package manager, and its accompanying packaging format, has been shaped by the needs of its respective ecosystem, but there is a growing need to make use of package metadata for more than software management: for example, in vulnerability scans, software bills of materials (SBOMs), and more. On May 19, Damián Vicino spoke at the Open Source Summit North America 2026 about his experiences in the past year trying to make sense of the varied metadata provided by more than 20 package managers.

Metadata mess

Vicino said that he has been using package managers for more than 20 years, and that he has been happy with them for most of that time. Two years ago, though, he started working in the open-source program office at Datadog; he needed to gather package metadata—such as the name, version, license, and dependencies of the software—as part of his duties there. For instance, he wanted to be able to grab the name and version of a package to see if there were any CVEs that applied to it; he also wanted to check licenses to ensure compliance.

This is information needed for all kinds of analysis, Vicino said: dependency updates, supply-chain security, vulnerability management, SBOM generation, and more. "If everyone needs to map that data together because you need to read it, why do we have one of the implementations in each of these tools independently?" He explained that there is no shared reference framework, and that it was much harder than he expected to try to get all of the data he needed from each package manager—much less all of them.

He thought that there might be "magical tools" to grab all of the data that he needed to do the analysis he wanted to do, but he could not find any. "So I started writing my own. I saw that was a really, really difficult thing." He showed example metadata (reproduced below from his slides) for Python and Node.js packages, as well as a FreeBSD port Makefile, to illustrate how similar information was expressed differently across package ecosystems.

Real projects today, he said, tend to be a combination of software from multiple sources; a Python project has C dependencies, or a Debian package contains a Python library. If, for instance, a developer is trying to create an SBOM for a project that combines artifacts from various package managers, then "you want the same representation for everything", but that does not exist.

Package manager chaos

Vicino said that there are, of course, reasons why each package manager does things differently: each one had evolved separately because it has its own purpose. Some are designed for operating systems, others are for programming languages; some package managers were created decades ago, others were created just a few years ago. A package manager may be designed around the assumption of a centralized registry, or expect distributed repositories of packages. There is a difference in how they are developed and extended; he pointed out that Python has the PEP specification process, while in other cases "some developer started something, and people started picking it up" without rigorous planning.

Package-manager developers set the rules for how everything works, "but they set different rules than every other package management developer in other ecosystems". Users of those package managers can't easily find the data they want because it isn't available. Maintainers of packages may want to publish specific information, but not know how to do so (if it is possible at all).

Developers want to provide information about all of the varied components in a project, but there is no good way to get all of that information in a unified format. "So who is paying the cost of this? Everyone is paying the cost, it's so confusing." Multi-platform, mixed ecosystem projects are broken, he said.

Not a project that doesn't build: they build, but you don't have the ability to respond to incidents because you don't know that you have an incident. You cannot go into that data and see that there is a CVE seven dependencies down the tree.

Package managers have existed since 1992, Vicino said, so why would the problem be solved now? (The exact date the first package manager was created or published is up for debate. Nevertheless, 1992 was the date he cited.) He believed that external pressure in the form of legal requirements would be the catalyst to push for solutions to the metadata mess. He pointed out that the Cyber Resilience Act (CRA) in the European Union has mandates for SBOMs, "and if you are going to make an SBOM, you need the data. It's going to be a legal requirement. That's real pressure."

CHAOSS working group

That said, after 30-plus years of package-manager development, he did not believe that package managers were going to change to be SBOM-compliant. When Vicino had been trying to find solutions for managing information from disparate package managers, he had asked around to see if anyone else had found the answers he was seeking. He didn't find the answers, but he did find other people asking similar questions. That, in part, led to the creation of the package metadata working group as part of the Community Health Analytics in Open Source Software (CHAOSS) project.

The working group's first meeting was in September 2025, Vicino said. The group wants to help different communities talk to one another and try to document practices across package managers and bridge the gaps between them. He emphasized that the working group is not trying to write new standards; there are already good standards for representing package metadata, such as the System Package Data Exchange (SPDX), CycloneDX, and OpenChain. "We just want to understand for each package manager what's the gap to be able to use that standard."

We thought it'd be easy

The plan is to work on one attribute at a time, he said. The group has a queue of 11 attributes it wants to analyze, beginning with license metadata. The group started with analyzing license metadata because "it was going to be easy and allow us to check how our process works". It turned out, he said, that "our assumptions were completely wrong". The group examined package managers for a number of language ecosystems, operating systems, and also looked at Dockerfiles. That involved looking at documentation, standards, and implementations to see how package managers expose license information.

The license-analysis research focused on the field name, whether the package managers validate license information, the structure for expressing the license, and how many attributes are supported. The group found little agreement on the field name for licenses: "there is 'license', 'licenses', 'licensing info', 'license file', 'license URL'. That's something we can work with, it's just mapping."

Package managers were grouped by how license information was declared in the package metadata. Eight ecosystems use unambiguous declarations, such as SPDX identifiers and expressions (AND, OR, WITH); Cargo, Homebrew, npm, and RPM all fall into that category. Unambiguous means "if you're a maintainer and you want to write the SPDX expression or equivalent, you can. That's the only requirement." It does not mean that the use of such identifiers is required or enforced.

Twelve of the ecosystems have a field for license declaration but use non-standard ways of expressing the license (e.g., free-form text); Debian packages, FreeBSD ports, Maven Central, and Ruby Gems are all in the ambiguous category. Three of the ecosystems do not include a field for license at all; Dockerfiles, for example, do not have a required license field. In addition, some of the package managers validate whether the license information exists, while others do not.

The group used data from the ecosyste.ms project that tracks open-source data to study the license declarations "in the wild" to see how many had valid SPDX data. There were 36,224 packages in the top 1% by popularity, he said, and 60% of those had valid SPDX data. In the top 10%, more than 2.3 million packages, only 47% had valid data. The CSV files used for the project's analysis are on GitHub, as are the scripts for processing and analyzing the data.

Vicino showed examples of good and bad license expressions. Maven Central provided a good example of what not to do with several different ways of expressing the Apache 2.0 license including (but not limited to) "Apache-2.0", "Apache License, Version 2.0", and "APACHE-2". He showed a slide with an example of what "good" would look like, a mandatory SPDX expression with optional escape hatch. SPDX has standard strings for many licenses, such as "Apache-2.0", but the "escape hatch" is needed for those licenses that the standard does not cover. For those, it offers the LicenseRef value, which can take a link to the custom license.

Recommendations

Vicino said it was easy to decide what the group would recommend to package-manager ecosystems: full SPDX expressions, including its operators, that are validated somehow. Currently most attributes, such as dependencies, are validated but "except for very few, nobody validates the license". It is not trivial to do, he said, but there are packages to do license validation for most languages: "so it should be something that can be introduced into package managers".

Where the validation step happens is another question. Some ecosystems have centralized registries; they could make validating the license a part of the publishing process. For distributed ecosystems, he said, it's harder. Validation should be enforced by the publishing tools for packages with a check by the "consumer" tools, but the group does not recommend enforcing a valid license expression by consumer tools "because they don't have a way to fix it" if the expression is invalid.

Of course, a package ecosystem cannot re-declare all of its license metadata overnight. He recommended a five-stage process to transition to valid SPDX identifiers, beginning with parallel support of valid and non-valid identifiers, then requiring new packages to have valid SPDX, updating existing packages, and ultimately deprecating the old format. Unfortunately, he did not mention the work Fedora has been doing to transition from its old short-license-name format to SPDX identifiers, which has followed a similar process. The project kicked off a multi-stage process in 2022, and has made significant progress in becoming SPDX-compliant.

Vicino said that every tool that depends on package metadata could benefit from the work that the group is doing. For example, it would make it easier to examine different ecosystems for known vulnerabilities. The idea is to reduce duplicated work and make better tools: users need not have "insider knowledge" about each package ecosystem to find information.

He said that the group is still looking for people to connect with for some of the package ecosystems, and encouraged interested people to get in touch. Licensing is only one of the attributes the group wants to tackle, though; he said there is also work ongoing related to metadata for source-code URLs, lock files, binary dependencies, funding links, and more. He pointed to information on the last slide of his presentation for links to the group's Slack channel and meeting schedule.

Questions

There was a little time left over for questions. I asked whether the group had looked at the Flatpak or Snap package formats; he said that they had not. I also asked if they had considered grouping the package managers by function: for example, looking at language package managers and operating system package managers separately to see how they were alike or different.

Vicino said that they had considered that at the start of the project, but the things that were most telling was whether an ecosystem used a centralized registry or not, and the age of the ecosystem. Perl's package manager and Debian's are not that different, he said, because they depend on a centralized registry. Older ecosystems have a more copyright- and attribution-centric approach, while newer ecosystems focus more on dependencies or enforcing version numbers. "But we don't see a big difference between how [an] OS-centric package manager works compared to how a language-specific one does."

At that point, the audience had run out of questions just as the session had run out of time.

[Thanks to the Linux Foundation, LWN's travel sponsor, for funding my travel to Minneapolis to attend the Open Source Summit.]

Comments (2 posted)

Policies for merging new filesystems

In a filesystem-track session at the 2026 Linux Storage, Filesystem, Memory Management, and BPF Summit, Amir Goldstein wanted to discuss his proposed documentation on adding new filesystems to the kernel. There are a number of unmaintained and untestable filesystems already in the kernel, which are a burden to VFS-layer developers who are trying to make sweeping changes, such as switching to folios and the "new" mount API. Goldstein's document is an attempt to head off the addition of filesystems that may increase that burden down the road.

The idea behind the document is to provide prospective filesystem projects a bit of a checklist for what is needed in order to be considered for inclusion. Two new filesystems, VMUFAT and FTRFS, were posted in the same week that he had proposed the document. Both of those projects needed to be asked the usual questions (such as "why not FUSE?"), pointed toward the need for adding fstests, and so on. Getting all of that down in one place that developers of new filesystems can be referred to will help.

Criteria

The filesystem developers need to decide what message they want to send to prospective new filesystems; he noted that the rejection of ZUFS sent a message, but he wondered if the community's message had changed since then. Christian Brauner said that for a few years there were multiple different container filesystems that were proposed, but that he never believed a single filesystem would work for all of the different kinds of containers the kernel supports. Pushing back under those circumstances makes sense, but there may come a time when there is a real need to add another filesystem to the kernel. "I just think the bar should be quite high to do that."

The community has seen the pain of bringing a filesystem in and needing to reverse course, Brauner said. There is a balance that needs to be found and criteria for new filesystems can help with that. Part of the problem, though, is that new filesystems are generally sent to Linus Torvalds for a decision; the filesystem community does not necessarily get a veto.

Ted Ts'o pointed out that the interest in using FUSE for filesystems has increased over the last 12-18 months. Much of that was driven by the security concerns with in-kernel filesystems. He pointed to the work that Darrick Wong has been doing on implementing ext4 as a FUSE filesystem that can still perform at close to native speed; that was aimed at providing better security for container workloads. Ts'o does not consider containers to provide a security boundary, but some who do are interested in accessing ext4 filesystems that way.

He does not think that the filesystem community should be pushing new filesystems to use FUSE, however. The bar should be set high enough for a new filesystem "so that it is long-term maintainable by the community". The new filesystem developers can then decide what approach they want to take.

An attendee suggested having a list that described the reasons why a filesystem should be in the kernel, as well as why it should not be. There are some existing in-kernel filesystems that do not need to be there, he said. The problem with that is that "someone has to write it", Jeff Layton said to laughter.

Damien Le Moal agreed with all of the criteria, such as needing tests and maintainers, calling it "a no brainer" for any proposed feature. In the special case of a filesystem, though, he said that the first question that the project needs to answer is "why?": "what is the exact thing you cannot do" with the existing choices? "What problems is it solving?" Without answers to those, there is no point in accepting a filesystem, no matter how good the code is or how committed the maintainers are.

Removal

It is extremely difficult to remove a filesystem once it has been merged, Josef Bacik said; addressing that is where the group should spend its time. The criteria for inclusion is reasonably well-known, but the process to deprecate and remove one is not. So it is not just criteria for merging that needs to be documented, but filesystem projects also need to continue with a certain maintenance level and keep up with the rest of the community, which should be described. If a filesystem falls behind, "this is how we are going to get rid of your filesystem".

Brauner said that there had been pushback from Greg Kroah-Hartman and others at a Maintainers Summit about leaving obsolete filesystems in the tree, as is done with drivers. There are drivers that "just sit and rot", perhaps filesystems should be treated the same way, they suggested. Jan Kara wondered if it made sense to simply implement a FUSE version of some of these unwanted filesystems as a way to be rid of them, though regressions might get in the way of the final step.

Brauner noted that he had put out a policy statement that bugs from corrupted filesystem images were not considered security problems. If that were not the case, there would be an unending stream of CVEs that would need to be addressed, particularly for older filesystems. He said that the flood of bug reports coming from LLMs has led to pulling out some kernel code from the networking subsystem, so "removal, suddenly, is something that can be done, apparently, without much objection".

Ts'o said that historically, "really simple" filesystems that do not use "any of the advanced interfaces" can remain in the tree and "rot" without causing much in the way of problems. There are certainly bugs lurking in some of those, which could be found by fuzzing or LLMs, "but nobody cares". The community could be explicit about that and simply list legacy filesystems with disclaimers like "no one cares about them from a security perspective, if you're a distro, we strongly recommend you don't build the suckers".

It makes sense to send a strong signal that some are second-class filesystems, he said. "The only reason why we haven't just simply created a FUSE driver and then ejected the in-kernel version is that it's not even worth the effort to do that." The list may be the lowest energy fix to the problem while allowing the few users of those filesystems to keep doing so. It is similar to support for the PA-RISC architecture in the kernel, which almost certainly has security problems of various sorts, but the kernel community seems unconcerned.

There was some fast-moving discussion around marking filesystems as deprecated or listing them as Ts'o suggested, though no real conclusion was reached. Meanwhile, Goldstein wondered if new filesystems should be required to be maintained out of the tree for a while as a proving ground for the code as well to show that there are interested users. That has been a successful path for some filesystems in the past.

Famfs

Goldstein raised the discussion about merging famfs. He said that there was never any real pushback against merging famfs in some form, it largely came down to whether it should use FUSE or not. Those questions allowed its developer, John Groves, to try both to see which made more sense; it was a kind of extended "design review", Goldstein thought, which was valuable as part of the process.

Groves did not disagree, though he hoped it would not be another year before famfs was merged in some form. There is a bit of an impedance mismatch since famfs uses memory, not storage, so data is not really persistent. But famfs is also low risk as a new kernel filesystem, since it cannot be used as a general-purpose filesystem. It is aimed as a solution to a new problem that comes about with a, say, multi-TB memory appliance.

Goldstein agreed, but noted that the process was meant to see if something more generalized could come out of it, which would make it a better fit for the upstream kernel. Groves said that two years ago he had presented the filesystem at LSFMM+BPF and switching to FUSE was suggested; he was resistant at first, but returned to the summit in 2025 with a FUSE-based implementation. He maintains both versions and is leaning toward suggesting a return to the standalone filesystem for merging.

Many of the requirements that are being suggested for new filesystems, such as working tests in fstests, only really apply to filesystems that have a persistent on-disk format, Ts'o said. That's important because there need to be different standards for, say, a new read-only or network filesystem. For example, EROFS is being used in ways that are "beyond its original use case, which is a sign that the people who did it got it right". It is unlikely that anyone will take on the pain of creating a new read-only or network filesystem unless there is truly a need; "it's just not worth the effort". The criteria for those, if they were to appear, are sure to be different than those in the document, which are geared toward regular, persistent on-disk filesystems.

[I would like to apologize for any errors here. The acoustics in the room were problematic for both hearing and recording. Misunderstanding and misidentification may have resulted.]

Comments (27 posted)

Separating memory descriptors from struct page

The kernel's memory-management subsystem is currently partway through a multi-year project to replace the page structure (which represents a page of physical memory) with memory descriptors. At the 2026 Linux Storage, Filesystem, Memory Management, and BPF Summit, Vishal Moola ran a fast-paced session in the memory-management track to describe the current state of that work and what is likely to happen next.

The page structure has been at the core of the kernel's memory-management subsystem since the 1.3.50 development release in late 1995, but it has a number of problems. It must be able to describe any type of page in the system, which limits how small the structure can be. These structures currently occupy a significant portion of the memory they represent. The size constraint has led to a lot of complexity as developers have used every trick they could to shoehorn more information into the structure. Use of struct page has leaked far beyond the memory-management code, making the structure harder for memory-management developers to change. These structures also contain a lot of redundant data in cases where pages are grouped into larger structures; a PMD-size folio can be described with a single folio structure, but requires 512 page structures.

The memory descriptors that will replace struct page are, eventually, intended to only occupy eight bytes. For many page types, that descriptor will contain a pointer to a type-specific structure — struct folio for folios, for example. Some other page types can be fully described with the eight bytes available. This descriptor-based future will be cleaner and more memory-efficient, but getting there is not a small task.

A number of descriptor types have been defined so far, Moola said, starting with the folio type. For pages used by the slab allocator, slab descriptors were added in 2021. There is struct ptdesc for page-table pages, and descriptor types for zsmalloc and netmem as well. Each of those descriptor types is currently carefully designed to overlay struct page but, when the switch is made, it will become possible to remove unneeded fields from each, making them smaller.

One concern he has with switching to descriptors is the double-allocation cost. When a page is needed for some purpose, kernel code will need to allocate that page, but it will also have to allocate the descriptor to manage it. With the existing system memory map, instead, the page structures are set up at boot time and are always present. He also wondered about how the change would be managed in general; he suggested adding a new configuration option (CONFIG_MEMDESC) that would provide access to all of the disruptive changes. It would be disabled by default. A participant said that folios will be the biggest user of descriptors in general; he asked whether it might make sense to convert all of the other types first as a relatively small change before making the big leap.

Matthew Wilcox said, though, that moving to the eight-byte descriptor is still a distant goal. There is "a long-tail problem" of code throughout the kernel using random fields in struct page for its own purposes. This code has worked so far, but all of it will have to be located and fixed before the change can be made. That said, he liked the idea of doing folios last. He suggested the creation of a "developer preview" configuration that would simply "disable ¾ of the kernel"; it would support the XFS filesystem but no networking. This configuration could be used initially to test out changes and run benchmarks. Converting page-table descriptors first, he said, might be a good start.

Moola said that he has tried moving to descriptors for a few types; it allowed him to shrink struct page to 32 bytes. One of the biggest challenges turns out to be page migration, which forces some fields to be kept in struct page, and also requires the allocation of a new descriptor during the operation. There was a detailed description of how migration uses struct page fields, and a conclusion that, initially, enabling descriptors would force migration to be disabled.

Moola's slides contain a number of examples of what descriptors might look like once they are separated from struct page; most of those details were not discussed in the session in any detail, though. The session concluded with a statement that this work appears to be making progress.

Comments (none posted)

Representing the true signatures of kernel functions

Optimizing compilers can, under some circumstances, infer when a parameter to a function is not needed, and remove it. This is all well and good until the kernel's tracing or BPF subsystems need information on how to call the function or where its arguments are stored. Alan Maguire and Yonghong Song spoke at the 2026 Linux Storage, Filesystem, Memory-Management, and BPF Summit about their work on recording information regarding changed function signatures in the kernel's BTF debugging information, to better support tracing such functions.

Song opened with three examples of ways that an optimizing compiler can transform a function's signature, each taken from the actual kernel. Here are simplified versions; the full examples can be seen in the slides. Firstly, an argument can simply be removed in its entirety:

    int something(int flags);
    // becomes
    int something();

Alternatively, if a function takes a structure as an argument, but only uses some of its fields, those fields can be extracted from the structure and passed on their own:

    typedef struct {
        int field1;
        int field2; // Not used
    } structure;

    void something(structure foo);
    // becomes
    void something(int field1);

Finally, if a function takes a pointer to a structure but does not mutate the structure, it can be passed by value instead. This can be worthwhile on its own merits if the structure is small (avoiding an extra load from memory), but it can also enable opportunities for the previous optimization to apply.

    void something(structure *flags);
    // becomes
    void something(structure flags);

Any code that is intended to accurately capture how optimized functions in the kernel are called needs to be able to handle these three cases. Together, they cover a large portion of the functions present in the kernel for which signature changes occur. The remaining functions are outliers in terms of complexity, and best addressed by follow-up work.

The DWARF debugging information format already has a way to represent these kinds of transformations, so when Song and Maguire started on this work in 2025, they thought of reusing that encoding. DWARF is intended to play a subtly different role, however: it represents the signatures of functions as they are in the source code, not as they are after optimization. Specifically, the DWARF data for a function argument describes how to find its value at run time. The tracing subsystem needs a different piece of information: what the type of an argument is at run time, so that it can reconstruct a representation of it for printing, or so that tracepoints can refer to and change arguments by name in a safe way. The DWARF maintainers rejected an extension to DWARF for that purpose on the grounds that the format is only intended for debugging information, however.

The necessary information on an optimized function's "true" signature (as Song calls it) is technically present in DWARF, but not in a usable way. Thus his current approach: teach the poke-a-hole (pahole) utility to reverse engineer the kernel's DWARF data and extract the true function signatures.

The core idea is to take the architecture's ABI, figure out how function arguments defined in the source code would be passed without optimization, and then check the DWARF information for discrepancies. The DW_AT_calling_convention annotation records what calling convention a given function uses; when Clang changes a function's signature during optimization, it sets the annotation to DW_CC_nocall to indicate that the function is called with a non-standard calling convention. Then, pahole walks through the DWARF information to find arguments that are passed in different registers than expected (or that are not passed at all) in order to reconstruct the changes to the signature. By focusing pahole's attention on functions that have the appropriate annotation, the process of reverse engineering the kernel's DWARF information can be made faster.

Out of 69,103 total kernel functions, 875 currently have their signatures changed by optimization. 532 of those are not currently represented in the kernel's BTF information at all, and 343 are represented incorrectly, Song said. With Song and Maguire's patches, only 18 total functions won't be able to be represented. Unfortunately, DWARF doesn't record whether function return values are changed in the same way as arguments, and there are problems with handling arguments passed on the stack, so there could still be problems with return values or functions with more than six arguments. Future work should be able to address both of those.

At that point, Maguire stepped up to speak about the GCC side of things. GCC indicates functions that have had their signatures changed by generating a new symbol name, but the same broad approach works for binaries it creates as well. José Marchesi warned that the symbol name is not a reliable indicator — some transformed functions do not receive a changed name, according to one of the other GCC maintainers. Marchesi didn't know the details, however, and encouraged Maguire to reach out to them to ask.

Another complication that Maguire has run into with GCC is functions that are inlined differently in separate call locations. Partially inlined functions have long been a source of complication for the kernel's BTF. Maguire went into more detail about the particulars of the problem and his potential solution in a session later in the conference.

Song raised the idea that, in the future, compilers could generate appropriate BTF information directly, without going via DWARF. The advantage is that compilers know exactly how they are changing function signatures, so there would be no risk of information lost in translation. Alexei Starovoitov liked the idea, since it would mean that the kernel community would not need to fight the DWARF standards, but wasn't sure how realistic it was. Ideally, a change to BTF would also work for Rust code (which implies it should work for link-time-optimized kernels as well, since CONFIG_RUST implies CONFIG_LTO). Starovoitov worried about the complexity, especially of deduplicating BTF declarations, but did think that extending BTF to handle more of the complexity was important. As the session wound down, the audience broke into scattered discussion about the particulars.

Comments (none posted)

Caching for extended attributes

Extended attributes (xattrs) provide a way to attach key/value metadata to inodes—files, directories, and the like—in a filesystem. As with many Linux filesystems, the FUSE filesystem supports xattrs. In a filesystem-track session at the 2026 Linux Storage, Filesystem, Memory Management, and BPF Summit, FUSE maintainer Miklos Szeredi led a discussion about caching xattrs in kernel memory; he would like to create some common infrastructure that could be used by FUSE and shared with other filesystems.

He began by listing the existing xattr-cache implementations in the kernel. There is a simple_xattr cache that is used by tmpfs, pidfs, and kernfs; it uses a hash table to store the data but it does not allow any shrinking. The nfs4_xattr_cache also uses a hash table to store the xattrs; it does allow shrinking. The mb_cache is used by ext2 and ext4 to deduplicate xattrs and is not a general-use xattr cache. Meanwhile, the EA inode feature is used by ext4 to store large xattrs that do not fit into the inode. Ted Ts'o cautioned that the EA inode was not intended as a cache at all, just as a way to store large xattrs, though it could perhaps be used as one. Amir Goldstein pointed out that the XFS buffer cache effectively acts as an xattr cache, which Szeredi acknowledged.

If there is to be a common implementation of the cache, it should probably use a hash table, Szeredi said. The basic idea of consolidating the two main xattr-cache implementations (simple_xattr and nfs4_xattr_cache) into a common code base that could then be used for FUSE seemed good to attendees. Currently FUSE requires a round-trip through user space to retrieve xattrs, so caching those that are used frequently will help its performance.

He went through some of the requirements for the shared xattr cache, including having low overhead for storing only a few xattrs and even lower overhead when not storing any. There is a need to be able to reclaim memory, especially that taken up by large xattr values; the NFS xattr cache has a complicated mechanism to do so already, he said.

Brauner asked about access-control list (ACL) caching, which is already done for filesystems. The i_acl field in the inode structure is used for that, he said, and wondered if that caching would also be part of this new shared scheme. Szeredi pointed out that i_acl is not a cache, it is simply an entry for the ACL for the inode; its use would be unaffected by the shared xattr cache.

Continuing on, Szeredi said that the xattr cache might globally deduplicate the keys and values in order to save memory. One use case for that, Ts'o said, is ACLs for Windows that are stored as xattrs; they can be "gargantuan", and are generally shared by every file in a directory. Deduplicating those would save substantial amounts of memory; that is also true for POSIX ACLs, he said.

Brauner asked what was meant by "global" and wondered if Szeredi meant per superblock. Szeredi said he was not yet sure about the granularity of the deduplication. Ts'o recommended doing it on a per-superblock basis in part because that would ease memory accounting for control groups; the xattr-cache use for a particular control group could be charged correctly.

There are some policy questions that go along with the cache, Ts'o said. Administrators may want to limit the size of xattrs that are added to the cache. For example, caching large xattrs, rather than more important ones like the security.* xattrs, may not be desired. Szeredi thought that restricting the cache to only store certain types of xattrs might be a way to do that.

There was also some hard-to-follow discussion on the implementation, but the overall feeling in the room was positive toward the idea. There were no objections to trying to find some common ground, rather than adding yet another xattr cache specifically for FUSE.

[I would like to apologize for any errors here. The acoustics in the room were problematic for both hearing and recording. Misunderstanding and misidentification may have resulted.]

Comments (3 posted)

BPF in the agentic era

Alexei Starovoitov gave "less of a presentation, more of a scream of realization" at the BPF track of the 2026 Linux Storage, Filesystem, Memory-Management, and BPF Summit. He shared a set of ideas for how BPF could change to avoid being swept away by the sea-change in programming represented by modern large language models (LLMs) and the coding agents based on them. In a follow-up session, the discussion covered more problems with how coding agents use tools like bpftrace, and the current deluge of patches in need of review in the BPF subsystem.

He wanted to be clear that these ideas were not something he was unilaterally deciding on, they were "just looking at how the world has to be". Coding agents do their best work when given tight feedback loops, he said. They write code, see errors, and make fixes. This makes them good at handling tedious jobs such as large refactoring operations. That feedback loop doesn't work well in the BPF ecosystem, especially when one has to boot a virtual machine in order to properly test a change.

Even when a BPF programmer has attempted to load their program into the kernel, errors from the verifier "are insane" — huge dumps of data that obscure the actual source of the problem. In Starovoitov's experience, writing BPF code is the only case he has observed where LLMs will give up rather than producing something.

BPF can never crash the kernel. "That's what BPF stands for." That is the one thing that makes the ecosystem unique. So, the BPF verifier isn't going away, and it needs to run in the kernel because user space is untrustworthy. This adds a certain amount of unavoidable latency and complexity between writing a BPF program and getting feedback on it. Given that constraint, how can BPF developers shorten the feedback loops between writing a BPF program and getting a useful error message?

Right now, the verifier essentially has two roles: finding places where the programmer made a mistake, and acting as a security boundary. The first part doesn't need to be done in the kernel; languages such as Rust do a great job of confronting programmers with their mistakes in user space. The BPF developers should be working to make BPF easy to use with Rust, so that simple mistakes can be caught quickly, Starovoitov said. Any program that does pass the Rust compiler — and doesn't do anything obnoxious with unsafe code or inline assembly — should pass the verifier as well. Obviously programmers will still be able to cause verifier errors if they try, but well-intentioned, normal BPF programs written in Rust should not cause verifier errors, he said.

Making this happen may require further work on the verifier, or the careful addition of run-time checks to BPF. But it's a goal that Starovoitov believes is possible and worthwhile. There are other ways to shorten the feedback loop for BPF programmers, of course. If BPF could be made to work in user-mode Linux, for example, there would be no more need to test BPF code in virtual machines.

This confused one member of the audience, who objected that Starovoitov had previously soundly rejected the idea of moving the BPF verifier to user space. The real kernel can't trust user space, Starovoitov said, but it should be possible to run the verifier in user-mode Linux in order to shorten the time it takes to receive a verifier error if one is forthcoming.

All of these changes are coming quickly. In a few months, "there will be no fighting the verifier anymore," Starovoitov promised, as long as one writes BPF programs in Rust. The verifier will use run-time checks where necessary to make everything work. José Marchesi asked whether that change was currently in development. Starovoitov confirmed that it was.

Daniel Borkmann asked how the output from the verifier would change. Current verifier errors are far too opaque, Starovoitov said. They say things like "register is not init" — what is a programmer supposed to do with that? The verifier needs error messages that say what happened and how to fix it. This is another area where Starovoitov wants to take inspiration from Rust, and, in particular, the formatting of its error messages. "You guys are all dinosaurs; Rust won as a language," he said, at least in part because of the quality of its error messages.

In the end, he wants there to be almost no verifier errors. The ones that remain should include details about what went wrong and how to fix it. This change isn't going to happen overnight, he said, but "it is where we have to go".

There was some discussion about even bolder proposals, such as integrating WebAssembly into the kernel. Starovoitov was supportive of the idea if it could be made to work, but didn't think that was likely. When I spoke to him after the session, he said that he thought previous attempts to bring WebAssembly to the kernel were too slow for serious use, but that if someone was willing to do the work to bring it up to par with the existing BPF JIT implementation, they should do so.

At that point, Starovoitov dove into the details of how his suggested transformation could be achieved. There are a number of limitations of the verifier that must be removed, he said. These include the inability to handle functions with more than six arguments, indirect function calls, returning structures by value, support for 128-bit integers, larger BPF program stacks, deeper call chains, removing the one-million-instruction limit, and long jumps in BPF programs. Also, the BPF developers will need to remove the discrepancies between the verification of static and global functions. "Of course, this helps humans too."

Some of these things, such as larger program stacks, have already had the groundwork laid by previous improvements to BPF. For the rest, Starovoitov hopes to move more data into BPF arenas, which have one flat address space. Once that is done, a "global" memory allocator (really one per arena) makes many of the other problems significantly simpler. That plan does pose problems for Rust code that uses virtual dispatch tables; currently, the Rust compiler emits these tables intermixed with the code, which BPF is not set up to handle. Starovoitov suggested that changing the compiler to emit the tables differently might be the simplest solution.

In fact, changing existing tools may be the only way to remain "in distribution" (within the area that LLMs have been trained on), he said. LLMs are not as good at handling tooling that isn't in their training sets; ensuring that BPF objects can be built and linked with existing tools is important to letting coding agents take advantage of the improvements.

Amery Hung asked whether that extended to introducing new concepts, such as new kinds of BPF maps. Starovoitov thought that it did. When it was added, he had thought that BPF arenas would be the last map type needed. Since then the BPF subsystem has also added rhashtable-based maps, but he doesn't envision the need for any more.

The biggest change that will be required in the verifier is improved handling of loops, Starovoitov continued. There have been lots of struggles with BPF loops over the years; their handling has improved, but some loops are still too much for the verifier. Currently, the verifier will go through a loop multiple times until it proves one of three things: that the loop will exit after a number of iterations, that the loop returns to (a subset of) a previously analyzed state, or that the program will hit the maximum number of verified instructions and trigger an error. Starovoitov's proposed solution, called widening, would make it so that when verifying a loop takes more than a small number of iterations, the verifier state is generalized to consider larger numbers of loop-variable values at once ("widening" the state). The downside is that widened states are less precise, and potentially require more explicit run-time checks. This change could let the verifier handle all loops with a fixed maximum number of iterations, at the cost of requiring run-time checks for the loops that require widening.

The early results of this approach are "really great", Starovoitov said, but there is a lot more work to do to make it usable. Still, he hopes to have something together in relatively short order. At that point, the scheduled session was coming to a close, but he had enough still to cover that the talk was extended to a second session the next day.

Tracing and code review

The bpftrace and DTrace tools use domain-specific languages for configuring tracepoints; this makes them good for humans, but bad for LLMs, Starovoitov said. LLMs struggle with the syntax of niche languages. He thinks that the future of tracing looks like having documentation for coding agents that explains how to use standard tools, with explicit examples. That could help bpftrace and DTrace remain usable, but would be even better for tools like drgn that use a standard language's syntax, just with extra capabilities. Ultimately, he wants to be able to tell a coding agent to "find a performance issue on this kernel" and have it use BPF and drgn to identify and debug the problem.

One audience member thought that more examples in the documentation would be a good thing, but didn't think that using the kernel's tracing infrastructure with coding agents was viable unless one does not care about crashing the machine on which they are running. Starovoitov thought that this was ultimately solvable by encoding information about the kernel's functions and types in a format that can be checked automatically by existing tools, such as providing a "vmlinux.rs" set of Rust bindings.

Starovoitov also wanted to discuss code reviews. The BPF subsystem is swamped with low-quality patches that look, on first impression, really good. At this point, he is ignoring the commit log on the assumption that it is written by LLMs and just looking directly at the patches themselves. But, in general, "maintainers don't scale". Starovoitov called on everyone who submitted patches to the BPF subsystem to also perform code reviews.

There is no official number or scoring system, but contributing to BPF does require a fair and honest effort to review code. Cupertino Miranda asked whether he could invite people to come review code in GCC as well, since he is contributing to the kernel as part of his GCC BPF work. Starovoitov said that if he had a patch for GCC, he would certainly do some code review there as well, but "I last touched GCC 20 years ago". He did suggest that perhaps people could consider using LLMs to review their patches, in GCC and elsewhere.

Steven Rostedt complained that not everyone has access to an infinite number of tokens, and that a dependency on cloud-hosted LLMs isn't sustainable. Starovoitov agreed that it was a dependency that needed to be managed, but didn't think that could be avoided: "We are done with coding manually. It will all be agents in the future." He then called it more of a way of life than a dependency.

Rostedt objected that LLM companies don't have a proven business model, and so they're giving things away for free that they will not be able to afford to continue providing to kernel developers in the long term. Starovoitov did not think that LLMs were going to go away, even if prices rose in the future. Another audience member proposed that these companies should put together a fund for open-source developers, and let programmers pick which model to use, so that they would not be locked into one model that could disappear. Someone else pointed out that the Cloud Native Computing Foundation already does give out licenses for things like this to the maintainers of their projects. Starovoitov pointed out that open-source models also exist, and predicted that "coding will be possible on a laptop, once the technology advances". Rostedt agreed that things will become cheaper, but that didn't entirely ameliorate his concern.

Borkmann said that the assembled BPF developers need to think about new people coming to the community. They should be able to review code, eventually, but there is a learning curve. How can they learn to do that when simple code review is done by LLMs? Starovoitov claimed that "all of the newcomers, they're all skilled in agentic coding". All of the patches that they are sending are written by coding agents, he said. It lowers the bar to kernel contribution.

Rostedt didn't think this was a good thing — if ten times as many people are contributing, but the same quantity are sending good code, "that's not a plus". Starovoitov said that this is why more code review is needed. Sashiko has been helpful for that; he doesn't review patches until they've had an LLM comment.

At that point, discussion broke up into a series of increasingly outré proposals for how to improve the review situation, such as asking LLMs to regenerate commit logs based on what is in the patches. Starovoitov gave one final announcement that BPF office hours (regular video meetings where one can discuss problems with the BPF maintainers) are now ad hoc, not regularly scheduled. People interested in attending one should send an email to the mailing list to find a time that works.

Comments (22 posted)

A loadable crypto module for FIPS certification

Many organizations require US Federal Information Processing Standard (FIPS) certification of the crypto code they are running. The certification process is lengthy, but the bigger problem is that the way the crypto subsystem is built into the kernel makes the result unable to be reused across kernel updates. I have proposed a patch series that decouples the crypto subsystem into a standalone loadable module, allowing a certified crypto module to be reused with multiple kernels and, thus, requiring fewer lengthy recertification delays.

The problem with FIPS

FIPS certification is granted at the binary level by the National Institute of Standards and Technology (NIST), and the validation process typically takes 12 to 18 months. In current kernels, much of the crypto subsystem is built directly into the main kernel image. That means even a non-crypto kernel update, say a scheduler fix or a driver addition, produces a new binary and invalidates the existing certification. Distributions are then forced through the full validation cycle again, making it difficult to deliver timely kernel updates while maintaining FIPS compliance.

A solution is to decouple the crypto subsystem from the main kernel and build it as a separate loadable module. The module, rather than the entire kernel, becomes the unit of certification. A subsequent kernel update that does not touch the module leaves the existing certification intact, and loading that certified module onto the updated kernel makes it FIPS-compliant automatically. When the crypto code itself needs updating, a new module can be submitted for certification while users continue running the previously certified one on newer kernels; the old module acts as a bridge, so users never have to choose between an updated kernel and FIPS compliance.

While the idea is attractive, a natural question is why the existing kernel's module system cannot simply be used here, since it already allows code to live outside the main kernel. Three obstacles stand in the way.

First, the crypto subsystem spans hundreds of source files across multiple directories, each with its own configuration-dependent rules. The existing module build system has no mechanism to collect built-in objects from across those directories and redirect them into a module. One might try collecting them in a dedicated module Makefile, but maintaining a separate Makefile alongside the original does not scale well. Replacing the original Makefiles outright is not viable either, since that would break normal builds where crypto algorithms are expected to be configurable as built-in or modular when the feature is not enabled.

Second, the kernel's symbol resolution for loadable modules is one-way: it supports symbols defined in the main kernel and referenced by modules, but not the reverse. Many parts of the main kernel call into the crypto subsystem, so moving that code into a module means the main kernel would reference symbols that no longer exist at link time. The build would simply fail.

Third, the crypto subsystem is needed early in the boot process, before the root filesystem is mounted, so a module that lives on disk cannot be loaded in time. On top of that, crypto algorithms register their initialization functions through the kernel's initcall mechanism, which coordinates their execution order during boot. A regular loadable module has no equivalent; simply calling all the init functions at module-load time could violate dependencies and panic the kernel.

These three obstacles translate into three problems that the patch series must solve: build integration, reverse symbol resolution, and early loading with correct initialization.

Building the module

Rather than maintaining a separate Makefile or replacing the original ones outright, the patch series introduces a new rule, crypto-objs-y, that replaces the standard obj-y in existing crypto Makefiles:

    // in crypto/asymmetric_keys/Makefile
    - obj-$(CONFIG_ASYMMETRIC_KEY_TYPE) += asymmetric_keys.o
    + crypto-objs-$(CONFIG_ASYMMETRIC_KEY_TYPE) += asymmetric_keys.o

When the feature is enabled, any crypto algorithm marked with this rule and configured as built-in (CONFIG_CRYPTO_*=y) is automatically collected into a single module, fips140.ko, rather than being linked into vmlinux. Since the rule works within the existing build infrastructure, each file keeps its original compilation flags, header paths, and Kconfig-dependent logic, with no duplication needed. When the feature is disabled, the rule falls back to the standard obj-y behavior, so normal kernel builds are unaffected. When the feature is enabled, the generated module is embedded into the kernel image as a data blob in a dedicated ELF section, preparing it for early-boot loading. The entire process is integrated into the normal make vmlinux build, so the module is built alongside the main kernel automatically.

Reverse symbol resolution

To allow the main kernel to reference crypto functions and variables that now live in the module, the patch series introduces a reverse symbol resolution mechanism. For each crypto symbol that the main kernel references, an address placeholder is created: a trampoline backed by a function pointer for functions, or a pointer indirection for variables, initially set to NULL. At link time the kernel references these placeholders, so the build succeeds. At run time, once the module is loaded, the placeholders are updated to point to the actual implementations and then marked __ro_after_init to prevent modification after boot.

With the placeholders in place, the remaining problem is directing the main kernel to use them rather than the original symbols. For exported functions, which make up the majority, this is automated using the linker's --wrap option; it renames all references in vmlinux to point at the generated trampolines without touching any source files. A smaller set of exported variables is handled with explicit macro wrappers in the relevant headers that redirect accesses through the placeholder pointers.

Early loading and initialization

Because the module binary is embedded in the kernel image, a dedicated loader maps it directly from memory at early boot, bypassing the filesystem entirely. After loading, the address placeholders are updated to the real crypto entry points. The crypto algorithms must then be initialized, and the ordering matters: algorithms register their init functions through mechanisms such as subsys_initcall() and module_init(), and these must execute in the same order as if the code were still built-in.

The patch series handles this by collecting the module's init functions into dedicated ELF sections and spawning a coordinator thread that executes them in lockstep with the main kernel's boot sequence. Synchronization barriers at each initcall level keep the module and the kernel in phase.

Reusing a certified module

With the crypto subsystem packaged as a loadable module, a previously certified module can be carried forward to newer kernels. One might worry that this would require a stable kernel ABI, effectively freezing crypto development, as was raised in the discussion of an earlier version of the patch series. That is not the intent; rather, distributions would periodically recertify updated modules instead of preserving a fixed interface indefinitely. The reuse of an already-certified module simply bridges the gap until the new certification arrives. How much to reuse, however, involves a tradeoff.

The most conservative option is no reuse at all: submit the updated module for a full FIPS certification, gaining access to the latest crypto improvements at the cost of the full 12-to-18-month wait. At the opposite end, a distribution can reuse the fips140.ko module binary unchanged on a newer kernel, achieving day-one validation.

To support this, the patch series provides a CONFIG_CRYPTO_FIPS140_EXTMOD_SOURCE option that alters the kernel build: given a previously certified binary, the build system embeds it into the kernel image rather than the module that would otherwise be compiled from the current crypto source code. The full procedure is documented in the technical documentation patch. The cost of this option is that engineers must maintain ABI compatibility between the module and the updated kernel.

A middle ground is to freeze the certified module's source code and recompile it against the new kernel. Because the source is unchanged, the module qualifies for a Non-Security Relevant (NSRL) recertification, which typically takes three to four months rather than 12 to 18. This path requires only source-level API compatibility, which distributions can maintain by patching the main kernel if internal interfaces change.

Going forward

The feature has already shipped with kernel 6.18 in Amazon Linux 2023. Other downstreams have pursued similar modularization efforts: Android's Generic Kernel Image (GKI) was the earliest to modularize kernel crypto for FIPS, and Oracle Linux developed its approach concurrently with this one. My patch series cover letter includes a detailed comparison of the three approaches. In short, the key differences are that the approach this patch series takes is more seamlessly integrated with the upstream Linux kernel, is less intrusive to the source tree, and can be more easily generalized across different kernel configurations and choices of which crypto algorithms to certify. This patch series is positioned as a unified upstream foundation that distributions can customize for their own certification and maintenance strategies to deliver timely kernel updates without disrupting FIPS compliance.

Comments (17 posted)