LWN.net Weekly Edition for January 29, 2026

Welcome to the LWN.net Weekly Edition for January 29, 2026

This edition contains the following feature content:

• Open source for phones: postmarketOS: an overview of an alternative Linux operating system for mobile devices.
• Linux Kernel Runtime Guard reaches its 1.0 release: runtime integrity checking for the Linux kernel.
• Who should vote in Fedora elections?: the project takes up the sticky question of what makes a contributor eligible to vote in FESCo elections.
• Filesystem medley: EROFS, NTFS, and XFS: status updates on three filesystems used with Linux.
• Fedora and GPG 2.5: as GNU Privacy Guard's 2.4 series approaches its end of life, Fedora discusses possible alternatives.
• Implicit arguments for BPF kfuncs: Ihor Solodrai attempts to solve the problem of passing arguments to kfuncs from BPF programs.

This week's edition also includes these inner pages:

• Brief items: Brief news items from throughout the community.
• Announcements: Newsletters, conferences, security updates, patches, and more.

Please enjoy this week's edition, and, as always, thank you for supporting LWN.net.

Comments (none posted)

Open source for phones: postmarketOS

Phones running Linux are ubiquitous these days and it has been that way since Android started working toward dominance in the smartphone market. Unfortunately, Android has slowly increased its freedom-unfriendliness and has become something of a privacy nightmare. In a talk entitled "We need an open-source phone OS" at Open Source Summit Japan 2025, Luca Weiss described the smartphone landscape and gave an overview of postmarketOS as an alternative Linux operating system for mobile handsets.

Weiss introduced himself as a software engineer at Fairphone, "which is a Dutch brand to make sustainable mobile phones". In his free time, he is a postmarketOS core contributor and a maintainer for the OpenRazer project to create Linux drivers for Razer gaming peripherals. He stressed that the talk was not being done for his employer; "these are my opinions".

Mobile phone OSes

There is a duopoly for mobile operating systems right now, he began: iOS for Apple phones and Android for phones from more than 100 manufacturers. With only a few exceptions, all of the Android phones are shipping with the proprietary Google Play Services so that the Google Play store is available. Most customers require this because they do not know where to get apps otherwise; in addition, some apps that they want are only available to phones with Google Play Services.

In order to be able to ship those services, Android phone makers must be bound by Google's rules, which is a double-edged sword. On the good side, there are some minimal security requirements for the Android version, as well as policies ensuring API compatibility for the apps. However, those manufacturers have far less independence in what they can do on their phones. It also requires a business relationship with a US company, which may be problematic for companies in some countries currently and it is unknown if those restrictions might expand over time. Beyond that, Google could change its rules at any time or even lose interest and stop providing Android source code at all.

There are multiple problem areas because there are no alternatives to that duopoly, he said. Google bundles and pushes privacy-invasive apps; his example was Google Photos, which defaults to sending photos to the Google cloud and, if that is disabled, regularly pops up messages to "warn" users that their photos are not being backed up—with a default action of re-enabling the feature to dismiss the popup.

While the Android Open Source Project (AOSP) is available under the Apache 2 license, development is done behind closed doors with releases tossed over the wall; getting code into AOSP is not impossible, he said, unless the feature does not align with Google's business interests. There is also the question of whether Android will stay open source, Weiss said. Meanwhile, installing apps that do not come from the Play store ("which they call 'sideloading'") is being restricted to apps made by developers registered with the company, which harms users who can no longer install whichever apps they want.

There is a large amount of proprietary code in any phone running Android and, even if that code was replaced with open-source software, there would still be proprietary code running in modems and other parts of the system. Android runs as user-space programs on top of the GPLv2-licensed Linux kernel, so the Android code can be released (or not) subject to the whims of Google, but the kernel code needs to be released when phones using it start shipping. Android and Linux are only running on a small part of the system-on-chip (SoC) for a phone, however; there are multiple other co-processors of various sorts (GPU, NPU, ...) that run proprietary firmware which cannot be replaced with free software because it is signed with a key that the vendor controls. Even if the signature checking is disabled, there are no efforts that he is aware of to create free alternatives for that firmware.

There are alternative Android distributions (or ROMs), such as LineageOS, /e/OS, CalyxOS, and others, Weiss said, which are more open source than stock Android, but still rely on proprietary blobs. Libraries and binaries that are shipped on stock Android are being incorporated into the community distributions. This includes GPU drivers, modem drivers, and the like, which sit between Android and the Linux kernel, but use non-standard kernel interfaces; he is not aware of any efforts to replace those components with open-source alternatives either.

Relying on those blobs means that all security updates have to come from the vendor; when the vendor stops updating a device because it is at its end of life, updates to those components will not be easily available, though they may be obtainable from updates to similar devices. Those alternative Android distributions are also reliant on Google releasing the AOSP code, which used to happen when the new code started shipping but had recently been delayed by two months. Since the talk, the company has announced that it will only be releasing AOSP code twice per year.

postmarketOS

The tagline for postmarketOS is "The Linux distribution for mobile phones and beyond...", he said to introduce the project. The goal is to make an operating system for phones with both user-space and kernel code that is fully open. PostmarketOS developers want to ensure "that the foundations are laid to actually be able to support devices for a long time"; that's possible because they are using the upstream kernel and not a fork of an older kernel. That allows updating all of the components "and if there's an incompatibility, you can actually fix it without any big issues".

The distribution is aimed at being largely generic, rather than having lots of device-specific parts. Right now, most devices have a device-specific root filesystem and boot image, but the plan is to have a single root filesystem for all devices, with just the kernel and some configuration that is customized for the device.

PostmarketOS is a volunteer-driven and donation-funded independent project, Weiss said. The donations are used to fund infrastructure, including "hacker phones to actually work on together", and to "have some stuff for the FOSDEM conference, where we're always present". It uses Alpine Linux and its package manager, so postmarketOS users "can install everything that you would ever want that is in the package repositories". On top of the Alpine base, postmarketOS provides some extra tooling, device packages, build infrastructure for creating install images, and documentation including an extensive wiki.

Android devices are based on a fork of a specific LTS kernel version that never changes for the life of the device. That lifetime is generally three years, which fits well within the lifecycle of the LTS kernels; once that is done, "you're stuck with this old Linux kernel version with millions of lines of code added and modified compared to the upstream version of Linux".

Instead of that path, postmarketOS is focused on using the upstream kernel by pulling the latest from kernel.org and adding the changes needed for specific devices. The developers are working to get the "ten to 100 patches" for each device into the upstream kernel, so that the diff gets smaller and they can more easily use the upstream kernels going forward. In addition, release candidates and linux-next can be tested to ensure that there are not any regressions and that devices will continue to function with upcoming kernel releases.

By sticking with upstream, the project gets the latest features in the kernel. "There's a lot of things happening there that are actually really cool, and there's no reason that you wouldn't want it on your phone as well". User-space components like to use the newer kernel features, so it is best not to get left behind.

Sometimes the postmarketOS developers are asked why they are not using parts of Android like some other projects do. For example, Ubuntu Touch is a contributor to the Halium project, which repackages "Android bits" and provides a translation layer that will allow Android hardware abstraction layers (HALs), such as the camera HAL, to work on top of a regular Linux user space. It will also allow binaries that are built with the Android C library (bionic) to be used from programs built with the GNU C library (glibc) via libhybris. "There are some very practical reasons for doing this", he said, because it allows relying on the hardware maker to provide a well-functioning camera driver for the hardware.

There are some downsides to that approach, however, including depending on Android updates and proprietary blobs for the functioning of the peripherals. It also would make the project dependent on a "sometimes a bit flaky layer between the Android components and the Linux components". Beyond that, it "doesn't encourage the development of native solutions", such as libcamera, which he noted had featured in the talk just prior to his. That does mean that the cameras that are supported have "quite limited" photo quality compared to, say, Ubuntu Touch, currently; the goal is to get to an acceptable photo quality while always using "100% open-source software".

Status and plans

Currently, postmarketOS "is not a finished product in any sense", Weiss said, but it is still being used. The project does not have any "report to the mothership" functionality or other analytics, so it relies on "anecdotal evidence" from social media and talking to users at conferences to get a sense for how it is being used. Some people do use postmarketOS as a daily driver phone, though some of those also carry a second phone for calls and, say, access to a banking app. "Banks are quite annoying with a lot of app requirements."

There are a lot of devices that are supported by postmarketOS: 111 in the "community" category, which covers the best-supported devices, 342 in "testing", which ranges from barely booting to mostly working for some use cases ("watching YouTube on your couch"), and 250 in the "downstream" category, which still use the Android kernel for the device. When the project began, there was little support for phone devices in the upstream kernel, so everything was in the downstream category; now the kernel supports around 450 devices, so those devices have moved to the other categories over time.

PostmarketOS is not ready for the average person at this point, he said, "but I would say that it is definitely ready for the average hacker". Those who want to hack on a phone will find it to be in good shape; even most of the phones in the testing category will boot and allow logging in via ssh. At that point, "you can do fun things with the phone". He noted that his FOSDEM 2025 talk went into more detail on device support for postmarketOS.

There are lots of plans for the future and various things being worked on, but there is no real roadmap because postmarketOS is volunteer-driven. He suggested following the blog. There are monthly progress reports posted there, which give a good sense of the progress being made. There is a big difference in what is working today compared to a year or two ago, he said.

"What can you actually do with postmarketOS today?" The first step is to install it; attendees probably already have an old phone—or can get a used one cheaply—that can run it. Once booted, the phone can be hacked on over ssh or with a GUI, such as Plasma Mobile, Phosh, GNOME Mobile, or Sxmo for a tiling-window-manager experience.

It is, after all, a Linux system, so users can do anything they might do with any other such system: run Docker containers, make a Kubernetes cluster of phones, or access USB peripherals of various sorts. The compost.party web site (which was not responding as this is written) runs on "a solar-powered old, broken phone"; postmarketOS could easily be used for something like that, he said. Other creative uses include a media-playback device, a musical instrument, or part of an audio-production system.

He noted that postmarketOS "could really use some help" on the technical side. Most of that work is on upstream projects, such as on the kernel for more hardware enablement and stability, middleware projects like libcamera or feedbackd (for haptic and visual feedback), improving the UI or adding features for various mobile GUI projects. Other possibilities are upgrading existing apps or developing new ones and helping to maintain packages in the Alpine aports repository and the postmarketOS-specific pmaports repository. Much of that work can be done from desktop Linux and some of it will not just improve mobile Linux or postmarketOS, but also Linux running on larger systems.

The project also welcomes contributions from non-developers, Weiss said, and there are many different ways to do so. If they are "tech curious", contributors could install postmarketOS on a phone; if they encounter any problems in doing so, they should point out where the documentation was inadequate—or propose updates to fix it. Translations for the user interfaces and apps would help as would getting the word out about the project in diverse spaces—including ones catering to non-technical people. Contributors can help out with the monthly blog posts, the podcast, which has languished a bit due to lack of time, or on applying for funding grants in different parts of the world.

Weiss took a few audience questions after completing his talk. One asked about a "golden device" that he would recommend for those who want to start working with postmarketOS. The OnePlus 6 and 6T were long recommended by the project, he said, but those are getting old enough they are hard to find in the used marketplace; the Pixel 3a is well-supported, as is the Fairphone 5. He suggested consulting the devices page on the wiki (linked above), which can help narrow things down based on which peripherals (camera, NFT, etc.) were most needed.

Interested readers may want to check out the YouTube video of the talk and Weiss's slides.

[ I would like to thank the Linux Foundation, LWN's travel sponsor, for assistance with traveling to Tokyo for Open Source Summit Japan. ]

Comments (32 posted)

Linux Kernel Runtime Guard reaches its 1.0 release

The Linux Kernel Runtime Guard (LKRG) is a out-of-tree loadable kernel module that attempts to detect and report violations of the kernel's internal invariants, such as might be caused by an in-progress security exploit or a rootkit. LKRG has been experimental since its initial release in 2018. In September 2025, the project announced the 1.0 version. With the promises of stability that version brings, users might want more information to decide whether to include it in their kernel.

Security through diversity

LKRG's mission is mildly hopeless — if an attacker has already compromised a running kernel, then there is no theoretical reason that the attacker could not also recognize and block or subvert the LKRG kernel module. In practice, however, an attacker would need to actually know to do that, and do it quickly enough to escape the periodic sweeps that LKRG performs. Using LKRG, therefore, raises the bar for attacks on the Linux kernel.

The project's documentation calls this "security through diversity" — if attackers haven't specifically addressed LKRG, running it can help spot attacks that would otherwise pass unnoticed. Of course, there is a cost to using LKRG as well, which makes the choice of whether to use it a tradeoff.

Running it

LKRG is packaged for Arch, Gentoo, NixOS, and Rocky Linux. On other systems, it can be built from source as a DKMS module. Once loaded, it provides a number of sysctl settings to control exactly what it should check for. LKRG supports two general categories of checks: those on the global state of the kernel and those on the state of a process.

The kernel has a lot of global state; right now, LKRG specifically checks a small but important subset of it. Specifically, it verifies whether code and read-only data within the kernel (including loaded modules) has the right permissions in the page table, whether the code or data has been modified (including whether the global SELinux settings have been tampered with), and some CPU-specific state registers. These last include the Supervisor Mode Execution Protection (SMEP) and Supervisor Mode Access Prevention (SMAP) settings, which prevent kernel code from accessing user-space memory. SMEP prevents the CPU from executing user-space code while in the kernel, and SMAP prevents other accesses. That protection must be disabled when the kernel is deliberately copying data to or from user-space memory, but the rest of the time it prevents an exploit in the kernel from making use of data stored in user space.

The SMEP and SMAP settings are trivial to check, but ensuring that the kernel's data has not been tampered with is more involved. When it first loads, LKRG takes a hash of all the data that should not change and stores this hash in its own memory. Then the data can be periodically re-hashed and checked against the stored hash. Some parts of the kernel can't be covered by the hash, because they do change during normal operation, but can still be checked for consistency against other parts of the kernel. For example, LKRG scans kernel memory to find loaded modules, and compares this to the kernel's own list of loaded modules. If a module is present but not in the list, it is probably trying to hide, and therefore likely to be nefarious. Each kernel module also gets its own hash calculated on first load and then checked periodically.

Kprobes also pose problems: they are dynamically inserted into otherwise-unchanging kernel code. LKRG can't ensure that these changes are correct using a hash, but it can check that the changes only show up when kprobes are actually enabled, and that the "enabled" setting is what it's expected to be.

By default, LKRG checks these global properties at 15-second intervals, but it can be configured to do so more or less frequently, or only when asked to do so. If it does find something suspicious, it can print a log message, restore the previous SELinux "enforcing" setting and SMEP/SMAP state, or panic the kernel.

The other set of checks it runs are per-process. These are also run periodically, but can additionally run just before any process attempts to do some privileged operation. LKRG verifies that the process has the credentials for the operation that it is about to perform, and that the stack pointer (and optionally the frames on the stack) make sense.

Each of these checks can be configured individually, but it is also possible to use LKRG "profiles". It offers several pre-configured levels of detection and enforcement. These range from "detect things only when requested" to "double-check everything whenever possible" (set via the lkrg.profile_validate sysctl) and from "when a violation is detected, log it" to "panic the kernel" (set via the lkrg.profile_enforce sysctl). The default is a reasonable middle ground: check kernel integrity without too much performance overhead, log for suspicious things, and crash the kernel when there's a definite problem.

Is it worth it?

On the one hand, LKRG comes with some admitted downsides: LKRG's default enforcement level is incompatible with running virtual machines VirtualBox; it works fine from within a virtual machine, but causes spurious alarms on the host when the guest CPU has a state that doesn't match the host. [An LKRG maintainer wrote in to point out that the problem exists only on VirtualBox, and that KVM works in the default configuration.] That's not a fundamental limitation of LKRG — it could be extended to track enters and exits from virtual machines, and adjust its expectations accordingly — but it does make it hard to recommend LKRG by default, given how many workloads involve some amount of virtualization.

There is also the matter of performance. A comparison of the Phoronix benchmark suite from 2020 with and without LKRG suggests that it introduced a performance overhead of about 4.4%. Since then, the LKRG developers have gotten this down to around 2.5%, but the error in the measurements makes it hard to be certain of the exact performance penalty, especially since it may be workload-dependent. There is also the fact that, as an out-of-tree module, it does not always support new kernel versions immediately. At the time of writing, it has been tested up through 6.17.0. [The project uses continuous integration testing, and keeps up to date with in-development kernels, but has somewhat infrequent releases.] The project's README suggests that the best use for LKRG would be systems that cannot always update to the latest kernel in response to disclosed vulnerabilities, for whatever reason.

Many users would welcome a small slowdown in the name of security, but there are easier ways to obtain some of the same benefits. For example, the kernel's built-in support for control flow integrity provides much the same benefit as LKRG's stack checking, through well-supported compiler and CPU features. The other things that it validates are less easily replaced, but the kernel has a long history of adopting new built-in ways to protect page-tables. There is always the possibility of bugs in the implementations of these ideas, but it's hard to judge whether LKRG would detect exploitation of those bugs reliably enough to be worthwhile.

On the other hand, the Singularity rootkit has plans to work around LKRG. The Metasploit framework has special code to deal with it and disable exploits that it would detect. It's hard to argue that LKRG is not worth using when it is demonstrably making attackers' lives harder.

Ultimately, LKRG has a relatively niche use case: when the performance overhead is acceptable, the computer's workload doesn't involve virtualization VirtualBox, and the system doesn't run bleeding-edge kernels [or the user is happy to run pre-release versions of LKRG], LKRG can provide defense in depth. Perhaps LKRG's data-validating features will one day make their way into the kernel proper; that does not currently seem to be a priority for the developers. Until LKRG's ideas do make their way into the kernel, users will have to make a judgment call about whether LKRG is right for their use case.

Comments (6 posted)

Who should vote in Fedora elections?

Creating fair governance models for open-source projects is not easy; defining criteria for participants to receive membership and voting rights is a particularly thorny problem for projects that have elections for representative bodies. The Fedora Council, the project's top-level governance body, is wrestling with that conundrum now. This was triggered by a Fedora special-interest group (SIG) granting temporary membership to at least one person for the sole purpose of allowing them to vote in the most recent Fedora Engineering Steering Council (FESCo) election. That opened a large can of worms about what it means to be a contributor and how contributors can be identified for voting purposes.

Fedora's elections are held after each release cycle; for example, the most recent was the FESCo Fedora 43 election, which ran from December 17, 2025 to January 7, 2026. It is not a high bar to be eligible to vote in a FESCo election. First, one must have membership in the "cla_done" group; that is available to anyone who has a Fedora account (which requires agreeing to the Code of Conduct) and has signed the Fedora Project Contributor Agreement. Secondly, a person must be a member of an additional group unrelated to the cla_done group. That could be membership in any of the eligible Fedora groups, and there are literally hundreds of Fedora groups that fit the bill. It is not difficult at all for someone who is active in the Fedora community in some fashion to gain membership to a group that would allow them to vote in an election.

Temporary SIG membership

Even though the barrier to voting was not high, some thought it should be lower still. During the Fedora 43 FESCo election cycle, Fedora user "JandeMus" asked why he was receiving an error when trying to vote:

It took me almost half an hour to find (some) groups but I have no idea which one to join. Why is this even necessary, can't I, with a FAS [Fedora Account System] account, not just vote? Joining a group means applying first, only for the sake of being able to vote. That's all I want to do.

Fedora does not make things easy.

Hristo Marinov replied with some links to the Fedora documentation about elections and groups. JandeMus replied that he would give up on the idea; he complained again that "Fedora makes things unnecessarily complicated" and that it was a shame that he, and others like him, who would not join a group for voting purposes, were prevented from participating in Fedora elections.

"MatH" replied on January 5 that the group membership requirement was merely "to keep the trolls out", and offered to grant JandeMus membership in the Fedora Join SIG so that he could vote. The Join SIG is a group that is involved in various efforts to welcome new people to Fedora. Marinov questioned whether that was a good or recommended practice, but MatH replied that JandeMus had been on Fedora's Discourse forum for more than two years and had made more than 400 posts, which had garnered lots of likes. "If that is the kind of person we want to keep out of Fedora, then please do let me know. [...] If some AI bots come and ask for membership I'll be sure to turn them down."

Christopher Klooz, one of the forum's moderators, joined in the discussion and objected to the idea that Fedora's group requirements were only to keep trolls out of the elections. He argued that the need to be sponsored into a group besides FAS was intentional and designed to ensure people voting for FESCo had the requisite understanding of the project. After further discussion, he said that MatH had abused his privileges in providing temporary group membership, and that he would report it if it happened again. JandeMus said that he was sorry he had asked the question and apologized "for all the commotion I caused". He also asked that the discussion thread be closed. Klooz responded that JandeMus had brought up a matter that needed attention, and he closed the topic thread.

On January 7, Klooz opened a ticket with the Fedora Council. In the ticket, he reported that MatH's position was shared by others in the Join SIG, and it was unclear how actively it would be used in the future. He asked the council to provide clear guidance on whether this was allowed or if the voting rules should be changed in some fashion.

Discussion

A discussion thread was opened on Discourse to allow the Fedora community to provide its thoughts on the matter. Join SIG member Ankur Sinha weighed in to say that he was uncomfortable with how the discussion about the SIG's actions had been handled to date. The Join SIG gave voting rights to people for contributions, such as editing the Fedora wiki, "because large parts of Fedora contribution do not rely on FAS groups". He felt that it would be putting process over people to bar the Join SIG from granting temporary memberships for voting.

Fedora Project Leader Jef Spaleta asked what the process is for establishing an entity with a FAS group. Could he simply create a Fedora Curling SIG that would allow creating a FAS group with voting rights? He said that he had deep concerns about tying voting rights to arbitrary FAS groups:

How many SIGs right now are in fact just ghost fas groups and are effectively non-functional with no attachment points for onboarding new contributors?

If you want to tie voting strongly with SIGs, then we need to have SIGs being functional self-governing groups.. and not the laissez-faire structures as they exist right now.

Eduard Lucena responded that the process to create a SIG is lightweight, but creating a FAS group requires opening a ticket with Fedora's infrastructure and release engineering team. "So basically is at discretion of the infra team what FAS groups are created". Kevin Fenzi, Fedora's infrastructure lead, said that there had been much discussion in the council and Fedora's community operations group to try to figure out what makes someone a contributor. The council has a goal of increasing the number of contributors, so it seemed like a good thing to come up with a commonly agreed criteria—especially if it could be expressed in a way that Fedora's election software could use to determine eligibility.

Spaleta said he would go farther than that; it seemed to him that it was not just a good thing, but a necessary thing to have such a criteria. He suggested it might be possible to use Fedora's badge system, which provides a leaderboard system for tracking contributions to various parts of Fedora, as a proxy for eligibility. Fenzi and Spaleta traded a few ideas, but no concrete proposal has been formed.

"Too complex"

Fedora Council member Justin Wheeler reported that the topic had been "the centerpiece of a significant debate" during the council's meeting on January 14. He said that there was tension between being inclusive and the potential that an election could be gamed via temporary memberships:

We discussed the difficulty of defining "active contributors" and noted that "legacy" voting rights for inactive members are just as problematic as new temporary ones. The Council determined this topic is too complex to resolve in a single meeting and requires detailed asynchronous discussion here on Fedora Discussion rather than waiting for the 2026 Strategy Summit.

The summit that Wheeler referred to is taking place in Tirana, Albania starting on February 3 and running through February 5; the participants are primarily council members and leads of council initiatives.

It is not clear how many people were granted voting rights by the Join SIG, but there was no sign of a major influx of new voters for the Fedora F43 FESCo election cycle. In fact, there were fewer voters than the previous cycle. According to the results page a total of 214 voters participated. There were 231 voters for the Fedora 42 election, only 166 for the Fedora 41 cycle, and 236 for Fedora 40. A spot check of the results going back to June 2008, shows that FESCo elections tend to draw a range somewhere between about 150 and 300 voters.

To put that in perspective, the FAS group for Fedora packagers has more than 1,400 members. That group is worth noting for two reasons. One, there is actually a fairly high bar for joining the packager group. Secondly, because there is a process for removing people from the packager group if they have been inactive for 12 months. The Fedora infrastructure team removes inactive packagers each release cycle, unlike other FAS groups that might have members who have not been active for years. Not to mention FAS groups that are, themselves, no longer related to active SIGs or working groups within Fedora. One might conclude that, at the moment, the problem is not that a few people are sneaking into the ballot booth for Fedora elections; it's that too few Fedora participants bother to vote at all.

For now, the discussion continues. Nailing down the right criteria to define contributors with the right to vote, and having a fair system in place to ensure they are able to do so, will not be easy.

Comments (5 posted)

Filesystem medley: EROFS, NTFS, and XFS

Filesystems seem to be one of those many areas where the problems are well understood, but there is always somebody working toward a better solution. As a result, filesystem development in the Linux kernel continues at a fast pace even after all these years. In recent news, the EROFS filesystem is on the path to gain a useful page-cache-sharing feature, there is a new NTFS implementation on the horizon, and XFS may be about to get an infrastructure for self healing.

EROFS page-cache sharing

The Enhanced Read-Only Filesystem (EROFS) is, as its name would suggest, a filesystem for read-only data. EROFS supports an assortment of advanced filesystem features and provides high-performance data compression. It can be found in a number of settings, perhaps most notably on Android devices. EROFS was merged for the 5.4 kernel release in 2019 and has been steadily developed since then.

One common use case for EROFS is as a base layer for container images. As such, an EROFS filesystem can often be mounted many times on a given machine, and there can be multiple variants of a given filesystem with minor differences. Several images may contain different application mixes, but all have the same C library, for example. While EROFS will deduplicate data within a given filesystem at creation time, it cannot do that for independently created filesystems. Duplication of files across filesystems can result in multiple open files containing the same data in the system as a whole, wasting a lot of memory as that data is placed in the page cache. It would be nice to find a way to deduplicate that page-cache data and eliminate that memory waste.

In early 2025, Hongzhen Luo posted a patch series implementing page-cache sharing for EROFS. Later on, Hongbo Li picked up and continued that work; the most recent posting is version 18. It works by assigning a "fingerprint" to each file within a filesystem at creation time; that fingerprint is later used to detect files containing the same data.

Specifically, each file within an EROFS filesystem is given an extended attribute; the name of that attribute can be set by the creator, but trusted.erofs.fingerprint appears to be standard usage. The actual contents of the fingerprint are not defined. A logical value to use would be a cryptographic hash of the file's contents but, as Gao Xiang said, it could simply be an integer value assigned by the image creator. It should, of course, be different for two files if they have different content.

The fingerprint is used if the filesystem is mounted with the inode_share option. Another option, domain_id, is used to separate users; only files mounted within the same domain ID will be deduplicated. In the absence of this restriction, an attacker who is able to mount an EROFS filesystem could attach arbitrary fingerprints to files with malicious content, possibly causing other users to obtain that content rather than the files they were hoping for.

When a fingerprinted file is opened, the EROFS code creates an internal inode that references the data within that file. The file opened by user space is redirected (within the kernel) to that deduplication inode, which is associated with the fingerprint internally. Subsequent opens of files with the same fingerprint will simply take a reference to the deduplication inode, and read operations will be redirected to that inode's backing store. As a result, all of the files will share the same folios in the page cache, eliminating the duplicate memory usage.

In the current implementation, at least, use of fingerprints is incompatible with direct I/O.

Depending on the workloads running within a system, page-cache sharing can reduce memory usage by as much as nearly half in the best cases. So the attractiveness of the idea is not entirely surprising. The patch series has evolved considerably over time and a lot of review concerns have been addressed, but some still remain. Christoph Hellwig, for example, is concerned about the security implications of this feature, and will likely need some convincing before coming around. So, while 18 revisions is quite a few, there will likely be more yet before this feature is merged.

A new NTFS

NTFS is the standard Windows filesystem format. One would think that Linux, which has long offered interoperability with as many systems as possible, would have a good NTFS implementation, but that has never really come to pass. For years, the kernel was limited to read-only support for NTFS. Linux users wanting full access to NTFS filesystems had to make do with ntfs-3g, running under FUSE in user space.

That situation appeared to change with the arrival of ntfs3 in 2021. This implementation offered full NTFS access and appeared to be what the community was waiting for, despite worries expressed at the time that the level of support behind this system was less than might be desired. It was merged for the 5.15 release, and the read-only NTFS filesystem was removed from the kernel for the 6.9 release in 2024.

The worries about the maintenance of ntfs3 have, to an extent, been realized since its merging. The ntfs3 code, having been freshly merged in 5.15, would normally be expected to see a fair rate of change in subsequent releases as bugs are fixed and more features added. In this case, though, there were exactly four ntfs3 patches in 5.16, only one in 5.17, five in 5.18, and so on. The pace picked up somewhat around 6.0, but has dropped off more recently. There have been 67 commits to fs/ntfs3 in the last year. Many of those changes are not by its maintainer (Konstantin Komarov), but by other developers fixing up ntfs3 for changes elsewhere in the tree.

In October 2025, Namjae Jeon surfaced with an alternative NTFS implementation called "ntfsplus"; the filesystem has since just been rebranded "ntfs", matching the old, read-only implementation. The work, Jeon said, was motivated by the fact that "ntfs3 still has many problems and is poorly maintained"; ntfs is meant to be a better-maintained and more functional NTFS implementation. The fifth revision of this patch set was posted on January 11.

The series begins by reverting the removal of the read-only NTFS filesystem. This code, Jeon says, "is much cleaner, with extensive comments, offers readability that makes understanding NTFS easier", so it makes a better base for ongoing development than the ntfs3 code base. From there, the series adds the code needed to turn the filesystem into a proper, read/write implementation of NTFS. Toward the end, the series removes the compatibility code that ntfs3 uses to emulate the old, read-only implementation.

There are, Jeon says, a number of advantages to this version. It uses the iomap layer to interface with the memory-management and block subsystems; ntfs3, instead, still uses the older buffer-head interface. (It should be said that there is a patch from Komorov adding iomap support to ntfs3 currently sitting in linux-next). There is an associated project adding a filesystem checker. Jeon claims that this filesystem passes many more fstests tests than ntfs3. A set of benchmarks shows better performance than ntfs3, as much as 110% better for some workloads.

The patch set has evolved quickly over the short time it has existed on the lists, and a lot of review comments have been addressed. The new filesystem appears to be mostly feature-complete, with one notable exception: it does not yet support journaling. The ntfs3 implementation is able to play back an existing journal (though Jeon says that this feature "in our testing did not function correctly"). The next step, according to the cover letter, will be full journaling support.

The kernel community is normally unenthusiastic about multiple implementations of the same functionality, and would rather see incremental improvements than wholesale replacements; that places a sizable obstacle in Jeon's path. Even so, the new code appears to be winning over the filesystem developers; there are a lot of review comments, still, but they are aimed at improving the code rather than questioning its existence. To replace the NTFS implementation again would be a big step, but it appears to not be an inconceivable one.

XFS self healing

The XFS filesystem tends to be associated with big systems; it is designed to scale to massive files in great number, on systems with a lot of CPUs. The organizations that run this type of system also tend to be concerned about reliability and data integrity. A lot of work has been done on XFS in those areas in recent times; one piece of that is the XFS autonomous self-healing infrastructure from Darrick Wong.

The kernel-based infrastructure will not, on its own, heal a problematic filesystem. It is primarily a reporting mechanism giving a user-space daemon the ability to learn about problems; the decision on how to respond to any particular problem can then be made in user space. That daemon might respond by killing and restarting a container, initiating some sort of scrub operation, or trying to correct an error in some other way. Regardless, this is a complex operation with policy decisions that are best made outside the kernel.

The series adds a new ioctl() operation, XFS_IOC_HEALTH_MONITOR, which will return a file descriptor to a suitably privileged process. Whenever an event of note takes place, a structure will be written to that file descriptor for user space to read and react to. For the curious, this patch starts with a justification of the decision to output events as binary C structures rather than using some sort of higher-level protocol encoding.

There is a wide variety of events that can be reported, starting with an "unmount" event; it indicates that the filesystem is no longer mounted, and no further events will be produced. There is a set of events for reporting metadata corruption, distinguishing between "sick" metadata (with corruption noted at run time) and "corrupt" metadata (detected by the filesystem checker). Other events report media and I/O errors. The series also adds an ioctl() operation to check whether a given file descriptor refers to a file on the filesystem that is being monitored; it can give the user-space daemon confidence that it is, indeed, operating on the right filesystem.

On the user-space side, Wong has a repository that includes an xfs_healer program designed to use the new interface. For the curious (and troff-capable), there is a minimal man page describing this utility; we have created a rendered version of that page that some may find a little easier on the eyes.

The kernel series is in its sixth revision, and it would appear to be stabilizing. It may find its way into the mainline in the near future. It might take a bit longer, though, to develop the user-space code to the point that administrators will trust it to operate on a live, production filesystem.

Comments (65 posted)

Fedora and GPG 2.5

The GNU Privacy Guard (GPG) project decided to break from the OpenPGP standard for email encryption in 2023, and instead adopted its own homegrown LibrePGP specification. The GPG 2.4 branch, the last one to adhere to OpenPGP, will be reaching the end of life in mid-2026. The Fedora project is currently having a discussion about how that affects the distribution, its users, and what to offer once 2.4 is no longer receiving updates.

gpg.fail

The Fedora discussion was prompted by the gpg.fail disclosures in late December. Christian Stadelmann learned about the GPG flaws reported by the gpg.fail researchers and noticed that Fedora's gnupg2 package had not received any updates to fix those flaws. In fact, the package had not been updated since July 2025. He asked about this on the Fedora devel mailing list and pointed to a bug that lists all the 2.5.x updates that have not been packaged for Fedora. "Is it possible that gnupg is unmaintained? This would pose a high security risk to the Fedora project."

The package was not unmaintained. Red Hat, which employs the GPG package's maintainer Jakub Jelen, has an annual shutdown at the end of the year during the holidays. Like many other Red Hat employees, Jelen was taking some time off. In addition, he explained on January 2, he was keeping the package on GPG's "oldstable" 2.4 branch deliberately. That was why the package had not seen updates in several months—its upstream had not pushed out any new 2.4 releases.

The 2.5 branch was originally an experimental branch that implemented LibrePGP. Jelen noted that LibrePGP is not compatible with anything else and would likely result in users shooting themselves in the feet. "Updating to 2.5 would result in new users generating incompatible LibrePGP keys, which I do not think is a good idea to do now for all Fedora users." Since that is undesirable, he had been staying on 2.4 and syncing with the FreePG repository that contains a set of patches that is maintained for downstream projects, such as Debian and Fedora. The gnupg2 package for Fedora 43 includes a number of patches from FreePG, though it does not include the patch for LibrePGP format support.

Jelen said that he hoped to have a better solution by the time that GPG 2.4 reaches the end of its life, "but I can not anticipate what it is going to be". Meanwhile, he also noted that he was pushing out a package with the 2.4 fixes from the GPG project.

Michael Gruber pointed out that some of the fixes were available since October on the 2.4 branch (but not yet in a release). Jelen said that he was unaware of that, because he tracked the CVE database and there had not been CVEs issued. He said it made him question, again, whether Fedora should invest time and resources into an upstream that is not disclosing vulnerabilities clearly.

The communication around 2.5 has been somewhat muddy in general. Until recently, the 2.5.x series was understood to be a development branch and the next stable version of GPG would be 2.6. In June 2025, Koch said in the announcement for GPG 2.5.8 that the release was "another one in a series of public testing releases leading to a new stable version 2.6". The 2.5.9 release in July had similar language. Versions 2.5.10 and 2.5.11 did not have release announcements sent to the GPG announcement list. In September, Koch announced 2.5.12 and said that the 2.5 series is fully supported and ready for production use. He also sent out an announcement for GPG 2.4.8 several months after it was actually released in August 2025; that announcement noted that 2.4 would reach end of life in June 2026. The news that 2.4's end of life was approaching was not quite in the bottom of a locked filing cabinet in a disused lavatory, but it would not be surprising if a number of users and packagers missed the message all the same.

Alternatives

In the discussion that followed, Clemens Lang pointed out that Red Hat was standardizing on the Sequoia project, and that Red Hat Enterprise Linux (RHEL) 10 had dropped GPG's libgcrypt as a core cryptography library. "RHEL 10 already contains RPM signing keys that cannot be understood by GnuPG." Gruber said that he was all for replacing GPG with something better, but complained that Red Hat was choosing key types that would force Sequoia adoption. He did not, however, offer a preferred replacement for GPG.

Neal Gompa said that Red Hat was not trying to drive Sequoia adoption with incompatible key types; it had chosen post-quantum cryptography (PQC) algorithms, and GPG simply does not support PQC. Jelen agreed and said that Red Hat was not forcing Sequoia adoption, and that any implementation of the OpenPGP standard should do. "We just want to stay on the 'standard' side".

Sequoia developer Neal H. Walfield said that there were two ways to replace GPG with Sequoia: convince application developers to switch, and finish the work-in-progress drop-in replacement (dubbed "Chameleon") for GPG that uses Sequoia.

The chameleon has the advantage that instead of forcing an application to switch to Sequoia, we give the user the freedom to choose what implementation they prefer. The disadvantage of the chameleon is that it has to be compatible with gpg, which means that we can't change the interface or the workflows. Further, since gpg's interface is quite complex with lots of subtle interactions, writing a perfect drop-in replacement will take a long time. So far we've implemented the functionality that some specific applications use. That has worked pretty well and I'd estimate that we have >90% coverage. But a lot more work is needed to cover the long tail of commands and their many options.

Petr Menšík brought up the %{gpgverify} macro used for Fedora RPM packaging. Fedora's packaging guidelines specify that packagers must use %{gpgverify} to perform verification of a source file's signature at the start of the build process for an RPM. Currently, that macro expands to an invocation of the gpgv signature tool. Gruber responded that the guidelines did not require that the macro call gpgv specifically, though; it could be changed to use Sequoia's sqv instead.

So far, no decision has been reached, though it would seem that the consensus is largely in favor of replacing GPG with Sequoia in instances where Fedora tooling requires signing and verification of OpenPGP signatures.

What to provide to users is still on the table. Arch Linux, Debian, Ubuntu, and other major distributions are still providing 2.4.x versions of GPG. Fedora tries to avoid deviating from upstream most of the time, but providing a version of GPG that does not implement the OpenPGP standard may not serve users well. The clock is ticking; the Fedora 44 release that is expected in April will likely ship with GPG 2.4.9, but a decision for Fedora 45 will need to be reached soon.

Comments (22 posted)

Implicit arguments for BPF kfuncs

The kernel's "kfunc" mechanism is a way of exporting kernel functions so that they can be called directly from BPF programs. There are over 300 kfuncs in current kernels, ranging in functionality from string processing (bpf_strnlen()) to custom schedulers (scx_bpf_kick_cpu()) and beyond. Sometimes these kfuncs need access to context information that is not directly available to BPF programs, and which thus cannot be passed in as arguments. The implicit arguments patch set from Ihor Solodrai is the latest attempt to solve this problem.

The BPF subsystem maintains a collection of context information in the massive bpf_prog_aux structure. Over the years it has grown to contain just about any sort of information that may be needed, including a list of maps the program is using, how long since it was loaded, the current stack depth, and dozens of other fields. It can almost be thought of as an analog to the self object passed to methods in some object-oriented languages. A number of kfuncs need to make use of this structure, but it is not something that BPF programs themselves can access.

The way this problem has been handled in the BPF subsystem so far — the __prog annotation — cannot be said to be overly elegant. Consider bpf_stream_vprintk(), which is defined as a macro:

    #define bpf_stream_printk(stream_id, fmt, args...)		\
    ({								\
	static const char ___fmt[] = fmt;			\
	unsigned long long ___param[___bpf_narg(args)];		\
								\
	___bpf_fill(___param, args);				\
	bpf_stream_vprintk_impl(stream_id, ___fmt, ___param,	\
		sizeof(___param), NULL);			\
    })

This macro marshals together the passed-in args and, in turn, calls bpf_stream_vprintk_impl, defined as:

    __bpf_kfunc int bpf_stream_vprintk_impl(int stream_id, const char *fmt__str, 
    					    const void *args, u32 len__sz,
					    void *aux__prog);

The BPF verifier sees the trailing __prog in the name of the final parameter and magically replaces whatever was passed (the NULL in the macro above) with a pointer to the bpf_prog_aux structure. It gets the job done, but what is going on here is not exactly obvious to the casual observer.

Solodrai first attempted to generalize this mechanism in late 2025 with a "magic function" abstraction. In this implementation, the __prog suffix was switched to __magic, and the in-kernel version of the API (the one with the __impl suffix) was generated automatically. The concept was well received, but there were a lot of comments on the implementation; as a result, the patches have evolved somewhat since then.

As the work stands now, a kfunc needing implicit arguments is declared normally, as it is expected to be invoked within the kernel. So, with the patch set applied, bpf_stream_vprintk() is declared as:

    __bpf_kfunc int bpf_stream_vprintk(int stream_id, const char *fmt__str, 
				       const void *args, u32 len__sz,
				       struct bpf_prog_aux *aux);

The pointer to the bpf_prog_aux structure is now without any magic adornments. Instead, when it comes time to inform the BPF subsystem about this kfunc, the declaration looks like:

    BTF_ID_FLAGS(func, bpf_stream_vprintk, KF_IMPLICIT_ARGS)

The KF_IMPLICIT_ARGS flag indicates that this function has an implicit argument that is not visible to BPF programs.

When the kernel loads a BPF program containing a call to one or more kfuncs, the verifier looks up those kfuncs in the BPF type format (BTF) data created when the kernel is built. That data describes the kfunc and lets the verifier ensure that the BPF program is calling it correctly. The patch series causes the kernel-build process to modify the generated BTF for KF_IMPLICIT_ARGS kfuncs in a couple of ways:

• An entry for a new function, using the name of the kfunc with _impl appended, is added to the BTF data. This entry has the prototype for the kfunc as declared, and links to that function within the kernel.
• The entry for the kfunc using the original name (without "__impl") is modified so that the last argument is omitted. That is the BTF data that is used to match kfuncs for BPF programs.

Whenever a call to a kfunc with implicit arguments is processed by the verifier, it modifies the call to add the bpf_prog_aux structure pointer as the final argument, essentially mapping the second form of the call above onto the first. So, in the end, the BPF program can access the kfunc with the expected API, with no need for wrapper macros to hide the work being done behind the scenes, and the kernel function can be declared under the name by which it will actually be used.

KF_IMPLICIT_ARGS is plural, implying that there could be more than one argument but, for the time being, the only thing that works is a struct bpf_prog_aux pointer as the final argument. Should that eventually prove to be insufficient, Solodrai said, "it is simple to extend this to more types". Given the apparent practice of shoving everything that might be needed into struct bpf_prog_aux, though, there may never be a need for anything else.

In the end, one could argue that the bpf_prog_aux pointer should have been implicitly passed to all kfuncs from the beginning. As it stands, this feature will make it easily and transparently available to the kfuncs that truly need it. There are already other patches (such as the sched_ext sub-scheduler series) that depend on this feature, so it seems likely to land in the mainline in the relatively near future.

Comments (none posted)