LWN.net Weekly Edition for March 20, 2025

Welcome to the LWN.net Weekly Edition for March 20, 2025

This edition contains the following feature content:

• Oxidizing Ubuntu: adopting Rust utilities by default: a tool that minimizes the risk of swapping out a set of core utilities.
• Better CPU vulnerability mitigation configuration: deciding which Spectre mitigations to build into a kernel is not a simple task; perhaps a more intuitive set of configuration options would help.
• Warming up to frozen pages for networking: a core memory-management transition runs into a bit of a snag.
• Looking forward to mapcount madness 2025: forward progress on one of the most challenging problems in the kernel's memory-management subsystem.
• The burden of knowledge: dealing with open-source risks: a FOSS Backstage talk on evaluating the risk associated with development projects.
• A look at /e/OS on tablet hardware: a tablet that is decoupled from most cloud services.

This week's edition also includes these inner pages:

• Brief items: Brief news items from throughout the community.
• Announcements: Newsletters, conferences, security updates, patches, and more.

Please enjoy this week's edition, and, as always, thank you for supporting LWN.net.

Comments (none posted)

Oxidizing Ubuntu: adopting Rust utilities by default

If all goes according to plan, the Ubuntu project will soon be replacing many of the traditional GNU utilities with implementations written in Rust, such as those created by the uutils project, which we covered in February. Wholesale replacement of core utilities at the heart of a Linux distribution is no small matter, which is why Canonical's VP of engineering, Jon Seager, has released oxidizr. It is a command-line utility that helps users easily enable or disable the Rust-based utilities to test their suitability. Seager is calling for help with testing and for users to provide feedback with their experiences ahead of a possible switch for Ubuntu 25.10, an interim release scheduled for October 2025. So far, responses from the Ubuntu community seem positive if slightly skeptical of such a major change.

Next 20 years of Ubuntu

Ubuntu celebrated 20 years since its first release in 2024 last year. Seager reflected on that milestone and published his vision for the next 20 years of Ubuntu in February. One of his themes for the future is modernization, calling on the project to constantly assess the foundations of the distribution against the needs of its users:

We should look deeply at the tools we ship with Ubuntu by default - selecting for tools that have resilience, performance and maintainability at their core. There are countless examples in the open source community of tools being re-engineered, and re-imagined using tools and practices that have only relatively recently become available. Some of my personal favourites include command-line utilities such as eza, bat, and helix, the new ghostty terminal emulator, and more foundational projects such as the uutils rewrite of coreutils in Rust. Each of these projects are at varying levels of maturity, but have demonstrated a vision for a more modern Unix-like experience that emphasizes resilience, performance and usability.

On March 12, Seager published a a follow-up to introduce his plan to start adopting some of the tools as defaults—with an eye to having them in place for the next Ubuntu long-term support (LTS) release, 26.04. The rationale for the switch is primarily "the enhanced resilience and safety that is more easily achieved with Rust ports". He cited a blog post by Rust core developer Niko Matsakis. The post, in a nutshell, is about Matsakis's vision for using Rust to write (or rewrite) foundational software; that is, "the software that underlies everything else".

Those who have been following the continuing debates and discussions about using Rust will find familiar themes in Matsakis's arguments in its favor: Rust provides the performance of C/C++ without demanding perfection from developers, it provides reliability, and it makes developers more productive regardless of experience level. Its reliability makes it particularly suitable for foundational software because "when foundations fail, everything on top fails also". Given Ubuntu's widespread adoption, Seager wrote, "it behooves us to be absolutely certain we're shipping the most resilient and trustworthy software we can".

Seager also thinks that embracing Rust will help meet another of his goals for Ubuntu, increasing the number of contributors. Not because Rust is necessarily easier to use than C, but because it provides a framework that makes it harder for contributors to commit potentially unsafe code. Presumably, though it was unsaid, that would make Rust a more attractive language for those interested in contributing but not interested in programming in C for whatever reason.

oxidizr

The abstract possibility that Rust utilities would be better, or even feasible, for Ubuntu is no substitute for hands-on experience. To that end, Seager created oxidizr as a way to quickly swap in (and out) Rust utilities in place of the traditional counterparts with relatively low risk. He released the first version, 1.0.0, on March 7. It is available under the Apache 2.0 license and, as one might expect, written in Rust.

The project is not yet packaged for Ubuntu, nor does Seager have a personal package archive (PPA) set up for users to install oxidizr with APT. There are binary releases on GitHub, or users can install the tool using cargo:

    $ cargo install --git https://github.com/jnsgruk/oxidizr

The binary releases may be the easiest way to get started, as oxidizr requires the exists function in the fs module, but exists was added in Rust 1.81.0 (released in September 2024) while the Rust version in Ubuntu 24.10 is still at 1.80.1. I used rustup to install the most recent stable version of Rust, and then used cargo to install oxidizr.

The oxidizr utility calls a set of utilities that can be independently replaced an "experiment". Experiments are Rust modules that define the packages to be installed (or removed) and handle renaming of the utilities to enable or disable use of the Rust versions. The current set of experiments include replacing GNU coreutils, findutils, or diffutils with the uutils coreutils, findutils, or diffutils, as well as replacing traditional sudo with the Rust-based sudo-rs.

For instance, to try out sudo-rs a user would run this command:

    # oxidizr enable --experiments sudo-rs

That will install the sudo-rs package from the Ubuntu package repository, back up the sudo binary, and create a /usr/bin/sudo symbolic link that targets the Rust binary (/usr/lib/cargo/bin/sudo). To enable all experiments, a user would use the all target instead:

    # oxidizr enable --experiments --all

Finally, to revert the system to the traditional utilities and remove the replacement packages from the system:

    # oxidizr disable --all

According to Seager, oxidizr works on all versions of Ubuntu after 24.04 LTS, but the uutils diffutils experiment is only supported on Ubuntu 24.10 or later. He did urge users to start testing on a virtual machine or other machine that is not their production workstation or server for safety's sake. Seager reported that he hasn't had many problems, but he has run into one incompatibility: the uutils cp, mv, and ls replacements don't support the -Z flag yet, which is used to set the SELinux context of a file or (in the case of ls) print a file's security context.

In my brief testing, I did not run into any problems with the uutils versions of the utilities or the changes oxidizr made to the system in order to swap them in. However, I did note that oxidizr does not make any changes to the system's man pages. Even when the GNU utilities have been replaced with the uutils versions, the GNU man pages are left in place, so "man cp" still displays the GNU version. It would be good to switch the man pages too in order to expose users to any gaps in the uutils documentation as well as the utilities themselves.

Reactions

Fern Dziadulewicz asked if the move toward uutils meant that "Ubuntu is actually kind of heading towards GNUlessness", as with some other Linux distributions that shy away from GNU components. Seager responded that people should not read too much into the change:

This is not symbolic of any pointed move away from GNU components - it's literally just about replacing coreutils with a more modern equivalent.

Sure, the license is different, and it's a consideration, but it's by no means a driver in the decision making.

That response did not satisfy Joseph Erdosy, who wrote that he would migrate to Fedora or Rocky Linux if Ubuntu goes through with the change. He said that he liked Rust and the idea of better, memory-safe alternatives, but that he was unhappy that the biggest "oxidized" project was an MIT-licensed rewrite of GPL-licensed code.

This decision seems to align with a broader trend of companies deprecating GPL software in favour of more permissively licensed alternatives, often under the guise of "modernization." However, the real-world impact is clear: free software is increasingly co-opted into proprietary ecosystems, weakening the principles that made Linux successful.

A few other users quickly agreed with Erdosy, then Ian Weisser announced that he was putting the topic into "Slow Mode" to "prevent piling-on until the developers have a chance to respond and keep this topic constructive". Shortly after, Seager responded that he did not agree that this potential move posed a threat to Ubuntu, or its community. He reiterated that it was not indicative of a political agenda or wider move away from GPL'ed software, and said that most of Canonical's own software is and would continue to be GPL'ed.

Ubuntu is a collection of software that we curate to build a distribution. It's a project dedicated to shipping the latest, and best open source we can find. There is no evidence of foul play, bad practice or poor intentions from the uutils maintainers - they're a thoughtful, dedicated community who are building their own software, and even contributing back to GNU coreutils in some cases. They are achieving things I think we should aspire to with Ubuntu in the coming years, and I remain committed to giving this a chance at success - noting that we and others will need to work closely with them to resolve issues with locales, selinux support and other issues.

If the current situation changes and we believe that the interests of the uutils project are no longer aligned with those of Ubuntu, we can change the coreutils package we choose to ship with Ubuntu.

Sergey Davidoff wondered why the Debian alternatives system, which is used to designate default applications when multiple programs with the same function are installed, was not sufficient for experimenting with Rust utilities. Julian Andres Klode replied that the alternatives system would not be suitable because the existing package would need to cooperate. He also responded to another user, "rain", who had floated the idea of allowing users to switch out individual commands. Klode said that it was a bad idea to allow users to select between Rust and non-Rust implementations on a per-command level, as it would make the resulting systems hard to support.

Liam Proven asked about support on versions of Ubuntu for architectures other than x86_64 and Arm, such as s390 and ppc64le, since "the LLVM Rust toolchain is still a little immature and code generation for other architectures is lacking". Uutils project founder and Ubuntu developer Sylvestre Ledru asked if Proven had any bug reports to share, since Firefox had been using the LLVM Rust toolchain to ship Firefox on those architectures for years. He pointed out that uutils had been successfully building on Debian and Ubuntu with those architectures as targets for a few years as well.

Next steps

Seager said that he had met with Ledru to discuss the idea of making uutils coreutils the default in Ubuntu 25.10, and Ledru felt that the project was ready for that level of exposure. Now it is just a matter of specifics, he said, and the Ubuntu Foundations team is already working up a plan to implement this in the next release cycle. He did acknowledge that there was a need for caution and was open to the possibility that he would need to "scale back on the ambition" if making the switch meant compromising stability or reliability in an Ubuntu LTS release. If the switch doesn't work out, it should be easy enough to revert in time for next year's LTS release.

To date, Ubuntu seems to be the first major Linux distribution that has seriously considered a switch to uutils. If Ubuntu 25.10 ships with uutils coreutils, it will be a significant win for the uutils project that grants exposure to a much larger user base than it has enjoyed so far. The "oxidize Ubuntu" experiment has the potential to accelerate Rust's adoption and inspire further attempts to replace C-based utilities with Rust, or it might have a chilling effect if Ubuntu runs into serious problems. Either way, the project should be instructive for the larger community.

Comments (126 posted)

Better CPU vulnerability mitigation configuration

Modern CPUs all have multiple hardware vulnerabilities that the kernel needs to mitigate; the 6.13 kernel has workarounds for 14 security-sensitive CPU bugs just on x86_64. Several of those have multiple variants, or multiple mitigations that apply on different microarchitectures. There are different kernel command-line options for each of these mitigations, which leads to a confusing situation for users trying to figure out how to configure their systems. David Kaplan recently posted a patch set that adds a single, unified command-line option for controlling mitigations and simplifies the logic for detecting, configuring, and applying them as well. If it is merged, the patch set could make it much easier for users to navigate the complicated web of CPU vulnerabilities and their mitigations.

The kernel provides information about the vulnerabilities it's aware of via sysfs. The /sys/devices/system/cpu/vulnerabilities/ directory contains a file for each known class of attack, with information about whether the system is vulnerable, and which mitigations have been enabled. Some of these are easy to interpret, and merely state whether a mitigation is turned on or off. Others are somewhat cryptic. For example, my x86_64 machine has the following to say about Spectre v2:

    Mitigation: Enhanced / Automatic IBRS; IBPB: conditional; RSB filling; \
        PBRSB-eIBRS: SW sequence; BHI: BHI_DIS_S

This line means that Enhanced Indirect Branch Restricted Speculation (Enhanced IBRS), Indirect Branch Predictor Barrier (IBPB), Return Stack Buffer (RSB) filling, Post-barrier Return Stack Buffer Enhanced IBRS (PBRSB-eIBRS), and Branch History Injection (BHI) are all enabled by the kernel. Enhanced IBRS is a technique that prevents the CPU from speculating through indirect branches. IBPB introduces a barrier to prevent code from before the barrier from influencing branch predictions after the barrier. RSB filling makes sure that any return addresses the CPU might speculate about when encountering a return instruction are overwritten on context switch. PBRSB-eIBRS extends that protection to cover exits from virtual machines on some CPUs. Finally, BHI protects against code that manipulates the branch predictor in order to speculate through a branch. Theoretically, all of those mitigations combined should render my computer mostly immune to Spectre-based attacks.

Each of those mitigations is controlled with one of the spectre_v2=, spectre_v2_user=, and spectre_bhi= kernel command-line options. The other mitigations have their own idiosyncratically named options. The kernel's documentation on hardware vulnerabilities describes the various options, but knowing what the best configuration for a given system is requires knowledge about how the different vulnerabilities can be abused. In practice, most users probably leave the settings on their defaults, which is good for security but comes with a noticeable performance penalty.

Kaplan's patch restructures the kernel's mitigation-configuration logic to depend on a single command-line option, mitigations=. That option already exists, but can currently only be set to "off" or "auto". Kaplan's patch extends it so that the user can append a semicolon and a comma-separated list of specific attack vectors not to bother mitigating:

• no_user_kernel — don't mitigate attacks from malicious user-space programs attempting to leak kernel data.
• no_user_user — don't mitigate attacks from one user-space program on another.
• no_guest_host — don't mitigate attacks from a malicious virtual machine trying to exfiltrate data from its hypervisor.
• no_guest_guest — don't mitigate attacks from one virtual machine on another.
• no_cross_thread — don't mitigate attacks from a program or virtual machine running on one simultaneous multithreading (SMT) core against code running on its sibling core.

The default is to mitigate all of these attack vectors, so there's no separate setting for "on". The exception is cross-thread attacks, which can generally only be completely prevented by disabling simultaneous multithreading. That's usually not desirable because of the large performance impact, so users need to specify "auto,nosmt" instead of just "auto" in order to get that behavior. Generally, leaving all of the mitigations enabled is a safe choice. But for systems that won't run virtual machines, for example, adding mitigations=auto;no_guest_host,no_guest_guest to the kernel command line could win back some lost performance. In Kaplan's cover letter, he justifies switching to this set of command-line options:

While many users may not be intimately familiar with the details of these CPU vulnerabilities, they are likely better able to understand the intended usage of their system. As a result, unneeded mitigations may be disabled, allowing users to recoup more performance.

When the user specifies one of the new attack-vector options, the kernel uses its knowledge about which bugs exist in the computer's CPU to turn on the best mitigations for that case. For now, they're only effective on x86 (and x86_64), pending input from the maintainers for other architectures about which mitigations should be associated with each option. The existing mitigation-specific command-line options will continue to function as normal, although they will probably see a good deal less use going forward.

This particular design is the result of an extended discussion on the previous version of Kaplan's patch set. That version had five different new command-line options, one for each potential attack vector. Reviewers found that design unnecessarily complicated, though, and eventually agreed on the simpler interface described above. Pawan Gupta, Josh Poimboeuf, and Kaplan all suggested various ideas, but Borislav Petkov made the suggestion that ended up being closest to what Kaplan adopted in the new version of the patch set.

The reviewers were not completely convinced by Kaplan's taxonomy of potential attacks, although they seemed to generally agree that it was better than the confusing status quo. Poimboeuf thought that the cross-thread group, in particular, didn't make much sense to separate from the others. In a cross-thread attack, the code that is running on the attacking SMT core (also called a hardware thread) must be either a user-space program or a virtual machine; therefore, Poimboeuf argued, any cross-thread attack should already fall under one of the other four categories that Kaplan set out.

Kaplan didn't disagree, although he had noted earlier in the discussion that disabling SMT has to be treated specially. Still, even with the complication around SMT, Kaplan's command-line options are a good deal less complicated than the existing set.

The internal logic for selecting mitigations — which also received substantial attention from reviewers, although only in the form of code style comments — has also been cleaned up. Previously, the code was somewhat labyrinthine. With Kaplan's changes, the functionality should be the same, but the implementation has been split up into a "select", "update", and "apply" function for each mitigation the kernel knows about. The select function determines whether the mitigation ought to be enabled, the update function allows the specific configuration for the mitigation to be changed depending on which other mitigations have been selected, and the apply function does the actual job of applying the mitigation to the kernel.

The most recent version of Kaplan's patch set was posted on March 10, and therefore hasn't had much time to attract any additional discussion. Given the positive reception of the previous version, it seems quite possible that these new command-line options will be available for x86 and x86_64 soon, hopefully with other architectures to follow.

Comments (19 posted)

Warming up to frozen pages for networking

When the 6.14 kernel is released later this month, it will include the usual set of internal changes that users should never notice, with the possible exception of changes that bring performance improvements. One of those changes is frozen pages, a memory-management optimization that should fly mostly under the radar. When Hannes Reinecke reported a crash in 6.14, though, frozen pages suddenly came into view. There is a workaround for this problem, but it seems there is a fair amount of work to be done that nobody had counted on to solve the problem properly.

The kernel uses reference counts to keep track of which pages of memory are in use. For example, a page in shared memory that is mapped into the address space of several processes will track a reference from each of those processes. As long as at least one of those processes exists and keeps the page mapped, that page will not be freed. The management of reference counts is not free, though; their manipulation requires expensive atomic operations, and the counts themselves take up memory. That has led to a desire to do without reference counting in places where it is not strictly necessary. The slab allocator, for example, tracks the objects it manages within each page and does not need a separate reference count for the page as a whole. In kernels prior to 6.14, though, slab pages are duly reference-counted anyway.

Frozen pages were introduced as a way to eliminate this overhead when possible; in a frozen page, the reference count is set to zero and stays there. Since the lifecycle of the page is tracked separately, there is no need to increment or decrement its count, so that overhead is avoided. Eventually, it will become possible to eliminate the reference count for frozen pages entirely (rather than just keeping it at zero), but there is work yet to be done to reach that point.

Reinecke encountered a kernel crash deep within the networking subsystem; after carefully bisecting the problem, he identified the commit switching the slab allocator to frozen pages as the culprit. Some extensive debugging and discussion ensued, and it eventually became clear that the networking code was trying to increase the reference count on a frozen page, leading to all kinds of internal confusion and an eventual crash.

Sending data through the network can be a complex operation involving pages scattered throughout physical memory. The networking subsystem, like others, handles this complexity by creating a vector describing the various chunks of data to be transferred. All of the pages contained within that vector need to remain present and valid while the operation is underway, so each page's reference count is incremented at the beginning, and decremented once the operation is complete. Many I/O paths within the kernel have traditionally followed that same pattern.

With the shift toward folios and the desire to avoid unneeded reference-count operations, though, that pattern has shifted. The I/O paths need to avoid reference-count manipulations whenever possible, and certainly when those manipulations cannot be done at all, so those paths have changed to adapt. At least, they have in some parts of the kernel; Matthew Wilcox expressed some surprise on learning that the job was only partially done:

I thought we'd done all the work needed to get rid of these pointless refcount bumps. Turns out that's only on the block side (eg commit e4cc64657bec). So what does networking need in order to understand that some iovecs do not need to mess with the refcount?

Reinecke answered that this kind of change was not going to be easy; the code is complex, and the place where a reference is taken may be far away from — and, indeed, in a completely different layer from — where that reference must be released. Wilcox, meanwhile, posted a patch adding some checks within the memory-management code that prevents attempts to manipulate reference counts on slab pages, which are the only frozen pages in the 6.14 kernel. That change, described as "a quick hack", was intended as a way to avoid having to revert the use of frozen pages entirely.

Even then, it took one more change from Vlastimil Babka, touching the networking code directly, to make the problem go away. Reinecke acknowledged the fix, but complained about the need to keep track of whether specific pages needed their reference counts updated or not:

Previously we could just do a bvec iteration, get a reference for each page, and start processing. Now suddenly the caller has to check if it's a slab page and don't get a reference for that. Not only that, he also has to remember to _not_ drop the reference when he's done. And, of course, tracing get_page() and the corresponding put_page() calls through all the layers. Really?

His complaint garnered little sympathy, though. Instead, Wilcox asserted that the networking layer needs to move away from using reference counts on pages, both to allow the memory-management hack to be removed and to improve networking performance. He added: "What worries me is that nobody in networking has replied to this thread yet. Do they not care?" In an attempt to provoke such a response, he changed the subject line to: "Networking people smell funny and make poor life choices".

Even that, though, has failed to motivate any sort of significant response from the networking subsystem. The only reply was from Cong Wang, who suggested that "using AI copilot or whatever automation tool" might help — a suggestion that does not appear to have gained any traction. Wilcox has posted his workaround as a separate patch that, one would expect, will find its way into 6.14 prior to its release.

As of this writing, that is where the situation stands. The immediate problem should be fixed, but the wider question of the management of reference counts for pages across the kernel remains unanswered. Perhaps the upcoming Linux Storage, Filesystem, Memory-Management, and BPF Summit will include a discussion on this issue; stay tuned.

Comments (11 posted)

Looking forward to mapcount madness 2025

One of the many important tasks that the kernel's memory-management subsystem must handle is keeping track of how pages of memory are mapped into the address spaces of the processes running on the system. As long as mappings to a given page exist, that page must be kept in place. As it turns out, tracking these mappings is harder than it seems it should be, and the move to folios within the memory-management subsystem is adding some complexities of its own. As a follow-up to the "mapcount madness" session that he ran at the 2024 Linux Storage, Filesystem, Memory-Management, and BPF summit, David Hildenbrand has posted a patch series intended to improve the handling of mapping counts for folios — but exact accounting remains elusive in some situations.

In theory, tracking mappings should be relatively straightforward: when a mapping to a page is added, increment that page's mapping count to match. The removal of a mapping should lead to an associated decrement of the mapping count. But huge pages and large folios complicate this scenario; they have their own mapping counts that are, essentially, the sum of the mapping counts for the pages they contain. It is often important to know whether a folio as a whole has mappings, so the separate count is useful, but it brings some complexities.

For example, one question that the kernel often asks is: how many processes have mapped a given page or folio? There are a number of operations that can be optimized if it is known that a mapping is exclusive — that the page or folio is mapped by a single process. The handling of copy-on-write pages is also hard to execute correctly if the exclusivity of a given mapping is not known; failures on this front have led to some unpleasant bugs in the past. For a single page, the exclusivity question is easily enough answered: if the mapping count is one, the mapping is exclusive, otherwise it is shared. That rule no longer applies if mapping counts are maintained at the folio level, though, since the folio-level count will almost never be just one.

The current scheme also has performance-related problems that folios could maybe help to improve. Mapping a traditional PMD-size huge page is equivalent to mapping 512 base pages; currently, if the entire huge page is mapped, the mapping count of each of its base pages must be incremented accordingly. Incrementing the mapping count on each of those base pages takes time the kernel developers would rather not spend; it would be a lot faster to only keep track of a single mapping count at the folio level. This optimization can only make the exclusivity question even harder to answer, though, especially in the presence of partially mapped folios (where only some of its pages are mapped into an address space).

Thus, it is not surprising the kernel developers have spent years trying to figure out how to properly manage the mapping counts as memory-management complexity increases.

Make room!

To better track mapping counts at the folio level, Hildenbrand first needed a bit more space in the folio structure for some additional information. struct folio is a bit of a complicated and confusing beast. As a way of facilitating the transition to folio use throughout the kernel, this structure is overlaid on top of struct page, which describes a single page. But folios often need to track more information than can be fit into the tightly packed page structure; this is especially true for large folios that contain many pages.

But, since a large folio does contain many pages — and physically contiguous pages at that — there are some tricks that can be employed. There is no real need to maintain a full page structure for every page within a folio, since they are managed as a unit; indeed, eliminating the management of all of those page structures is one of the objectives of the folio transition. But those page structures exist, laid out contiguously in the system's memory map. So a large folio does not have just one page structure's worth of memory at its disposal; it has the page structures for all of the component pages. The page structures for the "tail pages" — those after the first one — can thus be carefully put to use holding this additional information.

If one looks at the definition of struct folio, it quickly becomes clear that it is larger than a single page structure. After the initial fields that overlay the page structure for the head page, one will find this:

    union {
	struct {
	    unsigned long _flags_1;
	    unsigned long _head_1;
	    atomic_t _large_mapcount;
	    atomic_t _entire_mapcount;
	    atomic_t _nr_pages_mapped;
	    atomic_t _pincount;
    #ifdef CONFIG_64BIT
	    unsigned int _folio_nr_pages;
    #endif
	/* private: the union with struct page is transitional */
	};
	    struct page __page_1;
    };

This piece of the folio structure precisely overlays the page structure of the first tail page, assuming such a page exists. It contains information intended to help maintain the mapping count in current kernels, and other relevant fields. There is also a __page_2 component (not shown) that mainly holds information used by the hugetlbfs subsystem. As a result, the folio structure is actually the length of three page structures, though most of it is only valid for large (at least four pages) folios.

As sprawling as this seems, it still lacks the space Hildenbrand needed to better track mapping counts. To be able to handle order-1 (two-page) folios, he needed that space to fit within the page-1 union shown above. So the first six patches of the series are dedicated to shuffling fields around in the folio structure, adding a __page_3 union in the process. The __page_1 union gains some complexity, but the core of the work is in these new fields:

    mm_id_mapcount_t _mm_id_mapcount[2];
    union {
	mm_id_t _mm_id[2];
	unsigned long _mm_ids;
    };

They will be used to keep better track of the mapping for the folio to which they belong. Describing how that is done requires a bit more background, though.

One, two, or many

So how does all of this work help to improve the tracking of the mapping counts for large folios that may be shared between multiple processes and which can be partially mapped in any one of them? The starting point is the mm_struct structure that represents a process's address space. Any time a folio is mapped, that mapping will belong to a specific process, and thus a specific mm_struct structure. So the question of whether a folio is exclusively mapped comes down to whether all of its mappings belong to the same mm_struct. It is a simple matter of tracking which mm_struct structures hold mappings to the folio.

Of course, there could be thousands of those structures containing such mappings; consider that almost every process in the system will have the C library mapped, for example. Tracking all of those mappings without consuming a lot of time and memory would not be an easy task. But it is not really important to track every mapping to something like the C library; the purpose here is to stay on top of the folios that are exclusively mapped, and thus don't have all those mappings.

The _mm_id array that was added to page 1 of the folio structure is intended to serve this purpose; it can track up to two mm_struct structures that have mappings to the folio. The most straightforward way to do that would be to just store pointers to those mm_struct structures, but space in the folio structure is still at a premium. So, instead, a shorter "mm ID" is assigned to each mm_struct, using the kernel's ID allocator subsystem.

When a folio is first created, both _mm_id entries are set to MM_ID_DUMMY, indicating that they are unused. When the time comes to add a mapping, the kernel will search _mm_id for the appropriate mm ID, then increment the associated _mm_id_mapcount entry to record the new mapping(s). So, for example, if eight pages within a folio are mapped into the address space, the count will be incremented by eight to match. If the mm ID does not have an entry in _mm_id, the kernel will look for an MM_ID_DUMMY entry to use for this mm_struct, then start tracking the mappings there.

The kernel is now maintaining multiple mapping counts for this folio. The _large_mapcount field of the folio structure continues to count all of the mappings to the folio from any address space, as it does in current kernels. But there is also the _mm_id_mapcount count for each mm_struct tracking the number of mappings associated with that specific structure. The question of whether the folio is mapped exclusively is now easy to answer: if one of the _mm_id_mapcount counters is equal to _large_mapcount, then all of the mappings belong to the associated mm_struct and the kernel knows that the mapping is exclusive. Otherwise, the mapping is shared.

The ability to track two mm_struct structures handles the most common case of short-term shared mappings — when a process calls clone() to create a new child process. That new process will use the second _mm_id slot for the mapping that is now shared between the parent and the child. If, as often happens, the child calls execve() to run a new program, the shared mapping will be torn down, the child's _mm_id slot will be released, and the kernel will know that the folio is, again, mapped exclusively.

There is just one tiny gap in this mechanism, though: what happens when a third process comes along and maps the folio? There will be no _mm_id slot available for it, so its mapping(s) cannot be tracked. Should this happen, the kernel will set a special bit in the folio structure indicating that it no longer has a handle on where all the mappings to the folio come from, and will treat it as being shared. This could result in the kernel mistakenly concluding that a folio is mapped shared when it is mapped exclusively; the consequence will be worse performance, but no lack of correctness. If enough processes unmap the folio, there could come a time when _large_mapcount again aligns with one of the _mm_id_mapcount counts, and the kernel will once again know that the folio is mapped exclusively.

Per-page mapcounts and more

The result of all this work is that the kernel has a better handle on whether any given folio is mapped exclusively or shared, though it may still occasionally conclude that a folio is shared when it is not. But that was not the only objective of this work; Hildenbrand also would like to do away with the overhead of maintaining the per-page mapping counts in large folios. The final part of the patch series is an implementation of that goal; at the end, the per-page counts are no longer used or maintained.

The most significant consequence of dropping the per-page mapping counts appears to be making some of the memory-management statistics provided by the kernel (the various resident-set sizes, for example) a bit fuzzier. Hildenbrand suggests that this imprecision should not be a problem, but he also acknowledges that it will take time to see what the implications really are. To avoid surprises during that time, there is a new configuration parameter, CONFIG_NO_PAGE_MAPCOUNT, that controls whether these changes are effective. This work is considered experimental enough that, at this point, Hildenbrand does not want to have it enabled by default in production kernels.

There will be a desire to do that at some point, though; dropping the per-page map counts can make a clone() call up to 20% faster for some workloads, according to performance results included in the patch cover letter.

Meanwhile, this work enables another optimization with regard to how some transparent huge pages are used after a process forks. In current kernels, if the huge page (folio) is mapped at the base-page level ("PTE mapped"), it will not be reused after the fork. As the use of transparent huge pages — and, especially, in multi-size huge pages that must be PTE mapped — grows, reusing those huge pages will become increasingly important. Now, with the per-mm_struct mapping counts, the kernel can tell when a process has exclusive access to the huge page and can continue to use it as such. This reuse yields significant improvements in some benchmark results.

The use of large folios is expected to grow in the future; they are a more efficient way to manage much of the memory that any given process uses. So it is important to optimize that case as much as possible. Hildenbrand's patch set makes some steps in that direction while addressing a thorny problem that has resisted solution for years. These changes are currently in the linux-next repository, so there is a reasonable possibility that they could land in the mainline during the 6.15 merge window. If so, the 2025 Linux Storage, Filesystem, Memory-Management, and BPF Summit, which will be concurrent with that merge window, may be the last to feature a "mapcount madness" session.

Comments (10 posted)

The burden of knowledge: dealing with open-source risks

Organizations relying on open-source software have a wide range of tools, scorecards, and methodologies to try to assess security, legal, and other risks inherent in their so-called supply chain. However, Max Mehl argued recently in a short talk at FOSS Backstage in Berlin (and online) that all of this objective information and data is insufficient to truly understand and address risk. Worse, this information doesn't provide options to improve the situation and encourages a passive mindset. Mehl, who works as part of the CTO group at DB Systel, encouraged better risk assessment using qualitative data and direct participation in open source.

Mehl started with a few assumptions about the audience and open-source usage at the organizations they worked at. The first assumption was that audience members were in some way responsible for the use of open source in their organization. Next, those organizations have a five- to seven-digit number of open-source packages in use, spread out among a three- to five-digit number of internal projects. Many of the packages in use at those organizations are direct dependencies—the software the organization's developers actively chose to use—but the majority are indirect dependencies that are required for the software the organization wants to use.

Understanding risk

Those working with open source know that there are potential risks inherent in open-source projects. A project might have security vulnerabilities, or it might change licenses at some point and no longer be considered safe for the organization to use. Projects might also have sustainability issues, he said, which could take the form of an inactive maintainer, a completely dead project, or "other things that indicate that the longevity is not there".

Naturally, those responsible for open-source use need ways to measure the risk and communicate it to the organization. Mehl noted that there are many frameworks to choose from when assessing risk, but he chose to talk specifically about four methodologies: the Cybersecurity and Infrastructure Security Agency's (CISA) Framework for Measuring Trustworthiness, the OpenSSF Scorecard, the Community Health Analytics in Open Source Software (CHAOSS) framework metrics, and DB Systel's Open Source Red Flag Checker, which examines repositories for both red flags as well as activity and licensing conditions the group considers good.

Mehl put up a slide that highlighted a few quotes from the CISA blog post about its framework that, he said, helped to "understand the mindsets at play" in trying to codify risk around open-source software. For example, CISA claims that it is more complex to judge the trustworthiness of open-source software than proprietary software because there is "no direct relationship between the authors of software and those who use that software". Mehl said that the CISA framework is a bad framework for measuring risk, with a very narrow view on trust. "It puts me in a very passive relationship with open source, [it assumes] I have no direct relationship; I cannot change anything about this."

Passive metrics are not sufficient

Mehl said the problem with relying exclusively on the frameworks is that they only measure what can be measured and that passive metrics were not enough. He had a hot take on health metrics for open-source projects: "the people in this room can better assess the health of an open-source project than all the metrics". Metrics cannot replace an experienced "gut feeling" about a project. He did not say that organizations should not use them at all, but that they should not be the sole authority.

He brought up a paper from 2023, "Do Software Security Practices Yield Fewer Vulnerabilities?", by Nusrat Zahan, Shohanuzzaman Shohan, Dan Harris, and Laurie Williams. The OpenSSF Scorecard is an automated tool that assesses various factors in open-source projects hosted on GitHub, and assigns scores from 0-10. It is meant to be used by projects to assess and improve their security posture, or to be used by organizations to make decisions about the projects they use or may want to use. The paper found that a project's OpenSSF score "only explains 12% of vulnerabilities". In other words, the scorecard may be missing other factors that predict vulnerabilities.

The OpenSSF Scorecard and other metrics simply do not take into account many factors that are important when assessing an open-source project. For example, how is the maintainer behaving? How do they react to bug reports or pull requests? Is there a connection to an open-source foundation, or is the project a single-vendor effort? If it is funded by venture capital money, that is not sustainable and is predictive of license changes that will make the software non-free. Mehl pointed out that the CHAOSS framework covers some of these things, but it doesn't weight them.

Most importantly, Mehl said, "those passive metrics do not make us active". A huge problem inherent in using open source is that an organization does not have an alternative if one of these frameworks finds that an open-source package scores badly. "Most of the open source we're using is not controlled by ourselves." He said that he had software bill of materials (SBOMs) generated for software being used by DB Systel, and the software had about 125 dependencies on average. Many packages had more than 1,000 dependencies. In total, the project found 117,000 individual packages in use—and that is without considering versions of packages. If versions were taken into account, Mehl said that the number of packages would increase dramatically. Even worse, nine out of ten of the packages in use score worse than an aggregate five on the OpenSSF Scorecard. "If our rule were that you can only use open-source projects that score better than five out of ten, we would have a huge problem."

What do we do?

The frameworks can provide a lot of insight and knowledge, he said, which in some ways can be a burden. Now the organization has information to act on, but without clear recommendations. Using the frameworks to assess risks may grant an understanding that a project is flawed, but they do not provide the answers for addressing that risk. Replacing dependencies is not easy or economical. In organizations that are not experienced with open-source use, there may be a temptation to downplay the risks. "I wouldn't recommend that", he said. Another path is to become active and ask developers to replace dependencies, or even contact maintainers and ask them to update. "Or we could force them to fill out forms. You're laughing, but it has happened. Projects have been bombarded with forms to fill out." (Curl creator and maintainer Daniel Stenberg has written about exactly this scenario.)

Instead, Mehl suggested, "let's get more honest". Start with assessing the organization's risk profile and what is specifically important to it. Maybe the mere existence of a contributor license agreement is a risk, or dependence on a single vendor is a risk. "Risk profiles and risk assessment can be very individual. You should identify qualitative and quantitative data that matters to you" and group assets into different risk classes. For example, he said that DB Systel had some software that could be deployed for up to 60 years in critical infrastructure. Organizations could create risk "themes" instead of measuring all software use equally.

Organizations might come to the conclusion that they want to fund some projects financially by supporting maintainers or projects directly. He suggested that it would be wiser to do that through entities like Germany's Sovereign Tech Agency or to nudge software foundations to support specific projects and for organizations to come together and fund things collaboratively rather than one-off funding attempts. Money, however, does not solve everything. Mehl observed that some developers are not looking for or motivated by funding. Money can complicate things in open-source projects, and companies usually want something in return for their funding, which can be off-putting.

Another option is for organizations to contribute code to projects, perhaps even having employees become co-maintainers of projects. He also recommended that organizations could set up teams that provide support for open source and coordinate contributions to external projects, or even partner up with other organizations.

Recommended toolset

All of those options are on the more-reactive side of responding when it's clear an open-source project already in use has greater risk, though, and Mehl encouraged organizations to be more proactive. He proposed coming up with criteria for selecting projects based on risk assessments beforehand, so developers could make educated choices and choose open-source projects more wisely. Since he had knocked the CISA framework earlier, he highlighted what he considered a good example from the Biden administration's 2024 Cybersecurity Priorities. That memo, in part, recommends that agencies should improve open-source-software security and sustainability by directly contributing to maintaining open-source projects.

Organizations should have open-source program offices (OSPOs), he said, not for the sake of having an OSPO, but to help define roles and responsibilities and methods of engagement with open-source projects. "The opposite of CISA, this provides an active framework and mindset." Each organization should have tools in its toolbox to allow it to do four things: assess, sponsor, select, and engage with open-source projects that are important to the organization.

His final thoughts for the audience were that organizations need to collaborate more on assessment criteria and on how to share those efforts with other organizations. Too often, Mehl complained, that is duplicated work in thousands of organizations that could benefit from sharing.

Get active. Time's over for passive consumption of open source. We see in the world we can't rely on others to fix issues for us. We have to collaborate. We have to get active. We should do this in general but especially in open source.

Questions

The first question was, since Mehl was encouraging collaboration and reuse, whether DB Systel's internal guide on choosing open source was available. Mehl admitted that it wasn't, "but we should publish it, we should do that. It's a sensitive issue and it gets complicated, but yeah, I think we should share more."

Another audience member wanted to know how to tell internal developers "no" if a project doesn't meet the criteria, and asked if Mehl had seen pushback when telling someone that a developer's choice doesn't meet guidelines. Mehl said that DB Systel does not centralize that choice, that it gives guidance but the teams themselves make risk assessments. "This is a matter of an organization's risk assessment. It makes sense to centralize this in some organizations, but we have a different attack surface".

One member of the audience said that he was happy to hear someone say you can't judge risk solely by the numbers. He wanted to know if Mehl could publish the guidance that these decisions "have to be from the gut" somewhere that he could link to. Mehl didn't respond directly to the question of publishing the guidance but reiterated that "sometimes it makes sense to use something that scores badly" and then plan for the organization to engage with the project to improve it. "There is no other way around than getting active" in open source.

Some projects die over time, an audience member observed. He wanted to know what experience Mehl had with that and how to spot a project that is dying. Mehl said that there are a number of indicators that can hint a project is in trouble. That might include how many commits and active contributors a project has over time, or some of the OpenSSF metrics. But "sometimes software is just complete". Some projects do not need daily activity to sustain themselves; it might be good enough to have an update once per year. It depends on the project. Kubernetes "should be maintained more than once every 90 days", but other projects do not need that kind of activity. Of course, it also depends on where a project is being used, too. He said that an organization had to consider its strategy, identify the software it depends on most, and look at various things—such as the health of the community, the behavior of the maintainer, and CHAOSS metrics—as "a starting point of what you look at". Ultimately, it depends on each organization's risk profile.

While many of Mehl's guidelines can be boiled down to the unsatisfying answer no one likes ("it depends"), it's refreshing to see someone telling organizations they require more in-depth analysis to assess risk than can be had with one-size-fits-all frameworks and scorecards. It is even more encouraging that Mehl pushes organizations to be active in participating in open source, rather than treating projects like another link in the supply chain that can be managed like any other commodity.

Comments (27 posted)

A look at /e/OS on tablet hardware

/e⁠/⁠OS is a privacy-centric, open-source mobile operating system that has primarily been targeted at mobile phones, with only a few community supported images available for tablet devices. In December, Murena—a company that sells devices with /⁠e⁠/⁠OS preinstalled—announced that /⁠e⁠/⁠OS now officially supports tablets as well, starting with the Pixel tablet. The user experience is close enough to mainstream alternatives to make it attractive, but there are some under-the-hood problems that may give users pause.

/⁠e⁠/⁠OS is a "deGoogled" fork of LineageOS and continues to pull patches from Android Open Source Project as well. It was originally "Eelo", inspired by moray eels, but it was renamed due to a conflict with a company called eelloo. It is produced by the e Foundation, a non-profit organization in the public interest under French law. The foundation was registered by Mandrake Linux founder Gaël Duval in 2018. Mandrake Linux later became Mandriva Linux due to trademark issues, suggesting that Duval might want to pick future names more carefully. Murena is a privately held "sister company" that has run crowdfunding campaigns to put /⁠e⁠/⁠OS devices into the market. It helps to fund the e Foundation, which also receives donations from individuals and other organizations.

The most recent release is version 2.8, a minor security and bug-fix release, announced on February 27. It is based on LineageOS 21, which is based on Android 14. It incorporates updates from LineageOS up to January 29 and the Android security patches available as of January 2025. The project seems to put out its minor updates on a monthly cadence, and is about a month behind on integrating security updates from Android. The most recent major LineageOS release is 22 from December 2024, based on Android 15. This means that /⁠e⁠/⁠OS lags behind its upstreams by a fair amount for security updates, and won't be the best choice for users who want to get the newest Android features quickly.

/⁠e⁠/⁠OS also includes the microG project, which is an open-source implementation of proprietary Android apps and libraries. It replaces Google Play Services, and users do not need to authenticate with Google or even have a Google account.

Getting /⁠e⁠/⁠OS

Murena sent me a loaner Pixel tablet with /⁠e⁠/⁠OS for testing. Buying a device with /e/OS preinstalled is the easiest way to get started, but it is also possible to install it on a tablet one already owns. The project offers a graphical installer that works from Linux, macOS, and Windows. That project is in the process of being deprecated in favor of the web installer, which requires a Chromium-based browser with WebUSB support (it recommends Chrome, Edge, or Opera) to perform the installation.

The devices page has a searchable list of phones, tablets, and other hardware that have /⁠e⁠/⁠OS support. The page explains that there are three types of builds: official, community, and unofficial, but those names are a bit confusing. The documentation explains that official images are what one would expect: fully tested builds that are created on /e/OS build infrastructure and signed with /e/OS official keys. The community builds are also built and signed by /e/OS, and they are tested, but may be released even if they have critical bugs. The unofficial builds may be built on /e/OS infrastructure or by users, and they are not signed by /e/OS. The maintainer of the build may not even own the device that the build targets. The foundation does not provide over-the-air (OTA) update support for unofficial builds.

The graphical install method only supports 21 devices, currently. The FAQ describes how to get a list of devices that the installer will work with. If the target device isn't one of the lucky 21, all is not lost. There is also a command-line installer that supports more than 200 devices, although only a handful of the devices have "official" support.

Using /⁠e⁠/⁠OS

At first glance, /⁠e⁠/⁠OS looks (at least to my eyes) more like iOS than Android. The icons for its default apps bear a strong resemblance to those on iOS, and the default home screen layout is iOS-ish as well. But, aside from a bit of a paint job, the user experience is much more like Android—if one got rid of all the Google apps and services, that is. It only provides a handful of default apps for basic activities such as browsing the web, email, task management, calendaring, and contact management. These, optionally, can integrate with Murena Workspace, which is a replacement for Google Workspace based on Nextcloud and ONLYOFFICE. A Workspace account is free and includes 1GB of storage, though paid accounts with more storage are available. It is, at least in theory, possible to self-host the services offered by Murena. There is somewhat dated documentation for doing this on their GitLab instance.

Murena's services have had some growing pains due to an increasing number of active users in the past few years. The service suffered an outage in October that affected its storage, email, calendaring, its app store service, and OTA updates. Services were not fully restored until early March.

The stock mail app (just called "Mail") is forked from K-9 Mail. If a user is signed into a Murena account, the mail client is automatically configured with the right settings for Murena mail—but it can also be set up for third-party servers with support for IMAP or POP3. Since K-9 development will be coming to a close now that Thunderbird for Android has been released, one might wonder what the future holds for the /⁠e⁠/⁠OS mail client. There is a development branch that seems to be transitioning to Thunderbird for a future release.

Cromite is the default browser. It is a Chromium-based browser that limits or disables tracking features. Cromite is forked from Bromite, which was a browser for Android which seems to have stopped or stalled development, with the last release published in December 2022. Cromite seems usable enough, but I wound up installing Firefox after a few days of tinkering with it—not because using Cromite was a bad experience, but because I prefer Firefox and wanted to ensure that third-party apps ran properly on /⁠e⁠/⁠OS. So far, I haven't encountered any problems or noticed any difference from running Firefox on a stock Android device.

It is refreshing to have a device that is almost a clean slate, rather than being stuffed with unnecessary default apps and vendorware. My Motorola phone, for example, included more than 20 Google apps and a handful of Motorola's apps as well, most of which I have never needed or wanted to use. Leaving aside any considerations about whether these apps invade my privacy, they also take up precious space on a device with limited storage.

This leaves users more room to install whatever apps they see fit, using the App Lounge. It offers three sources of apps, Google's Play Store catalog, the F-Droid repositories of open-source Android apps, and a set of progressive web apps (PWAs) which are apps delivered as an installable web application rather than native Android apps. (See this article for a history of PWAs, and findPWA for a directory of popular PWAs.) The App Lounge is an open-source app, available under the GPLv3, and its source is available on the e Foundation's GitLab instance. The Lounge allows installing apps from Google's Play Store catalog with, or without, signing in.

Applications from F-Droid and PWAs are pulled from the CleanAPK repository rather than directly sourced from F-Droid or the provider of the PWA. CleanAPK mirrors applications and analyzes them with Exodus privacy software. The Exodus report is available via the App Lounge, which provides a detailed report on the permissions an app uses, as well as any trackers found in the app. For example, the report for Firefox finds three trackers and 32 permissions. The privacy report is a nice feature, but there are concerns about the use of CleanAPK, including the possibility that using it as an intermediary might allow spoofed apps to be installed on user devices. An analysis by "nervuri" last updated in March 2023 pointed out that the App Lounge does not verify that links to PWAs are correct. "CleanAPK can easily change any of these links and serve spoofed PWAs."

A number of the concerns raised by the analysis have since been addressed. However, in March 2022 Duval wrote that use of CleanAPK would be "totally abandoned this year". To date, it is still in use.

Aside from security concerns, I found the App Lounge to be slow and a bit buggy. It takes a few seconds to refresh and redraw all of the icons for the apps in its catalog. When searching for an application, it would often simply redo the last search rather than delivering results for a new search term. That is, if I searched for "Firefox" in the app and then performed a new search for "Netflix", it would deliver the same search for Firefox. Some work is needed to make this a better experience. It also could use some better curation; the Lounge's front page pulls categories like "top grossing apps" directly from Google's Play Store, instead of recommending apps that would be more suitable for users concerned with privacy or apps that are free software.

Stock Android devices have swipe or gesture typing: instead of having to tap each letter individually, the user can input text by running their finger over the word's letters on the virtual keyboard and get the right word. Usually, anyway. Sometimes the results are comically bad and one has to type the word out letter-by-letter. But, as a general rule, swipe typing is much faster and much less annoying. The /⁠e⁠/⁠OS virtual keyboard, unfortunately, does not have this feature. After using swipe typing for years, going back to bashing keys one at a time on a virtual keyboard is a regression in usability. Of course, there are options: users can install third-party keyboards, but the options are non-free and not privacy-friendly.

Gripes aside, I did find /⁠e⁠/⁠OS to be a usable Android alternative. The apps I use on other Android devices have worked just fine on /⁠e⁠/⁠OS, and it's good to have a device that doesn't require a sign-in or relationship with Google or Apple.

The hardware

The Murena tablet is a stock Google Pixel device with /⁠e⁠/⁠OS loaded in place of Android. In fact, the device really wants the user to know that it has been deGoogled. At each startup, it displays console text output in one corner of the screen to warn that it is booting into a different operating system. There is no way to get rid of that alert according to my contact at Murena.

The screen is a 10.95-inch display with 2560x1600 resolution (276 pixels per inch). It is crisp and clear, and the size works well for reading emails in Thunderbird, watching videos, reading ebooks, and other tasks. It works less well for doing those things while holding the tablet with just one hand, though. A tablet the size of the much-loved, but discontinued, Nexus 7 tablet would be even better. The Pixel has 8GB RAM and 128GB storage. (There is a 256GB storage model as well.) The Pixel has a decent feel to it—it's not heavy, but it has enough heft that it doesn't feel like a low-end toy.

Around the house, I mostly use a tablet for web browsing, skimming email lists, and reading ebooks. If I want to watch a movie or TV show, I have a perfectly decent television for that. But, if I'm traveling, then a tablet is my primary entertainment center. The Pixel, with /e/OS, is a pretty good device for that use case.

The music app with /e/OS is plain but usable, and it supports a decent range of formats such as MP3, AAC, and FLAC. It is not tied to any music service, so users have to bring their own music collection—which suits me fine. I loaded some of my music collection onto the tablet to road test the default music app and evaluate the speakers using a variety of music formats and genres. While I'm not an audiophile, I've sampled speakers on a wide variety of laptops and tablets over the past few decades and have suffered my share of tinny, subpar speakers. The Pixel's speakers, happily, are not in that category. They sound good and loud at about 70% volume, with respectable midrange and bass. Obviously, I'd use headphones instead of the speakers while on a plane or other public transport. The bad news here is that, like many devices today, the Pixel doesn't sport a headphone jack. Users have the options of connecting headphones via Bluetooth or buying a USB-C adapter for wired headphones.

/⁠e⁠/⁠OS, as one might expect, is not quite as polished a user experience as Android or iOS/iPadOS—but it's a good enough user experience that I wouldn't mind using it long-term. However, the speed of security updates and development in general gives me pause.

Maintaining a mobile operating system, several applications, and services is a big task. It seems that Murena and the e Foundation are a bit understaffed to support such an ecosystem. That is a bit of a chicken-and-egg problem, though. There is a need to attract more users, customers, and contributors to meet the challenge of becoming a fully sustainable ecosystem. But until /⁠e⁠/⁠OS is clearly a sustainable ecosystem, it's difficult to give a strong recommendation to jump in. Ultimately, I do hope for /⁠e⁠/⁠OS to succeed because we need privacy-conscious alternatives for mobile devices.

Comments (7 posted)