Leading items

Welcome to the LWN.net Weekly Edition for December 8, 2022

This edition contains the following feature content:

• Disunity at The Document Foundation: there are fundamental disagreements over the role that The Document Foundation should be playing.
• Juggling software interrupts and realtime tasks: another attempt to mitigate the longstanding problems with software interrupts.
• Losing the magic: a venerable kernel development technique fades away.
• Checking page-cache status with cachestat(): a proposed system call to determine whether (and how much of) a file is present in the page cache.
• Composefs for integrity protection and data sharing: a new filesystem for container use.

This week's edition also includes these inner pages:

• Brief items: Brief news items from throughout the community.
• Announcements: Newsletters, conferences, security updates, patches, and more.

Please enjoy this week's edition, and, as always, thank you for supporting LWN.net.

Comments (none posted)

Disunity at The Document Foundation

The Document Foundation (TDF) was created in 2010 to steward and support the development of the LibreOffice suite, which was then a new fork of OpenOffice.org. TDF has clearly been successful; unlike OpenOffice, which is currently under the Apache umbrella, LibreOffice is an actively developed and widely used project. But TDF has also been showing signs of stress in recent years, and the situation does not appear to be getting better. There are currently some significant disagreements over just what role TDF should play; if those cannot be resolved, there is a real chance that they could rip the Foundation apart.

Foundations and attics

Financial support for TDF comes primarily from a set of companies that are working with the LibreOffice code. All of these companies have a shared interest in a strong and active ecosystem around LibreOffice. That agreement tends to come to an end, though, when supporters feel that their contributions to TDF are being used to compete with them in the market. TDF supporters, it seems, want a development community that is strong, but not so strong that it doesn't leave them space for value-added services and products. Other members of the community, though, are mostly interested in the best LibreOffice that they can make and are indifferent to the prospect of impacting some company's revenue stream.

Tensions around this point had evidently simmered for some time before Michael Meeks (of Collabora) made them public in 2020:

Frustration with how TDF markets and positions its 'product' (LibreOffice) against the ecosystem that contributes the majority of the coding work is at an all-time high. That ecosystem itself is under long term stress.

In short, the version of LibreOffice created by TDF is sufficiently good that nobody felt the need to pay for support services. One approach for dealing with that problem was to brand the free LibreOffice as a "personal edition" and, perhaps, allow new features to go into paid "enterprise" editions (sold by TDF member companies) for some time before being added to the free version.

Needless to say, this idea was not universally popular. Even less popular was Collabora's decision to stop working on LibreOffice Online, a version of LibreOffice that provides collaborative editing of documents over the net. Collabora, which had been the principal (but not only) contributor to this code, is now developing it separately and marketing it under its Collabora Online branding instead. To Collabora, this move was a way to preserve the revenue stream from its LibreOffice work.

Others, though, saw it as a sort of looting of the LibreOffice commons by a single company. They were even less pleased in January 2022, when the Foundation's board proposed moving the LibreOffice Online code into an "attic", where it would no longer be part of the LibreOffice code base. Development of this code had stopped after Collabora left the building, so some saw it as an increasingly stagnant embarrassment to TDF. Others, though, saw the move as closing off any future community development of the online functionality. After months of tense discussion, the TDF board voted in July to delay the "atticization" of LibreOffice Online for now. This unresolved issue still hangs over the community.

TDF developers

The big argument over the last few months, though, is on a related topic: whether TDF should employ developers of its own and, if so, what those developers should work on. In February, board member Paolo Vecchi (Omnis Cloud Sarl) proposed that TDF should hire some developers of its own; the two suggested positions would work on creating a presence for LibreOffice in app stores, among many other things. (Then) board member Jan Holesovsky (Collabora), instead, argued that TDF needed mentors to support developers elsewhere: "teaching how to fish, not fishing itself".

There followed an intense conversation that continues to this day. Some participants feel that TDF should not be in the business of employing software developers — or even that, according to its bylaws, it cannot do so. Others see TDF-based developers as the core of a strong LibreOffice going forward. Yet others can accept developers employed by TDF, but want strong constraints on what those developers should be doing.

These viewpoints have been expressed in several interminable threads arguing over the proper role of TDF, with accusations of conflicts of interest flying in all directions. Much of the conversation was evidently in private, and it is hard to determine what the actual course of events was but, at some point, Vecchi and Holesovsky got together and put a serious effort into the creation of a proposal for the hiring of developers that would be acceptable to all involved. As part of this effort, Holesovsky backed down from the "not fishing" position and accepted that development could be done within TDF. Numerous versions of the proposal resulted from this dialog as various issues were worked out.

As of this writing, version 3.1 is the latest attempt. It makes the claim that TDF can support the community by employing developers to work on LibreOffice, especially if they focus on otherwise neglected areas. Suggested targets include better support of right-to-left and CJK (Chinese, Japanese, Korean) text, accessibility, interoperability with non-native file formats, and fixing of regressions: "there are 12.6K open bugs in TDF Bugzilla, of which 1.3K of them are regressions". The proposal also makes it clear that these developers will not work on long-term-support or "enterprise" versions of LibreOffice.

A sticking point

It seemed like things were getting closer to a conclusion, but there was, at a minimum, a disagreement over one stipulation in version 3.0:

TDF in-house developers will not compete with commercial contributors and will not develop alternative implementations of Open Source projects actively maintained by LibreOffice volunteer or corporate contributors – like Collabora Online, mdds, or cppunit.

This language goes to the core of the disagreement over TDF's role. Some participants do not feel that a vendor-neutral organization — that they are supporting — should be competing with them. Others, instead, do not accept the idea that member companies should be able to carve out pieces of the application space and prevent TDF-supported community development in those areas. Positions on both sides appear to be firmly entrenched.

At the beginning of November, Holesovsky publicly resigned from the TDF board:

It has come to the point when I am sick and tired of all the accusations and passive aggression against me and others. It has just become folklore in this community: "when you don't agree with somebody, accuse them that they do that because of commercial interests".

In other words, TDF has moved from a meritocracy (where the doers decide) to some kind of "shouting-ocracy" (where the vigorous shouters decide).

The resignation of one of its authors notwithstanding, on November 24, Vecchi put version 3.1 of the proposal to a vote of the board, suggesting that all of the disagreements had now been resolved. The controversial text quoted above is gone; in its place, the proposal reads:

Eventual limitations related to tasks, areas, projects or bugs on which the in-house developers should not work, eg. third parties are already engaged with them, shall be regulated through separate agreements and relevant communications between TDF and the third parties.

It quickly became clear that the promised consensus did not actually exist, though. Board member Cor Nouws (Nou&Off) stated that an alternative proposal was privately in the works; that proposal still has not been seen in public, but Emiliano Vavassori (another board member, whose affiliation is given as "a small company based in Bergamo") has decried the process under which it is being developed. Most board members, meanwhile, simply declined to vote on the proposal, with the result that it failed due to a lack of quorum. Board chair Thorsten Behrens (allotropia software GmbH) said that "this proposal is not ready to vote on", and Holesovsky resurfaced to say that: "That version is not balanced, and Paolo’s unwillingness to find balance there was one of the main reasons to my resignation". Meeks compared the discussion to "the Christmas pantomime season complete with comedy audience contradictions".

That is where the situation stands as of this writing.

The TDF board may yet find a way to hire a developer or two into the Foundation to work on neglected parts of that huge application. But that, honestly, is not TDF's biggest problem at the moment. Somehow, its membership has to find a way to clarify what the role of TDF should be and to rebuild trust and good will among its members. No organization can function indefinitely in a climate of distrust and acrimony, and that sort of climate is even more corrosive to a free-software development community. Those of us who depend on LibreOffice can only hope that this community will resolve its tensions before they lead to a breakdown and fork of the type that led to TDF's creation twelve years ago.

Comments (34 posted)

Juggling software interrupts and realtime tasks

The software-interrupt mechanism is one of the oldest parts in the kernel; arguably, the basic design behind it predates Linux itself. Software interrupts can get in the way of other work so, for almost as long as they have existed, developers have wished that they could be made to go away. That has never happened, though, and doesn't look imminent. Instead, Android systems have long carried a patch that tries to minimize the impact of software interrupts, at least in some situations. John Stultz is now posting that work, which contains contributions from a number of authors, in the hope of getting it into the mainline kernel.

Hardware interrupts (or just "interrupts") are initiated when a physical component in the system wants the kernel's attention; they will usually cause an immediate trap into a special handler function. Since interrupts take the system away from whatever else it was doing, interrupt handlers have to do their work quickly; there is not time for any sort of extended processing. This is not a new problem; pre-Linux Unix systems often included the concept of a "bottom half" as a way of deferring work that could not be done in an interrupt handler.

The Linux kernel, too, has had to develop mechanisms to defer processing until a more convenient time. One of those mechanisms is software interrupts (or "softirqs"). It was first introduced in the 0.99 kernel under the familiar "bottom half" name; the term "softirq" doesn't appear until the 1.1.77 development release. The abbreviation "bh" ("bottom half") can still be found in the names of kernel functions related to software interrupts.

In the current implementation, there is a handful of software-interrupt "vectors" assigned to specific subsystems. When one of those subsystems has to defer some work, it sets a bit in a per-CPU mask that will cause its software-interrupt handler to be called at a later time. Often, the software interrupt will run immediately after the completion of hardware-interrupt handling, meaning that it runs at a high priority, before just about any other work in the kernel. The software-interrupt mechanism is reserved for a handful of core-kernel subsystems, including networking and the block layer, with one big exception: the kernel's tasklet mechanism, which is available to any subsystem, executes tasklets in a software-interrupt handler.

Software interrupts can create latency in an otherwise well-functioning system, so they have long drawn the attention of developers interested in response times. In kernels configured for realtime preemption, software interrupts are handled in a process like any other, so their priority can be set relative to other important tasks. Most of us do not run realtime preemption kernels, though, so software-interrupt handling will usually steal time from other tasks that would like to be running. Even on normal systems, though, a system that is generating large numbers of software interrupts will eventually defer them to the per-CPU ksoftirqd kernel thread.

Normally, one expects realtime tasks to be the highest-priority work on the machine, even one that is not running a realtime kernel. But software-interrupt handling trumps even realtime tasks and can prevent them from meeting their deadlines. The latency that software interrupts can add has, evidently, been seen to cause audio glitches on Android systems, inspiring the work being proposed by Stultz now.

There are two components to the suggested change, the first of which is a scheduler tweak to modify its placement of realtime tasks. Currently, when a realtime task becomes runnable, the scheduler will search for a CPU that appears to have sufficient available capacity to run that task; among other things, that check will prevent the placement of a resource-hungry task on a slow CPU. Stultz's patch adds a new check to see whether a candidate CPU is handling (or is about to handle) a "slow" software interrupt that is expected to take a fair amount of processing time; for the purposes of this decision, "slow" is defined as being software interrupts from the networking or block subsystems. If the CPU is indeed busy in that way, the scheduler will try to place the realtime task elsewhere so that, with luck, it will gain access to the CPU more quickly.

The second piece changes how the kernel chooses to run software-interrupt handlers. Normally, those handlers are run as soon as possible on the current CPU unless the ksoftirqd thread is already running, in which case they will be queued for handling there instead. The patch series causes the kernel to check, before running a software-interrupt handler, whether there is currently a realtime task running on the current CPU; if so, and if the software interrupt is of the "slow" variety, it will be deferred to ksoftirqd so that the realtime task can continue running. Faster software interrupts (such as those for timers) will still execute immediately on the current CPU.

To effect that split, a subtle change was required. In current kernels, if ksoftirqd is running on a CPU, all software interrupts will be deferred to it. If these changes are active (there is a new kernel configuration option to control that), "fast" software interrupts will be handled immediately whether ksoftirqd is running or not. This change was necessary because the deferral of slower software interrupt handlers, as added by this patch series, may cause ksoftirqd to be running at times when the overall software-interrupt load is not high. With the old test, faster interrupt handlers could be delayed by an unnecessary deferral to ksoftirqd. The new code, though, could have the potential to not defer software interrupts at times when they are loading the system.

The end result of this work is much better latency results with realtime tests like cyclictest. The patches have been running on Android kernels "for a number of years" and presumably do not cause problems, at least in that context. Even so, Peter Zijlstra is not entirely happy with this work; it would be better, he suggested, to simply stop software-interrupt processing when there is a high-priority task to run. He has a series of patches that make that kind of change that he had posted previously, but admitted that the work "fell on its face due to regressions" at that time.

Stultz gave Zijlstra's series a try and concluded that, while it improves the situation, it does not fully address the problem. Specifically, it doesn't help if a single software-interrupt handler runs for a long time, and that is indeed something that happens. Thus, he said: "I'm not sure if it will let us move away from the softirq-rt placement optimization patches". Hopefully, some sort of solution that is acceptable to all of the developers involved will eventually be worked out. Otherwise, the outcome may end up being that Android continues to carry this patch series for several more years.

Comments (18 posted)

Losing the magic

The kernel project is now more than three decades old; over that time, a number of development practices have come and gone. Once upon a time, the use of "magic numbers" to identify kernel data structures was seen as a good way to help detect and debug problems. Over the years, though, the use of magic numbers has gone into decline; this patch set from Ahelenia Ziemiańska may be an indication that the reign of magic numbers may be reaching its end.

A magic number is simply a specific constant value that is placed within a structure, typically as the first member, to identify the type of that structure. When structures are labeled in this way, in-kernel debugging code can check the magic number and raise the alarm if the expected value is not found, thus detecting problems related to type confusion or data corruption. These numbers can also be located in hexadecimal data dumps (stack contents, for example) to identify known data structures.

The use of magic numbers in the kernel appears to have had its origin in the filesystem code, where it was initially used to identify (and verify) the superblock in the disk image. Even the 0.10 kernel release included a test against SUPER_MAGIC (0x137f) to verify that the boot disk was, indeed, a Minix filesystem. Other filesystems came later, starting with the "first extended" (ext), which used 0x137d for its EXT_SUPER_MAGIC value in the 0.96c release in July 1992.

In the 0.99 release (December 1992), the sk_buff structure that is still used in the networking subsystem to hold packets was rather smaller than it is now, but it did gain a magic field to identify the queue a packet was expected to be in. Toward the middle of 1993, the 0.99.11 release acquired an updated kmalloc() implementation that sprinkled magic numbers around as a debugging aid. That release, incidentally, is also the one where an attempt was made to use C++ to build the kernel; that only lasted until 0.99.13, a couple of months later.

The use of magic numbers in the kernel grew slowly after that. The 1.1.13 release, in May 1994, added a file called MAGIC in the top-level directory to keep track of the various numbers in use; it listed eight such numbers. This file was, incidentally, nearly the first documentation file in the kernel beyond the basic installation information; the kernel would not gain a directory for documentation until 1.3.22 in 1995. In this new file, Ted Ts'o wrote:

It is a *very* good idea to protect kernel data structures with magic numbers. This allows you to check at run time whether (a) a structure has been clobbered, or (b) you've passed the wrong structure to a routine. This last is especially useful --- particlarly when you are passing pointers to structures via a void * pointer. The tty code, for example, does this frequently to pass driver-specific and line discipline-specific structures back and forth.

The file went on to ask developers to follow this practice for future additions to the kernel. (For the curious: the typo of "particularly" lasted until 1.1.42 in August 1994; otherwise that text persists to this day).

The 1.3.99 release saw the movement of MAGIC to Documentation/magic-number.txt, perhaps as part of a general cleaning-up prior to the imminent 2.0 release. Some developers, at least, had clearly taken Ts'o's advice; there were, at this point, 21 entries in that file. The 2.2.0 version (January 1999) held 51 entries. Magic numbers appeared to be an established kernel-development practice.

The 2.4.0 release came almost exactly two years later. The 2.4 version of magic-number.txt was, except for one small change, identical to the 2.2 version; no new magic numbers had been added. That doesn't necessarily reflect a change in development practices so much as the eternal habit of letting documentation go out of date. Some effort went into updating the file during the 2.5 development series, and the 2.6.0 version contained an even 100 magic numbers. For the rest of the 2.6.x series, though, the only changes were small tweaks and the removal of a couple of obsolete entries; Documentation/magic-number.txt began to shrink.

In fact, no additions to that file have been made in the entire Git history. In 2016 the file was converted to the RST format and moved into the nascent development-process manual. It mostly sat idle and unnoticed until earlier this year, when the file began to shrink; the 6.1 version of Documentation/process/magic-number.rst is down to 14 entries. Where has the magic gone?

This change is the result of Ziemiańska's work, which is aimed at removing this file entirely; the current patch set describes it as "a low-value historical relic at best and misleading cruft at worst". In that series, Ziemiańska systematically deletes the final entries, often removing the associated structure fields and magic-number checks in the code, until the file is empty; the final patch in the series simply deletes it. Chances are that it will not be missed.

There is no clear point where the development community made a collective decision to move away from the magic-number practice; it just sort of faded away. The are probably a few reasons behind this change. The kernel community has, for many years now, tried to use type-safe interfaces rather than passing void pointers around, making it less likely that the wrong structure type will be passed into a function. Developers spend less time staring at hex dumps of data, preferring more structured output, tracepoints, and interactive debuggers as ways of tracking down problems. Debugging features in the kernel's memory allocators mean that many sorts of memory-corruption issues will be caught directly. Magic numbers are just not as helpful as they once were.

Magic numbers will still have their place; they still, for example, can help filesystem code be confident that it is dealing with the correct type of filesystem image. Even there, though, the use of checksums for on-disk data structures should provide better protection against many types of problems. But, for the most part, kernel development has lost some of the magic it once had; as often happens, it slipped away when nobody was paying attention.

Comments (55 posted)

Checking page-cache status with cachestat()

The kernel's page cache holds pages from files in RAM, allowing those pages to be accessed without expensive trips to persistent storage. Applications are normally entirely unaware of the page cache's operation; it speeds things up and that is all that matters. Some applications, though, can benefit from knowledge about how much of a given file is present in the page cache at any given time; the proposed cachestat() system call from Nhat Pham is the latest in a long series of attempts to make that information available.

In truth, even current kernels make it possible to learn which pages of a file are present in the page cache. The application just needs to map the file into its address space with mmap(), after which a call to mincore() will return a vector showing which pages in that file are resident. This is an expensive solution, though; it requires setting up a (possibly unneeded otherwise) mapping and returns information that, for many applications, has a higher resolution than is necessary.

The proposed cachestat() system call is rather simpler:

    struct cachestat {
        __u64 nr_cache;
        __u64 nr_dirty;
        __u64 nr_writeback;
        __u64 nr_evicted;
        __u64 nr_recently_evicted;
    };

    int cachestat(unsigned int fd, off_t offset, size_t len, size_t cstat_size, 
		  struct cachestat *cstat);

This call will check the pages of the file indicated by fd, starting at the given offset and going for len bytes, and count the number of pages that are in various states of residency. The offset must be page-aligned; len will be rounded up to a multiple of the page size if needed. The counts will then be returned in the structure pointed to by cstat. In that structure, nr_cache is the number of pages in the given range that are present in the page cache, nr_dirty is the number of those pages that are dirty (have been modified and not yet written back to persistent storage), and nr_writeback is the number of pages currently being written back.

The nr_evicted field provides the count of how many pages were once resident in the cache but have since been forced out, and nr_recently_evicted is the number of those that have been forced out in the recent past. In this case, the "recent past" is defined by the number of pages that have been evicted since the page in question was forced out; if that number is smaller than the process's working-set size, the eviction is deemed to be recent. These counts are obtained by looking at the shadow page-table information that was added to the kernel about ten years ago.

The size of the cachestat structure must be provided to cachestat() as cstat_size. This interface allows new fields to be added to that structure in the future; if cstat_size is smaller than the size as known within the kernel, data will only be provided up to the provided size, preserving compatibility. (If, instead, cstat_size is larger than what the kernel expects, the call will fail with an EINVAL error).

By not requiring the mapping and unmapping of the file(s) to be queried, cachestat() avoids most of the overhead created by the mincore() method. The fact that this call returns simple counts rather than detailed, by-page information is also helpful in the end; it seems that applications wanting this kind of information are interested in the number of cache-resident pages, but they don't really care about which pages are resident. So there is no point in returning the more detailed data.

One open question that is not well answered in this patch set, though, is: what kinds of applications will benefit from this information? When LWN covered a similar effort in 2010 (the system call was called fincore() then), the use case involved applications that call posix_fadvise() to bring data into the page cache prior to accessing it. These applications (SQLite is evidently one of them) know what their data-access patterns will be, but they have less information about how much of their data will fit into the page cache at any given time. By calling cachestat(), such an application can learn whether the pages it is prefetching into the cache are still there by the time it gets around to using them. If those pages are being evicted, the prefetching is overloading the page cache and causing more work overall; in such situations, the application can back off and get better performance.

So cachestat() appears to be useful, but whether there is room in the kernel for this new system call remains to be seen. Attempts to add this functionality have faltered for over a decade, perhaps due to the highly specialized nature of the use case. But, just maybe, the new interface and renewed push for inclusion will get it over the bar this time.

Comments (18 posted)

Composefs for integrity protection and data sharing

A read-only filesystem that will transparently share file data between disparate directory trees, while also providing integrity verification for the data and the directory metadata, was recently posted as an RFC to the linux-kernel mailing list. Composefs was developed by Alexander Larsson (who posted it) and Giuseppe Scrivano for use by podman containers and OSTree (or "libostree" as it is now known) root directories, but there are likely others who want the abilities it provides. So far, there has been little response, either with feedback or complaints, but it is a small patch set (around 2K lines of code) and generally self-contained since it is a filesystem, so it would not be a surprise to see it appear in some upcoming kernel.

Features

There is a lengthy introduction to composefs in the cover letter and more information in the documentation patch. Unlike many filesystems, composefs is not backed by a block device but instead by a set of regular read-only files: an image file that contains the directory structure and file metadata and a directory of content-addressed objects that are the file contents. Since the files themselves are in an object store, they are effectively deduplicated at the file level; files that have identical content are only stored once even if the metadata (e.g. owner, permissions, extended attributes) is different. In addition, when a file is read from composefs, the backing file is used—and cached in the page cache—so any read of another composefs file with the same content will use that page-cache entry.

The filenames used in the object store correspond to the fs-verity digests of the contents, so composefs can detect any changes to files using the fs-verity mechanism. The image file can contain the digests for the files to ensure they are not changed out from under composefs and the image file itself can be protected with fs-verity as well. The fs-verity digest of the image file could be retrieved from a secure location (e.g. a signed kernel command line) and passed to the mount command, allowing composefs to ensure that the entire filesystem is unchanged from its expected state.

The files needed for a composefs instance are created using the mkcomposefs tool:

    $ mkcomposefs --digest-store=objects rootfs/ rootfs.img

That command would process the rootfs directory, storing the image file corresponding to its directory structure and file metadata in rootfs.img, and create an object store with the file contents into the objects directory. As with Git, the objects directory has subdirectories named for the first two hex characters of the digest hash, each of which contains files named with the remaining characters of the hash.

A filesystem created that way could then be mounted as follows:

    $ mount -t composefs rootfs.img -o basedir=objects,verity_check=2 /mnt

The verity_check option governs whether to do fs-verity checking on the image and files; 0 disables fs-verity checking, 1 only checks images that specify it, and 2 requires fs-verity. There is also a digest option that can be used to pass the fs-verity digest hash for the image file to the mount command, which allows composefs to verify it. That provides end-to-end verification as noted in the cover letter: "So, given a trusted set of mount options (say unlocked from TPM), we have a fully verified filesystem tree mounted, with opportunistic fine-grained sharing of identical files."

Use cases

Multiple containers on a system often share many files, but that file data is typically replicated for each container image. Composefs can be used to create multiple, different, read-only container image files that all refer to the same object store, so that all of the files are only stored once. In addition, files that are being used by multiple containers simultaneously (e.g. shared libraries) will likely reside in the page cache, saving a slower retrieval from persistent storage.

The second use that Larsson and Scrivano envision is to replace the "link farm" that gets created for OSTree-based root filesystems with a composefs mount. Currently, OSTree uses hard links into its content-addressed object store, but that structure is not protected against changes because it does not get validated at run time, only when it is being created. By using fs-verity on the image file and object store, though, a composefs-based root filesystem would have its integrity protected automatically.

Beyond those two immediate uses for composefs, there are others on the horizon:

[...] there seems to be a wealth of other possible uses. For example, many systems use loopback mounts for images (like lxc or snap), and these could take advantage of the opportunistic sharing. We've also talked about using fuse [Filesystem in Userspace] to implement a local cache for the backing files. I.e. you would have a second basedir be a fuse filesystem, and on lookup failure in the first basedir the fuse one triggers a download which is also saved in the first dir for later lookups. There are many interesting possibilities here.

The reaction to the patches has been non-existent so far, other than some documentation updates suggested by Bagas Sanjaya. The facilities provided by composefs seem useful, however, and build on the fs-verity feature that is, after some fits and starts, already present in the kernel. There is also work in progress to support composefs in OSTree, but there is something of a chicken-and-egg problem until the filesystem lands in the kernel. With luck, and a continued lack of any serious opposition to composefs, that problem could be addressed—perhaps even early next year.

Comments (20 posted)