Leading items

Welcome to the LWN.net Weekly Edition for September 7, 2023

This edition contains the following feature content:

• Altering Python attribute handling for modules: a proposal for a mechanism to make some common mistakes easier to detect.
• The first half of the 6.6 merge window: a summary of the first batch of work pulled into the mainline for the 6.6 kernel release.
• Security topics: io_uring, VM attestation, and random-reseed notifications: patches to make io_uring optional, create a standard protocol for virtual-machine attestation, and notify user space that random-number generators need to be reseeded.
• Race-free process creation in the GNU C Library: integrating the pidfd mechanism into the posix_spawn() API.
• Reducing the bandwidth needs for fwupd: Fedora 40 may run a service to distribute firmware-update information on the local net.

This week's edition also includes these inner pages:

• Brief items: Brief news items from throughout the community.
• Announcements: Newsletters, conferences, security updates, patches, and more.

Please enjoy this week's edition, and, as always, thank you for supporting LWN.net.

Comments (none posted)

Altering Python attribute handling for modules

A recent discussion on the Python forum looked at a way to protect module objects (and users) from mistaken attribute assignment and deletion. There are ways to get the same effect today, but the mechanism that would be used causes a performance penalty for an unrelated, and heavily used, action: attribute lookup on modules. Back in 2017, PEP 562 ("Module __getattr__ and __dir__") set the stage for adding magic methods to module objects; now a new proposal would extend that idea to add __setattr__() and __delattr__() to them.

The idea came from a GitHub enhancement request that was filed for the mpmath Python module, which provides arbitrary-precision floating-point arithmetic. A common mistake is for users to import mpmath in an incorrect manner and then to end up setting an attribute on the module itself, which has no real effect, rather than on the context object mp. From the request:

    import numpy as np
    import mpmath as mp  # should be from mpmath import mp
    mp.dps = 50

The import statement for mpmath is incorrect, as noted, but it is a common mistake because of the way NumPy is usually (correctly) imported. The programmer thought they were setting the dps (decimal precision) attribute for the mpmath context object, but instead set the attribute on the module object itself, which has no effect on the precision.

The enhancement request notes that the problem is clearly a user error, but that it would be nice to somehow catch it and provide the user with an error message redirecting them to the proper setting (mp.mp.dps in this case). Since the functions of interest in mpmath are available from both the module object and the context object, the user may well not notice that their calculations are being done with the wrong precision. If the module could have a __setattr__() magic method that would be called when mp.dps is set (erroneously), the problem could be resolved by raising an exception. There is no support for __setattr__() on a module, however.

There is another way to get there, though, as is shown in the Python documentation. A module can create a subclass of ModuleType, which can contain other magic methods of interest, then change the class of the module object to be that subclass:

    import sys
    from types import ModuleType

    class VerboseModule(ModuleType):
	def __setattr__(self, attr, value):
	    print(f'Setting {attr}...')
	    super().__setattr__(attr, value)

    # change the class of the module instance __name__
    sys.modules[__name__].__class__ = VerboseModule

The problem with that approach, as pointed out by Sergey B Kirpichev in the initial discussion back in April, is that doing so negatively impacts the performance of attribute lookups for modules. He measured a nearly 3x difference in the speed of a lookup using the regular module object versus one with the __class__ modification (48ns versus 131ns). The lookup operation is commonplace for accessing anything that a module provides, of course.

The difference is that the __class__-based mechanism bypasses the internal C-based module object so all of its operations are done using the Python class. Allowing __setattr__() to be added to the module object, as was done in PEP 562 for __getattr__(), would mean that the C-based module object would only call out to Python when attributes are being set. It is worth noting that the module __getattr__() calls are only made if the attribute is not found on the object, which is also how it works for classes; so any performance penalty for adding __getattr__() is only paid for attributes that are being handled specially.

But, normally, looking up magic methods (also known as "dunders" for double underscore) like __getattr__() is done at the class level; those methods are defined for a class. PEP 562 added the ability to look up __getattr__() (and __dir__()) on module instances. The import machinery creates an instance from the ModuleType class when the module is imported. So a module that wants to add __getattr__() defines it at the top level of the module namespace , which is a big departure from the usual situation.

The April discussion showed a somewhat mixed reaction to the idea. Around the same time, PEP 713 ("Callable Modules") was proposed. It would add the ability to define a __call__() method for module instances, so that modules that only provide a single (main) interface can be called directly. The author of the PEP, Amethyst Reese, listed a number of standard library modules that could benefit from the feature. It also builds on the "precedent" of PEP 562.

In June, Victor Stinner posted a link to a GitHub issue in support of the module __setattr__() idea. He would like to protect the attributes of the sys module from accidental modification, which can result in strange error messages. For example, setting the sys.modules dictionary to an integer value will cause a later import to report:

    AttributeError: 'int' object has no attribute 'get'

He also suggested adding __delattr__() to the list of allowable module overrides so that efforts to delete attributes from modules like sys could be disallowed.

On August 31, Kirpichev proposed PEP 726 ("Module __setattr__() and __delattr__()"); he had drafted the PEP earlier, but it lacked a core-developer sponsor at that time. In the interim, he had worked on a reference implementation and gotten a sponsor. The reaction was generally favorable to the idea, but it was noted that there are ways around the protection for sys attributes that Stinner was looking for. As Oscar Benjamin put it:

This PEP does not in itself prevent anyone from replacing an object if they really want to. A determined monkeypatcher can still use sys.__dict__['modules'] = whatever. The intention here is more to prevent accidentally setting attributes. Otherwise the consenting adults principle still applies.

On September 5, both Jelle Zijlstra and steering council member Gregory P. Smith said that the PEP needed additional reasons to justify its addition to the language. Zijlstra said that Stinner's example was good, "but I'm not sure a single module is enough to justify a language change". Smith suggested that the PEP should change:

The current 726 text has a Motivation section that covers what one could do with it. But the thing I personally think is currently missing are specific use cases for why it'd be useful. ex: What problems are package or module maintainers having today that would be improved by having this in the future?

Smith noted that the council had just rejected PEP 713 (for adding __call__()) because there was "not enough compelling evidence of seemingly important practical uses for the feature". He suggested that PEP 726 would need to clear a similar bar in order to get accepted. Kirpichev thought that the example in the PEP (which is based around the mpmath problem that started the whole effort) was sufficiently concrete and that Stinner's example further showed the benefits.

He did update the PEP to try to make the motivation clearer; he also listed a few different real-world examples of the problem in his reply. Chris Markiewicz posted another use case, for handling a deprecated value for a module attribute, in the thread. Likewise, Andrej Klychin explained his use case for it, which involves invalidating a cache in the module when an attribute gets updated.

It is hard to say if there are enough "seemingly important practical uses" of the feature for the steering council to approve it. There is some discomfort with adding exceptions to the normal dunder lookup rules, though __setattr__() and __delattr__() seem like a reasonable extension of the PEP 562 change. There are a wide variety of ways that the proposed feature could be used, and the alternative mechanism, which slows down all module lookups, is not an attractive option—for something like sys in particular. It seems that there may be enough to clear the bar; the council is considering the PEP now, so a pronouncement should be coming fairly soon. If it is accepted, we should see it in Python 3.13, which is scheduled for October 2024.

Comments (5 posted)

The first half of the 6.6 merge window

As of this writing, 4,588 non-merge changesets have been pulled into the mainline repository for the 6.6 kernel release. The 6.6 merge window, in other words, is just getting started. Nonetheless, a fair amount of significant work has already been pulled, so the time has come to summarize what has happened so far in this development cycle.

Interesting changes pulled for 6.6, to date, include:

Architecture-specific

• The PA-RISC architecture has gained a just-in-time BPF compiler.

Core kernel

• The /sys/devices/system/cpu/smt/control knob has, until now, only been able to accept two values: on to turn simultaneous multithreading ("hyperthreading") on, or off to disable it. This knob has been enhanced to also accept a numeric value indicating the number of threads that should be enabled per core. Initially this feature will be used by the PowerPC architecture, which can (in some CPUs) run up to 16 threads in each core.
• The earliest eligible virtual deadline first (EEVDF) CPU scheduler has been merged; it should provide better performance and fairness while relying less on fragile heuristics. The merge message notes that there may be some rare performance regressions with some workloads, and that work is ongoing to resolve them.

Filesystems and block I/O

• There is a new flag, FSCONFIG_CMD_CREATE_EXCL, for the mount API; if it is present, it prevents a mount from sharing an in-kernel superblock with another mount. As explained in this commit, superblock sharing can result in mount options being ignored, which can lead to security problems. The intent is to enable the addition of a new --exclusive flag to the mount command so that administrators can request that superblock sharing not be allowed.
• The virtual filesystem layer now supports fine-grained timestamps on files. As described in this article, these timestamps allow filesystems like NFS to detect changes quickly for correct cache management while minimizing the overhead of maintaining timestamps in general. Currently fine-grained timestamps are supported by the Btrfs, ext4, tmpfs, and XFS filesystems.
• The tmpfs filesystem has gained a number of features, including quota support, user extended attributes, direct I/O, and stable directory offsets (fixing a problem with filesystems exported via NFS).
• The kernel will no longer allow the changing of permissions on symbolic links.
• The fchmodat2() system call has been added. This version supports the flags argument that is defined as part of fchmodat(), but which has never been supported by Linux. This new system call will enable the removal of some inelegant hacks that were needed by C libraries to implement the missing functionality. This merge message describes the currently supported flags.
• The erofs filesystem has gained a bloom filter that speeds up negative extended-attribute lookups and support for the Deflate compression algorithm.
• The XFS filesystem has gained the ability to use large folios in the page cache and some associated optimizations, all of which should improve performance significantly for some workloads.
• It is now possible to build a kernel without buffer-head support, though doing so greatly restricts the filesystems that are available. This is part of the ongoing effort to get rid of — or at least greatly reduce — the use of buffer heads in the kernel.
• The ublk user-space block driver has gained (undocumented) support for zoned-storage devices.

Hardware support

• GPIO and pin control: ADI DS4520 I2C-based GPIO expanders.
• Hardware monitoring: Renesas HS3001 humidity and temperature sensors.
• Miscellaneous: Loongson SPI controllers and Cirrus Logic CS42L43 SPI controllers.
• Networking: Broadcom ASP 2.0 Ethernet controllers, Marvell 88Q2XXX PHYs, and TI PRU ICSS Industrial Ethernet peripherals.
• Regulator: Qualcomm REFGEN voltage regulators, ADI MAX77857/MAX77831 regulators, Richtek RTQ2208 SubPMIC regulators, and Awinic AW37503 dual output power regulators.

Miscellaneous

• The kernel has moved forward to Rust 1.71.1 and bindgen 0.65.1. There have been a number of changes to low-level utilities; see this merge message for the details.

Networking

• The AF_XDP address family has gained the ability to deal with packets stored in multiple buffers; this changelog and this documentation patch have a fair amount of detail on how it works.
• There is a new BPF hook into packet defragmentation that makes it easier to see (and filter) complete packets; this merge message has a little more information.
• A new BPF hook (update_socket_protocol) allows a BPF program to change the requested protocol for a new socket. Its primary purpose seems to be to transparently cause programs requesting TCP connections to use multipath TCP instead. MPTCP has also gained some initial support for BPF programs that route packets to different subflows.
• The io_uring subsystem has gained partial support for network operations, though the full set will wait until the 6.7 cycle.

Security-related

• The seccomp() system call has gained a new flag (SECCOMP_USER_NOTIF_FD_SYNC_WAKE_UP) that indicates that events from the watched process will be handled synchronously; that allows the kernel to schedule the two processes more efficiently.
• The kmalloc() randomness patches have been merged. This work adds a bit of entropy to memory allocations, making exploits a bit harder.
• The SELinux subsystem, formerly "NSA SELinux", has been "de-branded". "While NSA was the original primary developer and continues to help maintain SELinux, SELinux has long since transitioned to a wide community of developers and maintainers".

Virtualization and containers

• The userfaultfd() system call has gained a new operation, UFFDIO_POISON, that can mark pages as being poisoned. The use case here is in virtual-machine migration; if a memory page has been corrupted (and thus "poisoned") on the old system, userfaultfd() can be used to retain that marking on the new system.

Internal kernel changes

• The "kunit" unit-testing framework can now run the Rust documentation tests along with the rest.
• The Frontswap mechanism was added to the kernel in 2010. In 2023, it's removal is now complete.
• The struct ptdesc patches have been merged. They take the page-table-related fields of struct page and split them into a separate structure, continuing the process of separating the various uses of struct page.
• A change to internal symbol handling was merged to make it harder for proprietary modules to bypass restrictions on GPL-only symbols.
• The dynamic software I/O TLB patch series has been merged, providing better support for devices that, for one reason or another, need to use bounce buffering.

The 6.6 merge window can be expected to remain open through September 10, after which the development community will spend seven or eight weeks stabilizing all of this work. As always, LWN will catch up with the rest of the merge window once it closes.

Comments (27 posted)

Security topics: io_uring, VM attestation, and random-reseed notifications

The kernel-development community has recently been discussing a number of independent patches, each of which is intended to help improve the security of deployed systems in some way. They touch on a number of areas within the kernel, including the question of how widely io_uring should be available, how to allow virtual machines to attest to their integrity, and the best way to inform applications when their random-number generators need to be reseeded.

Disabling io_uring

The io_uring interface has been a boon to users striving for the best performance with I/O-heavy workloads; it has finally given Linux an approach to asynchronous I/O (and more) that the community can be proud of. It has also brought a number of security-related bugs, to the point the Google recently described it as being "safe only for use by trusted components". It is thus not surprising that somebody (Matteo Rizzo, in this case) has put together a patch allowing the system administrator to disable io_uring entirely.

This patch adds a new sysctl knob (kernel.io_uring_disabled) that controls the availability of the io_uring feature. At the knob's default value of zero, io_uring remains available as always. Setting it to one disables it for unprivileged users, where "privileged" is defined as having the CAP_SYS_ADMIN capability. In response to a request from Andres Freund after a previous posting, Rizzo added another knob, kernel.io_uring_group, that can be set with a group number; any process that is a member of the indicated group is also allowed to use io_uring. Finally, setting kernel.io_uring_disabled to two turns the feature off entirely.

After five revisions, the patch seems about ready to go into the mainline; there does not seem to be any real opposition to it. One might wonder how long it will really be useful, though. As Ben Hawkes recently wrote, the bulk of the io_uring problems may have already been found:

The era of io_uring is probably coming to an end, but it's been a very popular area of research recently. It reminds me of the gold rush around unprivileged user namespaces. Basically these complex new kernel features are consistently more bug-prone than we'd like, and this pattern seems to repeat itself every few years.

In the case of io_uring, perhaps the worst problems have been found and the stream of vulnerabilities will begin to taper off.

Virtual-machine attestation

The field of confidential computing has put a lot of effort into the ability to run virtual machines that cannot be compromised or spied upon, even by the host computer on which those machines are run. Getting to that point requires a lot of system hardening, use of encryption, and hardware that provides features (such as encrypted memory) to protect virtual machines from the surrounding world. All that work will be for nothing, though, if a virtual machine is compromised in some way: if, for example, its data has been tampered with, or if the hardware features it is depending on are not really there.

Users of confidential-computing systems tend to start them and, after convincing themselves that all is well, entrusting them with the encryption keys or other secrets they need to get their job done. For a virtual machine, convincing an orchestration system is a matter of using the available integrity-measurement mechanisms and having the CPU attest to its own integrity using a secret key buried deeply inside. All of this information can be signed by a device like a trusted platform module, then passed out of the machine, where it can be verified externally.

Numerous vendors are working on this functionality and, naturally, each is solving the problem in its own way. This, as Dan Williams noted in this patch series, is not the best way forward:

The approach of adding adding new char devs and new ioctls, for what amounts to the same logical functionality with minor formatting differences across vendors, is untenable. Common concepts and the community benefit from common infrastructure.

Williams is working to provide that infrastructure. The result is a configfs interface where the orchestration system can create a directory, write nonce data to a special file (called inblob). The virtual machine will then read the nonce data, incorporate it into its attestation report, and make it available to be read from outblob. The orchestrator can then verify the signatures and nonce data; if everything checks out, the machine should be safe to use.

It's worth noting that this proposal says nothing about the format of the data written to and read from these configfs files; they are still specific to the confidential-computing mechanism that is in use. There is, evidently, a discussion underway concerning the standardization of this data, but it is not clear if or when that will happen. Meanwhile, though, there will at least be a uniform interface for working with this information.

Random reseeding

The kernel's random-number generator is meant to be fast, but it is still not fast enough for some users. In such cases, it is common to implement a pseudo-random-number generator in user space, which is seeded from the kernel at application startup. That can work well, but there is a problem: sometimes the random seed may be in danger of compromise and in need of replacement. This can happen, for example, if a virtual machine is snapshotted and later restored, resulting in two machines generating the same "random" number series from the same seed. This problem was addressed in the kernel in 2022, but it remains for user space.

The kernel is aware of events that may require reseeding a random-number generator; it is just a matter of making that information available to interested processes in user space. A system call to check whether reseeding is necessary could be added, but that would defeat the purpose of using the user-space generator in the first place; something faster is needed.

The approach currently under consideration can be seen in this short patch series from Babis Chalios. It allows a process to open /dev/random, invoke a new ioctl() to get a special-purpose file descriptor, then pass that descriptor to mmap() to map a single page of shared memory into the process's address space. That page contains a 32-bit value split into two fields: an eight-bit "notifier ID" and a 24-bit "epoch counter".

There are numerous notifiers in the kernel that may detect and signal the need to reseed the random-number generator; each of these is assigned a unique ID. Examples of notifiers might include the virtual-machine snapshot mechanism or a periodic timer. Whenever a notifier decides that a reseed is warranted, it increments the epoch counter and writes its own ID into the notifier-ID field; the combination of the two values ensures that the full 32-bit value will change with every update regardless of any races between notifiers. With this mechanism in place, a user-space process need only read this value before generating a random number; if it has changed since the last read, a reseed should happen before anything else.

Some discussion on the details of the reporting format are still ongoing (Greg Kroah-Hartman suggested using two 32-bit values), but otherwise this mechanism, which was evidently hashed out at the 2022 Linux Plumbers Conference, appears to be uncontroversial. Unless something surprising happens, reseed notifications should be ready for merging by the time the 6.7 merge window opens.

Comments (18 posted)

Race-free process creation in the GNU C Library

The pidfd API has been added to the kernel over the last several years to provide a race-free way for processes to refer to each other. While the GNU C Library (glibc) gained basic pidfd support with the 2.36 release in 2022, it still lacks a complete solution for race-free process creation. This patch set from Adhemerval Zanella seems likely to fill that gap in the near future, though, with an extension to the posix_spawn() API.

Unix systems refer to processes via an integer ID (the "process ID" or PID) that is assigned at creation time. The problem with PIDs is that they are reused over time; once a process with a given PID has exited and been reaped, that PID can be assigned to a new and unrelated process with the result that any given PID might not, in fact, refer to the process that the user thinks it does. To address this problem, the pidfd concept was introduced; a pidfd is a file descriptor that acts as a handle for a process. The process associated with a pidfd can never change, so many of the race conditions associated with PIDs do not exist with pidfds.

Current glibc releases include wrappers for a number of the low-level pidfd-related system calls, including pidfd_open(), pidfd_getfd(), and others. There is one piece missing, though: the ability to obtain a pidfd for a new process as that process is created. It is possible to use pidfd_open() to get a pidfd from a PID immediately after creation, but that still leaves a narrow window during which the process identified by a PID could exit and be replaced by another. Closing that window requires obtaining a pidfd from the kernel as a result of creating a new process, and glibc provides no way to do that.

That functionality could be provided by adding a wrapper for the clone3() system call, but there is some resistance to doing that. Instead, Zanella has taken the approach of enhancing the posix_spawn() API, which is seen by many as being a better approach to process creation (when immediately followed by an exec() call) than the Unix fork() model. The result is two new functions:

    int pidfd_spawn(int *restrict pidfd,
                    const char *restrict file,
                    const posix_spawn_file_actions_t *restrict facts,
                    const posix_spawnattr_t *restrict attrp,
                    char *const argv[restrict],
                    char *const envp[restrict]);

    int pidfd_spawnp(int *restrict pidfd,
                     const char *restrict path,
                     const posix_spawn_file_actions_t *restrict facts,
                     const posix_spawnattr_t *restrict attrp,
                     char *const argv[restrict_arr],
                     char *const envp[restrict_arr]);

Just like posix_spawn() and posix_spawnp(), these functions execute a combination of clone() and exec() to create a new process running the program indicated by file or path. The return value, though, will be a pidfd identifying the created process rather than a PID.

If the creator needs to know the new process's PID, that can be obtained by a new function added by the patch set:

    pid_t pidfd_getpid(int pidfd);

This function obtains the PID by looking at the /proc entry for the given pidfd.

The new functions are implemented with clone3() to obtain the pidfd during process creation, without a race window. Using clone3() makes some other things possible as well, specifically creating the new process in a different control group than the creator's. Zanella has made this capability available as well, via an extension to the posix_spawn() attribute mechanism. Creating into a different control group is available for posix_spawn() as well as pidfd_spawn().

While posix_spawn() is seen by many as a better model for the combination of fork() and exec(), it does not provide all of the functionality that is available. For cases where this API is not sufficient, earlier versions of the patch set included a function called fork_np() as a separate wrapper around clone3() that would return a pidfd identifying the new child process. Florian Weimer complained that this interface differs too much from what the kernel provides, though, and is "not future-proof at all". He asked Zanella to leave this function out of the series for now, and it has been duly removed from later versions of the series.

Rich Felker, instead, objected to the concept in general, claiming that any PID-related races are "purely programmer error" and that "making a new, complex, highly nonstandard interface to work around a problem that's programmer error, and getting this nonstandard and nonportable pattern into mainstream software, has negative value". It would be better, he said, to fix the software affected by this problem. Luca Boccassi disagreed, though, saying that "these are real race conditions, that cannot be solved otherwise". Weimer also said that there was value in introducing the pidfd functionality.

While there has been no definitive resolution to this particular disagreement, the fact remains that PID races can be a problem, and there are users (such as systemd) that would like to have this type of API to avoid those races. It thus seems reasonably likely that pidfd_spawn() (though perhaps not fork_np()) will eventually find its way into glibc.

Comments (91 posted)

Reducing the bandwidth needs for fwupd

The Linux Vendor Firmware Service (LVFS) provides a repository where vendors can upload firmware updates that can be accessed by the fwupd firmware update daemon on Linux systems. That mechanism allows users to keep the hardware components of their systems up to date with the latest firmware releases, but it has gotten so popular that the daily metadata queries are starting to swamp the LVFS content delivery network (CDN) server. So Richard Hughes, who developed fwupd and LVFS, suggested that it would make sense to start looking at ways to reduce that burden; the idea was discussed in a recent thread on the Fedora devel mailing list.

Hughes had a specific suggestion for an approach to spread the load by adding passim to Fedora 40 as an fwupd dependency and, potentially controversially, enable it by default. Passim will effectively allow systems on the same local network to share a single copy of the LVFS metadata, so that it is only downloaded once for a particular LAN. He wanted to discuss the idea before creating "lots of unnecessary drama". He described passim as follows:

The tl;dr: is I want to add a mDNS [multicast DNS] server that reshares the public firmware update metadata from the LVFS on your LAN. The idea is that rather than 25 users in an office downloading the same ~2MB file from the CDN every day, the first downloads from the CDN and the other 24 download from the first machine. All machines still download the [tiny] jcat file from the CDN still so we know the SHA256 to search for and verify.

It turns out that fwupd has been adopted by other operating systems, "ChromeOS, FreeBSD, Windows and macOS", which means that there is a "need to scale things up a couple of orders of magnitude". He thinks that passim could make a good test case for rolling out similar functionality for the DNF package manager and OSTree/libostree content. One important thing to note is that passim is not handling the actual firmware-update files, at least by default; it is only concerned with the metadata of the LVFS repository, which is roughly 2MB in size and is often downloaded daily.

Some of the concerns that were raised in the thread are, unsurprisingly, security-related. Getting some kind of malicious firmware metadata blob is the last thing anyone wants. But, as Stephen Smoogen put it, passim works on the assumption that "your local network (aka LAN) is a nice and friendly place, without evil people trying to overwhelm your system or feed you fake files", which may not actually be the case. He listed a few possible problem areas, including denial-of-service attacks or stolen signature keys that allow malicious versions of the metadata package to be created and continue to be distributed in the absence of some kind of key-revocation scheme. Hughes acknowledged those possibilities but noted that passim chooses a random source server if there are multiple offerings on the LAN, which may help reduce the scope of the problem if there is a single malicious provider.

The passim web page states that the metadata files are signature protected, and that clients will fall back to retrieving them from the CDN if the signature check fails. Files that are shared from a passim system are configured with a maximum age and a limit on the number of times they will be shared; once those are exceeded, the files are deleted and will no longer be advertised to others on the LAN. Clients will request the metadata file using HTTPS; local passim servers will use a self-signed TLS certificate for use in encrypting the connection.

As might be guessed, the use of a self-signed certificate was questioned; Leon Fauster asked what the benefit was since "it does not provide any 'security'". Hughes replied that it provides encryption so that other systems on the LAN cannot snoop on the traffic ("so client A can provide the file to client B without client C being aware what's being sent"). But, as Chris Adams pointed out, a self-signed certificate does not protect against a "man in the middle":

[...] there's no way for client B to know it is really talking to client A - it could be talking to client C with a man-in-the-middle attack and a different self-signed cert pretending to be client A.

It is not entirely clear what would be gained by snooping on the traffic, beyond knowing that a client is requesting the metadata, which might be inferred without actually snooping. Doing meaningful identification of LAN systems may be difficult as well. But the TLS protection is also meant to placate corporate security teams that would be unhappy with regular HTTP, "even if it's got two other layers of protection", Hughes said.

Simo Sorce had several suggestions, agreeing with Adams that the self-signed-certificate approach was only useful against passive, rather than active, attacks. "Trust on first use" (or TOFU) could be used on the clients to minimize the "window of impersonation", but there are still problems as it "requires clients to cache an association and then has weird failure modes to be dealt with if one of the actors get re-imaged or changes the cert for any reason". Overall, he did not think that using TLS added any real value; security teams that require it could provide their own certificates. Another option might be to use a pre-shared key, though Sorce did not really recommend it.

Using TOFU had some appeal, Hughes said, and asked for opinions on that. Sorce wondered how passim would deal with the new-certificate problem and asked about certificate expiration. Hughes's reply that currently the certificates effectively never expire (9999-year expiration) was not particularly well-received ("Ugh."). Meanwhile, certificate updates would cause that system to be ignored, Hughes said; that would eventually lead to all devices ignoring each other, Sorce pointed out.

Having a single host that was responsible for gathering the metadata file periodically and then serving it to the rest of the network was suggested by Peter Robinson. Hughes seemed amenable to that idea and asked Robinson to file a GitHub issue in order to track it. As the issue notes, some system administrators want to control where services run; there may also be a bastion host that is the only externally connected server, so it would be useful to be able to pin passim downloads to a particular host.

Enabling passim by default was questioned; Benson Muite said that it would be better to make passim opt-in, especially "for computers in an institutional setting where the LAN is well controlled". But Hughes replied that requiring people to enable it will simply lead to almost total lack of adoption and, thus, no real reduction in the network traffic. Marcus Müller agreed that opt-in was not really plausible, especially since it would need to enabled on a host-by-host basis.

He also asked about the plans for sharing firmware files in addition to the metadata; there are privacy concerns with advertising the firmware files that a system has downloaded. Hughes agreed, but firmware files are not included in phase one of his plan; if they are to be added at some point, there are some things to consider:

It's non-free software (which you have permission to redistribute, but still unpalatable for many) -- the compromise I've done for people changing the default to "metadata,firmware" is that you need to reboot into the new firmware before the published firmware gets shared; on the logic that you don't want to advertise to the world that you're currently running insecure firmware.

Privacy also came up in the context of Vit Ondruch's questions about the need to download all of the metadata daily. A given system will only have a handful of devices that get updated firmware, which should require far less than 2MB of metadata, he said. But Hughes replied that the whole database is always delivered from LVFS:

It is the metadata for every device -- every fwupd client deliberately gets the entire catalog rather than making a bespoke request like Windows update. This ensures that the LVFS doesn't know what hardware you have on your computer, and couldn't provide that kind of data even if compelled to by law enforcement. The entire architecture is privacy centric, and also allows it to scale to millions of devices without having thousands of servers.

There are, obviously, some problems that need to be addressed, but passim seems like a reasonable solution, at least for the metadata. The problem of scaling LVFS is real, so something will likely need to be done before too long. Those measures (whether passim-based or not) will need to be adopted widely, though, in order to make a serious reduction in the traffic burden. Fedora 40 is scheduled for April 2024, so even if the feature gets approved—Hughes is planning to proceed in making a change proposal—it will likely not make much of a dent until after that. A real dent in the traffic levels will require adoption in most or all of those other distributions or OSes that are now using fwupd.

Comments (56 posted)