LWN.net Weekly Edition for January 2, 2020
Welcome to the LWN.net Weekly Edition for January 2, 2020
Welcome to the first LWN Weekly Edition for the 2020's! This edition contains the following feature content:
- LWN's 2020 vision: our traditional selection of unlikely predictions for the coming year.
- Python first(): how to quickly find a matching string in Python — with several digressions.
- Cloning into a control group: a clone3() change to address a longstanding process-creation race condition.
- KRSI — the other BPF security module: a new approach to system monitoring using BPF.
- Fedora and fstrim: should Fedora systems run fstrim by default?
This week's edition also includes these inner pages:
- Brief items: Brief news items from throughout the community.
- Announcements: Newsletters, conferences, security updates, patches, and more.
Please enjoy this week's edition, and, as always, thank you for supporting LWN.net.
LWN's 2020 vision
January 1, 2020 marks the beginning of a new year and a new decade. Many things will doubtless change over the course of this year in the free-software community and beyond, while others will remain the same. One thing that will certainly hold true is LWN's tradition of starting the new year with some ill-advised predictions about what may be in store. Your editor has no special vision, but neither does he fear being proved badly wrong in a public setting — it's all in a day's work.The Python community will continue to chart its post-Guido course. One relatively unnoticed development in the Python project's 2020 steering-council election was the quiet withdrawal of project founder Guido van Rossum's nomination. While he will still participate, he no longer wants a leadership role in the project. Increasingly, Python will have to find its way without the developer who has guided it since the beginning.
Similarly, the GNU Project will have to decide what it will be in
the 2020s. This project's founder, Richard Stallman, retains his post as
its leader, and it's possible that he will still be there at the end of the
year. But there is ongoing
restlessness in the project that brings a desire for new leadership and
new directions. After all these years, it still sometimes seems like GNU
is stuck trying to reproduce the Unix workstations of the 1980s, leaving
much of the current computing environment to systems that are half-free at
best. We need a GNU project for mobile devices, private clouds, home
systems, embedded applications, and more. Stallman once famously said "I'm not really concerned with what's
running inside my microwave oven.
" The time has long since come to be
concerned about such things; wouldn't it be a great thing if a newly
reinvigorated GNU project were to take on this challenge?
There is a common theme to those last two items that may not really arise in 2020, but will certainly come about in the 2020s: many of our leaders in the free-software community got their start in the 1980s and 1990s. In talking with those people, your editor increasingly gets the sense that many of them are thinking they have done this sort of work for just about long enough. Retirements will increase in the coming years, and we will lose much of the skill and experience that has gotten us this far. There are plenty of skilled and motivated younger developers who can certainly pick up where these folks leave off, but the transitions will go more smoothly if they are properly planned for.
The presence of highly experienced developers is perhaps felt most strongly in the kernel project. This is a good thing in a setting where mistakes can be catastrophic on a large scale, but it also partially explains why kernel developers have a distinctly old-school workflow. This is, remember, the project that only started using a source-code management system in 2002. Change is afoot, though, and the kernel workflow efforts will begin to bear fruit in 2020. We'll still be sending patches over email at the end of the year, but we'll have an improved understanding of what a better solution that can work at the kernel project's scale will look like. As part of this effort, the kernel's testing tools will also continue to improve at a rapid rate.
For all its faults, though, the kernel's workflow produces reliable results. Thus, it is relatively safe to predict that the next long-term stable kernel release will be 5.9, on November 1, 2020. Most kernel release cycles have taken ten weeks in recent times. Should the process run a little faster and produce some nine-week releases, then the LTS release will, instead, be 5.10 in late December.
Strife in the Debian community will continue despite the project's recent decision on init-system policy. Debian suffers from a combination of hostile outsiders and certain project members who actively work to create turmoil on its mailing lists. The Debian community is resilient, and it will get past these problems too. But its mailing lists may not be a lot of fun to read in the meantime. The Devuan project, despite having picked up a post-vote defector or two from Debian, will continue to struggle for developer time and relevance.
It is now 20 years since the year-2000 bug failed to destroy civilization. Some now see y2k as an irrational panic, but without a focused effort there would have indeed been serious problems. The year-2038 bug looks distant and harmless, but it, too, requires a determined effort to avert real problems. The good news is that the year-2038 fixes will be mostly completed in 2020, at least at the kernel level. Yes, your editor has predicted this before and taken some good-natured grief for it, but this time it's for real.
BPF API issues will not go away in 2020. The in-kernel BPF virtual machine makes a great deal of power available to user space, and that will only increase in the coming year. Software providers will realize that it is often far easier to ship a BPF program to customers than to get a needed feature into the kernel itself. Sooner or later, one of those programs will break as the result of internal kernel changes and those customers will complain, quite possibly causing needed work to be reverted. The fact that much BPF functionality is available to proprietary programs is likely to create troubles of its own. A quick check shows 142 BPF helper functions in the 5.5-rc4 kernel; only 34 of those are marked GPL-only. Some developers are sure to see the growing set of BPF functions available to proprietary code as an erosion of whatever GPL protection the kernel has left.
Expect perturbations in the employment market as various economic realities catch up with us. Both the dotcom crash and the 2008 crisis impacted the development community; the next downturn will do the same. As in the past, Linux and free software as a whole will not be significantly hurt when this happens, but the same cannot be said for individual developers, at least in the short term.
Finally, though this is perhaps more of a wishlist item than a prediction: the free-software community has to think more deeply about what it is creating and how that will affect the world. At many levels free software has won; the world's computing infrastructure is built on the code we have created. We have much more control over our computing environments than we would have believed all those years ago; it's a great success.
But we have also created — and continue to develop — the foundation for the most intensive surveillance machine the world has ever known. We have helped to build a world where sex toys report on their usage patterns, video doorbells cooperate with police, companies are attacked via fish-tank thermometers, cars can be disabled remotely, elections can be attacked by hostile countries, televisions report viewing habits, home devices phone home with audio recordings, and more. All of this machinery is used to sell us things, or to direct our behavior in even less benevolent ways. It is increasingly concentrated in the hands of a tiny number of huge companies; using it will inevitably prove to be a temptation that governments are unable to resist. We wanted a world where we controlled our computers, but we have created a world where our computers are used to control us. It is not enough to blame the companies involved; we have helped to build this world.
Back in 1983, the GNU project helped to focus attention on the need to create a free operating system for our desktop computers. Now we desperately need a project that can bring a similar level of attention to the need for free computing in every aspect of our lives. Free software was able to demonstrate enough advantages to convince companies to adopt it, even if it was less convenient in some ways at the time. The next generation of free software (along with freedom-respecting devices built on it) has to succeed in convincing people that there is a better way. If the free-software community can rally around this project, the outlook for the 2020s in general will be much improved.
On a more mundane level, LWN is about to celebrate its 22nd birthday. Thanks to the solid support from you, our readers, we are still going strong after all these years. We wish for a great 2020 for all of you and we look forward to write for the best set of readers any publication could hope for.
Python first()
Python prides itself on being a newbie-friendly language; its developers have gone out of their way to try to ensure that easy tasks are straightforward to program. A recent discussion on the python-ideas mailing list looked at a use case that is common, but often implemented in an inefficient, incorrect fashion, with an eye toward making it easier to do correctly. Finding the first match for a regular expression in a body of text is where the conversation started, but it went in some other interesting directions as well.
findfirst()
Juancarlo Añez started things off with a post
about the use case, noting that there are "many ugly recipes
"
for getting the first match for a regular expression (regex), but that a
lot of programmers resort to:
matched = re.findall(regex, text)[0]
That formulation will fail with an IndexError if there are no matches. Even if it succeeds, it does a lot of extra work that gets thrown away to produce the first match; the entire list of matches will be created by re.findall() and all but the first will be wasted effort. He suggested:
def findfirst(regex, text, default=None, flags=0): return next(finditer(regex, text, flags=flags), default=default)
That would use the next() function with a default value (None by default) on the iterator returned by re.finditer(). That would produce the first match or the default value without causing an exception if the iterator raised StopIteration because it had no more values. As pointed out in the thread, though, re.finditer() returns Match objects, rather than a list of strings or tuples like re.findall(). The "obvious" switch to re.findall() will not work because a list is not an iterator, so next() will raise a TypeError.
There was some additional confusion about the two functions and their return values in the thread. Serhiy Storchaka pointed out that re.search() could be used in place of the re.finditer() in the example as follows:
re.search(regex, text, flags) or defaultHowever that returns a Match object (or the default), which Guido van Rossum said may not be what is wanted:
Storchaka was not convinced that a good case had been made for adding such a function, however. Añez tried to fill in the use case:
He pointed out that using the iter() function would make an iterator out of the list returned from re.findall(); that could be used to make his version of findfirst() behave as he wanted. But he also noted that a common extension to the itertools module is a function called first():
def first(seq, default=None): return next(iter(seq), default= default)
He suggested that first() would make a good addition as well:
Note that the case for findfirst() is weaker if first() is available. Yet findfirst() solves the bigger problem.
Andrew Barnert pointed out that using re.findall() is not particularly efficient, though:
To Van Rossum, that indicates
even more need for a findfirst(), presumably one that is optimized
for its task, rather than using re.findall(). But in analyzing some
examples posted
by Kyle Stanley, Storchaka found that they could generally be rewritten to
avoid the troublesome re.findall(...)[0] pattern, often using
re.search()—which is optimized to find the first
occurrence. "It seems that in most cases the author just [does] not know
about
re.search(). Adding re.findfirst() will not fix this.
"
But first()
Añez seemed to agree that basing findfirst() on re.search() made sense, but he also started a new thread to argue that first() should be added, perhaps as a builtin in the standard library. The example he gave in that message was unconvincing to a number of participants, but the general idea of adding a first() sparked a long discussion down several paths.
Steven D'Aprano thought that adding a builtin as a thin wrapper around next() was not providing much of an advantage. He also showed that there are some potential "gotchas" in first() as described:
# non-iterator iterable py> obj = [1, 2, 3, 4] py> [first(obj) for __ in range(5)] [1, 1, 1, 1, 1] # iterator py> obj = iter([1, 2, 3, 4]) py> [first(obj) for __ in range(5)] [1, 2, 3, 4, None]
That could be fixed by adding a way to peek ahead in an iterator, though
there are some oddities to that as well.
But Oscar Benjamin is concerned
that the StopIteration exception essentially leaks an
implementation detail to the users of a function that uses the one-argument
form of next(). It would make more sense to catch that exception
and return a ValueError instead. He pointed to PEP 479
("Change StopIteration handling inside generators
") as one
proposal that changed the behavior for generators, but did not do so for
iterators, which he sees as a problem.
It is interesting to note that there are some corners of Python that even
Van Rossum has lost track of.
Tim Peters said
that the simple one-line implementation of first() is equivalent
to a more complicated version that Van Rossum had posted. Peters
suggested that there should be two bars to pass before something gets added
as a builtin: "In other words, in that context, the bar for 'building it
in' consists far more of 'will it be used?' than 'is it difficult or
tricky or long-winded?'.
"
It turns out that the existence of the two-argument form of next(), which will not raise an exception and simply return the default (i.e. the second argument) if the iterator is exhausted, is something that Van Rossum only learned about in the thread. He thinks others may be in the same boat, which puts first() in the "tricky" category:
But Barnert asked:
"Are people who never find 2-arg next going to find
itertools.first?
"
Van Rossum acknowledged
the problem but thought that adding first() to itertools
would likely result in people finding out about it since that module tends
to attract some attention in blogs and tutorials. Storchaka wondered if
simply adding it to the itertools
recipes section of the documentation would be sufficient. Van Rossum
thought
otherwise: "Not nearly as much as just adding the function. We'll be
doing humanity a favor if we just add it.
"
The difference in behavior for first() on an iterator versus on an iterable (e.g. a list) that D'Aprano had pointed out showed up again later in the thread. Because next() changes the state of an iterator, first() will give surprising results. D'Aprano showed that first() is not idempotent so that calling it twice with the same iterator argument will not give the same results—which is not true for an iterable. He also said that this behavior makes the name "first" ambiguous:
return the first element of a sequence or container (in standard iteration order), OR the *next* element of an iterator
That set off some discussion of the right spelling for first() and whether if should raise an exception (something other than StopIteration) if there is no default passed to it. The latter would effectively remove one of the main features of first() that many proponents preferred, at least early on. Like the get() method for dictionaries, first(), as it was originally proposed, would never raise an exception. As Van Rossum put it:
Similarly, there is already a spelling of first() (or close enough) that raises: 1-arg next(). If 1-arg first() would also raise, it would fail the rule "similar things should be spelled similarly, and different things should be spelled differently".
I am not making this rule up -- it's an old design principle that Lambert Meertens used for ABC, Python's predecessor. It just didn't make it in the Zen of Python, unless you want to interpret "there should be one [...] way to do it" as its spiritual descendant.
Peters, who is in favor of a first() that can raise an exception,
suggested
that first(iterator) and next(iterator) "look
darned near identical
" but act quite differently if the iterator is
exhausted. He sees get() differently because it is a method on
the dict object, but Van Rossum was unconvinced
by that argument. He was not particularly interested in prolonging the
discussion, however: "But if
everyone thinks that first() should raise, fine, this thread is way too
long already (I should have kept it muted :-).
"
Others were not hesitant to keep the discussion going, though. In the end,
Añez came
to the conclusion that a one-argument first() should raise an
exception "so it can be applied to iterables that may yield None as
the legit, first value
". He also noted that the
the more-itertools
module from the Python Package Index (PyPI) has a first()
that raises ValueError on exhaustion if no default is given. Even
if first() is not "promoted" into itertools itself, he is
in favor of updating the itertools documentation and recipes to better address the
proper way to get the first entry in iterators and iterables.
Getting back to the original problem, Añez posted a findalliter() that would return strings or tuples like re.findall(), but do so with an iterator, rather than creating a whole list that might just be thrown away. He then defined findfirst() in terms of that (and first()).
It is not entirely clear where things go from here. In order to add either function to the standard library, it would seem that a PEP is needed; there doesn't seem to be a lot of opposition to the overall idea. A documentation change is likely even easier. But the discussion provided an interesting look into the Python development process, which tends to start out with postings to python-ideas to be fully hashed out before moving over to python-dev once a PEP has been drafted. The discussion also seems to have brought some semi-obscure corners of the language out of the darkness; much was learned in the process of working through it.
Cloning into a control group
The Linux control-group mechanism was designed to make it easy to assign processes to groups or move them around; it is a simple matter of writing a process ID to the appropriate cgroup.procs file in the control-group filesystem hierarchy. That only works for processes that actually exist, though. Adding the ability to place a new process into a control group at birth is the subject of this patch set from Christian Brauner.Before getting into the details, one might naturally ask why this capability is needed, given that Linux has done without it since control groups were first created. The answer is that current kernels present a bit of a race condition for process managers using control groups. A manager can create a process, and it can assign it to a group, but it has little control over what happens between those two actions. That means, for example, that a process might run briefly before being placed into a group where its resource usage can be accounted for properly. The amount of error introduced is likely to be small, but people who are into accounting tend to be intensely irritated by such things.
Placing a process into its intended control group at birth avoids that kind
of messiness. It also enables actions like creating a process in a frozen
group, ensuring that it will start in a frozen state and not run at
all until the process manager allows it to. Finally, as Brauner notes, this
feature "simplifies container
creation and exec logic quite a bit
".
Cloning a new process directly into a control group is only available with version-2 control groups, and it can only be invoked with the brand-new clone3() system call, which has this interface:
int clone3(struct clone_args *args, size_t size);
The decision to put all of the arguments to clone3() into a structure simplifies the prototype, but it also adds extensibility. That is demonstrated in this patch set, which adds a new field to struct clone_args:
u64 cgroup;
This new field naturally increases the size of the clone_args structure. Any call to clone3() must pass the size of the structure it is using as the size parameter, so the kernel is able to recognize whether the caller is using a version of the structure containing the cgroup field or not. This new field, in other words, can be added without breaking code that was built before its addition.
To create a process into a different control group than the parent, one places a file descriptor for the appropriate control-group directory into that cgroup field and adds CLONE_INTO_CGROUP to the flags field in that same structure. If all goes well, the child will be created in the indicated group; otherwise the call will fail.
Placing a process into a control group in this way must adhere to all of the usual rules. For example, processes are only allowed to be placed in leaf nodes in the hierarchy, not in the internal nodes; an attempt to create a process in an internal node will fail. Similarly, all threads of a process must normally be in the same control group (though thread mode can relax that restriction in some cases); an attempt to separate threads improperly will fail.
This patch set is quite new (this is its first posting), so there has been little time for the development community to review it. The feature seems useful enough that it is hard to imagine a great deal of opposition to the goal. Whether the implementation stands up to review remains to be seen, but it seems likely that this enhancement will find its way upstream in the relatively near future.
KRSI — the other BPF security module
One of the first uses of the BPF virtual machine outside of networking was to implement access-control policies for the seccomp() system call. Since then, though, the role of BPF in the security area has not changed much in the mainline kernel, even though BPF has evolved considerably from the "classic" variant still used with seccomp() to the "extended" BPF now supported by the kernel. That has not been for a lack of trying, though. The out-of-tree Landlock security module was covered here over three years ago. We also looked at the kernel runtime security instrumentation (KRSI) patch set in September. KP Singh has posted a new KRSI series, so the time seems right for a closer look.While KRSI is implemented as a Linux security module and is able to make access-control decisions, access control does not appear to be the core goal behind this work. Instead, KRSI exists to keep an eye on system behavior overall in order to detect attacks. It is, in a sense, better thought of as an extension of the kernel's audit mechanism that uses BPF to provide a higher level of configurability beyond what the audit subsystem can do.
The concept behind KRSI is simple enough: it allows a suitably privileged user to attach a BPF program to any of the hundreds of hooks provided by the Linux security module subsystem. To make this attachment easy, KRSI exports a new filesystem hierarchy under /sys/kernel/security/bpf, with one file for each hook. The bpf() system call can be used to attach a BPF program (of the new type BPF_PROG_TYPE_LSM) to any of these hooks; there can be more than one program attached to any given hook. Whenever a security hook is called, all attached BPF programs will be called in turn; if any BPF program returns an error status, then the requested action will be denied.
Many readers will be thinking that this mechanism sounds a lot like Landlock. While the fundamental idea — attaching BPF programs to security-module hooks — is the same, the underlying goals are different, and that leads to a different implementation. KRSI is a tool for system administrators who are interested in monitoring the behavior of the system as a whole; attaching a BPF program requires the CAP_SYS_ADMIN capability. Landlock, instead, is intended to allow unprivileged users to sandbox programs that they are running, so no privilege is needed to attach a BPF program to a hook via Landlock.
This difference fundamentally affects how these modules execute. Consider, for example, the hook that the kernel calls in response to an mprotect() call from user space:
int security_file_mprotect(struct vm_area_struct *vma, unsigned long reqprot, unsigned long prot);
In KRSI, the three parameters to this hook will be passed directly to any attached BPF programs; those programs can follow the vma pointer to learn all about the affected memory area. They can also follow vma->vm_mm to get to the calling processes top-level memory-management data (the mm_struct structure). There is, in short, a lot of information available to these programs.
The Landlock situation is different. Since Landlock hooks are under the control of unprivileged users, they cannot be allowed to just wander through kernel data structures. So a Landlock hook for mprotect() is passed a structure like this:
struct landlock_ctx_mem_prot { __u64 address; __u64 length; __u8 protections_current; __u8 protections_requested; };
In other words, the information passed to this hook contains nothing that user space did not already know. That makes it safe for the intended use case, but is likely to be too limiting for the global auditing case.
The advent of speculative-execution vulnerabilities, along with other
factors, has led to a slow simmer of questions about whether it can ever be
safe to allow unprivileged users to run extended BPF code in the kernel.
The BPF developers themselves have been coming
to the conclusion that it cannot be done, and have scaled back their
plans to make unprivileged BPF available. Indeed, even Mickaël Salaün, the
author of Landlock, now feels
that "it is quite challenging to safely expose eBPF to malicious
processes
". He went on to say:
So, while it may tempting to see KRSI and Landlock as being in competition with each other, that does not really appear to be the case.
There does not appear to be any fundamental opposition to KRSI — so far — but Casey
Schaufler did raise
the inevitable concern with this approach: "This effectively exposes
the LSM hooks as external APIs. It would mean that we can't change or
delete them.
" API issues often come up around BPF, especially in the
tracing area, so it is unsurprising that this question would arise here.
In this case, Singh replied: "we *do not* want to make LSM hooks
a stable API and expect the eBPF programs to adapt when such changes
occur
". It has repeatedly been made clear, though, that such
expectations do not override the kernel's stable-ABI rules. Given the
power that would be available to KRSI hooks, it is reasonable to expect
that they would be used for a range of purposes far beyond those envisioned
by its developers. If unrelated kernel changes break the resulting
programs, there is a good chance that they would be reverted.
Additionally, one could argue that this kind of problem is more likely to come about with KRSI than with, for example, tracepoints. While tracepoints have been added as an explicit way to make specific information available to user space, the security-module hooks were designed for internal use. They expose a lot of information, in internal-kernel formats, that one might otherwise not choose to make available even to privileged users. That can only make them more likely to break as those kernel data structures change. Changes to the security hooks are not that common, but they do happen (example); developers are unlikely to react well to the idea that they would no longer be able to make that kind of change.
The ABI issue could thus end up being the biggest obstacle to the merging of KRSI, even though such concerns have not (yet) stalled efforts in areas like tracing. It will be interesting to see what happens as the awareness of this functionality spreads. The usefulness of KRSI seems clear, but the potential for long-term problems it could bring is rather murkier.
Fedora and fstrim
A proposal to periodically run the fstrim command on Fedora 32 systems was discussed recently on the Fedora devel mailing list. fstrim is used to cause a filesystem to inform the underlying storage of unused blocks, which can help SSDs and other types of block devices perform better. There were a number of questions and concerns raised, including whether to change the behavior of earlier versions of the distribution when they get upgraded and if the kernel should be responsible for handling the whole problem.
The proposal
for a Fedora 32 system-wide change to "enable fstrim.timer by
default
" was posted by program manager Ben Cotton on behalf of its
owner, Chris Murphy. The fstrim.timer systemd unit file simply runs
fstrim.service (which runs fstrim) weekly on mounted filesystems.
But some devices have imperfect or broken support for the trim (or discard)
operation; even those devices with good support may take some time to
act on the trim commands, which delays other I/O to the device. The timer
is set for Monday at 00:00 local time,
which is meant to avoid affecting most users, though sleeping and
powered-off systems will run fstrim when they wake up or boot.
Under those circumstances, the delay may be more noticeable, but: "It's
expected most users won't notice this.
"
Some are concerned with changing this behavior on package updates or upgrades from previous Fedora versions, however. Stuart D. Gathman said that if the user disables the feature, it should not be re-enabled with updates. John M. Harris Jr. thought it should only be enabled for new installs and not change when upgrading to a new Fedora version. Murphy disagreed to some extent:
He also noted that customizations made to the timer and service unit files should be done in /etc, which would allow those changes to survive an upgrade to the base files that live in /usr. Harris complained that it would be hard for users to discover this change, but others pointed out that there would be an entry in the release notes, which could contain information on how to disable the functionality.
Lennart Poettering wondered
why the kernel did not simply handle the trimming itself. Waking up user
space to tell the kernel to do something strikes him as sub-optimal: "If this is generally
desirable, why is something as trivial as that not a kernel
functionality anyway?
" Vitaly Zaitsev suggested
that the trim operation may cause data corruption on some devices, but
Poettering pointed out
that moving the decision to user space did not change that one way or the
other. Beyond that, though, he asked about
the difference between this new Fedora feature and the discard
mount option for (at least) ext4, XFS, and Btrfs; "So
why do we need a userspace component to enqueue the same operation if
the file systems can do that anyway, at much better times?
"
The mount option does the trim/discard operation for each freed block, Louis Lagendijk said, which may lead to unpredictable latencies at unexpected times. Part of the problem is that there is insufficient information available from the devices that would allow the kernel to make the "right" decision, Murphy said. So other distributions (and operating systems) have simply been periodically doing a batch trim of the free blocks.
In another message, Murphy noted an LWN article from May that described some of the problems with discard. He is not opposed to eventually finding a kernel solution, but it is all rather messy at this point:
[...] Literally one idea kernel developers have, is to enable 'discard' mount option (online discard) by default. The idea is that inevitably irate end users will compel vendors to fix their shit via ridicule. I find this hilarious because the most civil "go fuck yourself" invented in the modern era is the expired warranty. I expect this idea would just lead to a lot of finger pointing, and they really should have just done this 9 years ago, hindsight being 20/20.
There was also some discussion of the discard option in /etc/crypttab that allows discard operations on LUKS-encrypted device-mapper volumes to be passed down to the underlying storage device. Setting that option was done for Fedora 27, but, since there is no periodic fstrim being done by default, it may not really have helped all that many users, Murphy said. There are some information leaks doing discards on LUKS volumes, however, as Zaitsev noted, which may cause some security-conscious users to disable the feature.
As the proposal notes, Ubuntu and openSUSE have been enabling fstrim.timer for some time now, which makes Fedora late to the table. It also means that the feature has been pretty well tested at this point, though. Sometime early in the new year, one would expect the Fedora Engineering Steering Committee (FESCo) to weigh in. Given that Fedora aims to be "first", it would seem likely that FESCo will not want the distribution to be left further behind on this.
Brief items
Security
Cloud Native Computing Foundation announces TUF graduation
The Cloud Native Computing Foundation (CNCF) is part of the Linux Foundation that is focused on Kubernetes and other cloud technologies. It has announced that The Update Framework (TUF) has graduated to a full member project. "TUF, an open-source technology that secures software update systems, is the first specification and first security-focused project to graduate. Justin Cappos, associate professor of computer science and engineering at NYU Tandon School of Engineering, initially developed the project in 2009. Cappos is also the first academic researcher to lead a graduated project and TUF is the first project born out of a university to graduate.Huang: Can We Build Trustable Hardware?
Andrew 'bunnie' Huang has posted a detailed article on why creating trustable hardware is so difficult and describing a project he's working on to do it anyway. "While open hardware has the opportunity to empower users to innovate and embody a more correct and transparent design intent than closed hardware, at the end of the day any hardware of sufficient complexity is not practical to verify, whether open or closed. Even if we published the complete mask set for a modern billion-transistor CPU, this 'source code' is meaningless without a practical method to verify an equivalence between the mask set and the chip in your possession down to a near-atomic level without simultaneously destroying the CPU."Garrett: Wifi deauthentication attacks and home security
Matthew Garrett works out how to avoid being recorded by "Ring" door cameras in his apartment building. "The most interesting one here is the deauthentication frame that access points can use to tell clients that they're no longer welcome. These can be sent for a variety of reasons, including resource exhaustion or authentication failure. And, by default, they're entirely unprotected. Anyone can inject such a frame into your network and cause clients to believe they're no longer authorised to use the network, at which point they'll have to go through a new authentication cycle - and while they're doing that, they're not able to send any other packets."Security quotes of the week
These aren't subtle vulnerabilities. These are stupid design decisions made by engineers who had no idea how to create a secure system. And this, in a nutshell, is the problem with the Internet of Things.— Bruce Schneier comments on a vulnerability report about DTEN video-conferencing devices
The arguably creepier surveillance system implemented by Degree Analytics doubles as a campus wifi network that tracks students' every move unless they check "no" on an opt-out window offering to "support student success, operations, and security." Degree Analytics not only tracks attendance, but it also monitors students' movements from dorms to dining halls to ostensibly identify unhealthy behavioral patterns (sleeping too much, not eating, avoiding student life programs).— Whitney Kimball at Gizmodo
Kernel development
Kernel release status
The current development kernel is 5.5-rc4, released on December 29. Linus said: "To absolutely nobody's surprise, last week was very quiet indeed. It's hardly even worth making an rc release, but there are _some_ fixes in here, so here's the usual weekly Sunday afternoon rc."
Previously, 5.5-rc3 was released on December 22.
Stable updates: 5.4.6, 4.19.91, 4.14.160, 4.9.207, and 4.4.207 were released on December 21, followed by 5.4.7, 4.19.92, and 4.14.161 on December 31.
Quotes of the week
Distributions
Alpine Linux 3.11 released
Version 3.11 of the lightweight Alpine Linux distribution is available. Changes include the 5.4 kernel, Raspberry Pi 4 support, GNOME and KDE support, and the deprecation of Python 2.The results from the Debian init-system GR
The results from the Debian general resolution vote on init systems are in; the project's developers chose the option titled "Systemd but we support exploring alternatives". It makes systemd into the preferred init system, and allows packages to use systemd-specific features; packagers are not required to support other init systems, but support for other systems is encouraged where it is practical.Górny: A distribution kernel for Gentoo
Michał Górny describes an effort to create something one might have never expected to see: a binary kernel package for the Gentoo distribution. "I have manually configured the kernels for my private systems long time ago. Today, I wouldn’t really have bothered. In fact, I realized that for some time I’m really hesitant to even upgrade them because of the effort needed to update configuration. The worst part is, whenever a new kernel does not boot, I have to ask myself: is it a real bug, or is it my fault for configuring it wrong?"
Distribution quote of the fortnight
So, I am sending hugs to all my fellow Debian colleagues no matter whether the are pro-systemd, anti-systemd, anything in between, or something completely different. Happy new year to all of you and the Debian project as whole.
Development
Development quotes of the fortnight
On the other hand, every single byte in this executable file can be accounted for and justified. How many executables have you created lately that you can say that about?
Page editor: Jake Edge
Announcements
Newsletters
Distributions and system administration
- DistroWatch Weekly (December 23)
- Linux Mint Monthly News (December)
- Lunar Linux Weekly News (December 20)
- Lunar Linux Weekly News (December 27)
- openSUSE Tumbleweed Review of the Week (December 23)
- openSUSE Tumbleweed Review of the Week (December 27)
- SparkyLinux News (December 31)
- Ubuntu Weekly Newsletter (December 21)
- Ubuntu Weekly Newsletter (December 28)
Development
- Emacs News (December 23)
- Emacs News (December 30)
- Git Rev News (December 25)
- Koha Community Newsletter (December)
- LLVM Weekly (December 23)
- LLVM Weekly (December 30)
- OCaml Weekly News (December 31)
- Perl Weekly (December 23)
- Perl Weekly (December 30)
- PostgreSQL Weekly News (December 22)
- PostgreSQL Weekly News (December 29)
- Python Weekly Newsletter (December 19)
- Python Weekly Newsletter (December 26)
- Weekly Rakudo News (December 24)
- Ruby Weekly News (December 19)
- This Week in Rust (December 17)
- This Week in Rust (December 24)
- Wikimedia Tech News (December 23)
Calls for Presentations
[CFP] Power Management and Scheduling in the Linux Kernel (OSPM-summit)
Power Management and Scheduling in the Linux Kernel (OSPM-summit) will take place May 11-13 in Pisa, Italy. "Although scheduler techniques for reducing energy consumption while meeting performance and latency requirements are the prime interest of the summit, we welcome anybody interested in having discussions on the broader scope of real-time systems, real-time and non-real-time scheduling, tooling, debugging and tracing." The call for papers closes February 10.
CFP Deadlines: January 2, 2020 to March 2, 2020
The following listing of CFP deadlines is taken from the LWN.net CFP Calendar.
Deadline | Event Dates | Event | Location |
---|---|---|---|
January 13 | May 18 May 20 |
[POSTPONED] Percona Live Open Source Database Conference 2020 | Austin, Tx, USA |
January 15 | August 13 August 21 |
Netdev 0x14 | Virtual |
January 19 | May 27 May 28 |
[ONLINE] PGCon 2020 | Ottawa, Canada |
January 31 | March 23 March 25 |
Supercomputing Frontiers Europe | Warsaw, Poland |
January 31 | June 17 June 18 |
[ONLINE] Open Source Data Center Conference | Berlin, Germany |
January 31 | April 16 April 17 |
[CANCELED] DevOps Days Toronto 2020 | Toronto, Canada |
February 9 | May 11 May 13 |
[RESCHEDULED] Linux Audio Conference | Bordeaux, France |
February 9 | March 29 April 1 |
[POSTPONED] foss-north | Göteborg, Sweden |
February 10 | May 11 May 13 |
[ONLINE] Power Management and Scheduling in the Linux Kernel | Pisa, Italy |
February 11 | July 6 July 12 |
[VIRTUAL] SciPy 2020 | Austin, TX, USA |
February 15 | June 17 June 18 |
[ONLINE] stackconf 2020 | Berlin, Germany |
February 15 | March 25 March 28 |
[POSTPONED] MiniDebConf Maceió 2020 | Maceió, Brazil |
February 16 | June 29 July 2 |
[VIRTUAL] Open Source Summit/Embedded Linux Conference North America | Austin, TX, USA |
February 28 | June 23 June 25 |
[CANCELED] IcingaConf 2020 | Amsterdam, Netherlands |
February 28 | May 26 May 28 |
[VIRTUAL] sambaXP 2020 | Berlin, Germany |
February 28 | June 23 June 25 |
[ONLINE] Postgres Vision 2020 | Boston, MA, USA |
If the CFP deadline for your event does not appear here, please tell us about it.
Upcoming Events
Events: January 2, 2020 to March 2, 2020
The following event listing is taken from the LWN.net Calendar.
Date(s) | Event | Location |
---|---|---|
January 13 | LCA Creative Arts Miniconf | Gold Coast, Australia |
January 13 | Open ISA (RISC-V, OpenPOWER, etc) Miniconf at Linux.conf.au | Gold Coast, Australia |
January 13 January 17 |
linux.conf.au | Gold Coast, Australia |
January 14 | lca Kernel Miniconf | Gold Coast, Australia |
January 24 January 26 |
devconf.cz | Brno, Czech Republic |
February 1 February 2 |
FOSDEM 2020 | Brussels, Belgium |
February 3 | Copyleft Conference | Brussels, Belgium |
February 18 February 20 |
PyCon Namibia | Windhoek, Namibia |
If your event does not appear here, please tell us about it.
Security updates
Alert summary December 26, 2019 to January 1, 2020
Dist. | ID | Release | Package | Date |
---|---|---|---|---|
Arch Linux | ASA-201912-6 | git | 2019-12-18 | |
Arch Linux | ASA-201912-5 | libgit2 | 2019-12-18 | |
Arch Linux | ASA-201912-4 | shadow | 2019-12-18 | |
CentOS | CESA-2019:4107 | C7 | firefox | 2019-12-24 |
CentOS | CESA-2019:4254 | C6 | freetype | 2019-12-24 |
CentOS | CESA-2019:4326 | C7 | fribidi | 2019-12-24 |
CentOS | CESA-2019:4256 | C6 | kernel | 2019-12-24 |
CentOS | CESA-2019:4190 | C7 | nss, nss-softokn, nss-util | 2019-12-24 |
CentOS | CESA-2019:4190 | C7 | nss, nss-softokn, nss-util | 2019-12-24 |
CentOS | CESA-2019:4190 | C7 | nss, nss-softokn, nss-util | 2019-12-24 |
CentOS | CESA-2019:4240 | C7 | openslp | 2019-12-24 |
CentOS | CESA-2019:4205 | C6 | thunderbird | 2019-12-24 |
CentOS | CESA-2019:4148 | C7 | thunderbird | 2019-12-24 |
Debian | DLA-2047-1 | LTS | cups | 2019-12-22 |
Debian | DSA-4590-1 | stable | cyrus-imapd | 2019-12-19 |
Debian | DLA-2044-1 | LTS | cyrus-sasl2 | 2019-12-20 |
Debian | DSA-4591-1 | stable | cyrus-sasl2 | 2019-12-20 |
Debian | DSA-4589-1 | stable | debian-edu-config | 2019-12-18 |
Debian | DSA-4595-1 | stable | debian-lan-config | 2019-12-27 |
Debian | DSA-4593-1 | stable | freeimage | 2019-12-27 |
Debian | DLA-2043-2 | LTS | gdk-pixbuf | 2019-12-20 |
Debian | DLA-2043-1 | LTS | gdk-pixbuf | 2019-12-19 |
Debian | DLA-2055-1 | LTS | igraph | 2019-12-31 |
Debian | DLA-2049-1 | LTS | imagemagick | 2019-12-29 |
Debian | DLA-2051-1 | LTS | intel-microcode | 2019-12-30 |
Debian | DLA-2054-1 | LTS | jhead | 2019-12-31 |
Debian | DLA-2052-1 | LTS | libbsd | 2019-12-30 |
Debian | DLA-1931-2 | LTS | libgcrypt20 | 2020-01-01 |
Debian | DLA-2048-1 | LTS | libxml2 | 2019-12-28 |
Debian | DSA-4592-1 | stable | mediawiki | 2019-12-27 |
Debian | DLA-2046-1 | LTS | opensc | 2019-12-25 |
Debian | DSA-4594-1 | stable | openssl1.0 | 2019-12-27 |
Debian | DLA-2053-1 | LTS | otrs2 | 2020-01-01 |
Debian | DLA-2050-1 | LTS | php5 | 2019-12-29 |
Debian | DLA-2042-1 | LTS | python-django | 2019-12-18 |
Debian | DLA-2045-1 | LTS | tightvnc | 2019-12-21 |
Debian | DSA-4596-1 | stable | tomcat8 | 2019-12-27 |
Debian | DLA-2056-1 | LTS | waitress | 2020-01-01 |
Debian | DLA-2038-2 | LTS | x2goclient | 2019-12-22 |
Fedora | FEDORA-2019-6bf27b45b3 | F30 | cacti | 2019-12-20 |
Fedora | FEDORA-2019-362f0e9710 | F31 | cacti | 2019-12-21 |
Fedora | FEDORA-2019-6bf27b45b3 | F30 | cacti-spine | 2019-12-20 |
Fedora | FEDORA-2019-362f0e9710 | F31 | cacti-spine | 2019-12-21 |
Fedora | FEDORA-2019-533a72fec5 | F30 | fribidi | 2019-12-20 |
Fedora | FEDORA-2019-adc8990386 | F31 | python-django | 2019-12-19 |
Mageia | MGASA-2019-0411 | 7 | 389-ds-base | 2019-12-25 |
Mageia | MGASA-2019-0407 | 7 | apache | 2019-12-25 |
Mageia | MGASA-2019-0399 | 7 | apache-commons-beanutils | 2019-12-19 |
Mageia | MGASA-2019-0410 | 7 | apache-mod_auth_openidc | 2019-12-25 |
Mageia | MGASA-2019-0418 | 7 | clamaw | 2019-12-31 |
Mageia | MGASA-2019-0415 | 7 | exiv2 | 2019-12-31 |
Mageia | MGASA-2019-0398 | 7 | fence-agents | 2019-12-19 |
Mageia | MGASA-2019-0417 | 7 | filezilla | 2019-12-31 |
Mageia | MGASA-2019-0396 | 7 | flightcrew | 2019-12-19 |
Mageia | MGASA-2019-0401 | 7 | freerdp | 2019-12-19 |
Mageia | MGASA-2019-0405 | 7 | ghostpcl | 2019-12-24 |
Mageia | MGASA-2019-0403 | 7 | htmldoc | 2019-12-19 |
Mageia | MGASA-2019-0421 | 7 | hunspell | 2019-12-31 |
Mageia | MGASA-2019-0414 | 7 | kernel | 2019-12-25 |
Mageia | MGASA-2019-0416 | 7 | libidn2 | 2019-12-31 |
Mageia | MGASA-2019-0404 | 7 | libmirage | 2019-12-24 |
Mageia | MGASA-2019-0409 | 7 | libofx | 2019-12-25 |
Mageia | MGASA-2019-0402 | 7 | libssh | 2019-12-19 |
Mageia | MGASA-2019-0413 | 7 | microcode | 2019-12-25 |
Mageia | MGASA-2019-0394 | 7 | pacemaker | 2019-12-19 |
Mageia | MGASA-2019-0419 | 7 | pdfresurrect | 2019-12-31 |
Mageia | MGASA-2019-0412 | 7 | php | 2019-12-25 |
Mageia | MGASA-2019-0420 | 7 | roundcubemail | 2019-12-31 |
Mageia | MGASA-2019-0400 | 7 | rsyslog | 2019-12-19 |
Mageia | MGASA-2019-0408 | 7 | ruby | 2019-12-25 |
Mageia | MGASA-2019-0397 | 7 | samba | 2019-12-19 |
Mageia | MGASA-2019-0406 | 7 | spamassassin | 2019-12-24 |
Mageia | MGASA-2019-0395 | 7 | sssd | 2019-12-19 |
Mageia | MGASA-2019-0422 | 7 | xpdf | 2019-12-31 |
openSUSE | openSUSE-SU-2019:2709-1 | 15.1 | LibreOffice | 2019-12-31 |
openSUSE | openSUSE-SU-2019:2712-1 | 15.1 | chromium | 2019-12-31 |
openSUSE | openSUSE-SU-2019:2712-1 | 15.1 | chromium | 2019-12-31 |
openSUSE | openSUSE-SU-2019:2698-1 | 15.1 | mariadb | 2019-12-22 |
openSUSE | openSUSE-SU-2019:2700-1 | 15.1 | samba | 2019-12-22 |
openSUSE | openSUSE-SU-2019:2710-1 | 15.1 | spectre-meltdown-checker | 2019-12-31 |
Oracle | ELSA-2019-4254 | OL6 | freetype | 2019-12-18 |
Oracle | ELSA-2019-4326 | OL7 | fribidi | 2019-12-20 |
Oracle | ELSA-2019-4326 | OL7 | fribidi | 2019-12-20 |
Oracle | ELSA-2019-4361 | OL8 | fribidi | 2019-12-24 |
Oracle | ELSA-2019-4356 | OL8 | git | 2019-12-20 |
Oracle | ELSA-2019-4256 | OL6 | kernel | 2019-12-18 |
Oracle | ELSA-2019-4884 | OL7 | python | 2019-12-20 |
Red Hat | RHSA-2019:4326-01 | EL7 | fribidi | 2019-12-19 |
Red Hat | RHSA-2019:4361-01 | EL8 | fribidi | 2019-12-23 |
Red Hat | RHSA-2019:4356-01 | EL8 | git | 2019-12-19 |
Red Hat | RHSA-2019:4360-01 | EL8 | libyang | 2019-12-22 |
Red Hat | RHSA-2019:4358-01 | OSP15 | openstack-keystone | 2019-12-19 |
Red Hat | RHSA-2019:4344-01 | OSP10.0 | qemu-kvm-rhev | 2019-12-20 |
Scientific Linux | SLSA-2019:4254-1 | SL6 | freetype | 2019-12-18 |
Scientific Linux | SLSA-2019:4326-1 | SL7 | fribidi | 2019-12-19 |
Scientific Linux | SLSA-2019:4256-1 | SL6 | kernel | 2019-12-18 |
Slackware | SSA:2019-354-01 | openssl | 2019-12-20 | |
Slackware | SSA:2019-354-02 | tigervnc | 2019-12-20 | |
Slackware | SSA:2019-353-01 | wavpack | 2019-12-19 | |
SUSE | SUSE-SU-2019:3390-1 | SLE12 | dia | 2019-12-27 |
SUSE | SUSE-SU-2019:3391-1 | SLE15 | dia | 2019-12-27 |
SUSE | SUSE-SU-2019:14260-1 | SLE11 | firefox, nspr, nss | 2019-12-20 |
SUSE | SUSE-SU-2019:3347-1 | OS7 OS8 SLE12 SES5 | firefox | 2019-12-19 |
SUSE | SUSE-SU-2019:3337-1 | SLE15 | firefox | 2019-12-19 |
SUSE | SUSE-SU-2019:3379-1 | OS8 SLE12 SES5 | kernel | 2019-12-21 |
SUSE | SUSE-SU-2019:3371-1 | SLE12 | kernel | 2019-12-20 |
SUSE | SUSE-SU-2019:3371-1 | SLE12 | kernel | 2019-12-20 |
SUSE | SUSE-SU-2019:3389-1 | SLE12 | kernel | 2019-12-27 |
SUSE | SUSE-SU-2019:3389-1 | SLE12 | kernel | 2019-12-27 |
SUSE | SUSE-SU-2019:3372-1 | SLE15 | kernel | 2019-12-20 |
SUSE | SUSE-SU-2019:3381-1 | SLE15 | kernel | 2019-12-23 |
SUSE | SUSE-SU-2019:3381-1 | SLE15 | kernel | 2019-12-23 |
SUSE | SUSE-SU-2019:3392-1 | SLE15 | libgcrypt | 2019-12-27 |
SUSE | SUSE-SU-2019:3370-1 | SLE12 | mariadb-100 | 2019-12-20 |
SUSE | SUSE-SU-2019:3369-1 | OS9 SLE12 | mariadb | 2019-12-20 |
SUSE | SUSE-SU-2019:3395-1 | SLE15 | mozilla-nspr, mozilla-nss | 2019-12-30 |
SUSE | SUSE-SU-2019:3394-1 | SLE12 | python-azure-agent | 2019-12-30 |
SUSE | SUSE-SU-2019:3393-1 | SLE15 | python-azure-agent | 2019-12-30 |
SUSE | SUSE-SU-2019:3386-1 | SLE15 | shibboleth-sp | 2019-12-23 |
SUSE | SUSE-SU-2019:3385-1 | SLE12 | slurm | 2019-12-23 |
SUSE | SUSE-SU-2019:3340-1 | SLE12 | spectre-meltdown-checker | 2019-12-19 |
SUSE | SUSE-SU-2019:3348-1 | SLE15 | spectre-meltdown-checker | 2019-12-19 |
SUSE | SUSE-SU-2019:3339-1 | SLE15 | thunderbird | 2019-12-19 |
SUSE | SUSE-SU-2019:3349-1 | SLE15 | trousers | 2019-12-19 |
SUSE | SUSE-SU-2019:3338-1 | SLE15 | xen | 2019-12-19 |
SUSE | SUSE-SU-2019:3341-1 | SLE12 | zziplib | 2019-12-19 |
Ubuntu | USN-4224-1 | 16.04 18.04 19.04 19.10 | python-django | 2019-12-18 |
Kernel patches of interest
Kernel releases
Architecture-specific
Core kernel
Device drivers
Device-driver infrastructure
Filesystems and block layer
Memory management
Networking
Security-related
Virtualization and containers
Miscellaneous
Page editor: Rebecca Sobol