LWN.net Weekly Edition for May 11, 2023
Welcome to the LWN.net Weekly Edition for May 11, 2023
This edition contains the following feature content:
- Faster CPython at PyCon, part one: adaptive instructions and other changes intended to make the CPython interpreter run faster.
- MicroPython 1.20: Python for microcontrollers: a Python variant for small systems.
- The end of the accounting search: after over a decade, LWN has finally settled on a free-software accounting solution.
- The ongoing trouble with get_user_pages(): one of the kernel's most troubled internal memory-management interfaces continues to give problems.
- The rest of the 6.4 merge window: more goodies for the next major kernel release.
This week's edition also includes these inner pages:
- Brief items: Brief news items from throughout the community.
- Announcements: Newsletters, conferences, security updates, patches, and more.
Please enjoy this week's edition, and, as always, thank you for supporting LWN.net.
Faster CPython at PyCon, part one
Two members of the Faster CPython team, which was put together at Microsoft at the behest of Guido van Rossum to work on major performance improvements for CPython, came to PyCon 2023 to report on what the team has been working on—and its plans for the future. PEP 659 ("Specializing Adaptive Interpreter") describes the foundation of the current work, some of which has already been released as part of Python 3.11. Brandt Bucher, who gave a popular talk on structural pattern matching at last year's PyCon, was up first, with a talk on what "adaptive" and "specializing" mean in the context of Python, which we cover here in part one. Mark Shannon, whose proposed plan for performance improvements in 2020 was a major impetus for this work, presented on the past, present, and future of the Python performance enhancements, which will be covered in part two.
Bucher started out by talking a bit about Python bytecode, but said that developers do not need to know anything about it to get faster code: "just upgrade to 3.11 and you'll probably see a performance improvement". In order to show how the team sped up the interpreter, though, it would help to look inside the Python code. He put up an example Point class that has two floating point attributes for its position (x, y) and a single shifted() method that takes two offsets to apply to the position of an instance, and returns a new instance of the class at the shifted position. He focused on two lines from the method:
y = self.y + dy cls = type(self)The first line applies the offset for the y axis, while the second gets a reference to the Point class (in preparation for returning a new instance by calling cls(x, y)). He used the dis module to disassemble the method; the relevant piece of bytecode is as follows:
# part of the output of dis.dis(Point.shifted) LOAD_FAST (dy) LOAD_FAST (self) LOAD_ATTR (y) BINARY_OP (+) STORE_FAST (y) LOAD_GLOBAL (type) LOAD_FAST (self) PRECALL (1) CALL (1) STORE_FAST (cls)Some of the binary opcodes (and some operations, such as method calls) have changed from those in Python 3.10 and earlier, so your output may be different than what he showed. The bytecode is a set of instructions for the Python stack-based virtual machine. In the first part above, the dy and self values are pushed onto the stack, then the LOAD_ATTR instruction retrieves the y attribute from the value popped from the stack (self), then pushes it. Next, the two values (self.y and dy) on top of the stack are added (and the result is pushed) and that result is popped to be stored in the variable y. In the second part, type and self are pushed, then PRECALL and CALL are two separate steps to perform the type(self) call; its result is popped to store into cls.
Adaptive instructions
If that code is run more than a handful of times, Bucher said, it becomes a
candidate for optimization in 3.11. The first step is something
called "quickening", which the PEP calls "the process of replacing slow
instructions with faster variants
". "Superinstructions" that combine
two related instructions into a single instruction are substituted into the
code. For example, the first two LOAD_FAST instructions can be
replaced with a single one:
LOAD_FAST__LOAD_FAST (dy, self)It is a simple change that results in a real performance boost, he said. The more interesting change is to replace some instructions with their adaptive counterparts. During the quickening process, five bytecodes are replaced with adaptive versions:
# part of the output of dis.dis(Point.shifted, adaptive=True) LOAD_FAST__LOAD_FAST (dy, self) LOAD_ATTR_ADAPTIVE (y) BINARY_OP_ADAPTIVE (+) STORE_FAST (y) LOAD_GLOBAL_ADAPTIVE (type) LOAD_FAST (self) PRECALL_ADAPTIVE (1) CALL_ADAPTIVE (1) STORE_FAST (cls)
The adaptive versions perform the same operation as their counterpart except that they can specialize themselves depending on how they are being used. For example, loading an attribute is "actually surprisingly complex", because the attribute could come from a number of places. It could be a name from a module, a class variable from a class, a method of a class, and so on, so the standard load-attribute code needs to be prepared for those possibilities. The simplest case is getting an attribute from an instance dictionary, which is exactly what is being done here.
![Brandt Bucher [Brandt Bucher]](https://static.lwn.net/images/2023/pycon-bucher-sm.png)
The LOAD_ATTR_ADAPTIVE operation can recognize that it is in the simple case and can ignore all of the rest of possibilities, so the adaptive instruction changes to LOAD_ATTR_INSTANCE_VALUE, which is a specialized instruction that only contains the code fast path for this common case.
The specialized instruction will then check to see if the class is unchanged from the last time this attribute was accessed and if the keys in the object's __dict__ are the same. Those much faster checks can be done in lieu of two dictionary lookups (on the class dictionary for a descriptor and on the object's __dict__ for the name); "dictionary lookups are fast, but they still cost something", while the other two checks are trivial. If the conditions hold true, which is the normal situation, the code can simply return the entry from the __dict__ at the same offset that was used previously; it does not need to do any hashing or collision resolution that comes when doing a dictionary lookup.
If either of the checks fails, the code falls back to the regular load attribute operation. Python is a dynamic language and the new interpreter needs to respect that, but the dynamic features are not being used all of the time. The idea is to not pay the cost of dynamism when it is not being used, which is rather frequent in many Python programs.
Similarly, the BINARY_OP_ADAPTIVE instruction specializes to BINARY_OP_ADD_FLOAT because floating-point values are being used. That operation checks that both operands are of type float and falls back to the (also surprisingly complex) machinery for an add operation if they are not. Otherwise, it can just add the raw floating point values together in C.
Normally, when a global name is being loaded, it requires two dictionary lookups; for example, when type() is being loaded, the global dictionary must be checked in case the function has been shadowed, if not (which is likely), then it must be looked up in the builtins dictionary. So LOAD_GLOBAL_ADAPTIVE takes a page from the attribute-loading specialization to check if the global dictionary or builtins dictionary have changed; if not, it simply grabs the value at the same index it used before.
It turns out that type() is called often enough that it gets its own specialized bytecode. It checks that the code for type() has not changed (by way of a monkey patch or similar) and, if not, simply returns the argument's class. There is a call in the C API to do so and "it's much much cheaper than making the call".
If the specialized instructions fail their tests enough times, they will revert back to the adaptive versions in order to change course. For example, if the Point class starts being used with integer values, BINARY_OP_ADD_FLOAT will revert to BINARY_OP_ADAPTIVE, which would replace itself with BINARY_OP_ADD_INT after a few uses. "This is what we mean by specializing adaptive interpreter".
It may seem like a "bunch of small localized changes"—and it is—but they add up to something substantial. For example, the shifted() method is nearly twice as fast in 3.11 versus 3.10. He obviously chose the example because it specializes well; a 2x performance increase is probably at the upper end of what can be expected. But it does show how replacing the generalized versions of these Python bytecode operations with their more specialized counterparts can lead to large improvements.
Bucher said that the various pieces of information that are being stored (e.g. the index of a dictionary entry) are actually being placed into the bytecode itself. He called those "inline caches" and they can be seen using the show_caches=True parameter to dis.dis(). But Python programmers should not really need to look at the caches, or even at the adaptive instructions, because all of that should not matter. The idea is that the interpreter will still do what it always did, but it can now do it faster in many cases.
For those who do want to dig under the covers more, though, Bucher recommended his specialist tool. Running some code with specialist will generate a web page with the source code of the program color-coded to show where the interpreter is optimizing well—and where it is not.
Coming soon
He then shifted gears to looking at things that will be coming in CPython 3.12. To start with, the dedicated quickening step has been eliminated and the superinstructions are simply emitted at compile time. In addition, there will be only a single call instruction rather than the two-step dance in 3.11.
Instead of having separate adaptive versions of the operations, the standard bytecodes implement the adaptivity. A bytecode only needs to be executed twice in order to become fully specialized. That is measured on individual bytecodes, rather than a function as a whole, so a loop will specialize immediately even if the surrounding code only gets executed once.
The team has also added more specialized instructions. "We have gotten better at specializing dynamic attribute accesses", so there are specializations for calling an object's __getattribute__() and for loading properties specified using the property() builtin. There are specializations for iterating over the four most-common objects in a for loop: list, tuple, range, and generator. There is also a single specialization for yield from in generators and await in coroutines. There are two others in the works that may make it into CPython 3.12.
Meanwhile, the inline caches are being reduced. Having more cached information makes the bytecode longer, which means longer jumps and more memory use, so shrinking the amount of cached data is welcome. The team has been able to eliminate some cache entries or reorganize the caches to make some significant reductions. All of that is coming in Python 3.12—and beyond.
Stay tuned for our coverage of Shannon's talk, which came on the second day of the conference. It will be the subject of part two, which is coming soon to an LWN near you.
[I would like to thank LWN subscribers for supporting my travel to Salt Lake City for PyCon.]
MicroPython 1.20: Python for microcontrollers
The MicroPython programming language implements a sizable subset of Python that can run on microcontrollers, thus bringing Python's easy-to-learn syntax, readability, and versatility to the embedded world. With its recent 1.20 release, MicroPython introduces a new package manager, reduces its code size, and adds support for many new boards, including the Raspberry Pi Pico W. The project has come a long way since its inception ten years ago, making it an easy-to-use tool for developing software for resource-constrained environments.
The MIT-licensed MicroPython was created in 2013 by Damien George as a Kickstarter project, along with the pyboard, a microcontroller board to run it on. LWN looked at MicroPython on the pyboard in 2015. There are still updated versions of the pyboard available in the MicroPython store; they are the official MicroPython boards and their sales support the ongoing development of the project.
By using MicroPython, developers can build on their knowledge of Python to create software for microcontrollers, where they typically would need to write C code. The resulting program may not be as fast as a compiled C program, but in many cases the tradeoff for ease of development is worth it. However, developers should be aware of the (well documented) differences from CPython, the canonical Python interpreter. MicroPython implements Python 3.4, as well as some select features of Python 3.5 and above. Some of Python's built-in types and standard library modules are not implemented fully. For example, slicing a string with a step other than one is not implemented, and user-defined attributes for built-in exceptions aren't supported either.
There's a reason for many of the differences from CPython: the interpreter needs to be able to run on devices with low resources. Additionally, the project applies several optimizations. One of these is that small integers are not allocated on the heap, as explained in the documentation about memory management. To avoid storing duplicate copies of the same string (including function and variable names), MicroPython uses string interning. The documentation also lists some tips to maximize speed.
Instead of letting MicroPython load a Python module, parse it, generate its bytecode and then execute it, developers can instead pre-compile their modules and load the resulting .mpy file on the board. This avoids the overhead of parsing and generating the bytecode on the microcontroller. This is similar to the CPython .pyc files, but MicroPython has its own bytecode representation.
A further optimization is frozen bytecode: this allows building the pre-compiled bytecode into the firmware image when compiling MicroPython for a specific board. When this firmware image is installed to the board, the bytecode will be executed from flash memory rather than copied into RAM, and objects of immutable types (such as strings and tuples) are loaded from flash memory as well. This results in less RAM usage and reduces heap fragmentation.
Supported hardware
MicroPython offers firmware downloads for more than 150 microcontroller boards. This includes boards with Espressif's ESP32 and ESP8266 microcontroller families, boards with an STM32 ARM Cortex-M processor, Microchip's SAMD microcontrollers with ARM Cortex-M0+ core, Nordic Semiconductor's nRF5 chips, and the Raspberry Pi Foundation's RP2040-based boards (including the Raspberry Pi Pico W). Each board's download page includes installation instructions. For example, a Raspberry Pi Pico with the UF2 bootloader requires the user to hold down the BOOTSEL button while plugging the board into USB, after which the firmware file can be copied to the USB mass storage device that appears. Other boards have their own specific requirements.
MicroPython has also been ported to the Zephyr real-time operating system. In MicroPython 1.20, this port is based on Zephyr 3.1.0, which was released in June 2022; the current Zephyr release is 3.3.0. A Zephyr development environment can be used to build MicroPython for every target board supported by Zephyr, although not all have been tested. It also gives MicroPython code access to Zephyr's uniform sensor API using the zsensor module. More information can be found in MicroPython's documentation about the Zephyr port.
The Zephyr port allows MicroPython to run on the BBC micro:bit v2, whereas there's only a direct port for the BBC micro:bit v1. The process of setting up the development environment, building the Zephyr port for the target board, and then flashing the firmware to the board is explained in the port's README file.
Development environment
When the MicroPython interpreter, which is written in C, is running on a microcontroller board, it offers a read-eval-print loop (REPL) shell. For most boards, the REPL is available using a serial connection over the board's USB port. This allows the user to connect to the MicroPython REPL with a tool such as GNU Screen and then interactively run commands on the board. The project also offers the mpremote tool to interact with a MicroPython device over a serial connection.
Thonny and Mu Editor are two popular development environments that not only support the MicroPython language and integrate the REPL, but also let the user install the MicroPython firmware on a connected board. Last year, the Arduino developers published an experimental pre-release of their own lightweight editor for MicroPython programs: Arduino Lab for MicroPython.
Using mpremote or one of the supported development environments, a developer can upload Python code to the board's filesystem. The MicroPython interpreter evaluates two files at boot: boot.py and main.py (or .mpy versions instead). The boot.py file is supposed to contain initialization code, for example to connect to a WiFi network. The main.py file should contain the main code of the program, which often executes the device's task indefinitely in an infinite loop.
As an example, here's some MicroPython code for a NeoPixel stick with its IN pin connected to the GPIO22 pin of a board (which I tested using an ESP32 board):
from machine import Pin from neopixel import NeoPixel from time import sleep PIXELS = 8 RED = (32, 0, 0) OFF = (0, 0, 0) np = NeoPixel(Pin(22), PIXELS) while True: for i in range(PIXELS): np[i] = RED np.write() sleep(0.1) np[i] = OFF np.write() sleep(0.1)
After importing the modules it needs, the program starts by declaring some constants. Since the neopixel library needs its colors specified as a tuple of red, green, and blue components, the color RED uses 32 as the red component (the maximum value 255 is blindingly bright on those LEDs). After this, the code creates a NeoPixel object for GPIO pin 22 and the number of pixels (LEDs) defined. This object is indexable, so the infinite loop just cycles through all the pixels, setting them one by one to the red color for 100 ms, before turning them off again.
Libraries
MicroPython comes with three types of libraries. Some modules are reimplementations of part of the Python standard library. These are modules such as collections, json, socket, and uasyncio (a subset of Python's asyncio module).
The second type of libraries implements functionality specific to MicroPython. These include the bluetooth module that offers low-level access to Bluetooth Low Energy, the machine module that is useful to control I/O pins, use a board's analog-to-digital converters, or use the I2C protocol, and the network module for network access. MicroPython also has some libraries that are specific to a board family.
Due to resource constraints or other hardware-specific limitations, MicroPython's libraries don't include all functions, classes or even modules on all devices. This information should always be consulted in the "Quick reference" for the device family's port in the sidebar of MicroPython's documentation.
While those two types of libraries are directly built into the MicroPython firmware, there are some additional external libraries that are maintained in the micropython-lib repository. Some of these libraries are implementations of parts of the CPython standard library that are not built in the firmware to save space. Others implement a subset of popular Python packages, such as the urequests version of the Requests HTTP library. It also hosts some MicroPython-specific packages, such as drivers for sensors, displays, and other hardware.
The recent MicroPython 1.20 release replaced the previous package manager upip with mip, which can be used to install libraries from the micropython-lib repository. The firmware includes the mip module on any network-capable board, after which libraries can be installed from the REPL as long as there's a network connection set up on the board:
>>> import mip >>> mip.install("neopixel")
The default behavior is to fetch .mpy files with pre-compiled bytecode (which is versioned) for libraries from micropython-lib. The package manager installs the files in the board's file system, where they are available to the developer's Python files.
Instead of supplying a package name from the micropython-lib repository, it's also possible to point to a GitHub repository or an arbitrary URL if it contains a package.json file with information about the files to install and the package's dependencies.
MicroPython on the web
Another interesting target is WebAssembly (Wasm), which allows MicroPython to run in a web browser. The WebAssembly port's README includes instructions to compile the MicroPython firmware to Wasm using Emscripten. When Anaconda founder Peter Wang announced PyScript at PyCon 2022, it was using CPython compiled to WebAssembly (from Pyodide) and making that available in the web browser so developers could create web apps in Python.
In November 2022, Anaconda followed up with the announcement of a second runtime for PyScript, based on MicroPython. This is currently a technical preview to explore the tradeoffs of using different Python runtimes. PyScript's MicroPython runtime has a total size of only 303KB and starts running the developer's script in less than 100ms, while the Pyodide runtime weights 11MB and is slower to start up.
Since the announcement in November, there hasn't been anything more
about this alternative runtime on Anaconda's blog, but it looks like some
things are cooking. In early April, Ted Patrick, PyScript's Engineering
Manager, opened a GitHub issue
in the PyScript repository with the message "MicroPython is going to be
supported as an alternative Python interpreter to Pyodide
", and assigning
it to the project's 2023-Q2
milestone. So it looks like MicroPython will be supported quite soon
as an alternative PyScript runtime.
Conclusion
For those looking for an easy way to program microcontrollers, MicroPython has much to offer. Together with alternatives like Adafruit's MicroPython fork CircuitPython and the education-focused Snek, it shows that Python has a place in the embedded world. The closest competitor for microcontroller development, Arduino, requires at least some knowledge of C++, but Python is generally easier to pick up. Moreover, as the WebAssembly port and its use as a smaller and faster PyScript runtime shows, MicroPython seems to be well suited for other constrained environments as well.
The end of the accounting search
Some things, it seems, just cannot be hurried. Back in 2007, your editor first started considering alternatives to the proprietary accounting system that had been used by LWN since the beginning. That search became more urgent in 2012, and returned in 2017 with a focused effort to find something better. But another five years passed before some sort of conclusion was reached. It has finally happened, though; LWN is no longer using proprietary software for its accounting needs.The system used until now — QuickBooks — was a legacy from the beginnings of the company in 1997. It has long exhibited a full litany of the sorts of misbehaviors endemic to proprietary systems. Regular (paid) updates were forced, despite the fact that new versions brought little in the way of new capabilities. Crashes were frequent; QuickBooks users know to make backups every few minutes to avoid losing work. The software had arbitrary limits meant to force "upgrades" to the "enterprise" version. More recently, Intuit has discontinued support for the desktop version entirely in an effort to force all users to its online, paid-by-the-month service.
Enough was finally enough; your editor took some time to go back and review the various systems that had been considered in the past. It turned out that the scripts developed to pry data out of QuickBooks still worked — at least, after a somewhat belated update to Python 3. In the end, the option that won out was GnuCash.
How the decision was made
There is a wealth of free accounting systems available for Linux systems. Many of them are highly complex and solve a much bigger problem; they include features like inventory tracking, customer-relation management, and more. Such systems are unwieldy to work with and difficult to get started with. In truth, many of them appear to have been developed as platforms for consulting businesses, though surely nobody would deliberately make their software complex and tricky just to motivate clients to purchase their services. LWN prefers to not spend its hard-earned pennies that way, and has no need for all of that complexity, so those systems were dropped from consideration early on.
At the other end, there are a number of accounting systems that are based on plain-text files and an associated set of programs. These systems are much easier to work with and are adaptable to most needs. Such a system could have been made to work for LWN, but they tend to lack some useful features. Check writing and report generation, for example, tend not to be well supported. The user interface for many operations is a text editor; this is often seen as an advantage, but it becomes less so as the number of transactions grows. There are advantages to a graphical interface designed for the job at hand.
GnuCash is not a perfect program by any means, but it does provide the accounting features that LWN needs, along with a reasonable graphical interface and a minimum of extra complexity.
The 2017 restart included a number of requirements that any new accounting system had to meet:
- Importing and exporting of data had to be doable — and relatively straightforward. Getting our QuickBooks data into GnuCash took a bit of scripting using the capable (if poorly documented) Python bindings. Another set of scripts has since been written to import data from the site and from our bank account. GnuCash has a reasonably capable import mechanism for the data formats exported by banks, complete with a Bayesian system to assign transactions to accounts, but a bit of scripting makes the process quicker yet.
- Basic accounting operations needed to be simple, and indeed they are.
- The generation of forms for our legal reporting requirements was on the list. GnuCash has some features to handle this, though not for US forms. In the end, this is a task that we now simply hand off to our accountant, who handles it with our annual taxes. A related requirement is check printing, which is important for a business to be able to do; more on that below.
- Ease of integration with the rest of LWN's operation was mentioned above; the Python bindings make most things fairly easy to do.
- The last requirement had to do with the future longevity of the system — how healthy is its community? The GnuCash project, having started in 1997, is slightly older than LWN; it is still here a quarter-century later and still making releases. GnuCash probably will not go away anytime soon. That said, its development community is smaller than one might like, and progress is not rapid.
All told, the requirements listed all those years ago have been met well enough for the change to be made.
Some further impressions
There are a number of aspects of GnuCash that seem not quite finished. For
example, it has had the ability to use an SQL database (PostgreSQL, MySQL,
or SQLite) for many years, but the native XML format is still
described
as "the most stable and established
" and recommended for most users.
Storing data in PostgreSQL does work, and startup time is marginally faster
when using it. Sadly, multi-user access is not supported, even when using
PostgreSQL. Another example is the reconciliation process, which doesn't
remember what the previous reconciled value was and, instead, makes up
something strange.
GnuCash has a report-generation mechanism that has clearly had a lot of effort put into it; the system is designed to be extensible by end users. The unfortunate part is that the extension language is Scheme, and it is, for the most part, undocumented. Scheme looks like Lisp but isn't really Lisp; the number of people who know it well is sure to be small. The prospect of creating a new report for GnuCash (or improving an existing one) can be a bit daunting; see the implementation of a balance sheet for an example. Your editor, in the end, found it easier to just create new reports outside of GnuCash in Python.
The ability to print checks has always been on the list. Even in the US, we write far fewer checks than we once did, but the need still does come up. GnuCash has that ability, and it can be made to work with just about any check format. The language used to add a new check format is rudimentary, but it gets the job done in the end. One annoying feature of GnuCash's check printing is that address information must be entered every time a check is printed, even if the recipient's address is stored as, for example, a vendor in GnuCash.
The biggest thing we lose in this move is the ability to export data in a format that our accountant can read directly into his proprietary tax-preparation system. He is willing to work with us, though, and believes that the reports generated from GnuCash will be enough to get the job done without a lot of additional effort (and expense). Perhaps he, too, is tired of hearing your editor's complaints about QuickBooks for all these years is prepared to do almost anything to be done with that.
In the end, GnuCash is good enough to do what LWN needs it to do. It is a nice feeling to have gotten rid of the one proprietary program used in LWN's operations, and to have better control over our accounting data. The new system is working well enough that your editor is kicking himself for having waited so long to commit to the change.
The ongoing trouble with get_user_pages()
The 2018 Linux Storage, Filesystem, and Memory-Management (LSFMM) conference included a session on get_user_pages(), an internal kernel interface that can, in some situations, be used in ways that will lead to data corruption or kernel crashes. As the 2023 LSFMM+BPF event approaches, this problem remains unsolved and is still the topic of ongoing discussion. This patch series from Lorenzo Stoakes, which is another attempt at a partial solution, is the latest focus point.
The problem
The get_user_pages() API comes in a number of variants; this API family is often referred to as "GUP". Its purpose is to provide the kernel with direct access to user-space memory; that involves ensuring that the relevant pages are resident in RAM and cannot be evicted for as long as that access is needed. The root of the problem with get_user_pages() is that it creates a situation where there are two separate paths to the memory in question.
User-space processes access their memory by way of virtual addresses mapped in their page tables. Those addresses are only valid within the process owning the memory. The page tables provide a convenient handle when the kernel needs to control access to specific ranges of user-space memory for a while. A common example is writing dirty file pages back to persistent store. A filesystem will mark those pages (in the relevant page table) as read-only, preventing further modification while the writeback is underway. If the owning process attempts to write to those pages, it will be blocked until writeback completes; thereafter, the read-only protection will cause a page fault, allowing the filesystem to be notified that the page has been dirtied again.
A call to get_user_pages() will return pointers to the kernel's page structures representing the physical pages holding the user-space memory, which can be used to obtain kernel-virtual addresses for those pages. Those addresses are in the kernel's address space, usually in the kernel's direct map that covers all of physical memory (on 64-bit systems). They are not the same as the user-space addresses, and are not subject to the same control. Direct-mapped memory that does not hold executable text is (almost) always writable by the kernel.
User space can use mmap() to map a file into its address space, creating a range of file-backed pages. Those pages will be initially marked read-only, even if mapped for write access, so that the filesystem can be notified when one of them is changed. If the kernel uses get_user_pages() to obtain write access to file-backed pages, the underlying filesystem will be duly notified that the pages have been made writable. At some future time, that filesystem will write the pages back to persistent storage, making them read-only. That protection change, though, applies only to the user-space addresses for those pages. The mapping in the kernel's address space remains writable.
That is where the problem arises: if the kernel writes to its mapping after the filesystem has written the pages back, the filesystem will be unaware that those pages have changed. Kernel code can mark the pages dirty, possibly leading to filesystem-level surprises when a seemingly read-only page has been written to. There are also a few scenarios in which the pages may never get marked as dirty, despite having been written to, in which case the written data may never find its way out to disk. Either way, the consequences are unfortunate.
This problem has been the subject of a long series of LSFMM discussions and an equally interminable set of LWN articles, but it is not an easy one to solve. There are times when the kernel simply needs access to user-space memory, often for performance purposes. A frequently repeated example is using RDMA to read data directly into file-backed pages. Allowing a DMA-capable device to write data directly into a user-space page requires pinning that page, perhaps for a long time. Finding a reliable way to enable this kind of back channel into user-space has proved difficult.
A partial solution?
In late April, Stoakes decided to face part of the problem head-on, posting
a
patch that would simply disallow get_user_pages() calls that
request write access to file-backed pages. Recognizing, though, that
there are some cases that require exactly this kind of mapping, he also
included a new flag, FOLL_ALLOW_BROKEN_FILE_MAPPING, to override
the prohibition; some InfiniBand controllers were updated to use that flag.
Making this change, Stoakes said, "is an important step towards a more
reliable GUP, and explicitly indicates which callers might encounter issues
moving forward
".
Over the following week or so, the series went through several revisions. The most significant, perhaps, was to drop the FOLL_ALLOW_BROKEN_FILE_MAPPING flag and, instead, only prohibit get_user_pages() calls that provide the FOLL_LONGTERM flag (and which request write access to file-backed pages), indicating that the mapping is likely to persist for a long time. Shorter-term mappings are not immune to the problem but, by virtue of being short-lived, they are much less likely to trigger it. This change was an acknowledgment that it is still not possible to fully solve — or even block — the problem.
This proposal has provoked a fair amount of discussion. Christoph Hellwig
worried that
it would break users who are using direct I/O to write into file-backed
mappings, but Jason Gunthorpe questioned whether
any such users exist, saying that people who tried it "didn't get very
far before testing uncovered data corruption and kernel crashes
".
David Hildenbrand, instead, suggested
that some virtualization setups could be broken by the change; once again,
Gunthorpe doubted that any
such use cases could be working successfully now.
Hildenbrand had
some other concerns about the patch, including the fact that it does
not solve the full problem: "If we want to sell this as a security
thing, we have to block it *completely* and then CC stable.
Everything else sounds like band-aids to me.
". He complained that it
does not address the "GUP-fast" subset of get_user_pages() APIs —
an omission that Stoakes later fixed. He suggested that bringing the topic
to this year's LSFMM+BPF conference (which starts on May 8) would be a
logical next step.
Ted Ts'o described
an ext4 bug that had resulted from this problem; the filesystem was not
prepared for pages to be marked dirty at unexpected times and could be made
to crash. A fix
was merged into 5.18 to prevent the crash but, Ts'o said, that might not
have been the right thing to do, since it "has apparently removed some
of the motivation of really fixing the problem instead of papering over
it
". He stated that, in the view of the filesystem
developers, writing to file-backed pages via get_user_pages() is a
bug and "you get to keep both pieces
".
Gunthorpe took Ts'o's words as yet another reason to block write access to file-backed pages:
This alone is enough reason to block it. I'm tired of this round and round and I think we should just say enough, the mm will work to enforce this view point. Files can only be written through PTEs. If this upsets people they can work on fixing it, but at least we don't have these kernel problems and inconsistencies to deal with.
There is still not a complete agreement, though, that even the partial
block that is on the table should be merged. The worries that it could end
up breaking user-space applications, or that merging the relatively easy
fix could delay the implementation of a complete solution, are not going to
just vanish. So it seems that yet another LSFMM+BPF discussion is
inevitable; indeed, Stoakes seems
to be looking forward to it: "I think discussion at LSF/MM is also a
sensible idea, also you know, if beers were bought too it could all work
out nicely :]
". So this long-term discussion is, it seems, not
over yet.
The rest of the 6.4 merge window
Linus Torvalds released 6.4-rc1 and closed the merge window on May 7. By that time, 13,044 non-merge changesets had found their way into the mainline repository for the 6.4 release. A little over 5,000 of those changesets came in after our summary of the first half of the merge window was written. Those changes brought a long list of new features and capabilities to the kernel.The most significant changes merged since the previous summary include:
Architecture-specific
- After many tries, the x86 linear address masking feature is finally supported by the kernel. In the end, only the LAM_U57 mode (which allows the storing of six bits of metadata in an address value) is supported; the LAM_U48 mode needs further consideration.
- The RISC-V architecture has gained support for the Svnapot extension. The purpose of this extension might not be obvious to all from its name; the "napot" part evidently stands for "naturally aligned power of two". It allows groups of 4K base pages to be tied together to create larger ("huge") pages.
- The RISC-V kernel can now be built as a PIE relocatable binary.
- RISC-V has also added a new system call, riscv_hwprobe(), to provide information about the hardware available on the running system. This commit has some documentation.
- Also added to RISC-V is support for hibernation.
- S390 now supports the STACKLEAK security feature (initially merged for x86 in 2018), which clears the kernel stack prior to returning to user space.
Core kernel
- The userfaultfd() system call has a new feature, UFFD_FEATURE_WP_UNPOPULATED, that allows unpopulated, anonymous memory areas to be write-protected. The new UFFDIO_CONTINUE_MODE_WP flag will cause page-table entries to be write-protected on minor faults.
- Process-level samepage merging control has been merged. This feature should make the kernel samepage merging feature more generally useful without compromising security.
- It is now possible for unprivileged processes to poll for pressure-stall information. In the unprivileged case, the polling period must be a multiple of two seconds.
- User trace events, a mechanism that allows user-space processes to create trace events through the kernel, have finally been merged after a year of reworking.
Filesystems and block I/O
- The new noswap mount option for tmpfs filesystems will cause it to not use the system swap space for data storage.
- The XFS filesystem has seen more work on its online scrub mechanism; this is said to be the last piece needed before the online filesystem-check functionality can be merged. That will come in a future cycle; meanwhile, though, the design documentation for XFS online fsck has been merged.
- The ntfs3 noacsrules mount option has been removed; it evidently never worked properly. Attempts to mount with that option will now fail with an error.
- The NFS server implementation has gained support for RFC 9289, which specifies remote procedure calls protected by TLS encryption.
Hardware support
- Clock: SkyWorks Si521xx PCIe clock generators, Sunplus SP7021 clocks, Loongson-2 clocks, MediaTek MT8188 clock controllers, Broadcom BCM63268 timer clock and reset controllers, StarFive JH7110 system clock controllers, Qualcomm SA8775P, SM6125, SM6375, and SM6115 graphics clock controllers, and Qualcomm SM7150, MSM8917/QM215, IPQ9574 and IPQ5332 global clock controllers.
- GPIO and pin control: Intel La Jolla Cove Adapter GPIO controllers, NXP S32G2 pin controllers, Qualcomm IPQ9574 and SM7150 pin controllers, and NVIDIA BlueField-3 pin controllers.
- Industrial I/O: Bosch Sensortec BMP580 pressure sensors, Texas Instruments ADS1100 and ADS1000 analog-to-digital converters, and ROHM BU27034 ambient light sensors.
- Input: NXP BBNSM power keys and Novatek NVT-ts touchscreens.
- Miscellaneous: Amlogic A1 SPI flash controllers, AMD composable DMA transfer (CDX) buses, non-volatile memory devices with "layouts", ASPEED AST2600 I3C controllers, Qualcomm PMIC flash LED modules, MediaTek MT6370 PMIC LED controllers, Rohm BD2606MVV LED drivers, Maxim 597x power switches, Renesas RZ/G2L MTU3a timers, Apple pulse-width modulators, and StarFive JH7100 and JH7110 watchdog timers.
- Sound: Freescale CPM time-slot assigners, Freescale CPM QUICC multichannel controllers, Cirrus Logic CS35L56 codecs, Analog Devices MAX98363 Soundwire speaker amplifiers, Realtek RT712 SDCA DMIC codecs, and AMD SoundWire managers.
- USB: UCSI Qualcomm PMIC GLINK interfaces.
- Also: the process of removing the kernel's ancient PCMCIA drivers has begun.
Miscellaneous
- A lengthy series removing MODULE_LICENSE() declarations from code that cannot be built as a module has been merged. Getting there was a long story; see this article and this merge message for details.
- The new virtual file /sys/kernel/tracing/touched_functions will provide a list of all kernel functions that have been traced or had a BPF program attached to them. This information is expected to be most useful to developers debugging problems (possibly) related to tracing or a BPF program.
- New Rust language support includes the pin-init core (meant to solve the problem of initializing pinned data structures), a number of locking primitives (LockClassKey, Lock and Guard, Mutex, and SpinLock), the ARef type for reference-counted objects, Task as a representation of the kernel's task_struct, LockedBy for data protected by an external lock, condition variables with CondVar (implemented using a wait queue), a UAPI crate for code dealing with user space, and a set of ioctl()-number manipulation functions.
- There is, as usual, a long list of changes to the perf tool, including the ability to filter events with BPF, improved lock-contention reporting, a new reference-count checking infrastructure, and more.
Virtualization and containers
- There is a new set of hypercalls defined to enable Hyper-V guests to access pass-through PCI devices. The kernel has also gained initial support for Hyper-V virtual trust levels.
- KVM has a new framework for handling SMCCC requests in user space. This, it is hoped, will enable the implementation of many virtualization-related operations in user space rather than adding more code to the kernel.
Internal kernel changes
- The way memory for loadable modules is managed within the kernel has changed significantly. The new module_memory mechanism increases the flexibility of memory management, with eventual improvements in both security and performance expected.
- The DEFINE_SEMAPHORE() macro now requires an argument for the initial value of the semaphore; see this article for more information on this change.
- The meaning of MAX_ORDER, which defines the size of the largest possible contiguous memory allocation in the kernel, has changed. While it was previously exclusive (the largest size was actually MAX_ORDER-1), it is now inclusive. The old meaning led to numerous subtle bugs that were fixed in the change.
- The per-VMA locks series has been merged; it should bring improved scalablity to the memory-management subsystem.
One significant feature that did not make it was user-space shadow stacks for the x86 architecture. Torvalds meditated on the pull request for almost the entire merge window before finding a bug and concluding that he did not want to pull it without some significant changes.
At this point, the merging of new features for 6.4 is complete, and the time has come to stabilize all of this work for the final release. Assuming that there are no surprises, that final release can be expected on June 25 or July 2.
Brief items
Kernel development
Kernel release status
The current development kernel is 6.4-rc1, released on May 7.
The one feature that didn't make it was the x86 shadow stack code. That side was probably a bit unlucky, in that it came in as I was looking at x86 issues anyway, and so I looked at it quite a bit, and had enough reservations that I asked for a couple of fairly big re-organizations.
This is just one more setback in the long-running shadow-stack story; see this article for some background.
In the end, 13,044 non-merge changesets were pulled during this merge window.
Stable updates: none have been released in the last week. The 6.3.2, 6.2.15, 6.1.28, and 5.15.111 stable updates are in the review process; they are due at any time.
Quote of the week
I'm already used to small differences, but *if* you send me pull requests, this does mean that it might be just slightly easier on me if you follow my lead on picking a diff algorithm, and do— Linus Torvaldsgit config diff.algorithm histogramin your kernel tree.
Distributions
Yocto Project 4.2 released
Version 4.2 of the Yocto Project distribution builder has been released. It features improved Rust support, a number of BitBake enhancements, lots of updated software, and numerous security fixes.Distributions quotes of the week
The X.org display server is deprecated, and will be removed in a future major RHEL release. The default desktop session is now the Wayland session in most cases.— Red HatThe X11 protocol remains fully supported using the XWayland back end. As a result, applications that require X11 can run in the Wayland session.
Red Hat is working on resolving the remaining problems and gaps in the Wayland session.
If you look at openSUSE's own statistics, there is absolutely no evidence that more users can lead to more contributors.— Richard BrownOur highest contribution numbers [were] when we had the least users. All of our periods of user growth have seen either a decline or stagnation in our contributor numbers.
Looking outside of our little bubble, this is not an isolated phenomenon.
Fundamentally, Projects that work hard to appeal to contributors, gain contributors.
Development
Google "We Have No Moat, And Neither Does OpenAI" (SemiAnalysis)
The SemiAnalysis site has what is said to be a leaked Google document on the state of open-source AI development. Open source, it concludes, is winning.
At the beginning of March the open source community got their hands on their first really capable foundation model, as Meta’s LLaMA was leaked to the public. It had no instruction or conversation tuning, and no RLHF. Nonetheless, the community immediately understood the significance of what they had been given.A tremendous outpouring of innovation followed, with just days between major developments (see The Timeline for the full breakdown). Here we are, barely a month later, and there are variants with instruction tuning, quantization, quality improvements, human evals, multimodality, RLHF, etc. etc. many of which build on each other.
(Thanks to Dave Täht).
Firefox 113.0 released
Version 113.0 of the Firefox browser is out. Changes include improved picture-in-picture support, blocking of third-party cookies in private windows, some accessibility improvements, and more. "A 13-year-old feature request was fulfilled and Firefox now supports files being drag-and-dropped directly from Microsoft Outlook".
New C features in GCC 13 (Red Hat Developer)
The Red Hat Developer site has an overview of some of the new C-language features supported by the GCC 13 release.
The nullptr constant first appeared in C++11, described in proposal N2431 from 2007. Its purpose was to alleviate the problems with the definition of NULL, which can be defined in a variety of ways: (void *)0 (a pointer constant), 0 (an integer), and so on. This posed problems for overload resolution, generic programming, etc. While C doesn’t have function overloading, the protean definition of NULL still causes headaches.
Julia 1.9 released
Version 1.9 of the Julia language has been released. Notable changes include improved caching of native code, faster load times via a "package extensions" mechanism, better memory-usage introspection, and more.Thunderbird 2022 financial report
The Thunderbird email-client project has put out a report describing its financial situation in 2022.
The breakout growth we enjoyed last year means hiring even more talented people to vastly improve the Thunderbird desktop experience. This past year we expended significant effort to dramatically improve Thunderbird’s UX and bring it in-line with modern expectations and standards. In 2022 we also laid the groundwork for large architectural changes for Thunderbird on the desktop. These changes address many years of technical debt that has limited our ability to add new features at a brisk pace. This work will largely pay off in our 2024 release, however it does power some of the improvements in the 115 “Supernova” release this summer.
Page editor: Jake Edge
Announcements
Newsletters
Distributions and system administration
Development
Meeting minutes
Miscellaneous
Calls for Presentations
CFP Deadlines: May 11, 2023 to July 10, 2023
The following listing of CFP deadlines is taken from the LWN.net CFP Calendar.
Deadline | Event Dates | Event | Location |
---|---|---|---|
May 28 | July 13 July 16 |
Free and Open Source Yearly | Portland OR, US |
June 4 | October 20 October 22 |
Linux Fest Northwest 2023 | Bellingham, WA, US |
June 5 | September 20 September 21 |
Linux Security Summit Europe | Bilbao, Spain |
June 12 | October 3 October 5 |
PGConf NYC | New York, US |
June 16 | September 17 September 18 |
Tracing Summit | Bilbao, Spain |
July 2 | November 3 November 5 |
Ubuntu Summit | Riga, Latvia |
July 7 | September 13 September 14 |
All Systems Go! 2023 | Berlin, Germany |
If the CFP deadline for your event does not appear here, please tell us about it.
Upcoming Events
Events: May 11, 2023 to July 10, 2023
The following event listing is taken from the LWN.net Calendar.
Date(s) | Event | Location |
---|---|---|
May 9 May 11 |
sambaXP | Goettingen, Germany |
May 10 May 12 |
Linux Security Summit | Vancouver, Canada |
May 10 May 12 |
Open Source Summit North America | Vancouver, Canada |
May 11 | NLUUG Spring Conference | Utrecht, The Netherlands |
May 12 | PGConf.BE | Haasrode, Belgium |
May 17 | Icinga Camp Berlin | Berlin, Germany |
May 17 May 20 |
BSDCan 2023 | Ottawa, Canada |
May 23 May 25 |
Red Hat Summit 2023 | Boston, US |
May 23 May 30 |
Debian Reunion Hamburg 2023 | Hamburg, Germany |
May 26 May 28 |
openSUSE Conference 2023 | Nürnberg, Germany |
May 30 June 2 |
PGCon 2023 | Ottawa, Canada |
June 13 June 15 |
Beam Summit 2023 | New York City, US |
June 13 June 15 |
Open Infrastructure Summit | Vancouver, Canada |
June 14 | Ceph Days Korea 2023 | Seoul, South Korea |
June 14 June 15 |
KVM Forum 2023 | Brno, Czech Republic |
June 15 June 17 |
Open Source Festival Africa | Lagos, Nigeria |
June 15 | Ceph Days Vancounver | Vancouver, Canada |
June 16 June 18 |
Devconf.CZ 2023 | Brno, Czech Republic |
June 20 June 22 |
SUSECON München 2023 | München, Germany |
June 27 | PostgreSQL Conference Germany | Essen, Germany |
June 28 June 30 |
Embedded Open Source Summit | Prague, Czech Republic |
If your event does not appear here, please tell us about it.
Security updates
Alert summary May 4, 2023 to May 10, 2023
Dist. | ID | Release | Package | Date |
---|---|---|---|---|
Debian | DSA-5398-1 | stable | chromium | 2023-05-04 |
Debian | DLA-3416-1 | LTS | emacs | 2023-05-10 |
Debian | DSA-5396-2 | stable | evolution | 2023-05-04 |
Debian | DSA-5399-1 | stable | odoo | 2023-05-05 |
Fedora | FEDORA-2023-659606fa84 | F38 | LibRaw | 2023-05-10 |
Fedora | FEDORA-2023-12b28d0d37 | F36 | chromium | 2023-05-10 |
Fedora | FEDORA-2023-2c4a95caf8 | F37 | chromium | 2023-05-10 |
Fedora | FEDORA-2023-af03a123b5 | F37 | community-mysql | 2023-05-10 |
Fedora | FEDORA-2023-06a49d4fb6 | F38 | community-mysql | 2023-05-10 |
Fedora | FEDORA-2023-3370eab930 | F38 | java-11-openjdk | 2023-05-05 |
Fedora | FEDORA-2023-0ab3a5423f | F37 | java-11-openjdk-portable | 2023-05-09 |
Fedora | FEDORA-2023-f839113811 | F37 | python-sentry-sdk | 2023-05-04 |
Fedora | FEDORA-2023-597f13ffb9 | F36 | rubygem-redcarpet | 2023-05-09 |
Fedora | FEDORA-2023-8682a0e17d | F37 | rubygem-redcarpet | 2023-05-09 |
Fedora | FEDORA-2023-44daa9c1d4 | F38 | rubygem-redcarpet | 2023-05-09 |
Fedora | FEDORA-2023-cc21019773 | F38 | rust-cargo-c | 2023-05-07 |
Fedora | FEDORA-2023-cc21019773 | F38 | rust-coreos-installer | 2023-05-07 |
Fedora | FEDORA-2023-cc21019773 | F38 | rust-fedora-update-feedback | 2023-05-07 |
Fedora | FEDORA-2023-cc21019773 | F38 | rust-git-delta | 2023-05-07 |
Fedora | FEDORA-2023-cc21019773 | F38 | rust-gst-plugin-reqwest | 2023-05-07 |
Fedora | FEDORA-2023-cc21019773 | F38 | rust-pore | 2023-05-07 |
Fedora | FEDORA-2023-cc21019773 | F38 | rust-rpm-sequoia | 2023-05-07 |
Fedora | FEDORA-2023-cc21019773 | F38 | rust-sequoia-octopus-librnp | 2023-05-07 |
Fedora | FEDORA-2023-cc21019773 | F38 | rust-sequoia-policy-config | 2023-05-07 |
Fedora | FEDORA-2023-cc21019773 | F38 | rust-sequoia-sq | 2023-05-07 |
Fedora | FEDORA-2023-cc21019773 | F38 | rust-sevctl | 2023-05-07 |
Fedora | FEDORA-2023-cc21019773 | F38 | rust-tealdeer | 2023-05-07 |
Fedora | FEDORA-2023-cc21019773 | F38 | rust-ybaas | 2023-05-07 |
Mageia | MGASA-2023-0158 | 8 | avahi | 2023-05-06 |
Mageia | MGASA-2023-0163 | 8 | git | 2023-05-06 |
Mageia | MGASA-2023-0161 | 8 | imagemagick | 2023-05-06 |
Mageia | MGASA-2023-0159 | 8 | libfastjson | 2023-05-06 |
Mageia | MGASA-2023-0157 | 8 | libxml2 | 2023-05-06 |
Mageia | MGASA-2023-0162 | 8 | parcellite | 2023-05-06 |
Mageia | MGASA-2023-0160 | 8 | virtualbox | 2023-05-06 |
Oracle | ELSA-2023-2127 | OL9 | samba | 2023-05-05 |
Red Hat | RHSA-2023:2204-01 | EL9 | Image Builder | 2023-05-09 |
Red Hat | RHSA-2023:2589-01 | EL9 | autotrace | 2023-05-09 |
Red Hat | RHSA-2023:2261-01 | EL9 | bind | 2023-05-09 |
Red Hat | RHSA-2023:2253-01 | EL9 | buildah | 2023-05-09 |
Red Hat | RHSA-2023:2193-01 | EL9 | butane | 2023-05-09 |
Red Hat | RHSA-2023:2222-01 | EL9 | conmon | 2023-05-09 |
Red Hat | RHSA-2023:2367-01 | EL9 | containernetworking-plugins | 2023-05-09 |
Red Hat | RHSA-2023:2478-01 | EL9 | curl | 2023-05-09 |
Red Hat | RHSA-2023:2650-01 | EL9 | curl | 2023-05-09 |
Red Hat | RHSA-2023:2459-01 | EL9 | device-mapper-multipath | 2023-05-09 |
Red Hat | RHSA-2023:2502-01 | EL9 | dhcp | 2023-05-09 |
Red Hat | RHSA-2023:2165-01 | EL9 | edk2 | 2023-05-09 |
Red Hat | RHSA-2023:2366-01 | EL9 | emacs | 2023-05-09 |
Red Hat | RHSA-2023:2626-01 | EL9 | emacs | 2023-05-09 |
Red Hat | RHSA-2023:2161-01 | EL9 | fence-agents | 2023-05-09 |
Red Hat | RHSA-2023:2166-01 | EL9 | freeradius | 2023-05-09 |
Red Hat | RHSA-2023:2326-01 | EL9 | freerdp | 2023-05-09 |
Red Hat | RHSA-2023:2202-01 | EL9 | frr | 2023-05-09 |
Red Hat | RHSA-2023:2487-01 | EL9 | fwupd | 2023-05-09 |
Red Hat | RHSA-2023:2216-01 | EL9 | gdk-pixbuf2 | 2023-05-09 |
Red Hat | RHSA-2023:2319-01 | EL9 | git | 2023-05-09 |
Red Hat | RHSA-2023:2357-01 | EL9 | git-lfs | 2023-05-09 |
Red Hat | RHSA-2023:2592-01 | EL9 | golang-github-cpuguy83-md2man | 2023-05-09 |
Red Hat | RHSA-2023:2167-01 | EL9 | grafana | 2023-05-09 |
Red Hat | RHSA-2023:2177-01 | EL9 | grafana-pcp | 2023-05-09 |
Red Hat | RHSA-2023:2260-01 | EL9 | gstreamer1-plugins-good | 2023-05-09 |
Red Hat | RHSA-2023:2312-01 | EL9 | jackson | 2023-05-09 |
Red Hat | RHSA-2023:2458-01 | EL9 | kernel | 2023-05-09 |
Red Hat | RHSA-2023:2148-01 | EL9 | kernel-rt | 2023-05-09 |
Red Hat | RHSA-2023:2570-01 | EL9 | krb5 | 2023-05-09 |
Red Hat | RHSA-2023:2532-01 | EL9 | libarchive | 2023-05-09 |
Red Hat | RHSA-2023:2179-01 | EL9 | libguestfs-winsupport | 2023-05-09 |
Red Hat | RHSA-2023:2122-01 | EL8 | libreswan | 2023-05-04 |
Red Hat | RHSA-2023:2126-01 | EL8.1 | libreswan | 2023-05-04 |
Red Hat | RHSA-2023:2124-01 | EL8.2 | libreswan | 2023-05-04 |
Red Hat | RHSA-2023:2125-01 | EL8.4 | libreswan | 2023-05-04 |
Red Hat | RHSA-2023:2123-01 | EL8.6 | libreswan | 2023-05-04 |
Red Hat | RHSA-2023:2120-01 | EL9 | libreswan | 2023-05-04 |
Red Hat | RHSA-2023:2633-01 | EL9 | libreswan | 2023-05-09 |
Red Hat | RHSA-2023:2121-01 | EL9.0 | libreswan | 2023-05-04 |
Red Hat | RHSA-2023:2340-01 | EL9 | libtiff | 2023-05-09 |
Red Hat | RHSA-2023:2453-01 | EL9 | libtpms | 2023-05-09 |
Red Hat | RHSA-2023:2582-01 | EL9 | lua | 2023-05-09 |
Red Hat | RHSA-2023:2621-01 | EL9 | mysql | 2023-05-09 |
Red Hat | RHSA-2023:2444-01 | EL9 | net-snmp | 2023-05-09 |
Red Hat | RHSA-2023:2655-01 | EL9 | nodejs nodejs-nodemon | 2023-05-09 |
Red Hat | RHSA-2023:2654-01 | EL9 | nodejs:18 | 2023-05-09 |
Red Hat | RHSA-2023:2645-01 | EL9 | openssh | 2023-05-09 |
Red Hat | RHSA-2023:2523-01 | EL9 | openssl | 2023-05-09 |
Red Hat | RHSA-2023:2652-01 | EL9 | pcs | 2023-05-09 |
Red Hat | RHSA-2023:2417-01 | EL9 | php:8.1 | 2023-05-09 |
Red Hat | RHSA-2023:2293-01 | EL9 | pki-core | 2023-05-09 |
Red Hat | RHSA-2023:2282-01 | EL9 | podman | 2023-05-09 |
Red Hat | RHSA-2023:2259-01 | EL9 | poppler | 2023-05-09 |
Red Hat | RHSA-2023:2378-01 | EL9 | postgresql-jdbc | 2023-05-09 |
Red Hat | RHSA-2023:2258-01 | EL9 | python-mako | 2023-05-09 |
Red Hat | RHSA-2023:2162-01 | EL9 | qemu-kvm | 2023-05-09 |
Red Hat | RHSA-2023:2136-01 | EL8.6 | samba | 2023-05-04 |
Red Hat | RHSA-2023:2127-01 | EL9 | samba | 2023-05-04 |
Red Hat | RHSA-2023:2519-01 | EL9 | samba | 2023-05-09 |
Red Hat | RHSA-2023:2137-01 | EL9.0 | samba | 2023-05-04 |
Red Hat | RHSA-2023:2283-01 | EL9 | skopeo | 2023-05-09 |
Red Hat | RHSA-2023:2234-01 | EL9 | sysstat | 2023-05-09 |
Red Hat | RHSA-2023:2257-01 | EL9 | tigervnc | 2023-05-09 |
Red Hat | RHSA-2023:2236-01 | EL9 | toolbox | 2023-05-09 |
Red Hat | RHSA-2023:2370-01 | EL9 | unbound | 2023-05-09 |
Red Hat | RHSA-2023:2256-01 | EL9 | webkit2gtk3 | 2023-05-09 |
Red Hat | RHSA-2023:2653-01 | EL9 | webkit2gtk3 | 2023-05-09 |
Red Hat | RHSA-2023:2373-01 | EL9 | wireshark | 2023-05-09 |
Red Hat | RHSA-2023:2248-01 | EL9 | xorg-x11-server | 2023-05-09 |
Red Hat | RHSA-2023:2249-01 | EL9 | xorg-x11-server-Xwayland | 2023-05-09 |
Slackware | SSA:2023-124-01 | libssh | 2023-05-04 | |
Slackware | SSA:2023-129-01 | mozilla | 2023-05-09 | |
SUSE | SUSE-SU-2023:1815-1 | MP4.0 MP4.1 MP4.2 MP4.3 SLE15 oS15.4 | amazon-ssm-agent | 2023-05-04 |
SUSE | SUSE-SU-2023:2152-1 | SLE12 | amazon-ssm-agent | 2023-05-09 |
SUSE | SUSE-SU-2023:1837-1 | OS9 SLE12 | apache2-mod_auth_openidc | 2023-05-04 |
SUSE | SUSE-SU-2023:2126-1 | SLE12 | cfengine, cfengine-masterfiles | 2023-05-08 |
SUSE | SUSE-SU-2023:1834-1 | MP4.3 SLE15 SLE-m5.3 SLE-m5.4 oS15.4 | cmark | 2023-05-04 |
SUSE | SUSE-SU-2023:2157-1 | MP4.3 SLE15 SLE-m5.3 SLE-m5.4 oS15.4 osM5.3 | conmon | 2023-05-10 |
SUSE | SUSE-SU-2023:1827-1 | MP4.3 SLE15 SLE-m5.1 SLE-m5.2 SLE-m5.3 SLE-m5.4 oS15.4 osM5.3 | containerd | 2023-05-04 |
SUSE | SUSE-SU-2023:1826-1 | SLE12 | containerd | 2023-05-04 |
SUSE | SUSE-SU-2023:2125-1 | SLE12 | containerd | 2023-05-08 |
SUSE | SUSE-SU-2023:2154-1 | MP4.3 SLE15 oS15.4 | distribution | 2023-05-09 |
SUSE | SUSE-SU-2023:2123-1 | OS8 OS9 SLE12 | dnsmasq | 2023-05-08 |
SUSE | SUSE-SU-2023:2153-1 | SLE12 | docker-distribution | 2023-05-09 |
SUSE | openSUSE-SU-2023:0102-1 | osB15 | editorconfig-core-c | 2023-05-04 |
SUSE | SUSE-SU-2023:2108-1 | MP4.3 SLE15 SLE-m5.3 SLE-m5.4 SES7 SES7.1 oS15.4 | ffmpeg | 2023-05-05 |
SUSE | SUSE-SU-2023:2115-1 | SLE15 SES6 | ffmpeg | 2023-05-05 |
SUSE | SUSE-SU-2023:2081-1 | SLE15 SES7 | git | 2023-05-08 |
SUSE | SUSE-SU-2023:2127-1 | MP4.3 SLE15 SES7.1 oS15.4 | go1.19 | 2023-05-08 |
SUSE | SUSE-SU-2023:2105-1 | MP4.3 SLE15 oS15.4 | go1.20 | 2023-05-05 |
SUSE | SUSE-SU-2023:2105-2 | SLE15 | go1.20 | 2023-05-08 |
SUSE | SUSE-SU-2023:2158-1 | SLE12 | google-cloud-sap-agent | 2023-05-10 |
SUSE | SUSE-SU-2023:1822-1 | MP4.2 SLE15 SLE-m5.2 SES7 SES7.1 | harfbuzz | 2023-05-04 |
SUSE | SUSE-SU-2023:1821-1 | OS9 SLE12 | harfbuzz | 2023-05-04 |
SUSE | SUSE-SU-2023:1820-1 | SLE15 | harfbuzz | 2023-05-04 |
SUSE | SUSE-SU-2023:1825-1 | SLE15 | helm | 2023-05-04 |
SUSE | SUSE-SU-2023:2139-1 | SLE-m5.4 | ignition | 2023-05-09 |
SUSE | SUSE-SU-2023:1962-1 | MP4.3 SLE15 SES7 SES7.1 oS15.4 | indent | 2023-05-08 |
SUSE | SUSE-SU-2023:2082-1 | SLE-m5.1 | installation-images | 2023-05-08 |
SUSE | SUSE-SU-2023:2083-1 | SLE-m5.4 | installation-images | 2023-05-08 |
SUSE | SUSE-SU-2023:2109-1 | SLE12 | java-11-openjdk | 2023-05-05 |
SUSE | SUSE-SU-2023:2110-1 | MP4.3 SLE15 oS15.4 | java-17-openjdk | 2023-05-08 |
SUSE | SUSE-SU-2023:1823-1 | OS9 SLE12 | java-1_8_0-ibm | 2023-05-04 |
SUSE | SUSE-SU-2023:2151-1 | MP4.0 SLE15 oS15.4 | kernel | 2023-05-09 |
SUSE | SUSE-SU-2023:2146-1 | MP4.1 SLE15 SES7 | kernel | 2023-05-09 |
SUSE | SUSE-SU-2023:2148-1 | MP4.2 SLE15 SLE-m5.1 SLE-m5.2 SES7.1 oS15.4 | kernel | 2023-05-09 |
SUSE | SUSE-SU-2023:2140-1 | MP4.3 SLE15 SLE-m5.3 SLE-m5.4 oS15.4 osM5.3 | kernel | 2023-05-09 |
SUSE | SUSE-SU-2023:2141-1 | MP4.3 SLE15 oS15.4 | kernel | 2023-05-09 |
SUSE | SUSE-SU-2023:2156-1 | SLE12 | kernel | 2023-05-10 |
SUSE | SUSE-SU-2023:2147-1 | SLE15 SLE-m5.1 SLE-m5.2 | kernel | 2023-05-09 |
SUSE | SUSE-SU-2023:2135-1 | MP4.3 SLE15 oS15.4 | libfastjson | 2023-05-09 |
SUSE | SUSE-SU-2023:1824-1 | OS9 SLE12 | liblouis | 2023-05-04 |
SUSE | SUSE-SU-2023:1829-1 | SLE15 SES7 SES7.1 oS15.4 | liblouis | 2023-05-04 |
SUSE | SUSE-SU-2023:1828-1 | SLE15 oS15.4 | liblouis | 2023-05-04 |
SUSE | SUSE-SU-2023:2097-1 | MP4.3 SLE15 SES7 SES7.1 oS15.4 | maven and recommended update for antlr3, minlog, sbt, xmvn | 2023-05-08 |
SUSE | SUSE-SU-2023:2111-1 | MP4.3 SLE15 SLE-m5.1 SLE-m5.2 SLE-m5.3 SLE-m5.4 oS15.4 osM5.3 | ncurses | 2023-05-08 |
SUSE | SUSE-SU-2023:2112-1 | SLE12 | ncurses | 2023-05-08 |
SUSE | SUSE-SU-2023:2096-1 | MP4.3 SLE15 SES7 SES7.1 oS15.4 | netty, netty-tcnative | 2023-05-08 |
SUSE | SUSE-SU-2023:2142-1 | MP4.3 SLE15 oS15.4 | ntp | 2023-05-09 |
SUSE | SUSE-SU-2023:1914-1 | OS9 SLE12 | openssl-1_0_0 | 2023-05-08 |
SUSE | SUSE-SU-2023:1814-1 | MP4.3 SLE15 SLE-m5.3 SLE-m5.4 oS15.4 osM5.3 | podman | 2023-05-04 |
SUSE | SUSE-SU-2023:2136-1 | SLE12 | prometheus-ha_cluster_exporter | 2023-05-09 |
SUSE | SUSE-SU-2023:2138-1 | SLE15 | prometheus-ha_cluster_exporter | 2023-05-09 |
SUSE | SUSE-SU-2023:2143-1 | MP4.2 MP4.3 SLE15 SES7 SES7.1 oS15.4 | protobuf-c | 2023-05-09 |
SUSE | SUSE-SU-2023:2080-1 | OS9 SLE12 | python-Django1 | 2023-05-08 |
SUSE | SUSE-SU-2023:2144-1 | OS8 SLE12 | python-cryptography | 2023-05-09 |
SUSE | SUSE-SU-2023:0722-2 | SLE-m5.4 | python-cryptography | 2023-05-09 |
SUSE | SUSE-SU-2023:2134-1 | MP4.3 SLE15 SLE-m5.3 SLE-m5.4 oS15.4 | python-ujson | 2023-05-09 |
SUSE | SUSE-SU-2023:2122-1 | MP4.3 SLE15 oS15.4 | redis | 2023-05-08 |
SUSE | SUSE-SU-2023:2137-1 | SLE12 | runc | 2023-05-09 |
SUSE | SUSE-SU-2023:2084-1 | MP4.2 MP4.3 SLE15 SLE-m5.1 SLE-m5.2 SLE-m5.3 SLE-m5.4 SES7.1 oS15.4 osM5.3 | shim | 2023-05-08 |
SUSE | SUSE-SU-2023:2150-1 | SLE12 | shim | 2023-05-09 |
SUSE | SUSE-SU-2023:2086-1 | SLE15 SES7 | shim | 2023-05-08 |
SUSE | SUSE-SU-2023:2100-1 | MP4.0 SLE15 | terraform-provider-helm | 2023-05-08 |
SUSE | SUSE-SU-2023:2103-1 | MP4.3 SLE15 SLE-m5.1 SLE-m5.2 SLE-m5.3 SLE-m5.4 oS15.4 osM5.3 | vim | 2023-05-04 |
SUSE | SUSE-SU-2023:2074-1 | SLE15 SLE-m5.1 SLE-m5.2 | zstd | 2023-05-08 |
Ubuntu | USN-6063-1 | 18.04 20.04 22.04 22.10 | ceph | 2023-05-09 |
Ubuntu | USN-6059-1 | 20.04 22.04 22.10 | erlang | 2023-05-08 |
Ubuntu | USN-6062-1 | 20.04 22.04 22.10 23.04 | freetype | 2023-05-09 |
Ubuntu | USN-6058-1 | 16.04 18.04 | linux-aws, linux-aws-hwe | 2023-05-05 |
Ubuntu | USN-6057-1 | 22.04 | linux-intel-iotg | 2023-05-05 |
Ubuntu | USN-6056-1 | 22.04 | linux-oem-6.1 | 2023-05-05 |
Ubuntu | USN-6060-1 | 18.04 20.04 22.04 22.10 23.04 | mysql-5.7, mysql-8.0 | 2023-05-08 |
Ubuntu | USN-6060-2 | 16.04 | mysql-5.7 | 2023-05-08 |
Ubuntu | USN-6065-1 | 16.04 18.04 20.04 | node-css-what | 2023-05-10 |
Ubuntu | USN-6054-1 | 18.04 20.04 22.04 22.10 23.04 | python-django | 2023-05-03 |
Ubuntu | USN-6055-1 | 16.04 18.04 20.04 | ruby2.3, ruby2.5, ruby2.7 | 2023-05-04 |
Ubuntu | USN-6055-2 | 16.04 18.04 20.04 | ruby2.3, ruby2.5, ruby2.7 | 2023-05-05 |
Ubuntu | USN-6061-1 | 20.04 22.04 22.10 23.04 | webkit2gtk | 2023-05-08 |
Kernel patches of interest
Kernel releases
Architecture-specific
Core kernel
Device drivers
Device-driver infrastructure
Documentation
Filesystems and block layer
Memory management
Networking
Security-related
Virtualization and containers
Miscellaneous
Page editor: Jonathan Corbet