|
|
Subscribe / Log in / New account

LWN.net Weekly Edition for May 11, 2023

Welcome to the LWN.net Weekly Edition for May 11, 2023

This edition contains the following feature content:

This week's edition also includes these inner pages:

  • Brief items: Brief news items from throughout the community.
  • Announcements: Newsletters, conferences, security updates, patches, and more.

Please enjoy this week's edition, and, as always, thank you for supporting LWN.net.

Comments (none posted)

Faster CPython at PyCon, part one

By Jake Edge
May 9, 2023

PyCon

Two members of the Faster CPython team, which was put together at Microsoft at the behest of Guido van Rossum to work on major performance improvements for CPython, came to PyCon 2023 to report on what the team has been working on—and its plans for the future. PEP 659 ("Specializing Adaptive Interpreter") describes the foundation of the current work, some of which has already been released as part of Python 3.11. Brandt Bucher, who gave a popular talk on structural pattern matching at last year's PyCon, was up first, with a talk on what "adaptive" and "specializing" mean in the context of Python, which we cover here in part one. Mark Shannon, whose proposed plan for performance improvements in 2020 was a major impetus for this work, presented on the past, present, and future of the Python performance enhancements, which will be covered in part two.

Bucher started out by talking a bit about Python bytecode, but said that developers do not need to know anything about it to get faster code: "just upgrade to 3.11 and you'll probably see a performance improvement". In order to show how the team sped up the interpreter, though, it would help to look inside the Python code. He put up an example Point class that has two floating point attributes for its position (xy) and a single shifted() method that takes two offsets to apply to the position of an instance, and returns a new instance of the class at the shifted position. He focused on two lines from the method:

    y = self.y + dy
    cls = type(self)
The first line applies the offset for the y axis, while the second gets a reference to the Point class (in preparation for returning a new instance by calling cls(x, y)). He used the dis module to disassemble the method; the relevant piece of bytecode is as follows:
    # part of the output of dis.dis(Point.shifted)
    LOAD_FAST      (dy)
    LOAD_FAST      (self)
    LOAD_ATTR      (y)
    BINARY_OP      (+)
    STORE_FAST     (y)

    LOAD_GLOBAL    (type)
    LOAD_FAST      (self)
    PRECALL        (1)
    CALL           (1)
    STORE_FAST     (cls)
Some of the binary opcodes (and some operations, such as method calls) have changed from those in Python 3.10 and earlier, so your output may be different than what he showed. The bytecode is a set of instructions for the Python stack-based virtual machine. In the first part above, the dy and self values are pushed onto the stack, then the LOAD_ATTR instruction retrieves the y attribute from the value popped from the stack (self), then pushes it. Next, the two values (self.y and dy) on top of the stack are added (and the result is pushed) and that result is popped to be stored in the variable y. In the second part, type and self are pushed, then PRECALL and CALL are two separate steps to perform the type(self) call; its result is popped to store into cls.

Adaptive instructions

If that code is run more than a handful of times, Bucher said, it becomes a candidate for optimization in 3.11. The first step is something called "quickening", which the PEP calls "the process of replacing slow instructions with faster variants". "Superinstructions" that combine two related instructions into a single instruction are substituted into the code. For example, the first two LOAD_FAST instructions can be replaced with a single one:

    LOAD_FAST__LOAD_FAST   (dy, self)
It is a simple change that results in a real performance boost, he said. The more interesting change is to replace some instructions with their adaptive counterparts. During the quickening process, five bytecodes are replaced with adaptive versions:
    # part of the output of dis.dis(Point.shifted, adaptive=True)
    LOAD_FAST__LOAD_FAST   (dy, self)
    LOAD_ATTR_ADAPTIVE     (y)
    BINARY_OP_ADAPTIVE     (+)
    STORE_FAST             (y)

    LOAD_GLOBAL_ADAPTIVE   (type)
    LOAD_FAST              (self)
    PRECALL_ADAPTIVE       (1)
    CALL_ADAPTIVE          (1)
    STORE_FAST             (cls)

The adaptive versions perform the same operation as their counterpart except that they can specialize themselves depending on how they are being used. For example, loading an attribute is "actually surprisingly complex", because the attribute could come from a number of places. It could be a name from a module, a class variable from a class, a method of a class, and so on, so the standard load-attribute code needs to be prepared for those possibilities. The simplest case is getting an attribute from an instance dictionary, which is exactly what is being done here.

[Brandt Bucher]

The LOAD_ATTR_ADAPTIVE operation can recognize that it is in the simple case and can ignore all of the rest of possibilities, so the adaptive instruction changes to LOAD_ATTR_INSTANCE_VALUE, which is a specialized instruction that only contains the code fast path for this common case.

The specialized instruction will then check to see if the class is unchanged from the last time this attribute was accessed and if the keys in the object's __dict__ are the same. Those much faster checks can be done in lieu of two dictionary lookups (on the class dictionary for a descriptor and on the object's __dict__ for the name); "dictionary lookups are fast, but they still cost something", while the other two checks are trivial. If the conditions hold true, which is the normal situation, the code can simply return the entry from the __dict__ at the same offset that was used previously; it does not need to do any hashing or collision resolution that comes when doing a dictionary lookup.

If either of the checks fails, the code falls back to the regular load attribute operation. Python is a dynamic language and the new interpreter needs to respect that, but the dynamic features are not being used all of the time. The idea is to not pay the cost of dynamism when it is not being used, which is rather frequent in many Python programs.

Similarly, the BINARY_OP_ADAPTIVE instruction specializes to BINARY_OP_ADD_FLOAT because floating-point values are being used. That operation checks that both operands are of type float and falls back to the (also surprisingly complex) machinery for an add operation if they are not. Otherwise, it can just add the raw floating point values together in C.

Normally, when a global name is being loaded, it requires two dictionary lookups; for example, when type() is being loaded, the global dictionary must be checked in case the function has been shadowed, if not (which is likely), then it must be looked up in the builtins dictionary. So LOAD_GLOBAL_ADAPTIVE takes a page from the attribute-loading specialization to check if the global dictionary or builtins dictionary have changed; if not, it simply grabs the value at the same index it used before.

It turns out that type() is called often enough that it gets its own specialized bytecode. It checks that the code for type() has not changed (by way of a monkey patch or similar) and, if not, simply returns the argument's class. There is a call in the C API to do so and "it's much much cheaper than making the call".

If the specialized instructions fail their tests enough times, they will revert back to the adaptive versions in order to change course. For example, if the Point class starts being used with integer values, BINARY_OP_ADD_FLOAT will revert to BINARY_OP_ADAPTIVE, which would replace itself with BINARY_OP_ADD_INT after a few uses. "This is what we mean by specializing adaptive interpreter".

It may seem like a "bunch of small localized changes"—and it is—but they add up to something substantial. For example, the shifted() method is nearly twice as fast in 3.11 versus 3.10. He obviously chose the example because it specializes well; a 2x performance increase is probably at the upper end of what can be expected. But it does show how replacing the generalized versions of these Python bytecode operations with their more specialized counterparts can lead to large improvements.

Bucher said that the various pieces of information that are being stored (e.g. the index of a dictionary entry) are actually being placed into the bytecode itself. He called those "inline caches" and they can be seen using the show_caches=True parameter to dis.dis(). But Python programmers should not really need to look at the caches, or even at the adaptive instructions, because all of that should not matter. The idea is that the interpreter will still do what it always did, but it can now do it faster in many cases.

For those who do want to dig under the covers more, though, Bucher recommended his specialist tool. Running some code with specialist will generate a web page with the source code of the program color-coded to show where the interpreter is optimizing well—and where it is not.

Coming soon

He then shifted gears to looking at things that will be coming in CPython 3.12. To start with, the dedicated quickening step has been eliminated and the superinstructions are simply emitted at compile time. In addition, there will be only a single call instruction rather than the two-step dance in 3.11.

Instead of having separate adaptive versions of the operations, the standard bytecodes implement the adaptivity. A bytecode only needs to be executed twice in order to become fully specialized. That is measured on individual bytecodes, rather than a function as a whole, so a loop will specialize immediately even if the surrounding code only gets executed once.

The team has also added more specialized instructions. "We have gotten better at specializing dynamic attribute accesses", so there are specializations for calling an object's __getattribute__() and for loading properties specified using the property() builtin. There are specializations for iterating over the four most-common objects in a for loop: list, tuple, range, and generator. There is also a single specialization for yield from in generators and await in coroutines. There are two others in the works that may make it into CPython 3.12.

Meanwhile, the inline caches are being reduced. Having more cached information makes the bytecode longer, which means longer jumps and more memory use, so shrinking the amount of cached data is welcome. The team has been able to eliminate some cache entries or reorganize the caches to make some significant reductions. All of that is coming in Python 3.12—and beyond.

Stay tuned for our coverage of Shannon's talk, which came on the second day of the conference. It will be the subject of part two, which is coming soon to an LWN near you.

[I would like to thank LWN subscribers for supporting my travel to Salt Lake City for PyCon.]

Comments (18 posted)

MicroPython 1.20: Python for microcontrollers

May 11, 2023

This article was contributed by Koen Vervloesem

The MicroPython programming language implements a sizable subset of Python that can run on microcontrollers, thus bringing Python's easy-to-learn syntax, readability, and versatility to the embedded world. With its recent 1.20 release, MicroPython introduces a new package manager, reduces its code size, and adds support for many new boards, including the Raspberry Pi Pico W. The project has come a long way since its inception ten years ago, making it an easy-to-use tool for developing software for resource-constrained environments.

The MIT-licensed MicroPython was created in 2013 by Damien George as a Kickstarter project, along with the pyboard, a microcontroller board to run it on. LWN looked at MicroPython on the pyboard in 2015. There are still updated versions of the pyboard available in the MicroPython store; they are the official MicroPython boards and their sales support the ongoing development of the project.

By using MicroPython, developers can build on their knowledge of Python to create software for microcontrollers, where they typically would need to write C code. The resulting program may not be as fast as a compiled C program, but in many cases the tradeoff for ease of development is worth it. However, developers should be aware of the (well documented) differences from CPython, the canonical Python interpreter. MicroPython implements Python 3.4, as well as some select features of Python 3.5 and above. Some of Python's built-in types and standard library modules are not implemented fully. For example, slicing a string with a step other than one is not implemented, and user-defined attributes for built-in exceptions aren't supported either.

There's a reason for many of the differences from CPython: the interpreter needs to be able to run on devices with low resources. Additionally, the project applies several optimizations. One of these is that small integers are not allocated on the heap, as explained in the documentation about memory management. To avoid storing duplicate copies of the same string (including function and variable names), MicroPython uses string interning. The documentation also lists some tips to maximize speed.

Instead of letting MicroPython load a Python module, parse it, generate its bytecode and then execute it, developers can instead pre-compile their modules and load the resulting .mpy file on the board. This avoids the overhead of parsing and generating the bytecode on the microcontroller. This is similar to the CPython .pyc files, but MicroPython has its own bytecode representation.

A further optimization is frozen bytecode: this allows building the pre-compiled bytecode into the firmware image when compiling MicroPython for a specific board. When this firmware image is installed to the board, the bytecode will be executed from flash memory rather than copied into RAM, and objects of immutable types (such as strings and tuples) are loaded from flash memory as well. This results in less RAM usage and reduces heap fragmentation.

Supported hardware

MicroPython offers firmware downloads for more than 150 microcontroller boards. This includes boards with Espressif's ESP32 and ESP8266 microcontroller families, boards with an STM32 ARM Cortex-M processor, Microchip's SAMD microcontrollers with ARM Cortex-M0+ core, Nordic Semiconductor's nRF5 chips, and the Raspberry Pi Foundation's RP2040-based boards (including the Raspberry Pi Pico W). Each board's download page includes installation instructions. For example, a Raspberry Pi Pico with the UF2 bootloader requires the user to hold down the BOOTSEL button while plugging the board into USB, after which the firmware file can be copied to the USB mass storage device that appears. Other boards have their own specific requirements.

MicroPython has also been ported to the Zephyr real-time operating system. In MicroPython 1.20, this port is based on Zephyr 3.1.0, which was released in June 2022; the current Zephyr release is 3.3.0. A Zephyr development environment can be used to build MicroPython for every target board supported by Zephyr, although not all have been tested. It also gives MicroPython code access to Zephyr's uniform sensor API using the zsensor module. More information can be found in MicroPython's documentation about the Zephyr port.

The Zephyr port allows MicroPython to run on the BBC micro:bit v2, whereas there's only a direct port for the BBC micro:bit v1. The process of setting up the development environment, building the Zephyr port for the target board, and then flashing the firmware to the board is explained in the port's README file.

Development environment

When the MicroPython interpreter, which is written in C, is running on a microcontroller board, it offers a read-eval-print loop (REPL) shell. For most boards, the REPL is available using a serial connection over the board's USB port. This allows the user to connect to the MicroPython REPL with a tool such as GNU Screen and then interactively run commands on the board. The project also offers the mpremote tool to interact with a MicroPython device over a serial connection.

Thonny and Mu Editor are two popular development environments that not only support the MicroPython language and integrate the REPL, but also let the user install the MicroPython firmware on a connected board. Last year, the Arduino developers published an experimental pre-release of their own lightweight editor for MicroPython programs: Arduino Lab for MicroPython.

Using mpremote or one of the supported development environments, a developer can upload Python code to the board's filesystem. The MicroPython interpreter evaluates two files at boot: boot.py and main.py (or .mpy versions instead). The boot.py file is supposed to contain initialization code, for example to connect to a WiFi network. The main.py file should contain the main code of the program, which often executes the device's task indefinitely in an infinite loop.

As an example, here's some MicroPython code for a NeoPixel stick with its IN pin connected to the GPIO22 pin of a board (which I tested using an ESP32 board):

    from machine import Pin
    from neopixel import NeoPixel
    from time import sleep

    PIXELS = 8
    RED = (32, 0, 0)
    OFF = (0, 0, 0)
    np = NeoPixel(Pin(22), PIXELS)

    while True:
	for i in range(PIXELS):
	    np[i] = RED
	    np.write()
	    sleep(0.1)
	    np[i] = OFF
	    np.write()
	    sleep(0.1)

After importing the modules it needs, the program starts by declaring some constants. Since the neopixel library needs its colors specified as a tuple of red, green, and blue components, the color RED uses 32 as the red component (the maximum value 255 is blindingly bright on those LEDs). After this, the code creates a NeoPixel object for GPIO pin 22 and the number of pixels (LEDs) defined. This object is indexable, so the infinite loop just cycles through all the pixels, setting them one by one to the red color for 100 ms, before turning them off again.

Libraries

MicroPython comes with three types of libraries. Some modules are reimplementations of part of the Python standard library. These are modules such as collections, json, socket, and uasyncio (a subset of Python's asyncio module).

The second type of libraries implements functionality specific to MicroPython. These include the bluetooth module that offers low-level access to Bluetooth Low Energy, the machine module that is useful to control I/O pins, use a board's analog-to-digital converters, or use the I2C protocol, and the network module for network access. MicroPython also has some libraries that are specific to a board family.

Due to resource constraints or other hardware-specific limitations, MicroPython's libraries don't include all functions, classes or even modules on all devices. This information should always be consulted in the "Quick reference" for the device family's port in the sidebar of MicroPython's documentation.

While those two types of libraries are directly built into the MicroPython firmware, there are some additional external libraries that are maintained in the micropython-lib repository. Some of these libraries are implementations of parts of the CPython standard library that are not built in the firmware to save space. Others implement a subset of popular Python packages, such as the urequests version of the Requests HTTP library. It also hosts some MicroPython-specific packages, such as drivers for sensors, displays, and other hardware.

The recent MicroPython 1.20 release replaced the previous package manager upip with mip, which can be used to install libraries from the micropython-lib repository. The firmware includes the mip module on any network-capable board, after which libraries can be installed from the REPL as long as there's a network connection set up on the board:

    >>> import mip 
    >>> mip.install("neopixel")

The default behavior is to fetch .mpy files with pre-compiled bytecode (which is versioned) for libraries from micropython-lib. The package manager installs the files in the board's file system, where they are available to the developer's Python files.

Instead of supplying a package name from the micropython-lib repository, it's also possible to point to a GitHub repository or an arbitrary URL if it contains a package.json file with information about the files to install and the package's dependencies.

MicroPython on the web

Another interesting target is WebAssembly (Wasm), which allows MicroPython to run in a web browser. The WebAssembly port's README includes instructions to compile the MicroPython firmware to Wasm using Emscripten. When Anaconda founder Peter Wang announced PyScript at PyCon 2022, it was using CPython compiled to WebAssembly (from Pyodide) and making that available in the web browser so developers could create web apps in Python.

In November 2022, Anaconda followed up with the announcement of a second runtime for PyScript, based on MicroPython. This is currently a technical preview to explore the tradeoffs of using different Python runtimes. PyScript's MicroPython runtime has a total size of only 303KB and starts running the developer's script in less than 100ms, while the Pyodide runtime weights 11MB and is slower to start up.

Since the announcement in November, there hasn't been anything more about this alternative runtime on Anaconda's blog, but it looks like some things are cooking. In early April, Ted Patrick, PyScript's Engineering Manager, opened a GitHub issue in the PyScript repository with the message "MicroPython is going to be supported as an alternative Python interpreter to Pyodide", and assigning it to the project's 2023-Q2 milestone. So it looks like MicroPython will be supported quite soon as an alternative PyScript runtime.

Conclusion

For those looking for an easy way to program microcontrollers, MicroPython has much to offer. Together with alternatives like Adafruit's MicroPython fork CircuitPython and the education-focused Snek, it shows that Python has a place in the embedded world. The closest competitor for microcontroller development, Arduino, requires at least some knowledge of C++, but Python is generally easier to pick up. Moreover, as the WebAssembly port and its use as a smaller and faster PyScript runtime shows, MicroPython seems to be well suited for other constrained environments as well.

Comments (1 posted)

The end of the accounting search

By Jonathan Corbet
May 5, 2023
Some things, it seems, just cannot be hurried. Back in 2007, your editor first started considering alternatives to the proprietary accounting system that had been used by LWN since the beginning. That search became more urgent in 2012, and returned in 2017 with a focused effort to find something better. But another five years passed before some sort of conclusion was reached. It has finally happened, though; LWN is no longer using proprietary software for its accounting needs.

The system used until now — QuickBooks — was a legacy from the beginnings of the company in 1997. It has long exhibited a full litany of the sorts of misbehaviors endemic to proprietary systems. Regular (paid) updates were forced, despite the fact that new versions brought little in the way of new capabilities. Crashes were frequent; QuickBooks users know to make backups every few minutes to avoid losing work. The software had arbitrary limits meant to force "upgrades" to the "enterprise" version. More recently, Intuit has discontinued support for the desktop version entirely in an effort to force all users to its online, paid-by-the-month service.

Enough was finally enough; your editor took some time to go back and review the various systems that had been considered in the past. It turned out that the scripts developed to pry data out of QuickBooks still worked — at least, after a somewhat belated update to Python 3. In the end, the option that won out was GnuCash.

How the decision was made

There is a wealth of free accounting systems available for Linux systems. Many of them are highly complex and solve a much bigger problem; they include features like inventory tracking, customer-relation management, and more. Such systems are unwieldy to work with and difficult to get started with. In truth, many of them appear to have been developed as platforms for consulting businesses, though surely nobody would deliberately make their software complex and tricky just to motivate clients to purchase their services. LWN prefers to not spend its hard-earned pennies that way, and has no need for all of that complexity, so those systems were dropped from consideration early on.

At the other end, there are a number of accounting systems that are based on plain-text files and an associated set of programs. These systems are much easier to work with and are adaptable to most needs. Such a system could have been made to work for LWN, but they tend to lack some useful features. Check writing and report generation, for example, tend not to be well supported. The user interface for many operations is a text editor; this is often seen as an advantage, but it becomes less so as the number of transactions grows. There are advantages to a graphical interface designed for the job at hand.

GnuCash is not a perfect program by any means, but it does provide the accounting features that LWN needs, along with a reasonable graphical interface and a minimum of extra complexity.

The 2017 restart included a number of requirements that any new accounting system had to meet:

  • Importing and exporting of data had to be doable — and relatively straightforward. Getting our QuickBooks data into GnuCash took a bit of scripting using the capable (if poorly documented) Python bindings. Another set of scripts has since been written to import data from the site and from our bank account. GnuCash has a reasonably capable import mechanism for the data formats exported by banks, complete with a Bayesian system to assign transactions to accounts, but a bit of scripting makes the process quicker yet.
  • Basic accounting operations needed to be simple, and indeed they are.
  • The generation of forms for our legal reporting requirements was on the list. GnuCash has some features to handle this, though not for US forms. In the end, this is a task that we now simply hand off to our accountant, who handles it with our annual taxes. A related requirement is check printing, which is important for a business to be able to do; more on that below.
  • Ease of integration with the rest of LWN's operation was mentioned above; the Python bindings make most things fairly easy to do.
  • The last requirement had to do with the future longevity of the system — how healthy is its community? The GnuCash project, having started in 1997, is slightly older than LWN; it is still here a quarter-century later and still making releases. GnuCash probably will not go away anytime soon. That said, its development community is smaller than one might like, and progress is not rapid.

All told, the requirements listed all those years ago have been met well enough for the change to be made.

Some further impressions

There are a number of aspects of GnuCash that seem not quite finished. For example, it has had the ability to use an SQL database (PostgreSQL, MySQL, or SQLite) for many years, but the native XML format is still described as "the most stable and established" and recommended for most users. Storing data in PostgreSQL does work, and startup time is marginally faster when using it. Sadly, multi-user access is not supported, even when using PostgreSQL. Another example is the reconciliation process, which doesn't remember what the previous reconciled value was and, instead, makes up something strange.

GnuCash has a report-generation mechanism that has clearly had a lot of effort put into it; the system is designed to be extensible by end users. The unfortunate part is that the extension language is Scheme, and it is, for the most part, undocumented. Scheme looks like Lisp but isn't really Lisp; the number of people who know it well is sure to be small. The prospect of creating a new report for GnuCash (or improving an existing one) can be a bit daunting; see the implementation of a balance sheet for an example. Your editor, in the end, found it easier to just create new reports outside of GnuCash in Python.

The ability to print checks has always been on the list. Even in the US, we write far fewer checks than we once did, but the need still does come up. GnuCash has that ability, and it can be made to work with just about any check format. The language used to add a new check format is rudimentary, but it gets the job done in the end. One annoying feature of GnuCash's check printing is that address information must be entered every time a check is printed, even if the recipient's address is stored as, for example, a vendor in GnuCash.

The biggest thing we lose in this move is the ability to export data in a format that our accountant can read directly into his proprietary tax-preparation system. He is willing to work with us, though, and believes that the reports generated from GnuCash will be enough to get the job done without a lot of additional effort (and expense). Perhaps he, too, is tired of hearing your editor's complaints about QuickBooks for all these years is prepared to do almost anything to be done with that.

In the end, GnuCash is good enough to do what LWN needs it to do. It is a nice feeling to have gotten rid of the one proprietary program used in LWN's operations, and to have better control over our accounting data. The new system is working well enough that your editor is kicking himself for having waited so long to commit to the change.

Comments (80 posted)

The ongoing trouble with get_user_pages()

By Jonathan Corbet
May 4, 2023
The 2018 Linux Storage, Filesystem, and Memory-Management (LSFMM) conference included a session on get_user_pages(), an internal kernel interface that can, in some situations, be used in ways that will lead to data corruption or kernel crashes. As the 2023 LSFMM+BPF event approaches, this problem remains unsolved and is still the topic of ongoing discussion. This patch series from Lorenzo Stoakes, which is another attempt at a partial solution, is the latest focus point.

The problem

The get_user_pages() API comes in a number of variants; this API family is often referred to as "GUP". Its purpose is to provide the kernel with direct access to user-space memory; that involves ensuring that the relevant pages are resident in RAM and cannot be evicted for as long as that access is needed. The root of the problem with get_user_pages() is that it creates a situation where there are two separate paths to the memory in question.

User-space processes access their memory by way of virtual addresses mapped in their page tables. Those addresses are only valid within the process owning the memory. The page tables provide a convenient handle when the kernel needs to control access to specific ranges of user-space memory for a while. A common example is writing dirty file pages back to persistent store. A filesystem will mark those pages (in the relevant page table) as read-only, preventing further modification while the writeback is underway. If the owning process attempts to write to those pages, it will be blocked until writeback completes; thereafter, the read-only protection will cause a page fault, allowing the filesystem to be notified that the page has been dirtied again.

A call to get_user_pages() will return pointers to the kernel's page structures representing the physical pages holding the user-space memory, which can be used to obtain kernel-virtual addresses for those pages. Those addresses are in the kernel's address space, usually in the kernel's direct map that covers all of physical memory (on 64-bit systems). They are not the same as the user-space addresses, and are not subject to the same control. Direct-mapped memory that does not hold executable text is (almost) always writable by the kernel.

User space can use mmap() to map a file into its address space, creating a range of file-backed pages. Those pages will be initially marked read-only, even if mapped for write access, so that the filesystem can be notified when one of them is changed. If the kernel uses get_user_pages() to obtain write access to file-backed pages, the underlying filesystem will be duly notified that the pages have been made writable. At some future time, that filesystem will write the pages back to persistent storage, making them read-only. That protection change, though, applies only to the user-space addresses for those pages. The mapping in the kernel's address space remains writable.

That is where the problem arises: if the kernel writes to its mapping after the filesystem has written the pages back, the filesystem will be unaware that those pages have changed. Kernel code can mark the pages dirty, possibly leading to filesystem-level surprises when a seemingly read-only page has been written to. There are also a few scenarios in which the pages may never get marked as dirty, despite having been written to, in which case the written data may never find its way out to disk. Either way, the consequences are unfortunate.

This problem has been the subject of a long series of LSFMM discussions and an equally interminable set of LWN articles, but it is not an easy one to solve. There are times when the kernel simply needs access to user-space memory, often for performance purposes. A frequently repeated example is using RDMA to read data directly into file-backed pages. Allowing a DMA-capable device to write data directly into a user-space page requires pinning that page, perhaps for a long time. Finding a reliable way to enable this kind of back channel into user-space has proved difficult.

A partial solution?

In late April, Stoakes decided to face part of the problem head-on, posting a patch that would simply disallow get_user_pages() calls that request write access to file-backed pages. Recognizing, though, that there are some cases that require exactly this kind of mapping, he also included a new flag, FOLL_ALLOW_BROKEN_FILE_MAPPING, to override the prohibition; some InfiniBand controllers were updated to use that flag. Making this change, Stoakes said, "is an important step towards a more reliable GUP, and explicitly indicates which callers might encounter issues moving forward".

Over the following week or so, the series went through several revisions. The most significant, perhaps, was to drop the FOLL_ALLOW_BROKEN_FILE_MAPPING flag and, instead, only prohibit get_user_pages() calls that provide the FOLL_LONGTERM flag (and which request write access to file-backed pages), indicating that the mapping is likely to persist for a long time. Shorter-term mappings are not immune to the problem but, by virtue of being short-lived, they are much less likely to trigger it. This change was an acknowledgment that it is still not possible to fully solve — or even block — the problem.

This proposal has provoked a fair amount of discussion. Christoph Hellwig worried that it would break users who are using direct I/O to write into file-backed mappings, but Jason Gunthorpe questioned whether any such users exist, saying that people who tried it "didn't get very far before testing uncovered data corruption and kernel crashes". David Hildenbrand, instead, suggested that some virtualization setups could be broken by the change; once again, Gunthorpe doubted that any such use cases could be working successfully now.

Hildenbrand had some other concerns about the patch, including the fact that it does not solve the full problem: "If we want to sell this as a security thing, we have to block it *completely* and then CC stable. Everything else sounds like band-aids to me.". He complained that it does not address the "GUP-fast" subset of get_user_pages() APIs — an omission that Stoakes later fixed. He suggested that bringing the topic to this year's LSFMM+BPF conference (which starts on May 8) would be a logical next step.

Ted Ts'o described an ext4 bug that had resulted from this problem; the filesystem was not prepared for pages to be marked dirty at unexpected times and could be made to crash. A fix was merged into 5.18 to prevent the crash but, Ts'o said, that might not have been the right thing to do, since it "has apparently removed some of the motivation of really fixing the problem instead of papering over it". He stated that, in the view of the filesystem developers, writing to file-backed pages via get_user_pages() is a bug and "you get to keep both pieces".

Gunthorpe took Ts'o's words as yet another reason to block write access to file-backed pages:

This alone is enough reason to block it. I'm tired of this round and round and I think we should just say enough, the mm will work to enforce this view point. Files can only be written through PTEs. If this upsets people they can work on fixing it, but at least we don't have these kernel problems and inconsistencies to deal with.

There is still not a complete agreement, though, that even the partial block that is on the table should be merged. The worries that it could end up breaking user-space applications, or that merging the relatively easy fix could delay the implementation of a complete solution, are not going to just vanish. So it seems that yet another LSFMM+BPF discussion is inevitable; indeed, Stoakes seems to be looking forward to it: "I think discussion at LSF/MM is also a sensible idea, also you know, if beers were bought too it could all work out nicely :]". So this long-term discussion is, it seems, not over yet.

Comments (none posted)

The rest of the 6.4 merge window

By Jonathan Corbet
May 8, 2023
Linus Torvalds released 6.4-rc1 and closed the merge window on May 7. By that time, 13,044 non-merge changesets had found their way into the mainline repository for the 6.4 release. A little over 5,000 of those changesets came in after our summary of the first half of the merge window was written. Those changes brought a long list of new features and capabilities to the kernel.

The most significant changes merged since the previous summary include:

Architecture-specific

  • After many tries, the x86 linear address masking feature is finally supported by the kernel. In the end, only the LAM_U57 mode (which allows the storing of six bits of metadata in an address value) is supported; the LAM_U48 mode needs further consideration.
  • The RISC-V architecture has gained support for the Svnapot extension. The purpose of this extension might not be obvious to all from its name; the "napot" part evidently stands for "naturally aligned power of two". It allows groups of 4K base pages to be tied together to create larger ("huge") pages.
  • The RISC-V kernel can now be built as a PIE relocatable binary.
  • RISC-V has also added a new system call, riscv_hwprobe(), to provide information about the hardware available on the running system. This commit has some documentation.
  • Also added to RISC-V is support for hibernation.
  • S390 now supports the STACKLEAK security feature (initially merged for x86 in 2018), which clears the kernel stack prior to returning to user space.

Core kernel

  • The userfaultfd() system call has a new feature, UFFD_FEATURE_WP_UNPOPULATED, that allows unpopulated, anonymous memory areas to be write-protected. The new UFFDIO_CONTINUE_MODE_WP flag will cause page-table entries to be write-protected on minor faults.
  • Process-level samepage merging control has been merged. This feature should make the kernel samepage merging feature more generally useful without compromising security.
  • It is now possible for unprivileged processes to poll for pressure-stall information. In the unprivileged case, the polling period must be a multiple of two seconds.
  • User trace events, a mechanism that allows user-space processes to create trace events through the kernel, have finally been merged after a year of reworking.

Filesystems and block I/O

  • The new noswap mount option for tmpfs filesystems will cause it to not use the system swap space for data storage.
  • The XFS filesystem has seen more work on its online scrub mechanism; this is said to be the last piece needed before the online filesystem-check functionality can be merged. That will come in a future cycle; meanwhile, though, the design documentation for XFS online fsck has been merged.
  • The ntfs3 noacsrules mount option has been removed; it evidently never worked properly. Attempts to mount with that option will now fail with an error.
  • The NFS server implementation has gained support for RFC 9289, which specifies remote procedure calls protected by TLS encryption.

Hardware support

  • Clock: SkyWorks Si521xx PCIe clock generators, Sunplus SP7021 clocks, Loongson-2 clocks, MediaTek MT8188 clock controllers, Broadcom BCM63268 timer clock and reset controllers, StarFive JH7110 system clock controllers, Qualcomm SA8775P, SM6125, SM6375, and SM6115 graphics clock controllers, and Qualcomm SM7150, MSM8917/QM215, IPQ9574 and IPQ5332 global clock controllers.
  • GPIO and pin control: Intel La Jolla Cove Adapter GPIO controllers, NXP S32G2 pin controllers, Qualcomm IPQ9574 and SM7150 pin controllers, and NVIDIA BlueField-3 pin controllers.
  • Industrial I/O: Bosch Sensortec BMP580 pressure sensors, Texas Instruments ADS1100 and ADS1000 analog-to-digital converters, and ROHM BU27034 ambient light sensors.
  • Input: NXP BBNSM power keys and Novatek NVT-ts touchscreens.
  • Miscellaneous: Amlogic A1 SPI flash controllers, AMD composable DMA transfer (CDX) buses, non-volatile memory devices with "layouts", ASPEED AST2600 I3C controllers, Qualcomm PMIC flash LED modules, MediaTek MT6370 PMIC LED controllers, Rohm BD2606MVV LED drivers, Maxim 597x power switches, Renesas RZ/G2L MTU3a timers, Apple pulse-width modulators, and StarFive JH7100 and JH7110 watchdog timers.
  • Sound: Freescale CPM time-slot assigners, Freescale CPM QUICC multichannel controllers, Cirrus Logic CS35L56 codecs, Analog Devices MAX98363 Soundwire speaker amplifiers, Realtek RT712 SDCA DMIC codecs, and AMD SoundWire managers.
  • USB: UCSI Qualcomm PMIC GLINK interfaces.
  • Also: the process of removing the kernel's ancient PCMCIA drivers has begun.

Miscellaneous

  • A lengthy series removing MODULE_LICENSE() declarations from code that cannot be built as a module has been merged. Getting there was a long story; see this article and this merge message for details.
  • The new virtual file /sys/kernel/tracing/touched_functions will provide a list of all kernel functions that have been traced or had a BPF program attached to them. This information is expected to be most useful to developers debugging problems (possibly) related to tracing or a BPF program.
  • New Rust language support includes the pin-init core (meant to solve the problem of initializing pinned data structures), a number of locking primitives (LockClassKey, Lock and Guard, Mutex, and SpinLock), the ARef type for reference-counted objects, Task as a representation of the kernel's task_struct, LockedBy for data protected by an external lock, condition variables with CondVar (implemented using a wait queue), a UAPI crate for code dealing with user space, and a set of ioctl()-number manipulation functions.
  • There is, as usual, a long list of changes to the perf tool, including the ability to filter events with BPF, improved lock-contention reporting, a new reference-count checking infrastructure, and more.

Virtualization and containers

  • There is a new set of hypercalls defined to enable Hyper-V guests to access pass-through PCI devices. The kernel has also gained initial support for Hyper-V virtual trust levels.
  • KVM has a new framework for handling SMCCC requests in user space. This, it is hoped, will enable the implementation of many virtualization-related operations in user space rather than adding more code to the kernel.

Internal kernel changes

  • The way memory for loadable modules is managed within the kernel has changed significantly. The new module_memory mechanism increases the flexibility of memory management, with eventual improvements in both security and performance expected.
  • The DEFINE_SEMAPHORE() macro now requires an argument for the initial value of the semaphore; see this article for more information on this change.
  • The meaning of MAX_ORDER, which defines the size of the largest possible contiguous memory allocation in the kernel, has changed. While it was previously exclusive (the largest size was actually MAX_ORDER-1), it is now inclusive. The old meaning led to numerous subtle bugs that were fixed in the change.
  • The per-VMA locks series has been merged; it should bring improved scalablity to the memory-management subsystem.

One significant feature that did not make it was user-space shadow stacks for the x86 architecture. Torvalds meditated on the pull request for almost the entire merge window before finding a bug and concluding that he did not want to pull it without some significant changes.

At this point, the merging of new features for 6.4 is complete, and the time has come to stabilize all of this work for the final release. Assuming that there are no surprises, that final release can be expected on June 25 or July 2.

Comments (none posted)

Page editor: Jonathan Corbet

Brief items

Kernel development

Kernel release status

The current development kernel is 6.4-rc1, released on May 7.

The one feature that didn't make it was the x86 shadow stack code. That side was probably a bit unlucky, in that it came in as I was looking at x86 issues anyway, and so I looked at it quite a bit, and had enough reservations that I asked for a couple of fairly big re-organizations.

This is just one more setback in the long-running shadow-stack story; see this article for some background.

In the end, 13,044 non-merge changesets were pulled during this merge window.

Stable updates: none have been released in the last week. The 6.3.2, 6.2.15, 6.1.28, and 5.15.111 stable updates are in the review process; they are due at any time.

Comments (none posted)

Quote of the week

I'm already used to small differences, but *if* you send me pull requests, this does mean that it might be just slightly easier on me if you follow my lead on picking a diff algorithm, and do
     git config diff.algorithm histogram
in your kernel tree.
Linus Torvalds

Comments (none posted)

Distributions

Yocto Project 4.2 released

Version 4.2 of the Yocto Project distribution builder has been released. It features improved Rust support, a number of BitBake enhancements, lots of updated software, and numerous security fixes.

Comments (1 posted)

Distributions quotes of the week

The X.org display server is deprecated, and will be removed in a future major RHEL release. The default desktop session is now the Wayland session in most cases.

The X11 protocol remains fully supported using the XWayland back end. As a result, applications that require X11 can run in the Wayland session.

Red Hat is working on resolving the remaining problems and gaps in the Wayland session.

Red Hat

If you look at openSUSE's own statistics, there is absolutely no evidence that more users can lead to more contributors.

Our highest contribution numbers [were] when we had the least users. All of our periods of user growth have seen either a decline or stagnation in our contributor numbers.

Looking outside of our little bubble, this is not an isolated phenomenon.

Fundamentally, Projects that work hard to appeal to contributors, gain contributors.

Richard Brown

Comments (10 posted)

Development

Google "We Have No Moat, And Neither Does OpenAI" (SemiAnalysis)

The SemiAnalysis site has what is said to be a leaked Google document on the state of open-source AI development. Open source, it concludes, is winning.

At the beginning of March the open source community got their hands on their first really capable foundation model, as Meta’s LLaMA was leaked to the public. It had no instruction or conversation tuning, and no RLHF. Nonetheless, the community immediately understood the significance of what they had been given.

A tremendous outpouring of innovation followed, with just days between major developments (see The Timeline for the full breakdown). Here we are, barely a month later, and there are variants with instruction tuning, quantization, quality improvements, human evals, multimodality, RLHF, etc. etc. many of which build on each other.

(Thanks to Dave Täht).

Comments (69 posted)

Firefox 113.0 released

Version 113.0 of the Firefox browser is out. Changes include improved picture-in-picture support, blocking of third-party cookies in private windows, some accessibility improvements, and more. "A 13-year-old feature request was fulfilled and Firefox now supports files being drag-and-dropped directly from Microsoft Outlook".

Comments (11 posted)

New C features in GCC 13 (Red Hat Developer)

The Red Hat Developer site has an overview of some of the new C-language features supported by the GCC 13 release.

The nullptr constant first appeared in C++11, described in proposal N2431 from 2007. Its purpose was to alleviate the problems with the definition of NULL, which can be defined in a variety of ways: (void *)0 (a pointer constant), 0 (an integer), and so on. This posed problems for overload resolution, generic programming, etc. While C doesn’t have function overloading, the protean definition of NULL still causes headaches.

Comments (43 posted)

Julia 1.9 released

Version 1.9 of the Julia language has been released. Notable changes include improved caching of native code, faster load times via a "package extensions" mechanism, better memory-usage introspection, and more.

Comments (1 posted)

Thunderbird 2022 financial report

The Thunderbird email-client project has put out a report describing its financial situation in 2022.

The breakout growth we enjoyed last year means hiring even more talented people to vastly improve the Thunderbird desktop experience. This past year we expended significant effort to dramatically improve Thunderbird’s UX and bring it in-line with modern expectations and standards. In 2022 we also laid the groundwork for large architectural changes for Thunderbird on the desktop. These changes address many years of technical debt that has limited our ability to add new features at a brisk pace. This work will largely pay off in our 2024 release, however it does power some of the improvements in the 115 “Supernova” release this summer.

Comments (2 posted)

Page editor: Jake Edge

Announcements

Newsletters

Distributions and system administration

Development

Meeting minutes

Miscellaneous

Calls for Presentations

CFP Deadlines: May 11, 2023 to July 10, 2023

The following listing of CFP deadlines is taken from the LWN.net CFP Calendar.

DeadlineEvent Dates EventLocation
May 28 July 13
July 16
Free and Open Source Yearly Portland OR, US
June 4 October 20
October 22
Linux Fest Northwest 2023 Bellingham, WA, US
June 5 September 20
September 21
Linux Security Summit Europe Bilbao, Spain
June 12 October 3
October 5
PGConf NYC New York, US
June 16 September 17
September 18
Tracing Summit Bilbao, Spain
July 2 November 3
November 5
Ubuntu Summit Riga, Latvia
July 7 September 13
September 14
All Systems Go! 2023 Berlin, Germany

If the CFP deadline for your event does not appear here, please tell us about it.

Upcoming Events

Events: May 11, 2023 to July 10, 2023

The following event listing is taken from the LWN.net Calendar.

Date(s)EventLocation
May 9
May 11
sambaXP Goettingen, Germany
May 10
May 12
Linux Security Summit Vancouver, Canada
May 10
May 12
Open Source Summit North America Vancouver, Canada
May 11 NLUUG Spring Conference Utrecht, The Netherlands
May 12 PGConf.BE Haasrode, Belgium
May 17 Icinga Camp Berlin Berlin, Germany
May 17
May 20
BSDCan 2023 Ottawa, Canada
May 23
May 25
Red Hat Summit 2023 Boston, US
May 23
May 30
Debian Reunion Hamburg 2023 Hamburg, Germany
May 26
May 28
openSUSE Conference 2023 Nürnberg, Germany
May 30
June 2
PGCon 2023 Ottawa, Canada
June 13
June 15
Beam Summit 2023 New York City, US
June 13
June 15
Open Infrastructure Summit Vancouver, Canada
June 14 Ceph Days Korea 2023 Seoul, South Korea
June 14
June 15
KVM Forum 2023 Brno, Czech Republic
June 15
June 17
Open Source Festival Africa Lagos, Nigeria
June 15 Ceph Days Vancounver Vancouver, Canada
June 16
June 18
Devconf.CZ 2023 Brno, Czech Republic
June 20
June 22
SUSECON München 2023 München, Germany
June 27 PostgreSQL Conference Germany Essen, Germany
June 28
June 30
Embedded Open Source Summit Prague, Czech Republic

If your event does not appear here, please tell us about it.

Security updates

Alert summary May 4, 2023 to May 10, 2023

Dist. ID Release Package Date
Debian DSA-5398-1 stable chromium 2023-05-04
Debian DLA-3416-1 LTS emacs 2023-05-10
Debian DSA-5396-2 stable evolution 2023-05-04
Debian DSA-5399-1 stable odoo 2023-05-05
Fedora FEDORA-2023-659606fa84 F38 LibRaw 2023-05-10
Fedora FEDORA-2023-12b28d0d37 F36 chromium 2023-05-10
Fedora FEDORA-2023-2c4a95caf8 F37 chromium 2023-05-10
Fedora FEDORA-2023-af03a123b5 F37 community-mysql 2023-05-10
Fedora FEDORA-2023-06a49d4fb6 F38 community-mysql 2023-05-10
Fedora FEDORA-2023-3370eab930 F38 java-11-openjdk 2023-05-05
Fedora FEDORA-2023-0ab3a5423f F37 java-11-openjdk-portable 2023-05-09
Fedora FEDORA-2023-f839113811 F37 python-sentry-sdk 2023-05-04
Fedora FEDORA-2023-597f13ffb9 F36 rubygem-redcarpet 2023-05-09
Fedora FEDORA-2023-8682a0e17d F37 rubygem-redcarpet 2023-05-09
Fedora FEDORA-2023-44daa9c1d4 F38 rubygem-redcarpet 2023-05-09
Fedora FEDORA-2023-cc21019773 F38 rust-cargo-c 2023-05-07
Fedora FEDORA-2023-cc21019773 F38 rust-coreos-installer 2023-05-07
Fedora FEDORA-2023-cc21019773 F38 rust-fedora-update-feedback 2023-05-07
Fedora FEDORA-2023-cc21019773 F38 rust-git-delta 2023-05-07
Fedora FEDORA-2023-cc21019773 F38 rust-gst-plugin-reqwest 2023-05-07
Fedora FEDORA-2023-cc21019773 F38 rust-pore 2023-05-07
Fedora FEDORA-2023-cc21019773 F38 rust-rpm-sequoia 2023-05-07
Fedora FEDORA-2023-cc21019773 F38 rust-sequoia-octopus-librnp 2023-05-07
Fedora FEDORA-2023-cc21019773 F38 rust-sequoia-policy-config 2023-05-07
Fedora FEDORA-2023-cc21019773 F38 rust-sequoia-sq 2023-05-07
Fedora FEDORA-2023-cc21019773 F38 rust-sevctl 2023-05-07
Fedora FEDORA-2023-cc21019773 F38 rust-tealdeer 2023-05-07
Fedora FEDORA-2023-cc21019773 F38 rust-ybaas 2023-05-07
Mageia MGASA-2023-0158 8 avahi 2023-05-06
Mageia MGASA-2023-0163 8 git 2023-05-06
Mageia MGASA-2023-0161 8 imagemagick 2023-05-06
Mageia MGASA-2023-0159 8 libfastjson 2023-05-06
Mageia MGASA-2023-0157 8 libxml2 2023-05-06
Mageia MGASA-2023-0162 8 parcellite 2023-05-06
Mageia MGASA-2023-0160 8 virtualbox 2023-05-06
Oracle ELSA-2023-2127 OL9 samba 2023-05-05
Red Hat RHSA-2023:2204-01 EL9 Image Builder 2023-05-09
Red Hat RHSA-2023:2589-01 EL9 autotrace 2023-05-09
Red Hat RHSA-2023:2261-01 EL9 bind 2023-05-09
Red Hat RHSA-2023:2253-01 EL9 buildah 2023-05-09
Red Hat RHSA-2023:2193-01 EL9 butane 2023-05-09
Red Hat RHSA-2023:2222-01 EL9 conmon 2023-05-09
Red Hat RHSA-2023:2367-01 EL9 containernetworking-plugins 2023-05-09
Red Hat RHSA-2023:2478-01 EL9 curl 2023-05-09
Red Hat RHSA-2023:2650-01 EL9 curl 2023-05-09
Red Hat RHSA-2023:2459-01 EL9 device-mapper-multipath 2023-05-09
Red Hat RHSA-2023:2502-01 EL9 dhcp 2023-05-09
Red Hat RHSA-2023:2165-01 EL9 edk2 2023-05-09
Red Hat RHSA-2023:2366-01 EL9 emacs 2023-05-09
Red Hat RHSA-2023:2626-01 EL9 emacs 2023-05-09
Red Hat RHSA-2023:2161-01 EL9 fence-agents 2023-05-09
Red Hat RHSA-2023:2166-01 EL9 freeradius 2023-05-09
Red Hat RHSA-2023:2326-01 EL9 freerdp 2023-05-09
Red Hat RHSA-2023:2202-01 EL9 frr 2023-05-09
Red Hat RHSA-2023:2487-01 EL9 fwupd 2023-05-09
Red Hat RHSA-2023:2216-01 EL9 gdk-pixbuf2 2023-05-09
Red Hat RHSA-2023:2319-01 EL9 git 2023-05-09
Red Hat RHSA-2023:2357-01 EL9 git-lfs 2023-05-09
Red Hat RHSA-2023:2592-01 EL9 golang-github-cpuguy83-md2man 2023-05-09
Red Hat RHSA-2023:2167-01 EL9 grafana 2023-05-09
Red Hat RHSA-2023:2177-01 EL9 grafana-pcp 2023-05-09
Red Hat RHSA-2023:2260-01 EL9 gstreamer1-plugins-good 2023-05-09
Red Hat RHSA-2023:2312-01 EL9 jackson 2023-05-09
Red Hat RHSA-2023:2458-01 EL9 kernel 2023-05-09
Red Hat RHSA-2023:2148-01 EL9 kernel-rt 2023-05-09
Red Hat RHSA-2023:2570-01 EL9 krb5 2023-05-09
Red Hat RHSA-2023:2532-01 EL9 libarchive 2023-05-09
Red Hat RHSA-2023:2179-01 EL9 libguestfs-winsupport 2023-05-09
Red Hat RHSA-2023:2122-01 EL8 libreswan 2023-05-04
Red Hat RHSA-2023:2126-01 EL8.1 libreswan 2023-05-04
Red Hat RHSA-2023:2124-01 EL8.2 libreswan 2023-05-04
Red Hat RHSA-2023:2125-01 EL8.4 libreswan 2023-05-04
Red Hat RHSA-2023:2123-01 EL8.6 libreswan 2023-05-04
Red Hat RHSA-2023:2120-01 EL9 libreswan 2023-05-04
Red Hat RHSA-2023:2633-01 EL9 libreswan 2023-05-09
Red Hat RHSA-2023:2121-01 EL9.0 libreswan 2023-05-04
Red Hat RHSA-2023:2340-01 EL9 libtiff 2023-05-09
Red Hat RHSA-2023:2453-01 EL9 libtpms 2023-05-09
Red Hat RHSA-2023:2582-01 EL9 lua 2023-05-09
Red Hat RHSA-2023:2621-01 EL9 mysql 2023-05-09
Red Hat RHSA-2023:2444-01 EL9 net-snmp 2023-05-09
Red Hat RHSA-2023:2655-01 EL9 nodejs nodejs-nodemon 2023-05-09
Red Hat RHSA-2023:2654-01 EL9 nodejs:18 2023-05-09
Red Hat RHSA-2023:2645-01 EL9 openssh 2023-05-09
Red Hat RHSA-2023:2523-01 EL9 openssl 2023-05-09
Red Hat RHSA-2023:2652-01 EL9 pcs 2023-05-09
Red Hat RHSA-2023:2417-01 EL9 php:8.1 2023-05-09
Red Hat RHSA-2023:2293-01 EL9 pki-core 2023-05-09
Red Hat RHSA-2023:2282-01 EL9 podman 2023-05-09
Red Hat RHSA-2023:2259-01 EL9 poppler 2023-05-09
Red Hat RHSA-2023:2378-01 EL9 postgresql-jdbc 2023-05-09
Red Hat RHSA-2023:2258-01 EL9 python-mako 2023-05-09
Red Hat RHSA-2023:2162-01 EL9 qemu-kvm 2023-05-09
Red Hat RHSA-2023:2136-01 EL8.6 samba 2023-05-04
Red Hat RHSA-2023:2127-01 EL9 samba 2023-05-04
Red Hat RHSA-2023:2519-01 EL9 samba 2023-05-09
Red Hat RHSA-2023:2137-01 EL9.0 samba 2023-05-04
Red Hat RHSA-2023:2283-01 EL9 skopeo 2023-05-09
Red Hat RHSA-2023:2234-01 EL9 sysstat 2023-05-09
Red Hat RHSA-2023:2257-01 EL9 tigervnc 2023-05-09
Red Hat RHSA-2023:2236-01 EL9 toolbox 2023-05-09
Red Hat RHSA-2023:2370-01 EL9 unbound 2023-05-09
Red Hat RHSA-2023:2256-01 EL9 webkit2gtk3 2023-05-09
Red Hat RHSA-2023:2653-01 EL9 webkit2gtk3 2023-05-09
Red Hat RHSA-2023:2373-01 EL9 wireshark 2023-05-09
Red Hat RHSA-2023:2248-01 EL9 xorg-x11-server 2023-05-09
Red Hat RHSA-2023:2249-01 EL9 xorg-x11-server-Xwayland 2023-05-09
Slackware SSA:2023-124-01 libssh 2023-05-04
Slackware SSA:2023-129-01 mozilla 2023-05-09
SUSE SUSE-SU-2023:1815-1 MP4.0 MP4.1 MP4.2 MP4.3 SLE15 oS15.4 amazon-ssm-agent 2023-05-04
SUSE SUSE-SU-2023:2152-1 SLE12 amazon-ssm-agent 2023-05-09
SUSE SUSE-SU-2023:1837-1 OS9 SLE12 apache2-mod_auth_openidc 2023-05-04
SUSE SUSE-SU-2023:2126-1 SLE12 cfengine, cfengine-masterfiles 2023-05-08
SUSE SUSE-SU-2023:1834-1 MP4.3 SLE15 SLE-m5.3 SLE-m5.4 oS15.4 cmark 2023-05-04
SUSE SUSE-SU-2023:2157-1 MP4.3 SLE15 SLE-m5.3 SLE-m5.4 oS15.4 osM5.3 conmon 2023-05-10
SUSE SUSE-SU-2023:1827-1 MP4.3 SLE15 SLE-m5.1 SLE-m5.2 SLE-m5.3 SLE-m5.4 oS15.4 osM5.3 containerd 2023-05-04
SUSE SUSE-SU-2023:1826-1 SLE12 containerd 2023-05-04
SUSE SUSE-SU-2023:2125-1 SLE12 containerd 2023-05-08
SUSE SUSE-SU-2023:2154-1 MP4.3 SLE15 oS15.4 distribution 2023-05-09
SUSE SUSE-SU-2023:2123-1 OS8 OS9 SLE12 dnsmasq 2023-05-08
SUSE SUSE-SU-2023:2153-1 SLE12 docker-distribution 2023-05-09
SUSE openSUSE-SU-2023:0102-1 osB15 editorconfig-core-c 2023-05-04
SUSE SUSE-SU-2023:2108-1 MP4.3 SLE15 SLE-m5.3 SLE-m5.4 SES7 SES7.1 oS15.4 ffmpeg 2023-05-05
SUSE SUSE-SU-2023:2115-1 SLE15 SES6 ffmpeg 2023-05-05
SUSE SUSE-SU-2023:2081-1 SLE15 SES7 git 2023-05-08
SUSE SUSE-SU-2023:2127-1 MP4.3 SLE15 SES7.1 oS15.4 go1.19 2023-05-08
SUSE SUSE-SU-2023:2105-1 MP4.3 SLE15 oS15.4 go1.20 2023-05-05
SUSE SUSE-SU-2023:2105-2 SLE15 go1.20 2023-05-08
SUSE SUSE-SU-2023:2158-1 SLE12 google-cloud-sap-agent 2023-05-10
SUSE SUSE-SU-2023:1822-1 MP4.2 SLE15 SLE-m5.2 SES7 SES7.1 harfbuzz 2023-05-04
SUSE SUSE-SU-2023:1821-1 OS9 SLE12 harfbuzz 2023-05-04
SUSE SUSE-SU-2023:1820-1 SLE15 harfbuzz 2023-05-04
SUSE SUSE-SU-2023:1825-1 SLE15 helm 2023-05-04
SUSE SUSE-SU-2023:2139-1 SLE-m5.4 ignition 2023-05-09
SUSE SUSE-SU-2023:1962-1 MP4.3 SLE15 SES7 SES7.1 oS15.4 indent 2023-05-08
SUSE SUSE-SU-2023:2082-1 SLE-m5.1 installation-images 2023-05-08
SUSE SUSE-SU-2023:2083-1 SLE-m5.4 installation-images 2023-05-08
SUSE SUSE-SU-2023:2109-1 SLE12 java-11-openjdk 2023-05-05
SUSE SUSE-SU-2023:2110-1 MP4.3 SLE15 oS15.4 java-17-openjdk 2023-05-08
SUSE SUSE-SU-2023:1823-1 OS9 SLE12 java-1_8_0-ibm 2023-05-04
SUSE SUSE-SU-2023:2151-1 MP4.0 SLE15 oS15.4 kernel 2023-05-09
SUSE SUSE-SU-2023:2146-1 MP4.1 SLE15 SES7 kernel 2023-05-09
SUSE SUSE-SU-2023:2148-1 MP4.2 SLE15 SLE-m5.1 SLE-m5.2 SES7.1 oS15.4 kernel 2023-05-09
SUSE SUSE-SU-2023:2140-1 MP4.3 SLE15 SLE-m5.3 SLE-m5.4 oS15.4 osM5.3 kernel 2023-05-09
SUSE SUSE-SU-2023:2141-1 MP4.3 SLE15 oS15.4 kernel 2023-05-09
SUSE SUSE-SU-2023:2156-1 SLE12 kernel 2023-05-10
SUSE SUSE-SU-2023:2147-1 SLE15 SLE-m5.1 SLE-m5.2 kernel 2023-05-09
SUSE SUSE-SU-2023:2135-1 MP4.3 SLE15 oS15.4 libfastjson 2023-05-09
SUSE SUSE-SU-2023:1824-1 OS9 SLE12 liblouis 2023-05-04
SUSE SUSE-SU-2023:1829-1 SLE15 SES7 SES7.1 oS15.4 liblouis 2023-05-04
SUSE SUSE-SU-2023:1828-1 SLE15 oS15.4 liblouis 2023-05-04
SUSE SUSE-SU-2023:2097-1 MP4.3 SLE15 SES7 SES7.1 oS15.4 maven and recommended update for antlr3, minlog, sbt, xmvn 2023-05-08
SUSE SUSE-SU-2023:2111-1 MP4.3 SLE15 SLE-m5.1 SLE-m5.2 SLE-m5.3 SLE-m5.4 oS15.4 osM5.3 ncurses 2023-05-08
SUSE SUSE-SU-2023:2112-1 SLE12 ncurses 2023-05-08
SUSE SUSE-SU-2023:2096-1 MP4.3 SLE15 SES7 SES7.1 oS15.4 netty, netty-tcnative 2023-05-08
SUSE SUSE-SU-2023:2142-1 MP4.3 SLE15 oS15.4 ntp 2023-05-09
SUSE SUSE-SU-2023:1914-1 OS9 SLE12 openssl-1_0_0 2023-05-08
SUSE SUSE-SU-2023:1814-1 MP4.3 SLE15 SLE-m5.3 SLE-m5.4 oS15.4 osM5.3 podman 2023-05-04
SUSE SUSE-SU-2023:2136-1 SLE12 prometheus-ha_cluster_exporter 2023-05-09
SUSE SUSE-SU-2023:2138-1 SLE15 prometheus-ha_cluster_exporter 2023-05-09
SUSE SUSE-SU-2023:2143-1 MP4.2 MP4.3 SLE15 SES7 SES7.1 oS15.4 protobuf-c 2023-05-09
SUSE SUSE-SU-2023:2080-1 OS9 SLE12 python-Django1 2023-05-08
SUSE SUSE-SU-2023:2144-1 OS8 SLE12 python-cryptography 2023-05-09
SUSE SUSE-SU-2023:0722-2 SLE-m5.4 python-cryptography 2023-05-09
SUSE SUSE-SU-2023:2134-1 MP4.3 SLE15 SLE-m5.3 SLE-m5.4 oS15.4 python-ujson 2023-05-09
SUSE SUSE-SU-2023:2122-1 MP4.3 SLE15 oS15.4 redis 2023-05-08
SUSE SUSE-SU-2023:2137-1 SLE12 runc 2023-05-09
SUSE SUSE-SU-2023:2084-1 MP4.2 MP4.3 SLE15 SLE-m5.1 SLE-m5.2 SLE-m5.3 SLE-m5.4 SES7.1 oS15.4 osM5.3 shim 2023-05-08
SUSE SUSE-SU-2023:2150-1 SLE12 shim 2023-05-09
SUSE SUSE-SU-2023:2086-1 SLE15 SES7 shim 2023-05-08
SUSE SUSE-SU-2023:2100-1 MP4.0 SLE15 terraform-provider-helm 2023-05-08
SUSE SUSE-SU-2023:2103-1 MP4.3 SLE15 SLE-m5.1 SLE-m5.2 SLE-m5.3 SLE-m5.4 oS15.4 osM5.3 vim 2023-05-04
SUSE SUSE-SU-2023:2074-1 SLE15 SLE-m5.1 SLE-m5.2 zstd 2023-05-08
Ubuntu USN-6063-1 18.04 20.04 22.04 22.10 ceph 2023-05-09
Ubuntu USN-6059-1 20.04 22.04 22.10 erlang 2023-05-08
Ubuntu USN-6062-1 20.04 22.04 22.10 23.04 freetype 2023-05-09
Ubuntu USN-6058-1 16.04 18.04 linux-aws, linux-aws-hwe 2023-05-05
Ubuntu USN-6057-1 22.04 linux-intel-iotg 2023-05-05
Ubuntu USN-6056-1 22.04 linux-oem-6.1 2023-05-05
Ubuntu USN-6060-1 18.04 20.04 22.04 22.10 23.04 mysql-5.7, mysql-8.0 2023-05-08
Ubuntu USN-6060-2 16.04 mysql-5.7 2023-05-08
Ubuntu USN-6065-1 16.04 18.04 20.04 node-css-what 2023-05-10
Ubuntu USN-6054-1 18.04 20.04 22.04 22.10 23.04 python-django 2023-05-03
Ubuntu USN-6055-1 16.04 18.04 20.04 ruby2.3, ruby2.5, ruby2.7 2023-05-04
Ubuntu USN-6055-2 16.04 18.04 20.04 ruby2.3, ruby2.5, ruby2.7 2023-05-05
Ubuntu USN-6061-1 20.04 22.04 22.10 23.04 webkit2gtk 2023-05-08
Full Story (comments: none)

Kernel patches of interest

Kernel releases

Linus Torvalds Linux 6.4-rc1 May 07
Sebastian Andrzej Siewior v6.3.1-rt13 May 09
Tom Zanussi 5.4.242-rt81 May 08

Architecture-specific

Core kernel

Device drivers

Jacky Huang Introduce Nuvoton ma35d1 SoC May 04
Mårten Lindahl regulator: Add support for TPS6287x May 04
Daniel Wagner Unifying fabrics drivers May 04
Kuogee Hsieh add DSC 1.2 dpu supports May 04
Sebastian Reichel mfd/pinctrl/regulator: Add RK806 Support May 04
Cristian Ciocaltea Add RK3588 OTP memory support May 04
Varadarajan Narayanan Enable IPQ9754 USB May 05
Jiawen Wu TXGBE PHYLINK support May 05
Jorge Lopez HP BIOSCFG driver May 05
Walker Chen Add TDM audio on StarFive JH7110 May 06
Jakob Hauser Add RT5033 charger device driver May 06
Dmitry Baryshkov drm/msm: add support for HDMI on msm8974 May 07
Matti Vaittinen Support ROHM BU27008 RGB sensor May 08
Paweł Anikiel Add Chameleon v3 ASoC audio May 08
Bryan O'Donoghue Add Qualcomm PMIC TPCM support May 08
Emil Tantilov Introduce Intel IDPF driver May 08
Ji-Ze Hong (Peter Hong) can: usb: f81604: add Fintek F81604 support May 09
Srinivas Kandagatla ASoC: qcom: add display port module support. May 09
Elliot Berman Drivers for Gunyah hypervisor May 09
Florian Fainelli Support for Wake-on-LAN for Broadcom May 09
Changhuang Liang Add JH7110 AON PMU support May 09
Sarath Babu Naidu Gaddam net: axienet: Introduce dmaengine May 10
Aidan MacDonald ASoC: Add ESS ES9218P codec driver May 10
Rohit Agarwal Add pinctrl support for SDX75 May 10
Sricharan Ramabadhran Add minimal boot support for IPQ5018 May 10

Device-driver infrastructure

Documentation

Filesystems and block layer

Memory management

Networking

Security-related

Virtualization and containers

Miscellaneous

Michal Kubecek ethtool 6.3 released May 08

Page editor: Jonathan Corbet


Copyright © 2023, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds