|
|
Subscribe / Log in / New account

LWN.net Weekly Edition for September 21, 2023

Welcome to the LWN.net Weekly Edition for September 21, 2023

This edition contains the following feature content:

This week's edition also includes these inner pages:

  • Brief items: Brief news items from throughout the community.
  • Announcements: Newsletters, conferences, security updates, patches, and more.

Please enjoy this week's edition, and, as always, thank you for supporting LWN.net.

Comments (none posted)

Using the limited C API for the Python stdlib?

By Jake Edge
September 20, 2023

The "limited" C API for CPython extensions has been around for well over a decade at this point, but it has not seen much uptake. It is meant to give extensions an API that will allow binaries built with it to be used for multiple versions of CPython, because those binaries will only access the stable ABI that will not change when CPython does. Victor Stinner has been working on better definition for the API; as part of that work, he suggested that some of the C extensions in the standard library start using it in an effort for CPython to "eat its own dog food". The resulting discussion showed that there is still a fair amount of confusion about this API—and the thrust of Stinner's overall plan.

The limited API comes from PEP 384 ("Defining a Stable ABI"), but that is largely a historical document at this point. The C API Stability document and developers guide both have more up-to-date information. There are several APIs available that extensions can use, but only the limited API provides ABI stability between major releases of CPython (e.g. from 3.11 to 3.12); packages using the other APIs will need to be rebuilt in order to ensure that they work with a new major (or even minor, in the case of the unstable API) release.

At the end of August, Stinner wondered about switching some of the C-based extensions in the standard library to use the limited API. The goal is to more extensively test the API and to promote it by example: "Using private C API functions and the internal C API should be the exception, not the default in Python stdlib". While the standard library itself is rebuilt and packaged with a new CPython release, other extensions will benefit from moving to the stable ABI (also known as "abi3"), which comes from using the limited API:

The stable ABI makes the distribution of package binaries easier. For example, binaries are already available before the new Python is being released! It makes newer Python usable since the first day of its release, because it's simply the same binary for all Python versions. (One binary per platform+architecture is still needed.)

It turns out that at least a few standard library modules are already using the limited API, so all that is needed in order to "convert" them is a line that declares that the module uses the limited API:

    #define Py_LIMITED_API 0x030d0000  /* value is version 3.13 */
Defining Py_LIMITED_API hides symbols (functions and other interface elements) that are not part of the limited API, so that they cannot be used in the extension code. Other modules can be converted with only minor changes to them, Stinner said. There are some standard library modules that will not be changed, yet, because their performance suffers from being unable to use the internal API. He reported a performance regression in the C extension for the statistics module as part of closing his pull request to switch it to the limited API; in the end, he decided that doing so made little sense. But, for many other modules, "there is no significant impact on performance"

Stinner tried to start converting standard library modules back in 2020, but ran into a few different problems, which have now largely been resolved. Beyond performance degradation, he also has encountered API calls that are not part of the limited API, but perhaps should be considered for inclusion. He wondered what other core developers thought about converting some standard library modules.

Barry Warsaw liked the idea as a way for the project to test out the limited API itself. He thought that doing so might also help if it was decided to move some modules out of the standard library, since a Python Package Index (PyPI) replacement could then have a binary wheel ready and waiting for new CPython releases. Alex Gaynor was also in favor:

I like the idea of eating our own dog food. I'm also the author of several of those packages that use abi3 wheels, so I have a strong [interest] in the limited API becoming better :)

I'm also sympathetic to the people who will say, "eating our own dog food isn't a good enough reason to lose performance", so I think it would be a very good outcome of this process if, wherever we identify areas for improvement by eating our own dog food, we make the dog food taste better.

But Guido van Rossum was less enthused with the idea; he thought it would lead to a lot of churn and a bunch of pull requests (PRs) "that few people care to review, and that will increase everybody's frustration (not just yours) with how hard it is to get people to review PRs". The standard library modules are not broken, so he wondered why they were being "fixed":

"Eat your own dogfood" is a fine idea, and I think it's great to apply it to new modules. Just like we sometimes add [type] annotations to new code, despite our general reluctance to add annotations to existing code (especially stdlib code). I feel the same ought to apply here: let's not try to "fix" existing modules, because they aren't broken, and ultimately there is no reason for the stdlib to use the limited API.

Stinner replied with a list of links to commits and issue discussions as background about the effort, going back to 2018, but Van Rossum was concerned that the underlying motivation was somewhat suspect:

The argument seems to be "dogfooding is good" and possibly that stdlib modules are used widely as "example code" so best practices should be followed? Those aren't technical reasons though – IMO this smells like technical solutions for social problems.

But Stinner said that there is an underlying technical reason as well: "the limited C API is badly tested by Python itself". That has led to finding bugs after a release had already been made; if some parts of the standard library were built and tested with the limited API, those bugs could be found and fixed well before a release is made. In addition, converting real extensions will help show any gaps in the API functions available in the limited API.

Van Rossum suggested moving slowly with any changes to the standard library and wanted to discuss the C API at the upcoming core developer sprint in October. He also outlined his understanding of the different APIs that are available for CPython, along with what the guarantees are for extensions that use them. It is a somewhat complicated picture that Stinner is trying to clarify as part of his work.

But the stable ABI has been around for quite some time at this point, Marc-André Lemburg said, and has seen limited adoption, so perhaps the effort should be redirected into helping extensions remain compatible with a range of Python versions. Those extensions would need to be recompiled for new CPython major versions, "but that's easily done using cibuildwheel". That tool can build and test binary wheels for multiple operating systems and Python versions as part of a project's continuous-integration (CI) process.

The tooling for building extensions has not helped with adoption of the limited API, though, Paul Moore said. Currently, the tools default to using the full C API for extensions, so that is generally what extension authors do; if that changed, adoption might grow substantially. Beyond that, Petr Viktorin pointed out that cibuildwheel only helps with extensions that are on PyPI; applications that use CPython as a way to create their own plugins and extensions want to use the stable ABI so they can work with multiple CPython versions. A Vim commit outlined the situation well, he said.

In a lengthy message, Stinner described the overall problem he is trying to help solve: having extensions be more quickly available at the time a new CPython is released. He works on Fedora, which will be shipping the newly minted CPython 3.12 (due in early October) in Fedora 39, which is slated for mid-October; the hope is to have up-to-date versions of most of the popular extensions available by that time. He sees the limited API (thus stable ABI) as being a key facilitator of that for future CPython releases. "If we can help maintainers to move towards the limited C API, you can expect having more C extensions to be usable at day 1 of Python 3.13 release." That will also help the maintainers of the extensions, since users will not be clamoring for them to update their extension as soon as a new Python is available.

In Van Rossum's mind, it is "the requirement that once 3.x.0 is released all 3rd party packages should be instantly available" that is the root cause of the problem; he suggested resetting user expectations since that is never going to be achievable. Viktorin wondered what kind of time frame would be reasonable to expect most third-party extensions. Van Rossum replied that it has generally taken a few months after the release to get to that point, but thought that package maintainers should be encouraged to start putting together wheels for their modules once the first release candidate of a new CPython is released. He is also "unhappy about the pressure I am currently feeling to make it our fault if not every 3rd party package works on day one".

As might be guessed, Stinner disagreed with much of that. He has no silver bullet, but getting more packages to use the limited API will lead to more of them being available on day one. Meanwhile, maintainers should not be subjected to additional pressure to update their builds; they "prefer to work on new features, or fix their own bugs, rather than following Python C API changes". Van Rossum tired of the discussion, however, and wanted to wait until they could talk about the issue face-to-face in October.

There were other sub-topics in the thread, of course, but the question of what to do for the standard library, if anything, will presumably be the subject of a lively discussion at the sprint. Van Rossum seems unconvinced that the stable ABI has much to offer ("I still feel that the Stable ABI is a solution largely in search of a problem"), but other core developers (and extension authors) disagree. In the end, it seems unlikely that there will be any movement away from supporting the limited API, though the effort to broaden its reach—in CPython itself at least—is still up in the air.

Comments (5 posted)

The European Cyber Resilience Act

September 19, 2023

This article was contributed by Marta Rybczyńska

The security of digital products has become a topic of regulation in recent years. Currently, the European Union is moving forward with another new law, which, if it comes into effect in a form close to the current draft, will affect software developers worldwide. This new proposal, called the "Cyber Resilience Act" (CRA), brings mandatory security requirements on all digital products, both software and hardware, that are available in Europe. While it aims at a worthy goal, the proposal is causing a stir among open-source communities.

There is a reason why the open-source world has concerns: the legislation indirectly defines who is responsible for the security of open source and who should pay to improve the current state. In addition, it puts the responsibility on individual developers and foundations hosting open-source projects instead of the manufacturers of goods embedding the software. It could have important consequences for open source across the globe.

The original proposal of the CRA (the main text and the annexes) brings several requirements (enumerated in Annex I) for "products on the market" (more on definitions a little later). Most requirements are generally accepted best-practices, such as a secure default configuration, providing security updates, protection from unauthorized access, and releases free of known vulnerabilities; others could be considered somewhat vague like "(g) minimise their own negative impact on the availability of services provided by other devices or networks".

Each product (which means every release for software) would need to provide "an assessment of the cybersecurity risks" along with the release-related documentation (listed in Annex II). The security assessment covers the software itself and all its dependencies; the way to perform the assessment could fall under one of three categories, from self-assessment to the involvement of a third party. Self-assessment is the default. However, the regulation requires a stricter approach when the product's "core function" falls into the category of a "critical product with digital elements" (listed in Annex III), which is further divided into Class I (less critical) and Class II (more critical). Depending on their class, products must undergo a mandatory external security assessment; all Class II products are required to do so, while Class I products must only if they do not follow the (not yet defined) "harmonised standards, common specifications or cybersecurity certification schemes".

The release-related documentation needs to cover (among other items) the product's expected uses, its support time frame, and the way to install security updates. All manufacturers must have a vulnerability-reporting procedure and release security updates free of charge for users. If a manufacturer learns about an "actively exploited vulnerability", it is expected to notify authorities rapidly (24 hours in many cases; followed by a complete analysis one month later). Finally, a party not complying with the requirements may be subject to fines of up to €15-million or up to 2.5 percent of the worldwide annual turnover (gross revenue), whichever is higher.

Commercial activity

As the devil is in the details, definitions have an important impact on understanding the CRA. The most important term is, without a doubt, "commercial activity" as found in the following sentence:

In order not to hamper innovation or research, free and open-source software developed or supplied outside the course of a commercial activity should not be covered by this Regulation.

The definition for "commercial activity" comes from a document called "The Blue Guide", which has examples on page 21 that provide a little more explanation. The text states that commercial activity means providing goods in "a business related context" and the decision if a particular activity is commercial or not should be done:

on a case by case basis taking into account the regularity of the supplies, the characteristics of the product, the intentions of the supplier, etc. In principle, occasional supplies by charities or hobbyists should not be considered as taking place in a business related context.

Without further clarification, one might consider many open-source projects as commercial activities, especially mature ones that are used widely and make regular releases. Those with developers hired to work on the project might also qualify.

The definition of commercial activity is the main point affecting open-source projects. It will not affect hobby projects run by unpaid volunteers only. It could (depending on the any future judgments) affect all others. If that wording is not changed in the final version of the CRA, it could cause uncertainty, leading to lesser involvement in open source.

Who is the manufacturer?

The discussion of commercial activity brings us to the other primary term of the CRA: the "manufacturer". Again, according to the "Blue Guide" (page 34): "The manufacturer is any natural or legal person who manufactures a product or has a product designed or manufactured, and places it on the market under his own name or trademark". The CRA also defines other roles like the distributor or importer, but "manufacturer" will be the most important one in our case. The manufacturer is critical because they are responsible for fulfilling requirements and could face the fines mentioned above.

Let us take the example of a project hosted by an open-source foundation. That foundation could be considered the manufacturer, even if it might have limited impact on the development process and concentrates on governance and organizational aspects. The definition of the manufacturer might be even more complicated when there is no formal legal entity. Is the person tagging a release the manufacturer in this case? The person with the most commits? All the maintainers together? Their employers?

If foundations or other supporting organizations are to be classified as manufacturers, they will be required to put additional constraints on projects and how they develop and release. That could cause significant tension, especially for projects with no established security culture.

Another point is the need for a budget in case an external assessment is required; currently, operating systems and networking software both require an external assessment. A typical budget for a security assessment is in tens of thousands of dollars (or euros). The exact scope of the audit for CRA requirements will be defined further (the general description appears in Annex VI), but we might assume a need for a similar budget for every non-bugfix release. It will be vital for many projects to figure out who will pay for assessments. Many of them, including virtually all without an organization backing them, will not be able to pay that fee.

Even if a project falls under self-assessment only, its release must be accompanied by several documents, including a risk analysis. Writing and verifying this information would mean additional work for projects, though the exact content of this documentation still needs to be fully defined.

Vulnerability and exploitation reporting

Under the CRA, each manufacturer needs to have a vulnerability-reporting process and, in most cases, provide security updates. Logically, they would also have to provide updates of all dependencies, especially when they are linked into the main product. A recent discussion in the Linux kernel community showed that timely delivery of security fixes is not yet a solved problem; regulation like the CRA might actually push device vendors to publish fixes more rapidly.

One clause also requests all manufacturers to report each "actively exploited vulnerability" whenever they learn about it. The report must happen within strict time limits, such as a first notification in 24 hours (though it is not required for small companies). The company should provide a detailed analysis with a fix within one month. These reports are sent to the European Union Agency for Cybersecurity (ENISA). With that pile of 0-days, ENISA will become a juicy target for attacks (though one might argue that a service like GitHub's private advisories is already such a target).

The obligation to notify about all issues also breaks normal disclosure processes. These days, vendors disclose vulnerabilities only after a fix is available. Also, the one-month limit for a complete analysis might sometimes be hard to meet. The industry typically uses 90 days, but some vulnerabilities (notable examples include hardware issues like speculative execution bugs) take months from the discovery to the fix.

The reaction

After the original proposal was published in September 2022, the open-source community started rapidly responding to it. One of the first reactions came from a foundation, NLnet Labs, in a blog post describing the impact on its domain: network and routing protocols. Many of the projects the foundation works on fall into the "critical products" category and would require significant additional work, possibly including an external security audit. Also, they note that some of these tools, which may have been available for dozens of years, could be considered "commercial" so they would fall under the regulation even though the organization gets no income from that software.

Other organizations have done their own analysis as well. A blog post from Mike Milinkovich of the Eclipse Foundation lists all of the documentation and assessment that the Foundation would have to do for each released software version; he also mentions the uncertainty of which tools would be classified as critical or highly critical. During FOSDEM 2023, a panel (WebM video) took place where European Commission representatives answered questions from the community. This session mostly concentrated on the impact of the definition of "commercial activity" that is a condition for the product to fall under the scope of the CRA. It was said that even for charities, it is going to be case-by-case determination. Also, a Commission representative said that the goal of the regulation is to force manufacturers to do more due diligence on the components that they include; they will be obligated to do so as part of their security assessment.

In April 2023, multiple organizations released an open letter asking the Commission to "engage with the open source community and take our concerns into account". In addition, some specialized communities such as Content Management Systems (CMSes) wrote their own open letter, which was signed by representatives of WordPress, Joomla, Drupal, and TYPO3. Interested readers may also want to look at the list of reactions that is maintained by the Open Source Initiative (OSI).

Possible response

If the regulation is put in place in its current form, some projects may not want to risk being classified as "commercial activity" and may decide to state that their code will not be available to the EU (enforced by technical means or not). That also raises interesting licensing questions; for example, it is not clear that GPL-covered code can have such a restriction. When code is restricted in that way, the security assessment for any downstream projects using that project in the EU will become more complicated — it could even mean that each downstream project needs to perform that audit independently. Or remove the dependency.

Some projects might decide to stop accepting contributions from developers employed by companies, or not take donations from companies, in order to not be classified as commercial. That could seriously impact those projects, reducing both their funding and the development base. But, even if such a project falls under the non-commercial category, downstream users might be using it otherwise.

Finally, there may be an impact on the number of new open-source projects. Convincing a (big) company to open-source new work is a daunting task; if there is more burden related to liability and documentation, fewer companies will release projects that are not crucial to their goals. This could affect tools like those for testing, continuous integration, for programming embedded devices, and so on. An increase in the number of forks is also likely; companies may want a version of some project's code with changes for CRA compliance. That could in fact decrease overall security, instead of improving it.

The current state

In addition to the initial version, as of August 2023 there are currently two sets of amendments, one from the EU Council, and another from the EU Parliament, that resulted from the work of Parliament committees. The most recent vote in the EU Parliament committees took place in July 2023. The next step is negotiations (called a "trilogue") between the Council, Commission, and Parliament to come up with a final version.

The two set of amendments change certain details, but not the general thrust of the regulation. Both of them change lists of the "critical product with digital elements" (those that might require an external audit). The amendments from the Council shorten both lists, while ones from the Parliament move all routers to Class II and add new categories to Class I (home automation, smart toys, etc.).

They also both modify the "open-source exception". The Parliament version seems to move requirements to companies and gives a set of examples of commercial and non-commercial activities. An indication of a non-commercial activity is a fully distributed model and lack of control by one company. On the other hand, if "the main contributors to free and open-source projects are developers employed by commercial entities and when such developers or the employer can exercise control as to which modifications are accepted in the code base", that is an indication of commercial activity, the same as regular donations from companies. It also states that individual developers "should not be subject to obligations pursuant to this Regulation". The Council's version seems to cover more products by the exception:

this Regulation should only apply to free and open-source software that is supplied in the course of a commercial activity. Products provided as part of the delivery of a service for which a fee is charged solely to recover the actual costs directly related to the operation of that service [...] should not be considered on those grounds alone a commercial activity.

The start of the negotiation on the final version is likely to happen after the summer break (which means in September 2023). Note that European Elections will happen in early June 2024 which means that the process is likely to be rushed to completion before that date.

The outcome?

Improving the security state of the digital world is a worthy goal, and many ideas the CRA brings are reasonable best practices. However, the impact of the current form of the regulation is difficult to predict. In the open-source world, it could be putting all the burden on upstream projects. These projects are frequently underfunded, so they might not have the resources to perform all the required work; that analysis and documentation work is worth doing, but funding has to be available in order to make it happen. FOSS developers, especially those working in the embedded space, should be paying attention to this legislation, as there is more to it than our summary above covers. Readers in the EU may want to contact their representatives about the CRA, as well.

Comments (109 posted)

Shrinking shrinker locking overhead

By Jonathan Corbet
September 15, 2023
Much of the kernel's performance is dependent on caching — keeping useful information around for future use to avoid the cost of looking it up again. The kernel aggressively caches pages of file data, directory entries, inodes, slab objects, and much more. Without active measures, though, caches will tend to grow without bounds, leading to memory exhaustion. The kernel's "shrinker" mechanism exists to be that active measure, but shrinkers have some performance difficulties of their own. This patch series from Qi Zheng seeks to address one of the worst of those by removing some locking overhead.

Kernel subsystems that maintain caches should register a shrinker that can be called when the kernel needs to free memory for other uses. A shrinker is described by struct shrinker; among other things, it contains a pair of callbacks that the kernel can use to query how many cached objects could be freed, and to ask that they actually be freed. Shrinkers can be asked to focus on a specific NUMA node or memory control group, but not all shrinkers implement that functionality. Since shrinkers are called from the reclaim path when memory is tight, they should be quick and refrain from allocating memory themselves.

Shrinkers can be registered and deleted as the system runs, creating a concurrency problem: a shrinker should not be deleted while it is running, and the list of shrinkers must be changed carefully given that other CPUs may be traversing it at the same time. In current kernels, the shrinker list is protected by a reader/writer semaphore (rwsem); traversing the list to run shrinkers requires read access, while changing the list requires exclusive write access. This was meant to be a fast solution; frequent traversals of the list (reads) can run concurrently, while changes to the list that would require write access are relatively rare.

This rwsem, it turns out, can be a performance bottleneck on busy systems. It is a global lock, so frequent acquisitions and releases can create a lot of cache-line bouncing, slowing the system even if the lock itself is not contended. Things can get worse if a shrinker runs (or is blocked) for a long time. If a writer comes along, it will request a write lock, which will have to wait until all existing read locks are dropped; meanwhile, the write-lock request blocks any additional read locks from being granted. In this situation, a long-running shrinker can clog up the works for some time.

Performance problems of this type come up often in the kernel, and the path to their solution is reasonably well-worn at this point; it almost inevitably involves using read-copy-update (RCU) to defer changes to existing structures until all users are gone.

In this case, the patch series starts by changing the shrinker registration interface so that all shrinkers are allocated dynamically — even those that are present from boot and cannot be removed. This change allows all shrinkers to be treated uniformly, getting rid of special cases, and sets the stage for changing how shrinker registration is handled. As seen in this patch, a new shrinker instance is created with shrinker_alloc(), made active with shrinker_register(), and released with shrinker_free().

There are a couple of implications here. One, as noted in the cover letter, is that this change will break all out-of-tree modules that implement shrinkers; they will have to be converted to the new API or they will fail to load. This is a deliberate change to ensure that, in kernels implementing the new mechanism, no old-style shrinkers are in use. A more quiet change is that, while the existing register_shrinker() interface is exported to all modules, the new functions are exported as GPL-only. As a result, proprietary kernel modules that implement shrinkers will not be fixable at all.

The bulk of this 45-part patch series is focused on converting all in-kernel shrinkers to the new API, after which the old one is deleted. The real purpose of the patch set is only achieved in patch 42, where the lockless algorithm is introduced. The shrinker structure gains three new fields: a reference count, a completion to be used for removals, and an rcu_head structure.

When a shrinker is registered, its reference count is set to one, and it is added (in an RCU-safe manner) to the shrinker list; it is then available to be called when the memory-management subsystem needs to find some memory. The traversals of the shrinker list are performed with the RCU lock held, meaning that the entries in the list will not disappear at an inconvenient time. To invoke a shrinker, the kernel will first attempt to increment its reference count; that attempt will only succeed if the count is already greater than zero. The RCU lock will then be dropped, and the shrinker invoked. Once its work is done, the RCU lock will be reacquired, and the reference count decremented. Since the reclaim code held a reference, the shrinker will not have disappeared while the lock was dropped.

When the time comes to remove a shrinker, shrinker_free() will drop the reference acquired at registration time, then use the completion to wait until all other references (if any) are also dropped. At this point, the fact that the reference count is zero means that shrinker will not acquire any more users, since an attempt to increment the reference count only succeeds if that count is greater than zero. But there may still be threads traversing the shrinker list and seeing this shrinker's entry there, so its removal has to be handled with care. That, of course, is what RCU is for; the entry is taken off the list, but then handed to RCU until a grace period passes, after which it is known that the shrinker structure can be safely freed.

With these changes made, the shrinker rwsem is no longer used during the invocation of shrinkers; it is only taken for write access when changes are being made to the shrinkers themselves. The final patch in the series turns the rwsem into a lower-overhead mutex, and the work is done.

This series is in its sixth revision, and the stream of comments appears to be slowing down. Benchmark results show no regressions from this change, unlike previous attempts to address the locking bottleneck that created problems elsewhere. Unless new problems turn up somewhere — always a possibility with this kind of low-level code — it looks like lockless shrinking may be reaching a point where it is ready for wider testing in linux-next.

Comments (18 posted)

Moving physical pages from user space

By Jonathan Corbet
September 18, 2023
Processes in a Linux system run within their own virtual address spaces. Their virtual addresses map to physical pages provided by the hardware, but the kernel takes pains to hide the physical addresses of those pages; processes normally have no way of knowing (and no need to know) where their memory is located in physical memory. As a result, the system calls for memory management also deal in virtual addresses. Gregory Price is currently trying to create an exception to this rule with a proposal for a new system call that would operate on memory using physical addresses.

When physical placement matters

Most of the time, user space is entirely happy to let the kernel worry about where memory should be mapped; all physical pages are alike, so it really does not matter which ones are used by a given process. That situation can change, though, in situations where all physical pages are not alike. Non-uniform memory-access (NUMA) machines are a case in point; these machines are split into multiple nodes, each of which normally contains one or more CPUs and some physical memory. For a process executing on a given node, memory attached to that same node will be faster than memory on other nodes, so the placement of memory matters.

Kernel developers have been working on the NUMA problem for years, and have developed a number of mechanisms to try to keep processes and their memory together. System calls can be used to bind processes to specific nodes, to ask that memory be allocated on specific nodes, and to move pages of memory from one node to another when needed. There is always room for improvement, but NUMA systems work well most of the time.

Hardware engineers are creative folks, though, and they have been busily working on other ways to create different types of memory. Contemporary systems still have traditional RAM, but memory might also be located on a peripheral device, in a non-volatile RAM array, in a bank of high-bandwidth memory, or in an external CXL device. In each of those cases, the memory involved will have different performance characteristics than ordinary RAM, once again making the physical placement of a process's pages into an important concern.

Since the NUMA concept already exists and is able to represent different classes of memory, it has been extended to handle these newer memory types as well. Each bank of "different" memory is normally organized into its own CPU-less NUMA node. The existing system calls controlling memory allocation can then be used to locate pages within these special nodes. That solves the management problem at a low level, but it is really only the beginning.

In many cases, the desired result for systems with multiple memory types is some form of memory tiering, where pages are migrated between memory types depending on how heavily they are used. Ideally, heavily used pages should be located in the system's fastest memory, while rarely used pages can be put out to pasture in slower memory. Finding an optimal way to move pages between memory tiers is an area of active development, and a number of questions remain open.

Tiering in user space

In this context, Price is seeking to add a new system call to allow some of those migration decisions to be made in user space. There are some existing interfaces that can be used to determine which physical pages are (or are not) in active use; devices providing memory can also, sometimes, provide this information. Using this data, a user-space management process could decide to move pages into the type of memory that is best suited to their current usage profile.

There is a problem, though. That information, being tied to physical pages, lacks any connection to the processes using those pages. A user-space program wanting to force page migrations based on this information would first have to convert the physical page addresses into (process, virtual-address) pairs for use with the existing system calls. That is a non-trivial and expensive task. Price is looking for a way to move pages between memory types without the need for an awareness of which processes are using those pages.

The result is a new system call, move_phy_pages(), that is patterned after the existing move_pages() call (which uses virtual addresses); it is otherwise completely undocumented at this point. The interface appears to be:

    int move_phy_pages(unsigned long count, void **pages, int *nodes,
    		       int *status, int flags);

This call will attempt to move count pages, the physical addresses of which are stored in the pages array; each page will be moved to the NUMA node indicated by the appropriate entry in nodes. The status array will be filled in with information about what happened to each page; on success, the status entry will contain the page's new node number. The only relevant flags value appears to be MPOL_MF_MOVE_ALL, which instructs the call to move pages that are mapped by multiple processes; otherwise only singly mapped pages are moved.

If the nodes array is NULL, the system call will, instead, just store the status of each of the indicated pages in status. There are limits to how useful that is, since the node number of physical pages is already described by their physical address and does not normally change over time.

In reviewing the patch, Arnd Bergmann questioned the use of the void * type for the pages array. The values provided there are not actually pointers that can be dereferenced in any context; instead, they are used by the kernel to obtain the page-frame numbers (PFNs) for the pages of interest. Since, in some 32-bit configurations, full physical addresses may not fit within a normal pointer type, Bergmann suggested using the __u64 type instead.

That conversation also raised the question of whether, instead, user space should be providing PFNs to move_phy_pages(). As Bergmann pointed out, there are no system calls that accept PFNs now, so that would be breaking new ground. That, though, reflects the fact that, until now, system calls do not normally deal with physical addresses at all. If this work goes forward, finding a consensus on the best way to refer to such addresses, for move_phy_pages() and anything that might follow — will be important.

Whether this work will actually move forward remains to be seen. It is, almost by definition, an interface to move pages around without knowing which processes are using them; otherwise, move_pages() could be used instead. Perhaps the information regarding physical memory and its utilization that is available to user space (Price provided a list of information sources in this message) is sufficient to make useful decisions, but that would probably need to be demonstrated somehow. This patch provides access to functionality that is normally kept deeply within the memory-management subsystem; developers will want to see that the benefits it provides justify that intrusion.

Comments (13 posted)

Why glibc's fstat() is slow

By Jonathan Corbet
September 14, 2023
The fstat() system call retrieves some of the metadata — owner, size, protections, timestamps, and so on — associated with an open file descriptor. One might not think of it as a performance-critical system call, but there are workloads that make a lot of fstat() calls; it is not something that should be slowed unnecessarily. As it turns out, though, the GNU C Library (glibc) has been doing exactly that, but a fix is in the works.

Mateusz Guzik has been working on a number of x86-related performance issues recently. As part of that work, he stumbled into the realization that glibc's implementation of fstat() is expressed in terms of fstatat(). Specifically, a call like:

    result = fstat(fd, &stat_buf);

is turned into:

    result = fstatat(fd, "", &stat_buf, AT_EMPTY_PATH);

These calls are semantically equivalent; by the POSIX definition, a call to fstatat() providing an empty string for the path and the AT_EMPTY_PATH flag operates directly on the provided file descriptor. But the difference in the kernel is significant; implementing fstat() in this way is significantly slower, for a couple of reasons.

One of those is that fstatat() is a more complex system call, so it does preparatory work that is not useful for the simple fstat() case. Once alerted to the problem, Linus Torvalds posted a patch that detects this case and avoids that extra work. But the result is still, according to Guzik's measurements, about 12% slower than calling fstat() directly.

That performance loss is the result of the second problem: fstatat() must check the provided path and ensure that it is empty. One might think that it makes no sense to even look at the path when the user has provided a flag (AT_EMPTY_PATH) that says there is nothing to be seen there but, as Al Viro pointed out, POSIX mandates this behavior. Checking the path means accessing user-space data from the kernel; that, in turn, can require disabling guardrails like supervisor mode access prevention. It all adds up to a significant amount of overhead to check an empty string.

Torvalds made it clear that he thought glibc's behavior made no sense and wondered why things were done that way. A bit later, though, he found a plausible explanation for this choice. On an x86-64 system, the kernel exports a number of related system calls, including fstat() (number 5) and newfstatat() (number 262). Torvalds concluded:

The glibc people found a "__NR_newfstatat", and thought it was a newer version of 'stat', and didn't find any new versions of the basic stat/fstat/lstat functions at all. So they thought that everything else was implemented using that 'newfstatat()' entrypoint.

But on x86-64 (and most half-way newer architectures), the regular __NR_stat *is* that "new" stat.

The "new" fstat(), after all, came about in the 0.97 release in 1992, so there was no reason for the x86-64 architecture (which arrived rather later than that) to use anything else. But, if Torvalds's explanation reflects reality, the glibc developers were fooled by the "new" part of the newfstatat name and passed over the entry point they should have used to implement fstat().

There are a few observations that one could make from this little bit of confusion:

  • The system calls (and their names) provided at the kernel boundary are not the same as those expected by user-space programmers. The glibc people know this better than anybody else, since part of their job is to provide the glue between those two interfaces, but confusion still seems to happen.
  • The fact that the kernel's documentation of the interface it presents to user space is ... mostly nonexistent ... certainly does not help prevent confusion of this type.
  • Using qualifiers like "new" in the naming of functions, products, or one's offspring tends to be unwise; what is new today is old tomorrow.

Be that as it may, even with Torvalds's change (which was merged for the 6.6-rc1 release and will presumably show up in a near-term stable update), fstat() is slower than it needs to be when glibc is being used. In an attempt to improve the situation, Guzik raised the issue on the libc-alpha list. Adhemerval Zanella Netto responded that the library developers are trying to simplify their code by using the more generic system calls whenever possible, that the AT_EMPTY_PATH problem is likely to affect all of the *at() system calls, and that, as a consequence, the problem would be "better fixed in the kernel".

Torvalds pointed out that, while other system calls have to handle AT_EMPTY_PATH, fstatat() is the only one that is likely to matter from a performance perspective; none of the others should be expected to show up as problems in real-world programs. Meanwhile, despite the misgivings expressed previously, Zanella put together a patch causing glibc to use ordinary fstat() when appropriate. Torvalds agreed that it looked correct, but complained that the implementation was messy; he seemed to prefer an alternative implementation that Zanella posted later.

As of this writing, neither version of the patch has found its way into the glibc repository; the latter version is under consideration. It is probably safe to assume that a version of this patch will be applied at some point; nobody has an interest in glibc being slower than it needs to be. This particular story has a happy ending, but it does stand as an example of what can happen in the absence of clarity around the interfaces between software components.

Comments (24 posted)

Page editor: Jonathan Corbet

Brief items

Kernel development

Kernel release status

The current development kernel is 6.6-rc2, released on September 17. Linus said:

I think the most notable thing about 6.6-rc2 is simply that it's exactly 32 years to the day since the 0.01 release. And that's a round number if you are a computer person.

Because other than the random date, I don't see anything that really stands out here.

Stable updates: 6.5.4, 6.1.54, 5.15.132, and 5.10.195 were released on September 19.

Comments (none posted)

Quote of the week

As you may have gathered from the MAINTAINERS patch I just sent out, I have resigned from my position at Red Hat, and will be stepping back from nouveau development.

This is a personal decision that I've been mulling over for a number of years now, and I feel that with GSP-RM greatly simplifying support of future HW, and the community being built around NVK, that things are in good hands and this is the right time for me to take some time away to explore other avenues.

Ben Skeggs

Comments (1 posted)

Distributions

The Debian Project mourns the loss of Abraham Raji

The Debian project is mourning Abraham Raji, who died in an accident on September 13.
Abraham was a popular and respected Debian Developer as well a prominent free software champion in his home state of Kerala, India. He was a talented graphic designer and led design and branding work for DebConf23 and several other local events in recent years. Abraham gave his time selflessly when mentoring new contributors to the Debian project, and he was instrumental in creating and maintaining the Debian India website.

The Debian Project honors his good work and strong dedication to Debian and Free Software. Abraham’s contributions will not be forgotten, and the high standards of his work will continue to serve as an inspiration to others.

Comments (1 posted)

Development

JDK 21 released

JDK 21, the reference implementation of the Java 21 language specification, has been released. "This release includes fifteen JEPs [1], including the final versions of Record Patterns (440), Pattern Matching for switch (441), and Virtual Threads (444)".

Comments (25 posted)

PostgreSQL 16 released

Version 16 of the PostgreSQL database manager has been released.

PostgreSQL 16 contains many new features and enhancements, including:
  • Allow parallelization of FULL and internal right OUTER hash joins
  • Allow logical replication from standby servers
  • Allow logical replication subscribers to apply large transactions in parallel
  • Allow monitoring of I/O statistics using the new pg_stat_io view
  • Add SQL/JSON constructors and identity functions
  • Improve performance of vacuum freezing
  • Add support for regular expression matching of user and database names in pg_hba.conf, and user names in pg_ident.conf

Comments (6 posted)

Miscellaneous

Videos from FOSSY released (Software Freedom Conservancy)

The Software Freedom Conservancy (SFC) has announced the availability of videos from the first-ever Free and Open Source Yearly (FOSSY) conference, which was held in July in Portland, Oregon in the US.
During the four days of the conference, there were a wide variety of talks from speakers with a range of experience and backgrounds, and amazing community focused discussions. Featuring wide ranging topics such as a panel discussion about software coops, what is life like without a smartphone (where the picture on the right is from), and thinking about FOSS from a systems theory perspective. Our track organizers brought together communities from all over, and led by example choosing speakers, topics and setting up panels for important conversations. There is definitely a talk that will interest you, whether you are interested in nonprofit board structure, an introduction to Reproducible Builds or maybe you are looking to have more nature adventures with free software.

Comments (1 posted)

Forty years of GNU

The Free Software Foundation looks forward to the 40th anniversary of the GNU project, coming soon:

On September 27, 1983, a computer scientist named Richard Stallman announced the plan to develop a free software Unix-like operating system called GNU, for "GNU's not Unix." GNU is the only operating system developed specifically for the sake of users' freedom, and has remained true to its founding ideals for forty years.

Comments (74 posted)

Page editor: Jake Edge

Announcements

Newsletters

Distributions and system administration

Development

Emacs News September 18
This Week in GNOME September 15
This Week in KDE September 16
OCaml Weekly News September 19
Perl Weekly September 18
Weekly Rakudo News September 18
Ruby Weekly News September 14
This Week in Rust September 13
Wikimedia Tech News September 18

Calls for Presentations

CFP Deadlines: September 21, 2023 to November 20, 2023

The following listing of CFP deadlines is taken from the LWN.net CFP Calendar.

DeadlineEvent Dates EventLocation
September 21 February 27
February 29
22nd USENIX Conference on File and Storage Technologies Santa Clara, CA, US
September 28 November 28 NLUUG Fall Conference Utrecht, The Netherlands
October 2 October 6
October 8
Qubes OS Summit Berlin, Germany
October 10 October 28 China Linux Kernel conference 2023 Shenzhen, China
November 1 November 11 Clang-built Linux Meetup Richmond, VA, US
November 1 March 14
March 17
SCALE 21x Pasadena, CA, US

If the CFP deadline for your event does not appear here, please tell us about it.

Upcoming Events

Events: September 21, 2023 to November 20, 2023

The following event listing is taken from the LWN.net Calendar.

Date(s)EventLocation
September 19
September 21
Open Source Summit, Europe Bilbao, Spain
September 20
September 21
Linux Security Summit Europe Bilbao, Spain
September 21
September 23
LibreOffice Conference Bucharest, Romania
September 22
September 24
GNU Tools Cauldron Cambridge, UK
September 22
September 24
Jesień Linuksowa 2023 Gliwice, Poland
September 22
September 24
VideoLAN Dev Days 2023 Dublin, Ireland
September 25
September 26
GStreamer Conference 2023 A Coruña, Spain
September 25
September 27
Kernel Recipes 2023 Paris, France
September 26
September 29
Alpine Linux Persistence and Storage Summit Lizumerhuette, Austria
September 27 GNU Project 40th Anniversary celebration Biel, Switzerland
October 1 Hackday celebrating forty years of GNU at the Free Software Foundation Boston, US
October 3
October 5
PGConf NYC New York, US
October 5
October 6
PyConZA Durban, South Africa
October 6
October 8
Qubes OS Summit Berlin, Germany
October 6
October 8
LibreOffice Conf Asia x UbuCon Asia 2023 Surakarta, Indonesia
October 7
October 8
LibreOffice - Ubuntu Conference Asia 2023 Surakarta, Indonesia
October 15
October 17
All Things Open 2023 Raleigh, NC, US
October 17 Icinga Camp Milan Milan, Italy
October 17
October 19
X.Org Developers Conference 2023 A Coruña, Spain
October 20
October 22
Linux Fest Northwest 2023 Bellingham, WA, US
October 21
October 23
openSUSE.Asia Summit 2023 Chongqing, China
October 24
October 26
Linux Foundation Member Summit Monterey, CA, USA
October 28 China Linux Kernel conference 2023 Shenzhen, China
October 30
November 3
Netdev 0x17 Vancouver, Canada
November 3
November 4
Seattle GNU/Linux Conference 2023 Seattle, US
November 3
November 5
Ubuntu Summit Riga, Latvia
November 4
November 5
OpenFest 2023 Sofia, Bulgaria
November 7
November 9
Open Source Monitoring Conference Berlin, Germany
November 9
November 10
Fourth LibreOffice Latin America Congress Mexico City, Mexico
November 11 Clang-built Linux Meetup Richmond, VA, US
November 13
November 15
Linux Plumbers Conference Richmond, VA, US

If your event does not appear here, please tell us about it.

Security updates

Alert summary September 14, 2023 to September 20, 2023

Dist. ID Release Package Date
Debian DLA-3567-1 LTS c-ares 2023-09-15
Debian DSA-5499-1 stable chromium 2023-09-18
Debian DLA-3568-1 LTS firefox-esr 2023-09-16
Debian DSA-5496-1 stable firefox-esr 2023-09-13
Debian DSA-5500-1 stable flac 2023-09-18
Debian DLA-3573-1 LTS frr 2023-09-19
Debian DSA-5501-1 stable gnome-shell 2023-09-18
Debian DLA-3570-1 LTS libwebp 2023-09-18
Debian DSA-5497-1 stable libwebp 2023-09-13
Debian DSA-5497-2 stable libwebp 2023-09-17
Debian DLA-3572-1 LTS libyang 2023-09-19
Debian DLA-3571-1 LTS openjdk-11 2023-09-19
Debian DLA-3565-1 LTS ruby-loofah 2023-09-13
Debian DLA-3566-1 LTS ruby-rails-html-sanitizer 2023-09-13
Debian DLA-3563-1 LTS samba 2023-09-14
Debian DLA-3569-1 LTS thunderbird 2023-09-17
Debian DSA-5498-1 stable thunderbird 2023-09-15
Debian DSA-5502-1 stable xrdp 2023-09-18
Fedora FEDORA-2023-34411d8f77 F37 borgbackup 2023-09-15
Fedora FEDORA-2023-555f9fac30 F38 borgbackup 2023-09-15
Fedora FEDORA-2023-3bfb63f6d2 F38 chromium 2023-09-18
Fedora FEDORA-2023-b1253907f1 F38 curl 2023-09-16
Fedora FEDORA-2023-31fe7ee034 F37 firefox 2023-09-15
Fedora FEDORA-2023-c7af372e2e F38 firefox 2023-09-15
Fedora FEDORA-2023-bf8423a373 F37 flac 2023-09-16
Fedora FEDORA-2023-1b5f6f4eb2 F38 giflib 2023-09-18
Fedora FEDORA-2023-cf176d02d8 F39 golang-github-prometheus-exporter-toolkit 2023-09-20
Fedora FEDORA-2023-cf176d02d8 F39 golang-github-xhit-str2duration 2023-09-20
Fedora FEDORA-2023-cf176d02d8 F39 golang-gopkg-alecthomas-kingpin-2 2023-09-20
Fedora FEDORA-2023-f5a6136ac8 F37 libpano13 2023-09-20
Fedora FEDORA-2023-90ed807e04 F38 libpano13 2023-09-20
Fedora FEDORA-2023-f5680e3b4b F37 libtommath 2023-09-18
Fedora FEDORA-2023-3388038193 F37 libwebp 2023-09-16
Fedora FEDORA-2023-c4fa8a204d F38 libwebp 2023-09-15
Fedora FEDORA-2023-c0696d7b53 F37 matrix-synapse 2023-09-18
Fedora FEDORA-2023-9b1a1023ac F37 open-vm-tools 2023-09-20
Fedora FEDORA-2023-df375d0634 F38 open-vm-tools 2023-09-14
Fedora FEDORA-2023-c0696d7b53 F37 python-matrix-common 2023-09-18
Fedora FEDORA-2023-0e9e7544df F37 redis 2023-09-16
Fedora FEDORA-2023-03422cb8de F38 redis 2023-09-16
Fedora FEDORA-2023-c0696d7b53 F37 rust-pythonize 2023-09-18
Fedora FEDORA-2023-ac1aa963e4 F37 salt 2023-09-14
Gentoo 202309-07 binwalk 2023-09-17
Gentoo 202309-03 ghostscript 2023-09-17
Gentoo 202309-08 python-requests 2023-09-17
Gentoo 202309-04 rar 2023-09-17
Gentoo 202309-06 samba 2023-09-17
Gentoo 202309-02 wireshark 2023-09-17
Oracle ELSA-2023-5144 OL8 .NET 6.0 2023-09-15
Oracle ELSA-2023-5143 OL9 .NET 6.0 2023-09-15
Oracle ELSA-2023-5145 OL8 .NET 7.0 2023-09-14
Oracle ELSA-2023-5146 OL9 .NET 7.0 2023-09-14
Oracle ELSA-2023-5061 OL9 dmidecode 2023-09-13
Oracle ELSA-2023-5197 OL7 firefox 2023-09-19
Oracle ELSA-2023-5197 OL7 firefox 2023-09-19
Oracle ELSA-2023-5184 OL8 firefox 2023-09-19
Oracle ELSA-2023-5200 OL9 firefox 2023-09-19
Oracle ELSA-2023-5046 OL8 flac 2023-09-14
Oracle ELSA-2023-5048 OL9 flac 2023-09-13
Oracle ELSA-2023-5194 OL9 frr 2023-09-19
Oracle ELSA-2023-12788 OL8 gcc 2023-09-13
Oracle ELSA-2023-12788 OL9 gcc 2023-09-13
Oracle ELSA-2023-5050 OL8 httpd:2.4 2023-09-13
Oracle ELSA-2023-12800 OL6 kernel 2023-09-16
Oracle ELSA-2023-12799 OL7 kernel 2023-09-18
Oracle ELSA-2023-12792 OL7 kernel 2023-09-14
Oracle ELSA-2023-12792 OL7 kernel 2023-09-14
Oracle ELSA-2023-12799 OL7 kernel 2023-09-18
Oracle ELSA-2023-12803 OL7 kernel 2023-09-16
Oracle ELSA-2023-12800 OL7 kernel 2023-09-16
Oracle ELSA-2023-12798 OL8 kernel 2023-09-16
Oracle ELSA-2023-12803 OL8 kernel 2023-09-16
Oracle ELSA-2023-12803 OL8 kernel 2023-09-16
Oracle ELSA-2023-12798 OL9 kernel 2023-09-16
Oracle ELSA-2023-5069 OL9 kernel 2023-09-15
Oracle ELSA-2023-12798 OL9 kernel 2023-09-16
Oracle ELSA-2023-12802 OL7 kernel-container 2023-09-16
Oracle ELSA-2023-12801 OL8 kernel-container 2023-09-16
Oracle ELSA-2023-5080 OL9 keylime 2023-09-13
Oracle ELSA-2023-5071 OL9 libcap 2023-09-13
Oracle ELSA-2023-5081 OL9 librsvg2 2023-09-13
Oracle ELSA-2023-5094 OL9 qemu-kvm 2023-09-13
Oracle ELSA-2023-5191 OL7 thunderbird 2023-09-19
Oracle ELSA-2023-5191 OL7 thunderbird 2023-09-19
Red Hat RHSA-2023:5142-01 EL7 .NET 6.0 2023-09-13
Red Hat RHSA-2023:5144-01 EL8 .NET 6.0 2023-09-13
Red Hat RHSA-2023:5143-01 EL9 .NET 6.0 2023-09-13
Red Hat RHSA-2023:5145-01 EL8 .NET 7.0 2023-09-13
Red Hat RHSA-2023:5146-01 EL9 .NET 7.0 2023-09-13
Red Hat RHSA-2023:5178-01 EL6 busybox 2023-09-18
Red Hat RHSA-2023:5193-01 EL8.6 dbus 2023-09-18
Red Hat RHSA-2023:5252-01 EL8 dmidecode 2023-09-19
Red Hat RHSA-2023:5197-01 EL7 firefox 2023-09-18
Red Hat RHSA-2023:5184-01 EL8 firefox 2023-09-18
Red Hat RHSA-2023:5183-01 EL8.1 firefox 2023-09-18
Red Hat RHSA-2023:5187-01 EL8.2 firefox 2023-09-18
Red Hat RHSA-2023:5192-01 EL8.4 firefox 2023-09-18
Red Hat RHSA-2023:5198-01 EL8.6 firefox 2023-09-18
Red Hat RHSA-2023:5200-01 EL9 firefox 2023-09-18
Red Hat RHSA-2023:5205-01 EL9.0 firefox 2023-09-18
Red Hat RHSA-2023:5219-01 EL8 frr 2023-09-19
Red Hat RHSA-2023:5196-01 EL8.4 frr 2023-09-18
Red Hat RHSA-2023:5195-01 EL8.6 frr 2023-09-18
Red Hat RHSA-2023:5194-01 EL9 frr 2023-09-18
Red Hat RHSA-2023:5244-01 EL8 kernel 2023-09-19
Red Hat RHSA-2023:5238-01 EL8.1 kernel 2023-09-19
Red Hat RHSA-2023:5255-01 EL8 kernel-rt 2023-09-19
Red Hat RHSA-2023:5221-01 EL8 kpatch-patch 2023-09-19
Red Hat RHSA-2023:5235-01 EL8.1 kpatch-patch 2023-09-19
Red Hat RHSA-2023:5190-01 EL8.2 libwebp 2023-09-18
Red Hat RHSA-2023:5222-01 EL8.4 libwebp 2023-09-19
Red Hat RHSA-2023:5189-01 EL8.6 libwebp 2023-09-18
Red Hat RHSA-2023:5214-01 EL9 libwebp 2023-09-19
Red Hat RHSA-2023:5204-01 EL9.0 libwebp 2023-09-18
Red Hat RHSA-2023:5236-01 EL8.1 libwebp: critical 2023-09-19
Red Hat RHSA-2023:5245-01 EL8 linux-firmware 2023-09-19
Red Hat RHSA-2023:5259-01 EL8 mariadb:10.3 2023-09-19
Red Hat RHSA-2023:5249-01 EL8 ncurses 2023-09-19
Red Hat RHSA-2023:5217-01 EL7 open-vm-tools 2023-09-19
Red Hat RHSA-2023:5213-01 EL8.1 open-vm-tools 2023-09-19
Red Hat RHSA-2023:5210-01 EL8.2 open-vm-tools 2023-09-19
Red Hat RHSA-2023:5216-01 EL8.4 open-vm-tools 2023-09-19
Red Hat RHSA-2023:5220-01 EL8.6 open-vm-tools 2023-09-19
Red Hat RHSA-2023:5218-01 EL9.0 open-vm-tools 2023-09-19
Red Hat RHSA-2023:5269-01 EL8 postgresql:15 2023-09-19
Red Hat RHSA-2023:5191-01 EL7 thunderbird 2023-09-18
Red Hat RHSA-2023:5201-01 EL8 thunderbird 2023-09-18
Red Hat RHSA-2023:5188-01 EL8.1 thunderbird 2023-09-18
Red Hat RHSA-2023:5186-01 EL8.2 thunderbird 2023-09-18
Red Hat RHSA-2023:5185-01 EL8.4 thunderbird 2023-09-18
Red Hat RHSA-2023:5202-01 EL8.6 thunderbird 2023-09-18
Red Hat RHSA-2023:5224-01 EL9 thunderbird 2023-09-19
Red Hat RHSA-2023:5223-01 EL9.0 thunderbird 2023-09-19
Red Hat RHSA-2023:5264-01 EL8 virt:rhel and virt-devel:rhel 2023-09-19
Red Hat RHSA-2023:5239-01 EL8.1 virt:rhel and virt-devel:rhel 2023-09-19
Scientific Linux SLSA-2023:5197-1 SL7 firefox 2023-09-19
Scientific Linux SLSA-2023:5217-1 SL7 open-vm-tools 2023-09-19
Scientific Linux SLSA-2023:5191-1 SL7 thunderbird 2023-09-19
Slackware SSA:2023-256-02 libarchive 2023-09-13
Slackware SSA:2023-257-01 libwebp 2023-09-14
Slackware SSA:2023-256-04 mozilla 2023-09-13
Slackware SSA:2023-261-01 netatalk 2023-09-18
Slackware SSA:2023-258-01 python3 2023-09-15
SUSE SUSE-SU-2023:3695-1 SLE12 binutils 2023-09-20
SUSE SUSE-SU-2023:3689-1 SLE12 bluez 2023-09-20
SUSE openSUSE-SU-2023:0249-1 osB15 chromium 2023-09-19
SUSE openSUSE-SU-2023:0246-1 osB15 chromium 2023-09-13
SUSE openSUSE-SU-2023:0247-1 osB15 chromium 2023-09-15
SUSE SUSE-SU-2023:3692-1 SLE12 curl 2023-09-20
SUSE SUSE-SU-2023:3610-1 MP4.3 SLE15 SES7.1 oS15.4 oS15.5 firefox 2023-09-15
SUSE SUSE-SU-2023:3626-1 SLE12 firefox 2023-09-15
SUSE SUSE-SU-2023:3609-1 SLE15 firefox 2023-09-15
SUSE SUSE-SU-2023:3635-1 MP4.2 MP4.3 SLE15 SLE-m5.3 SLE-m5.4 SLE-m5.5 oS15.4 oS15.5 flac 2023-09-18
SUSE SUSE-SU-2023:3640-1 SLE12 gcc12 2023-09-18
SUSE SUSE-SU-2023:3686-1 MP4.3 SLE15 gcc7 2023-09-19
SUSE SUSE-SU-2023:3700-1 MP4.3 SLE15 oS15.4 oS15.5 go1.20 2023-09-20
SUSE SUSE-SU-2023:3701-1 MP4.3 SLE15 oS15.4 oS15.5 go1.21 2023-09-20
SUSE SUSE-SU-2023:2783-2 SLE15 grpc, protobuf, python-Deprecated, python-PyGithub, python-aiocontextvars, python-avro, python-bcrypt, python-cryptography, python- cryptography-vectors, python-google-api-core, pyt 2023-09-20
SUSE SUSE-SU-2023:3688-1 SLE15 gstreamer-plugins-good 2023-09-19
SUSE SUSE-SU-2023:3680-1 MP4.0 SLE15 oS15.4 oS15.5 kernel 2023-09-19
SUSE SUSE-SU-2023:3687-1 MP4.1 SLE15 kernel 2023-09-19
SUSE SUSE-SU-2023:3684-1 MP4.2 SLE15 SLE-m5.1 SLE-m5.2 SES7.1 oS15.4 kernel 2023-09-19
SUSE SUSE-SU-2023:3683-1 MP4.3 SLE15 SLE-m5.3 SLE-m5.4 oS15.4 kernel 2023-09-19
SUSE SUSE-SU-2023:3682-1 MP4.3 SLE15 oS15.4 kernel 2023-09-19
SUSE SUSE-SU-2023:3681-1 SLE12 kernel 2023-09-19
SUSE SUSE-SU-2023:3705-1 SLE12 kernel 2023-09-20
SUSE SUSE-SU-2023:3601-1 SLE12 kernel 2023-09-14
SUSE SUSE-SU-2023:3600-1 SLE15 SLE-m5.3 SLE-m5.4 oS15.4 kernel 2023-09-14
SUSE SUSE-SU-2023:3704-1 SLE15 SLE-m5.5 oS15.5 kernel 2023-09-20
SUSE SUSE-SU-2023:3599-1 SLE15 SLE-m5.5 oS15.5 kernel 2023-09-14
SUSE SUSE-SU-2023:3656-1 SLE15 oS15.5 kernel 2023-09-18
SUSE SUSE-SU-2023:3690-1 SLE12 libcares2 2023-09-20
SUSE SUSE-SU-2023:3639-1 MP4.2 SLE-m5.1 SLE-m5.2 libeconf 2023-09-18
SUSE SUSE-SU-2023:3634-1 MP4.2 MP4.3 SLE15 SLE-m5.3 SLE-m5.4 SLE-m5.5 SES7.1 oS15.4 oS15.5 libwebp 2023-09-18
SUSE SUSE-SU-2023:3698-1 MP4.2 SLE15 SLE-m5.1 SLE-m5.2 SES7 SES7.1 oS15.4 oS15.5 libxml2 2023-09-20
SUSE SUSE-SU-2023:3699-1 MP4.3 SLE15 SLE-m5.3 SLE-m5.4 oS15.4 libxml2 2023-09-20
SUSE SUSE-SU-2023:3665-1 SLE12 libxml2 2023-09-18
SUSE SUSE-SU-2023:3666-1 SLE15 oS15.5 libxml2 2023-09-18
SUSE SUSE-SU-2023:3691-1 SLE12 mdadm 2023-09-20
SUSE SUSE-SU-2023:3702-1 SLE12 mutt 2023-09-20
SUSE SUSE-SU-2023:3669-1 MP4.0 MP4.1 SLE15 python-brotlipy 2023-09-19
SUSE SUSE-SU-2023:3670-1 SLE12 python-brotlipy 2023-09-19
SUSE SUSE-SU-2023:3664-1 MP4.3 SLE15 SLE-m5.3 SLE-m5.4 SLE-m5.5 oS15.4 oS15.5 thunderbird 2023-09-18
Ubuntu USN-6375-1 22.04 23.04 atftp 2023-09-15
Ubuntu USN-6381-1 14.04 16.04 18.04 binutils 2023-09-18
Ubuntu USN-6376-1 20.04 c-ares 2023-09-18
Ubuntu USN-6363-1 23.04 curl 2023-09-13
Ubuntu USN-6372-1 16.04 dbus 2023-09-14
Ubuntu USN-6367-1 20.04 firefox 2023-09-14
Ubuntu USN-6373-1 14.04 16.04 18.04 20.04 22.04 gawk 2023-09-14
Ubuntu USN-6364-1 16.04 18.04 20.04 ghostscript 2023-09-13
Ubuntu USN-6389-1 20.04 22.04 23.04 indent 2023-09-20
Ubuntu USN-6377-1 20.04 libraw 2023-09-18
Ubuntu USN-6371-1 14.04 16.04 18.04 20.04 libssh2 2023-09-14
Ubuntu USN-6369-1 20.04 22.04 23.04 libwebp 2023-09-14
Ubuntu USN-6386-1 20.04 22.04 linux, linux-aws, linux-aws-5.15, linux-azure, linux-azure-5.15, linux-azure-fde, linux-azure-fde-5.15, linux-gcp, linux-gcp-5.15, linux-gke, linux-gkeop, linux-gkeop-5.15, linux-hwe-5.15, linux-ibm, linux-ibm-5.15, linux-kvm, linux-lowlatency, linux-lowlatency-hwe-5.15, linux-nvidia 2023-09-20
Ubuntu USN-6387-1 18.04 20.04 linux, linux-aws, linux-aws-5.4, linux-azure, linux-azure-5.4, linux-gcp, linux-gcp-5.4, linux-gkeop, linux-hwe-5.4, linux-ibm, linux-ibm-5.4, linux-iot, linux-kvm, linux-oracle, linux-oracle-5.4, linux-xilinx-zynqmp 2023-09-20
Ubuntu USN-6383-1 22.04 23.04 linux, linux-aws, linux-aws-6.2, linux-azure, linux-azure-6.2, linux-azure-fde-6.2, linux-gcp, linux-gcp-6.2, linux-hwe-6.2, linux-ibm, linux-kvm, linux-lowlatency, linux-lowlatency-hwe-6.2, linux-oracle, linux-raspi, linux-starfive 2023-09-19
Ubuntu USN-6388-1 14.04 16.04 linux, linux-aws, linux-kvm, linux-lts-xenial 2023-09-20
Ubuntu USN-6339-4 22.04 linux-intel-iotg 2023-09-18
Ubuntu USN-6385-1 22.04 linux-oem-6.0 2023-09-19
Ubuntu USN-6384-1 22.04 linux-oem-6.1 2023-09-19
Ubuntu USN-6382-1 16.04 18.04 20.04 memcached 2023-09-19
Ubuntu USN-6370-1 14.04 16.04 18.04 20.04 22.04 modsecurity-apache 2023-09-14
Ubuntu USN-6374-1 16.04 18.04 20.04 22.04 23.04 mutt 2023-09-15
Ubuntu USN-6380-1 16.04 18.04 20.04 nodejs 2023-09-19
Ubuntu USN-6365-1 20.04 22.04 23.04 open-vm-tools 2023-09-13
Ubuntu USN-6366-1 16.04 postgresql-9.5 2023-09-13
Ubuntu USN-6378-1 20.04 22.04 23.04 python-django 2023-09-18
Ubuntu USN-6368-1 20.04 22.04 23.04 thunderbird 2023-09-14
Ubuntu USN-6379-1 20.04 vsftpd 2023-09-18
Full Story (comments: none)

Kernel patches of interest

Kernel releases

Linus Torvalds Linux 6.6-rc2 Sep 17
Greg Kroah-Hartman Linux 6.5.4 Sep 19
Greg Kroah-Hartman Linux 6.1.54 Sep 19
Clark Williams 6.1.54-rt15 Sep 19
Clark Williams 6.1.46-rt14 Sep 17
Greg Kroah-Hartman Linux 5.15.132 Sep 19
Greg Kroah-Hartman Linux 5.10.195 Sep 19
Luis Claudio R. Goncalves 5.10.194-rt94 Sep 14
Luis Claudio R. Goncalves 4.14.325-rt154 Sep 14

Architecture-specific

Core kernel

Device drivers

Brad Larson Support AMD Pensando Elba SoC Sep 13
Alexander Usyskin drm/xe/gsc: add initial gsc support Sep 14
Adrián Larumbe Add fdinfo support to Panfrost Sep 14
Matti Vaittinen Support ROHM BM1390 pressure sensor Sep 15
Victor Shih Add support UHS-II for GL9755 Sep 15
Manikandan Muralidharan Add support for XLCDC to sam9x7 SoC family. Sep 15
Ilpo Järvinen Add PCIe Bandwidth Controller Sep 15
Sricharan Ramabadhran Add support for IPQ5018 tsens Sep 15
Konrad Dybcio MM8013 fg driver Sep 15
William-tw Lin mtk-socinfo driver implementation Sep 15
Sebastian Fricke Wave5 codec driver Sep 15
Tony Nguyen Introduce Intel IDPF driver Sep 13
Lorenzo Bianconi Add WED support for MT7988 chipset Sep 14
Pawel Chmielewski ice: Add basic E830 support Sep 15
Wentong Wu Add Intel LJCA device driver Sep 17
Meng Li amd-pstate preferred core Sep 18
Hsiao Chien Sung Support IGT in display driver Sep 18
Markus Schneider-Pargmann soc: mediatek: MT8365 power support Sep 18
wangweidong.a@awinic.com ASoC: codecs: Add aw87390 amplifier driver Sep 18
John Watts Add FS035VG158 panel Sep 18
Yi-De Wu GenieZone hypervisor drivers Sep 19
Yahui Cao Add E800 live migration driver Sep 18
Jason-JH.Lin Add CMDQ secure driver for SVP Sep 19
Matthew Brost DRM scheduler changes for Xe Sep 18
tumic@gpxsee.org Digiteq Automotive MGB4 driver Sep 19
cy_huang@richtek.com ASoC: Add rtq9128 audio amplifier Sep 20

Device-driver infrastructure

Documentation

Filesystems and block layer

Memory management

Networking

Security-related

Herbert Xu crypto: Add lskcipher API type Sep 14
David Gstir DCP as trusted keys backend Sep 18
Dmitry Safonov net/tcp: Add TCP-AO support Sep 18
Andrii Nakryiko BPF token and BPF FS-based delegation Sep 19

Virtualization and containers

Miscellaneous

Jozsef Kadlecsik ipset 7.18 released Sep 19

Page editor: Jonathan Corbet


Copyright © 2023, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds