Leading items

Welcome to the LWN.net Weekly Edition for November 7, 2024

This edition contains the following feature content:

• Building secure images with NixOS: a talk on how to use UEFI secure boot to verify the integrity of NixOS installations.
• The OpenWrt One system: an open system designed from the beginning to run OpenWrt.
• Safety in an unsafe world: using Rust's type system to encode (and enforce) arbitrary constraints.
• The Overture open-mapping project: a different approach to gathering and integrating open mapping data.
• OSI board AMA at All Things Open: the Open Source Initiative board discusses the Open Source AI Definition and more.

This week's edition also includes these inner pages:

• Brief items: Brief news items from throughout the community.
• Announcements: Newsletters, conferences, security updates, patches, and more.

Please enjoy this week's edition, and, as always, thank you for supporting LWN.net.

Comments (none posted)

Building secure images with NixOS

Image-based Linux distributions have seen increasing popularity, recently. They promise reliability and security, but pose packaging problems for existing distributions. Ryan Lahfa and Niklas Sturm spoke about the work that NixOS has done to enable an image-based workflow at this year's All Systems Go! conference in Berlin. Unfortunately, LWN was not able to cover the conference for scheduling reasons, but the videos of the event are available for anyone interested in watching the talks. Lahfa and Sturm explained that it is currently possible to create a NixOS system that cryptographically verifies the kernel, initrd, and Nix store on boot — although doing so still has some rough edges. Making an image-based NixOS installation is similarly possible.

Lahfa started by giving a brief overview of NixOS for those attendees who were unfamiliar with it. He described the distribution as a "standard systemd-based Linux", but with some differences mostly centered around the fact that it does not follow the filesystem hierarchy standard. In NixOS, all of the binaries on the system live in /nix/store, and are configured to use a path and library path that are tightly scoped to only their declared dependencies. This has a lot of benefits, Lahfa said, including NixOS's ability to run multiple versions of the same software. But it also has consequences for secure boot.

Lahfa explained that secure boot "controls who is allowed to run software on your computer". It relies on using signed binaries; the computer will only boot into the provided kernel if the signature on it is valid. On systemd systems, it is possible to use unified kernel images (UKIs), which package a unified extensible firmware interface (UEFI) boot stub, the kernel, and its initrd together. This has security benefits, because it means that secure boot validates the initrd as well as the kernel. But it causes problems for NixOS, which needs to present many more options in the bootloader than most other distributions in order to support its efficient rollback features.

NixOS's separation of binaries into individual paths under /nix/store — and ability to share libraries between different versions — allows the distribution to keep a large number of previous configurations around. Every time a NixOS system has its configuration changed, from a software update, for example, the complete state of the installed programs is saved as a "generation". In the bootloader, the user can select any previous generation they would like (at least until the old generations are cleaned up to reclaim their storage space), and the kernel will load the appropriate initrd for that generation, which in turn sets up all of the configuration files from that generation. This allows for fearless upgrades, since the previous configuration is available in the boot menu — a value proposition quite similar to image-based distributions. Unfortunately, this ability doesn't work well if the initrd needs to be bundled with the kernel, because that increases both the size of each kernel image, and the number of different kernel images that must be stored. Doing so will quickly fill up the EFI (Extensible Firmware Interface) system partition (ESP).

So, to work around this, Lahfa, Sturm, and other contributors wrote Lanzaboote, a NixOS-aware reimplementation of systemd-stub, the UEFI secure-boot stub included in the systemd project. Lanzaboote is signed, for secure boot, and then separately verifies the hashes of the kernel and initrd that it loads. Now, generations can share kernels and initrds when possible, making it possible to have more generations available. Unfortunately, Lanzaboote is hard to upstream. The systemd project suggested using systemd-boot addons instead — binaries that are verified by secure boot but not directly run. If initrds and kernels could be made into addons, then Lanzaboote's functionality may not be needed, and NixOS could just use systemd's loader.conf to mix-and-match initrds and kernels. In the future, efforts to "denormalize" UKIs into their component parts like this without losing their attendant security benefits could potentially benefit image-based distributions as well, by enabling more sharing between images.

Building images

For the second half of the talk, Sturm spoke about how NixOS could actually be used as an image-based distribution. He started by describing why — when NixOS already supports atomic rollbacks — users might want to use an image-based model. Ultimately, it comes down to security, he said. An image-based workflow enables more security features — "things we really want, but cannot offer by default to everyone". The core benefit is cryptographically enforced immutability, which makes a computer more tamper-resistant.

Sturm then explained how to use NixOS to build images. Nix (NixOS's package manager and build system) can produce ISO files directly as an output from the build process, but he instead suggested an approach based on changing how NixOS stores generations. Normally, generations are stored as files inside /nix/store. Any data that is shared between generations (such as a program that was not updated) can be referenced by multiple generations. To produce images based on NixOS, he suggested having a separate partition for each generation. This loses NixOS's ability to share data between generations, but it also brings the distribution closer to what existing image-based distributions do, meaning that existing tooling can be reused. Sturm specifically called out systemd-repart, ukify, and systemd-sysupdate as existing tools that fit naturally into this approach.

Not every tool can be directly adapted, though. Sturm said that mkosi — systemd's OS image builder — would "break the developer flow" on NixOS, and that it was better to build everything directly with Nix. Eventually, the folks working on image-based NixOS may want mkosi support to enable more comprehensive testing by the upstream systemd project, but for now it is not a goal.

Sturm then gave an example of the partition layout that a computer using this approach might use: two separate sets of partitions for generations, so that the one not in use can be updated, each protected by dm-verity. Each set of partitions has a partition for the dm-verity metadata, and one for the store itself. The second set of store partition can be created upon first boot, so a system using this approach can be installed just by copying the EFI system partition (ESP) and first store partition into place. Finally, a persistant partition for user data takes up the rest of the disk.

In a computer set up like this, the Nix store could be mounted using nix-store-veritysetup-generator, or it could be mounted under /usr with the normal systemd-veritysetup-generator and bind-mounted into place. In either case, the Nix store would be protected by dm-verity. None of this work is enabled by default on NixOS, Sturm cautioned. A user needs to opt-in to get it. But it "lets you kind of build your own distro" in that you can potentially use Nix to produce system images that don't rely on any NixOS tools at run time with this method.

One audience member asked whether it was possible to have the Nix store both verified by dm-verity and encrypted. Sturm said that it was "not supported by repart but doable". Specifically, the user would need to create an encrypted block device and then layer dm-verity over the top. Another audience member asked whether users would really have to give up on sharing data between generations to use this approach — wouldn't it be possible to have a shared baseline that multiple generations could reference? Sturm agreed that would be possible as well, but not while simultaneously using dm-verity. The tradeoff between efficiency and security is something that each user will need to consider when deciding which one is right for them.

NixOS breaks a lot of assumptions about Linux systems, starting with the filesystem hierarchy standard and going from there. Despite that, a lot of the same tools that are being used with other image-based distributions can be used to create an image-based NixOS installation. Users who want the benefits of an image-based distribution at the same time as NixOS's unique advantages may have to do a bit of tweaking, since support is still a work in progress, but should nonetheless find that combining the two is possible.

Comments (10 posted)

The OpenWrt One system

OpenWrt is, despite its relatively low profile, one of our community's most important distributions; it runs untold numbers of network routers and has served as the base on which a lot of network-oriented development (including the bufferbloat-reduction work) has been done. At the beginning of 2024, a few members of the project announced a plan to design and produce a router device specifically designed to run OpenWrt. This device, dubbed the "OpenWrt One", is now becoming available; the kind folks at the Software Freedom Conservancy were kind enough to ship one to LWN, where the desire to play with a new toy is never lacking.

The OpenWrt One was designed to be a functional network router that would serve as a useful tool for the development of OpenWrt itself. To that end, the hope was to create a device that was entirely supported by upstream free software, and which was as unbrickable as it could be. The developers involved concluded that the Banana Pi boards were already reasonably close to those objectives, so that was the chosen starting point. Banana Pi is also the manufacturer of the board that resulted.

The OpenWrt One comes with a two-core Arm Cortex-A53 processor, 1GB of RAM, and 256MB of NAND flash memory. There is also a separate, read-only 16MB NOR flash array in the device. Normally, the OpenWrt One will boot and run from the NAND flash, but there is a small switch in the back that will cause it to boot from the NOR instead. This is a bricking-resistance feature; should a software load break the device, it can be recovered by booting from NOR and flashing a new image into the NAND array.

The device only comes with two Ethernet ports, one configured by default to talk to an upstream router. That choice raised a few eyebrows, since a router can normally be expected to have more ports; in an FAQ posting in January, John Crispin defended that choice this way:

We didn't want to impose additional complexity and costs by including an external managed switch IC. One port is 1GBit/s capable, while the other features a speed up to 2.5GBit/s. This is a limitation of the chosen SoC.

The router has 2.4GHz and 5GHz WiFi radios and three antennas. There is a USB 2.0 port on the front. Within the box, there is a mikroBUS connector for miscellaneous peripheral devices and an M.2 PCIe connector for the attachment of a solid-state drive. The space for M.2 cards is limited to the 2242 or 2230 formats, meaning that it allows the addition of up to 2TB of relatively fast storage with currently available drives. That, in turn, would make it possible to use the router as a network-attached storage server, or to run a more traditional Linux distribution on it.

The OpenWrt one arrived with no documentation whatsoever. Perhaps surprisingly for a device shipped by the SFC, it also lacked the required written offer to provide source for the GPL-licensed software contained within — not that said source is difficult to find. [Correction: the offer is there, printed on the outside of the box. I missed that, and apologize for the mistake.] There was no power supply provided; the user needs to supply a USB-C charger or power over Ethernet. It draws about three watts when operating.

The device had a minimal OpenWrt snapshot installed. Evidently, it was not the intended image and did not work properly; it would boot and allow logins via SSH, but lacked the ability to install the LuCI web interface. That made life difficult for those of us who, having never taken the time to fully understand OpenWrt's unique package-management and configuration tools, tend to lean hard on LuCI. That, in turn, led to an early exercise of the NOR-flash recovery mechanism, described on this page, to flash a new image. The ritual is a bit cumbersome, involving holding a front-panel button down while connecting power to the rear. But, then, completely reflashing a device should not be something one can do accidentally.

After booting into the new image, the One behaved like any other OpenWrt router. It came up running a 6.6.50 kernel and with LuCI installed; with just a bit of configuration, it was sitting on the local network and providing WiFi service. Nothing exciting to report. That is the thing about working hardware; it just quietly does what is expected of it.

What could be more interesting is seeing this router get into the hands of developers and enthusiasts who will use it to make OpenWrt (and other small-system distributions) better. The combination of free software (with the exception, seemingly, of the firmware blob loaded into the WiFi adapter) and brick-proof hardware should make the OpenWrt One a relatively easy and pleasant device to develop for. That will, hopefully, encourage more developers to join the project and make the system better for everybody.

Getting there, though, will require some significant documentation improvements. The device lacks even the usual "getting started" page, much less more detailed information on how to connect it to the network or what the three LEDs mean. A nice tutorial on how to build a custom image for this device would be welcome. OpenWrt as a whole has reasonably complete (if sometimes hard to navigate) documentation, so presumably these gaps will be filled over time.

Given the wide range of devices that OpenWrt can run on, the addition of one more might seem like a drop in the bucket. But most of those systems were designed to run something else, even if it is often OpenWrt with a proprietary interface grafted on top, and replacing the software is not always easy or risk-free. A device that was designed to run OpenWrt from the beginning, without limiting the user's freedom, is a welcome addition and a suitable present to the OpenWrt project as it celebrates 20 years of development.

[The OpenWrt One is expected to be generally available by the end of November. The price of each system will include an (unspecified) donation to the OpenWrt project. Schematics and data sheets for the device are also available.]

Comments (47 posted)

Safety in an unsafe world

Joshua Liebow-Feeser took to the stage at RustConf to describe the methodology that his team uses to encode arbitrary constraints in the Rust type system when working on the Fuchsia operating system (slides). The technique is not unknown to the Rust community, but Liebow-Feeser did a good job of both explaining the method and making a case for why it should be used more widely.

He began with the motivation for his presentation, which was netstack3, Fuchsia's new networking stack written entirely in Rust. The project started six years ago, and Liebow-Feeser led the project for four years. Networking stacks are "very serious", he said. They are responsible for almost all traffic, often implement dozens of different protocols, and are the first line of defense against attackers. They're also just plain large pieces of software.

Netstack3 encompasses 63 crates and 60 developer-years of code. It contains more code than the top ten crates on crates.io combined. Over the last year, the code has finally become ready to deploy. But deploying it to production requires care — networking code can be hard to test, and the developers have to assume it has bugs. For the past eleven months, they have been running the new networking stack on 60 devices, full time. In that time, Liebow-Feeser said, most code would have been expected to show "mountains of bugs". Netstack3 had only three; he attributed that low number to the team's approach of encoding as many important invariants in the type system as possible.

The method

This isn't a new idea, Liebow-Feeser said. Many people have tried to make it so that buggy programs simply don't compile. But the netstack3 team has a concrete, general framework for approaching this kind of design. He broke the process into three steps: definition, enforcement, and consumption. For definition, the programmer must take something that Rust can reason about (usually types) and attach the desired property to it. This is usually done via documentation — describing that a particular trait represents a particular property, for example. Then the programmer enforces the property by making sure that all of the code that directly deals with the type upholds the relevant invariant.

Critically, the programmer must at some point use field privacy or unsafe markers to ensure that these invariants cannot be broken elsewhere. For example, a structure with all public fields can be constructed from anywhere, so it isn't possible to rely on invariants tied to that type. If the structure includes a private field, it can only be constructed in that specific module, which means the programmer can audit all of the uses. When adding a private field isn't a solution, for some reason, Rust programmers can fall back on marking things unsafe, so that other programmers will hopefully read the documentation before using a structure or trait. Finally, other code can rely on the property as a guarantee, such as by making optimizations that require it.

In a codebase using this method, each guarantee will have a single Rust module that deals with the internals of the property. Then, the rest of the code can rely on the type checker to ensure that the property holds everywhere. For a more concrete understanding of what that looks like in practice, Liebow-Feeser introduced several examples.

The first example he gave was nearly trivial: a binary tree that needs to enforce ordering invariants between nodes. Of course Rust cannot natively check for this. So how might the programmer ensure that an invalid tree is never produced? Following Liebow-Feeser's process, they must first document the requirement. Then, they ensure that the code which deals directly with creating and modifying nodes in the tree respects the invariant. They must also ensure that relevant details are hidden behind private fields, so that code outside the module cannot interfere. Finally, the rest of the code can now rely on the fact that the tree is always in order.

That approach will be familiar to any programmer who has implemented a data structure. But the approach generalizes to more subtle properties, Liebow-Feeser said. His second example was actually from Rust's standard library. Rust promises thread safety, but the language itself actually knows nothing about thread-safety properties. Those are all implemented in the standard library.

One relevant property that the authors of the standard library would like to guarantee is that non-thread-safe closures are never passed to std::thread::spawn(). To do this, they created an unsafe trait (Send) that represents code which is safe to run on another thread. Send is unsafe not because it is inherently dangerous, but just because it represents a property that the compiler cannot check. The trait could then be used as a bound to restrict which functions can be passed to spawn(). Finally, while safe Rust code cannot implement the trait directly, the standard library adds various trait-inference rules and derive macros to let other code implement the trait. (In fact, this is a slight simplification because Send is an auto trait — an unstable feature that instructs the compiler to implement traits automatically for compatible types, although Liebow-Feeser did not mention this in his talk.)

Automatic deadlock prevention

The final example Liebow-Feeser gave was significantly more involved, and was taken directly from the netstack3 code. The networking stack needs to be multithreaded with fine-grained locking, for performance, he explained. Rust's existing libraries can guarantee that the code is thread-safe, but they can't guarantee that it won't deadlock. With so many developers working on such a large codebase, being able to know that the code won't deadlock is an important property that isn't easy to guarantee. Netstack3 has 77 mutexes, across thousands of lines of code. Its predecessor, netstack2 (implemented in Go) had a lot of deadlocks.

For the first step, the developers added names to each mutex, in the form of a generic Id type parameter. The relevant IDs are all zero-sized types, so they don't exist at run time, and have no run-time overhead. Then, they defined two unsafe traits: LockAfter and LockBefore. These represent exactly what they sound like — for a specific mutex to be locked after another mutex, LockAfter must be implemented for the relevant types. They added a derive macro, so that there's little boilerplate needed to add to each ID type, and added blanket trait implementations such that if there is a cycle in the graph of locks, it results in a compile time error because of overlapping trait definitions.

In order to actually make the existence of the locking traits useful, however, the developers also needed to add methods that use them. In this case, they created a new Context type that carries around one of the ID types. The lock() method on their mutex type takes a mutably borrowed context, so the original context cannot be used until the mutex is unlocked. At the same time, it provides a new context with the ID of the lock, which can only be used to lock mutexes with the correct LockAfter implementation.

So, from the perspective of all of the code outside the module that implements this, mutexes can only be locked with an appropriate, un-borrowed context object. The context objects impose a global ordering on how mutexes can be locked, and attempting to add an incorrect LockAfter implementation (one that would permit a cycle) is a compile error. Programmers are free to lock whatever mutexes they can get past the compiler, secure in the knowledge that this can't cause a deadlock. In turn, this makes it easier to justify adding more fine-grained locking to the implementation. At run time, all of the type-level machinery associated with this guarantee has been compiled out, so there is no run-time overhead.

Conclusion

In practice, there are actually some problems with the simplified deadlock-prevention example as presented, Liebow-Feeser said. But generally, this ability to take a property that the language does not know about and "teach" it to Rust, so that now it is enforced at compile time, is why he likes to call Rust an "X-safe" language. It's not just memory-safe or thread-safe, but X-safe for any X that one takes the time to implement in the type system.

He concluded his talk by asking people to try this out for themselves, especially in new domains that nobody has tackled yet. He encouraged people to think of functions that panic or return an Option as a code smell — those are all places where the code could encode the necessary invariants at compile time instead. He called on the audience to "make your APIs exactly match the structure of the problem". At the same time, he cautioned people that the appropriate solution may not be the same every time. For deadlocking in netstack3, they used traits and trait inference rules to ensure that there are no cycles. But not every circumstance will warrant the use of traits; the important thing is to follow the method he laid out.

Liebow-Feeser thinks "this can reshape how we do software engineering". There are lots of domains where correctness is important, and successfully scaling software in those areas will require making it possible for tools to verify that correctness, he said. Whether his prediction is right remains to be seen — but in any case, the method he spoke about seems like a nice framework to unify different techniques for ensuring program safety.

Comments (90 posted)

The Overture open-mapping project

OpenStreetMap tends to dominate the space for open mapping data, but it is not the only project working in this area. At the 2024 Open Source Summit Japan, Marc Prioleau presented the Overture Maps Foundation, which is building and distributing a set of worldwide maps under open licenses. Overture may have a similar goal to OpenStreetMap, but its approach and intended uses are significantly different.

Once upon a time, not too long ago, Prioleau began, map making was mostly done by surveying — sending somebody out to measure where things were. That has changed over the last couple of decades with the advent of location-aware mobile devices; map making is now driven by sensors, not surveyors. That has changed the nature of maps and how they are used; Overture Maps was created to take advantage of (and support) those changes. The project is still in its early days, but it has the support of a long list of companies and is already being used by some of them.

Overture Maps was created to change the model for map production. The amount of available mapping data has exploded, he said, and the types of that data have changed. Maps that were once concerned with just the road network have grown to include information about speed limits, road signage, lane usage, and more. There may never be a world where all of this data is open, but there is space for a set of common base layers to tie it all together. These base layers might contain information about roads — their route, geometry, and directionality, for example — while letting others attach value-added data like traffic information. As has been seen in other areas, companies can cooperate in the creation of the foundational layers, while adding their special products on top.

The companies that initially sponsored Overture Maps had all been working on OpenStreetMap previously, but they had needs that were not being fully met there. For example, they want to use all of the available mapping data, much of which does not appear in OpenStreetMap; this data includes some government mapping data and the increasing amount of data generated by machine-learning systems. There is a need for a high degree of validation of this data; maps reflect facts, and there need to be protections to keep people from changing those facts. Among other things, Overture Maps uses machine-learning systems to validate map data.

That data also must be presented in an organized scheme, in a way that makes it easy for others to attach additional data to it. This process is called, for better or worse, "conflation". Overture Maps supports this functionality via a mechanism called the Global Entity Reference System (GERS). Every item of interest in a map is assigned a permanent GERS ID, which is a 128-bit identifier. GERS, he said, might be the most significant part of the entire Overture Maps effort; it makes conflation simple, easing the use of map data.

Mapping data can be divided into over a dozen data types, called "themes"; Overture Maps currently supports six of them. The transportation theme covers the road network, including information on public transit, cycling, and more. Places covers points of interest — primarily businesses. This is the most annoying data type to manage, since it is highly volatile and businesses often don't bother to notify the world when they cease to exist. Divisions are administrative boundaries, from national borders to neighborhood boundaries. Buildings is exactly what it seems, as is the addresses theme. Finally, the base theme covers geographical features, including ground cover, water features, and more.

One of the key features of Overture Maps, he said, is integrating data from multiple sources. As an example, the latest release included over 20 million new Japanese addresses, along with building data from both OpenStreetMap and the Microsoft building footprints database, which was machine-generated from satellite data. It turns out that the OpenStreetMap data is more complete in the cities, where the contributors live, while the Microsoft database is more complete in rural areas.

Returning to GERS, Prioleau said that mapping is being driven by an explosion of data types. But only some types are "exploding". The road network, while constantly changing, is relatively easy to keep up with; the same is true of addresses and such. This kind of data is manageable as a shared base layer that all can contribute to and use. The attached layers, though, which might include traffic data, restaurant reviews, opening hours, or any other sort of add-on data, are far too volatile to be managed in an open layer. This data will, he said, continue to be a proprietary product indefinitely.

For years, these add-on data types have been fragmented into numerous incompatible formats. Integrating the data into a useful product has often been more expensive than acquiring it in the first place. If everybody could agree on a global ID for mapping data, though, a lot of these problems would go away; GERS is meant to be that ID.

Being in Japan, Prioleau took a moment in his conclusion to talk about the automotive industry, which is collecting vast amounts of data as its cars phone home. (He did not address the privacy implications of this data collection). Manufacturers see this data as valuable, but have not always been able to realize that value. Identifying this data with GERS IDs would make it easy for these manufacturers to contribute some of the data to Overture Maps, and for them to use the rest to add value to their products. One manufacturer, for example, is looking at detecting a car's wheels slipping on ice, and sending an icy-roads notification to other cars operating in the area.

Once the talk was done, I could not resist asking the obvious question: why create a new project rather than focusing these resources on making OpenStreetMap better? Prioleau answered that he has been an OpenStreetMap contributor for nearly 20 years; it is far from clear, he said, that the project wants much of the work that is being done in Overture Maps. OpenStreetMap is focused on creating a community of mappers; dumping a bunch of AI-generated data into the project is not the best way to encourage that community.

The companies that launched Overture Maps first tried hard to turn OpenStreetMap into the sort of project they needed, he said; the OpenStreetMap contributors "successfully fended that off". So Overture Maps was created with a focus on the end users of the data rather than on the contributors. While a bit over half of the project's data comes from OpenStreetMap now, he said, he expects that proportion to fall in the future.

An interesting way to look at these two projects, perhaps, is to see OpenStreetMap as being analogous to a typical free-software development project, while Overture Maps is more like a distributor. The former is focused on its development community, while the latter is working on integrating the results of various mapping projects, performing quality control, and producing something that is easily usable by others. So, while the two projects might appear to be in competition at one level, there may actually be a useful role for both of them in the end.

[ Thanks to the Linux Foundation, LWN's travel sponsor, for supporting our travel to this event. ]

Comments (20 posted)

OSI board AMA at All Things Open

Members of the Open Source Initiative (OSI) board sat down for a 45-minute "Ask Me Anything" (AMA) session at All Things Open in Raleigh, NC on October 29. Though the floor was open to any topic the audience might want to ask of the OSI board, many of the questions were focused on the Open Source AI Definition (OSAID), which was announced the day before. The new definition has been somewhat controversial, and the board spent a lot of time addressing concerns about it during the session, as well as questions on open washing, and a need for more education about open source in general.

The session was held in one of the smaller rooms at the venue, with about 30 people in attendance (not counting OSI board members or staff). The session kicked off with some ground rules from the moderator, Mer Joyce, who had also worked as a facilitator for the OSAID drafting process. The first order of business was introducing the board members in attendance; OSI vice-secretary Anne-Marie Scott, vice chair Thierry Carrez, Gaël Blondelle, Pamela Chestek, Sayeed Choudhury, and Tracy Hinds.

Deborah Bryant, a former board member who currently works with the OSI as its US policy director, got the ball rolling with a question about the most interesting challenge the board expected to face in the next year.

Scott answered that the recent effort to create the OSAID had broadened the OSI's community, which was good but also "problematic". She said that the organization had encouraged a group of people to participate in the OSAID process who "may not have always affiliated themselves with open source" but were "incredibly valuable to the work we have done". Now, the OSI needed to work on keeping them engaged and to make connections to the existing community.

Choudhury agreed, adding that it had been an active year, and that the OSI now needed to get back to core things the organization had focused on in the past with the legal and developer communities. He also drew attention to the policy work the OSI had done, particularly with regard to the EU with the Cyber Resilience Act (CRA). "We've been doing that work, but I think we need to focus on it a lot more."

Bespoke licenses

The first audience question was about the trend of "bespoke" licenses, such as the CockroachDB License, that claimed to be open-source licenses but had restrictions such as "a certain number of users, or under a certain amount of revenue per year". He wanted to know how to navigate those "because right now we have to evaluate every single one of them".

Chestek was the first to respond to the question. Bespoke licenses, she said, defeat the purpose of the Open Source Definition (OSD). The idea behind the OSD was to make adopting open-source software frictionless because users know immediately that they have all the freedoms that they need:

These [bespoke] licenses are not designed to build community, they're designed to extract value out of software by free-riding on the concept of open source. But they're not open source.

What the OSI does, she said, is to occasionally talk about those licenses and point out that there's a reason that open source works, "and that is frictionless adoption". Without that, there is no chance of building community, so what is the point of the license "other than to maybe look like you're a good citizen", without being willing to make the real commitment to an open-source development model.

Carrez said that these new licenses focus on a single benefit of open source, which is the availability of code. As an organization, he said, OSI needed to do more to educate the public that there are additional benefits in terms of innovation and sustainability. He added that developers now take for granted a lot of the benefits of open source.

I can tell you that developing today is very different than developing then. I don't want us to go back to the dark ages of the '90s where you had to basically look at [the license of] every piece of code in order to use it.

How to move forward

The next question was from Nithya Ruff, head of the open-source program office (OSPO) at AWS. She said that "not everyone agrees with the new Open Source AI Definition". There were some points of disagreement, she said, but "a lot of places where we agree". She wanted to know how the community could work together to move forward.

Scott said that the simple answer was to keep talking. Some people felt the OSAID was too open, others felt it was too closed, "and then we've got everything in the middle". That is driven, she said, by the kinds of organizations that people work for, as well as the values that they hold. She went on to say that there were more conversations to be had, some in the open and some under the Chatham House Rule, for the "next phase of engagement". Ruff was right, she said, "there's more agreement than there is difference, but on the points of difference, they are strongly held".

Carrez responded that the board didn't have a strong position one way or the other when it started the OSAID process, and that he was "very sympathetic" to the dissenting voices that were heard during the process. However, he said that he realized "the ultimate goal is really to replicate the success we've seen in open-source software to AI" and not to simply translate the OSD to AI.

Some of the tension we've seen is in people that haven't made that mind shift to the wider picture and are just trying to apply what they are very familiar with, that they're experts at, and that made it more difficult.

"Nobody disagrees about the principles [behind the OSAID], where we see disagreement is implementation", Chestek said. The OSI had gone further than others in trying to define a fully open implementation, and tried to put a stake in the ground that defined what it considered open right now. That implementation, she said, is "the piece of it that we know is going to change" but not the principles. Maybe when the industry was more stable, "then I do hope we'll really come to a unified place". She added that it was a "wild experience" to do the OSAID work at the peak of a hype cycle while an industry was being regulated.

Education

The next question came from Carson Shaar, who introduced himself as the co-founder of a company in the open-source space, Zero-True, and a recent college graduate. He said that he'd observed "quite a lack of education" around open source. Universities were doing a lot of teaching around entrepreneurship and building products, "but not a lot of work around contributing to open source and working in open source". He wanted to know what work the OSI was doing to educate and involve students.

Choudhury, who is director of the OSPO at Carnegie Mellon Libraries and director of an Alfred P. Sloan Foundation grant for coordination of University OSPOs, began by saying "I feel your pain". At least in the US, he said, education about open source is "deeply lacking" and fails to help students understand open source beyond the computer-science perspective. The Sloan Foundation is "providing support to the domestic movement" around open source to help university OSPOs give students, faculty, and technology staff a better understanding of the broader open-source ecosystem. "How to navigate in that from a technical perspective, legal perspective, community perspective" as part of the actual educational experience and not just something "on the side". It is, however, early days. "I'm not going to pretend we solve[d] this problem."

Left behind

I asked the next, somewhat long-winded, question. After introducing myself, I noted that I had observed many comments and responses that expressed a feeling that the OSI had chased a "shiny ball nobody asked it to chase", as well as disappointment with the OSAID process and final definition. There seemed to be a loss of trust in the OSI as a result, by the community that put the OSI where it is today. What was the plan to deal with that?

Hinds said that the board recognized that community members were upset, and felt they were not heard. However, "this [definition] was something that was being, I would say, asked and even demanded. We had people saying, 'we need this yesterday'." There was, she said, an underlying assumption that there could be a "translation from OSD to open-source AI" and that the OSI was being trusted to take the process seriously and try to facilitate it.

Choudhury said that the OSI was spurred on by pending regulation. He quoted Mike Milinkovich, executive director of the Eclipse Foundation, as saying that "we just have to get used to the fact that software is about to be regulated" in the context of the CRA. There were clear signals that regulation was about to start, including the use of the term "open-source AI" without defining it. What other group, he asked, is really better positioned to define it? But that meant reaching out to new sectors involved in regulating AI, "which was always going to be messy".

Carrez replied that the OSI "may not have done a great job promoting" the work that it has done to "have the back of developers" in the face of regulation such as the CRA. The regulatory landscape, he said, would be very different without the work done by the OSI and others, that would have put open source at risk.

There is also the fact that the OSI had to work with a lot of stakeholders, and that people from the OSI's traditional constituency were some of the voices heard from during the process, just not the only voices. That, he said, caused some frustration.

Blondelle argued that the OSI was not chasing a shiny ball, but trying to protect the original definition. The OSI had seen vendors using the term open source for things that were clearly not open source, "so I think we had to define open source AI because otherwise we would have lost some ground" on the OSD.

Hinds replied again that she wanted to make it clear that the OSI board had "felt that letdown". It would be spending energy on "reinforcing the value we provide to legal and developer communities, because we feel the pain of them feeling let down and need to do that repair" while making sure to include newer communities to figure out how they can all work together when needed.

"Stable, but not permanent"

The next question came from an attendee who said he was acting deputy chief AI officer for the Cybersecurity and Infrastructure Security Agency (CISA). He said that CISA leads the effort to manage risk in cyber and physical infrastructure, and said that the OSD serves as "a sort of risk tolerance and risk statement" about the acquisition and security risks of software. He said that helps him to advocate for open-source software in government as the best solution that will give the best outcomes to mitigate risk. However, risk was not one of the things that was mentioned about the OSAID, he said, and he wanted to know how CISA could help "sort of drive towards definitions that are equivalent risk-management postures" that would help him with security decisions and recommendations for AI systems.

Carrez said that was an interesting question, and that the OSI was staying ready to evolve the definition. One of the questions, he said, is what is the best way to patch and run AI systems? Comparing pure software to AI systems is difficult, for example the economic cost of generating the AI system's models. "The reality is that if only a handful of companies and a handful of governments have the resources" to rebuild models, it is not a practical goal for open-source AI. There is more and more evolution, he said, on the ways that models are fine-tuned or patched in a less costly way. It was important, though, to "put a stake in the ground" with the OSAID to have something to work from to have the discussions.

Choudhury also noted that the OSAID was the start of a journey and "a stable definition, but it's not permanent".

Open washing

The final question was about combating open washing, and how the OSI, government, and developer communities should be trying to prevent bad actors or others from misrepresenting software or AI systems as open if they are not.

Chestek said that this was not a new thing for open-source software and had probably been going on as long as it had existed. The OSI relies a lot on the community to do communal shaming, which is "probably the most powerful" way to combat open washing. When a company misrepresents its software, the OSI usually finds out about the situation from the community. Then the OSI would say something publicly, if it was appropriate to do so. That, she said, would probably carry over to the OSAID as well and she hoped "we can all converge at least on the principles of it". For example, if a system has a commercial limitation on it, "that's just not open source, and we don't even need to get into the weeds about whether or not you provided all the information about the data".

"Can I fork it?" asked Hinds. From a practical standpoint, she said, the right to fork translates to AI, and that is what the OSI is going for. "Can I look at this model? Can I work with this? Can I do something with it? I think that's really easy to resonate with our existing communities" as well as new ones coming into the open-source space.

The board also had an opportunity to give parting thoughts to the audience, which Carrez used to thank attendees and encouraged people to join the OSI and run for the board if they were interested in helping to make open source better.

[ Thanks to the Linux Foundation, LWN's travel sponsor, for supporting our travel to this event. ]

Comments (39 posted)