Leading items

Welcome to the LWN.net Weekly Edition for August 15, 2024

This edition contains the following feature content:

• COSMIC desktop makes its debut: a Rust-based desktop environment for System76's Pop!_OS.
• Changes coming in PostgreSQL 17: changes to the next version of the database system, including more incremental-backup support and some security improvements.
• Meeting the Debian Technical Committee: a DebConf talk gives an overview of what the committee does and how it does it.
• Distinguishing Debian testing from unstable: a small proposed change to the /etc/os-release file for Debian sets off a lengthy, sometimes acrimonious discussion.
• Standards for use of unsafe Rust in the kernel: an attempt to describe the requirements for documenting the use of unsafe blocks in kernel code.
• Endless OS aimed at educational and offline environments: a new release of a distribution focused on education, especially for rural areas where internet connectivity may be sporadic or non-existent.
• Zettlr: note-taking and publishing with Markdown: a tool for editing and organizing documents created using Markdown.

This week's edition also includes these inner pages:

• Brief items: Brief news items from throughout the community.
• Announcements: Newsletters, conferences, security updates, patches, and more.

Please enjoy this week's edition, and, as always, thank you for supporting LWN.net.

Comments (none posted)

COSMIC desktop makes its debut

Linux hardware vendor System76 started promoting its work on a Rust-based, Wayland desktop environment for its Pop!_OS Ubuntu-derivative distribution almost two years ago. On August 8, the company released an alpha version of the COSMIC desktop environment for users to test out. While it has rough edges and missing features, it is stable enough to get a good feel for what the finished product has in store—and the initial results are promising.

The road to COSMIC

System76 sells Linux laptops, desktops, servers, and other hardware. The company was founded in 2005, and preinstalled Ubuntu as the default operating system on its offerings for many years. In 2017, the company created its Pop!_OS distribution after Canonical discontinued its work on the Unity desktop. Pop!_OS started out using GNOME with just a few tweaks, but over the years the distribution has diverged farther and farther from stock GNOME. In 2018, with the Pop!_OS 18.10 release, the company added its own software "store" called the Pop Shop. The 20.04 LTS release introduced tiling features to the desktop with a GNOME extension called Pop!_OS Shell.

The 21.04 release introduced the "COSMIC" name by using it to describe its customizations to GNOME. The name was initially expanded as "Computer Operating System Main Interface Components", but the company seems to have abandoned that unwieldy backronym, which is probably for the best. Changes in 21.04 included the addition of a dock (a fork of the Dash to Dock extension), trackpad gestures, and other various and sundry modifications.

According to the README for the tiling shell extension, the original plan was to propose integration of the tiling window management features to GNOME upstream. It is unclear if that ever happened, but a recent blog post about building the desktop indicates that it was unsustainable to maintain COSMIC as an extension, since upstream GNOME changes could cause the extension to break. The team also considered forking GNOME, but that would entail "taking on decades of legacy code and adapting it to fit what we want to build". That was ruled out, which left building a custom desktop from the ground up.

COSMIC components

The team working on COSMIC was already familiar with Rust and, according to the blog post, chose to use it for its "readability, stability, and memory safety". The developers chose the iced cross-platform graphical-user-interface (GUI) library, and created their own libraries for text handling, animations, and additional Wayland support. COSMIC uses its own Wayland compositor, which is built on top of the Smithay project. COSMIC's compositor does have XWayland support, so X applications should work just fine on the desktop.

The team has also created a cross-platform libcosmic platform toolkit for applications and applets that is based on iced. According to its documentation, the goal of libcosmic is to "enable the creation of a cross-platform ecosystem of applications that are easy to port from one OS to another". The docs for libcosmic are a work in progress, but interested developers will find a number of examples for creating COSMIC applications in its GitHub repository using the library, as well as an application template.

Desktop impressions

Even though COSMIC is no longer based on GNOME, it still takes a lot of visual cues from GNOME. The default applications and system tools seem to share GNOME's "less is more" approach to toolbars and user-interface elements. For example, COSMIC's file manager has just two navigation buttons (forward and backward), the ability to edit the filesystem path, and (of course) lets users click on directories and files to open them. It has only two file views: grid (thumbnails) or a list view that includes the modification date and size of the file. Other file managers like Mate's Caja or KDE's Dolphin offer several view options, a toolbar full of controls, such as "up" or "home" navigation buttons, and allow users to customize the icons/actions that are presented on the toolbar. Currently that is not possible with COSMIC's default applications.

Some of the desktop's minimalism may be due to the fact that some features are just not implemented yet. One would usually expect, for instance, right-clicking the mouse on the desktop would bring up options to change the background or manage display settings. Right now, right-clicking COSMIC's desktop does nothing. It seems unlikely that will be the case by the time the desktop has a stable release. Still, the overall theme seems to be "keep it simple" when it comes to the user interface.

Simple does not mean that the user can't have it their way, though. Like the GNOME-based iteration of COSMIC, the new desktop environment has a top panel and dock that can be customized to suit most users' preferences. Users can resize and reposition both, add or remove applets (such as the date/time widget, or a button to manage tiling), configure them to automatically hide until they are moused over, and even set light/dark mode for those elements independently of the desktop theme. If a user has a multi-monitor setup, the dock and panel can be set to appear on all monitors or a specific monitor. The settings also have a handy "reset to default" option in case customization gets out of hand. Users can choose to simply turn off the panel and/or dock entirely if desired.

COSMIC's tiling mode is the feature that is likely to be most compelling for users. Tiling is completely optional in COSMIC, and it is off by default. Users who do not like a tiling window manager can just ignore the feature and it will not get in the way. For those who do want it, though, COSMIC's implementation is flexible and user-friendly.

If tiling is turned on, it will attempt to automatically arrange windows on the desktop, but it is easy to adjust and override the automatic placements using the keyboard or mouse. In the extension-based tiling implementation, rearranging windows required a shortcut to start the "adjustment mode" and then use shortcuts to rearrange windows. Now, users only have to use the shift-super-arrow shortcut or just use the mouse to rearrange windows. (The "super" key is the one with the Windows logo on most PC keyboards these days.)

Windows can be stacked as well, by dragging one window on top of another and letting go. This creates a tabbed interface at the top of the window for switching between stacked applications. If a new application window is opened while a stack has focus, then the window will be added to the stacked window automatically. Finally, the tiling mode also supports floating windows. Users can click the title bar and select "Float window" or use the super-g shortcut to toggle the float mode for any window. Using super-f for "float" would have been easier to remember, but that shortcut is used to open a new file manager window. It is possible to change that in the keyboard shortcut settings, though.

For an alpha release, the tiling mode seems quite solid. The one complaint that some users might have is that tiling mode does not seem to work well with multi-monitor setups. In testing, tiling mode could only be activated for one monitor. Other than that, it seemed to work perfectly.

Desktop user-experience folks seem unable to resist the temptation to tinker with the behavior of alt-tab. Historically, one could expect that alt-tab would cycle through windows on the desktop. Now that behavior is desktop-dependent. Sometimes alt-tab does what one would expect, sometimes it cycles through applications instead of windows. In the case of COSMIC, though, it does an unexpected third thing: alt-tab brings up a list of up to eight application windows that the user can cycle through. If there are more than eight windows open, they simply are not displayed. Hopefully this convention will evolve to be more usable before the desktop reaches a stable release. Aside from that one grumble, though, COSMIC is already quite usable when it comes to window-management features.

The desktop only ships with two theme options at the moment, for light mode and dark mode. It also has a settings option to switch between modes automatically at sunset. Users can create and install custom themes as well. For example, Reddit user "Bilu47" posted a link to a set of custom themes that they had created.

The new COSMIC App Store seems more stable and responsive than the Pop Shop included 22.04 LTS. I've run 22.04 on one of my laptops for more than a year, and the Shop can be sluggish at times, and unstable too. Despite being an alpha release, the new App Store has been problem-free for installing, removing, and updating software. It can install applications from the Pop!_OS repositories and Flatpaks from Flathub, and it also lets users install updates. The catalog of software displays applications by category (e.g., "Create", "Work", "Develop", and so on), and includes basic descriptions of the applications. It is missing some information compared to the previous iteration, though, such as the application's homepage and license.

COSMIC supports fractional scaling for displays, including fractions less than 100%. This is an interesting feature for users who might want to squeeze a bit more desktop space out of smaller, lower-resolution screens. The options range from 50% to 200%. The 75% option for a 1920x1080 display produced good results while testing the desktop. The 50% option was too small, but that may have more to do with my eyes than the feature. Younger users may find it perfectly usable. Likewise the larger fractional settings worked well, though the desktop does not rearrange windows correctly within the available space. Increasing the display scaling may result in some windows being partially or completely off screen.

COSMIC $thing

In addition to the desktop environment's components, COSMIC comes with its own file manager, terminal application, and text editor. They all follow the naming convention "COSMIC $thing"—COSMIC Text Editor, COSMIC Files, and COSMIC Terminal. All of the applications are usable, but not quite full-featured enough for daily use. They do, however, provide a good showcase of the libcosmic toolkit for creating applications and applets for the desktop.

COSMIC's sample applications only have three top-level menu choices: File, Edit, and View. This means that users may not find things, such as settings, where they would normally expect to find them. COSMIC's homegrown applications also have a somewhat novel way of displaying settings and other secondary pages. Instead of opening a new window or a dialog for settings, etc., applications have a side panel that opens on the right-hand side of the window as seen in the screenshot to the left.

The terminal and text editor applications are fine for light use, but probably will not win over users who already have a favorite editor or terminal application. The file manager would need a few additional features to be suitable for daily use, such as the ability to choose from any installed application to open a file. Right now, selecting "Open with" for a file only displays a list of applications that COSMIC thinks are suitable for the file.

Users can set the color scheme for applications independently of the desktop, so even if the desktop is set to a light theme, one might choose to use the dark theme for the terminal application. This would also be useful when using COSMIC applications with another desktop environment, since they do not inherit their color scheme from GNOME, KDE, or other desktop environments.

Fragmentation

On the topic of running applications on different desktop environments: testing COSMIC as my primary desktop for more than a week really emphasized just how inconsistent the Linux desktop user-interface experience can be. Using Firefox, Claws Mail, Emacs, and other applications alongside native COSMIC applications, GNOME applications, and a few KDE applications, means having a bunch of inconsistent menu types, application title bars, and so forth. Claws Mail, for example, did not inherit the minimize and maximize title bar buttons in COSMIC, nor did Firefox. Some applications use COSMIC's native file dialog when opening files, while Emacs and the Kate text editor use different file choosers. The list of small, but real, incompatibilities is lengthy.

As a person who has been using Linux since the 1990s, I've grown accustomed to this and it is generally nothing more than a mild annoyance, if that. For users new to Linux, which would include much of System76's target audience, it is likely more confusing. This, of course, is not System76's fault or problem to solve alone, but the introduction of yet-another-desktop does exacerbate the fragmentation problem. The company is not wrong in creating COSMIC, by any means, but more unity (pun somewhat intended) on the Linux desktop would be a good thing for appealing to new users. Short of dropping all of the open-source-desktop teams into an arena and making them sort out their differences Highlander style, though, there doesn't seem to be a solution to the fragmentation problem.

Getting COSMIC

ISOs for Pop!_OS 24.04 LTS alpha are available in two flavors: one for systems with NVIDIA graphics chipsets (which includes the proprietary NVIDIA drivers), and one for systems with Intel or AMD graphics. If the system has hybrid graphics with Intel/NVIDIA, choose the NVIDIA ISO. There is a Raspberry Pi version of Pop!_OS for the current stable release, but so far there is no alpha ISO for Raspberry Pi.

The company has been open about wanting to see COSMIC adopted by other Linux distributions, and the desktop is already packaged for a number of them. There are instructions for installing COSMIC on Arch, Fedora, NixOS, or Serpent OS. OpenSUSE is listed as "coming soon" on the landing page for COSMIC, and Redox OS (which LWN covered in June) already includes some COSMIC components. One note for users who try out the desktop on another distribution: the packages may be behind those available for Pop!_OS.

Users should temper their expectations before installing and testing COSMIC. It is reasonably stable for an alpha release, but it does have plenty of minor bugs, usability paper cuts, and enough instability that it's a bad bet for a production system. It has problems waking from the suspend state, as just one example. This is a known problem, so it is likely to be fixed soon, but it is no fun having to restart a laptop every time its lid closed.

A smaller, fixable, issue is that it's necessary to set the system time zone manually using timedatectl or similar:

    $ sudo timedatectl set-timezone 'America/Denver'

All of those problems, of course, are to be expected from an alpha-quality release. Users who would like to test COSMIC and give feedback are encouraged to join the Pop!_OS Mattermost server. Note that this requires setting up an account on the server.

Grumbles about desktop fragmentation aside, COSMIC is an ambitious project and shows a lot of promise. It also has a great deal of maturing to do before it will be suitable for daily use, but it has made significant progress in just two years. Watching it evolve, and seeing whether users and developers take to it, should be interesting.

Comments (58 posted)

Changes coming in PostgreSQL 17

The PostgreSQL project has released beta versions of PostgreSQL 17 containing several interesting security and usability improvements, alongside the usual performance improvements and bug fixes. If the release proceeds according to the usual timeline, the full release of version 17 is expected in September or October. The most important changes are in what PostgreSQL does when a database supervisor has their credentials revoked, and added support for incremental database backups.

Security

PostgreSQL 17 includes a few changes that could affect compatibility, in the name of security. Most noticeable is a change by Jeff Davis to how functions invoked during maintenance operations locate objects in the database. Normally, when an SQL statement refers to a table, data type, function, or other object by a name, PostgreSQL determines which object is being referred to using a search path that defines the order in which schemas are searched. This presents a possible security risk: if someone could influence the contents of the search path, they could make a privileged database operation call a function of their design with the same name as some built-in function. So, in PostgreSQL 17, privileged maintenance operations will default to using a safe search path — either the default, which should be fine for most users, or a search path declared during the creation of a function, so that the function author can be sure that each name will always refer to the same object.

That isn't the only change that makes running maintenance operations safer, however. Nathan Bossart added a new permission, MAINTAIN, that can be granted per-table or per-role. This means that now maintenance operations don't need general administration privileges, and the permissions can be more narrowly scoped. Unlike the search-path change, this change is backward compatible — although it may prompt some administrators to restructure their permission systems.

Another security-related change is a tweak to how PostgreSQL handles the SET SESSION AUTHORIZATION command that allows superusers to temporarily act as other users — the database equivalent of su. In previous versions, anyone who was a superuser at the time their database connection opened could issue the command. This led to the theoretical problem of someone opening a privileged connection, losing their superuser status, and then using their open connection to get it back. Of course, the far more common case is that of a new administrator having to close and reopen their connection to be able to use it. In PostgreSQL 17, Joseph Koshakow fixed this behavior and the command only cares about a user's superuser status at the time it is issued.

A similar issue affecting logical replicas has also been fixed by Vignesh C. If the user who owns a logical replication subscription has their superuser access revoked, the process that applies the updates for logical replication, the apply worker, will now be automatically restarted, instead of continuing to run with superuser-level access.

Administration

There is also one feature that "must not be regarded as a security feature" — a configuration option to disable ALTER SYSTEM that LWN covered in April, by Jelte Fennema-Nio and Gabriele Bartolini. The feature is a boon for system administrators, however, because it allows them to ensure that PostgreSQL is only configured via files, not via ad-hoc invocations of ALTER SYSTEM.

Administrators may also enjoy the addition of incremental backups to the pg_basebackup tool by Robert Haas, Jakub Wartak, and Tomas Vondra. Incremental backups are likely to be faster than full backups for most databases, since they only need to record pages of the database that have changed. Which pages have changed is determined using the database's write-ahead log (WAL) — the log of database operations that is used to ensure that writes to the database are durable. A new background process reads the WAL files and produces a summary of which pages have changed. The administrator can configure its behavior using some new settings. Incremental backups may not be possible if something is slowing that process down too much (such as the database dealing with too many writes).

Other complications include the fact that PostgreSQL does not track the relationship between incremental backups — that responsibility is left to the server administrator. They must retain the last full backup, and all of the incremental backups made after it, and the WAL segment and history files that must be retained for a normal backup as well. This complexity is somewhat nerve-racking to see in a backup system, but the project does provide pg_combinebackup, a utility that can turn a full backup and a series of incremental backups into a full backup. So one possible workflow would be to take incremental backups, and use them to update an existing full backup offline, without impacting the operation of the database.

Finally, PostgreSQL 17 adds some additional options to the COPY FROM command and EXPLAIN queries. The COPY FROM command allows users to import data from a file into a table. It now has the option to discard rows that would cause an error (such as failing to parse a given type, or causing a constraint violation), as well as options to increase the verbosity of what is logged about the copying process. These changes required more work than one might assume; the final version included in the release ended up including changes from Damir Belyalov, Atsushi Torikoshi, Alex Shulgin, Jian He, Yugo Nagata, and Bharath Rupireddy.

EXPLAIN can now be asked to report the memory used by the SQL-query optimizer during execution of a query, and to separately report the cost of converting data for transmission over the network. Its output has also been cleaned up slightly, which should hopefully make EXPLAIN results easier to read. This work also involved contributions from many people: Ashutosh Bapat, Stepan Rutz, Matthias van de Meent, Nazir Bilal Yavuz, Tom Lane, and Dean Rasheed.

Optimizations

Databases are usually made more performant in one of two ways: optimizing the basic operations of the database, or making the SQL-query planner smarter, so that the database can do fewer basic operations in the first place. PostgreSQL 17 includes both kinds of optimization. Vondra and Van de Meent made it so that block-range indexes can be built in parallel; Peter Geoghegan and Van de Meent made using B-Tree indexes more efficient when looking up groups of values.

Vacuuming the database — removing unused rows and pages from the middle of a table — has also become much more efficient. Melanie Plageman, Heikki Linnakangas, Masahiko Sawada, John Naylor, and Thomas Munro all contributed various fixes that should speed up vacuuming in one way or another. The amount of memory used during vacuuming may go up slightly, because the default buffer size has been increased.

Sometimes developers will use object-relational mappers, or other abstractions that sit on top of the database. Those abstractions can introduce redundancies in their SQL queries that a knowledgeable human is unlikely to write by hand. Some of the improvements to the optimizer in PostgreSQL 17 are aimed at removing those redundancies. For example, David Rowley, Richard Guo, and Andy Fan made it so that IS NOT NULL and IS NULL checks can be automatically removed from queries when a column is known to be NOT NULL. Rowley also added partition pruning for IS UNKNOWN and IS NOT UNKNOWN conditions on boolean columns.

The other improvements are more generally applicable, and include things such as allowing for greater parallelism, allowing some kinds of sub-queries to be converted to joins, and allowing the optimizer to consider the statistics of columns referenced in previous clauses. Overall, users of PostgreSQL should expect to see some modest speedups on queries where the new optimizations apply. The project's recommendations for getting good performance from a database remain the same — primarily, that one should start by running their queries through EXPLAIN to see what steps the database will actually have to go through to answer the query.

Overall, PostgreSQL 17 is more of the same from the PostgreSQL project — a large release with improvements to performance, maintenance, and security, that nonetheless preserves backward compatibility in most cases. Information on where to download the beta release can be found on the PostgreSQL snapshots page. The project does not recommend upgrading production servers yet, but does recommend installing the beta release in non-production environments to test it and ensure there are no lurking regressions.

Comments (11 posted)

Meeting the Debian Technical Committee

It is something of a DebConf tradition that members of the Debian Technical Committee (TC) take the stage to talk about the work that the committee does—and more. DebConf24 in Busan, South Korea was no exception, as TC chair Sean Whitton, who will complete his term at the end of the year, and one of its newest members, Stefano Rivera, described the constitutional underpinnings of the TC, how it tries to make decisions when it needs to, and the constant process of recruiting new members. After that, they took a few questions from the audience. The session provided a nice overview of the TC and its role in Debian, but it may well be of interest further afield.

The presentation was given by Whitton, who began with something of a "roll call", listing the eight current members of the committee. He also recognized and thanked former members, both those who had been appointed after the general resolution that limited terms to around four years and those who had served prior to that. There are some whose original appointment dates are lost in the mists of time, he said.

Powers

The powers that the TC has are enumerated in the Debian Constitution, specifically in section 6.1. The committee is tasked with making technical policy decisions for the distribution; the TC determines "how policy ends up shaking out in packages in the real world", he said. When developers' jurisdictions overlap, and they disagree, the TC can make a decision between them.

Normally, the TC is approached by both (or all) sides in a dispute, asking for a resolution to their disagreement. Sometimes, though, the TC has to decide whether to overrule a developer at the request of another. That requires a 3:1 majority vote in the committee, so it does not happen often.

In addition, the TC has an advice-giving role. The committee will often offer advice, rather than making a decision, especially if the members feel that it is too early to force a solution. Even if the dispute was not brought to the TC requesting advice, it often chooses to offer advice, instead of taking another course of action, Whitton said.

There are some constraints placed on the committee by the constitution as well. All of the discussion of an issue before the committee must be done in public, for one thing. The TC is meant to "choose between options on the table, as opposed to coming up with new ones", which is a "pretty significant restriction" because it means that if there are no good options offered, the TC has to bow out. The committee is only meant to be used as a last resort, but it has to decide "whether it is a 'last resort' yet, that's part of the challenge", he said.

Another constraint, which is sometimes surprising to Debian developers, is that the TC cannot override the decision of delegates appointed by the Debian Project Leader (DPL). So, for example, a dispute with the release team cannot be resolved by the committee, since that team is appointed by the DPL.

Interestingly, the TC is the only organization in Debian that can change the maintainer of a package; even if the Debian Account Manager (DAM) team determines that someone is no longer a member of the project, the developer can still upload their packages. But the TC never uses this power, Whitton said, which means that it is "this informally defined role of the 'package maintainer' who ends up having almost all of the power unless there is a relevant delegate". It is a somewhat odd consequence of the way the powers are distributed in the constitution.

Technical versus social

The TC is meant to "decide technical matters and not social matters, but that almost never happens"; things that come before the committee "almost always have a social component". The recent TC bug report around the question of distinguishing the testing and unstable versions is likely a case in point.

Ian Jackson wrote the Debian constitution as a graduate student in the 1990s, Whitton said; Jackson had this idea of separating technical from social, "but that doesn't work". The TC tries to coordinate with the DAM and Community teams to handle the social side of things but, each of the organizations has its own point of view, so it is difficult. This attempt to separate the technical from the social is considered by Jackson to be "a great mistake" in retrospect, Whitton said. There are advantages to the split, because some TC members, including him, would not be interested in serving on the DAM team, for example, "but it also causes us no end of problems".

There are some other projects that do not have that separation or have it to a different extent. He noted that Matthew Garrett, who is a current TC member, has also served on the Fedora Engineering Steering Committee (FESCo), which has a role in setting the technical direction of the project beyond the option-choosing role of the TC; he was not sure what role FESCo played in disciplining project members, however. The FreeBSD project has a core team that is elected for two- or three-year terms; it is "explicitly empowered to deal with issues that are both technical and social". Whitton thinks that Debian should perhaps look at other projects for ways that it could improve its situation with regard to the intertwining of technical and social considerations in intra-distribution disputes.

Over the four years or so that he has been on the TC, it has considered changing how it does its work, but its members do not want to do so unilaterally. The project as a whole should be involved in any kind of change like that. Meanwhile, the members of the committee are also busy with the day-to-day work of the TC. He is not quite sure where that should lead, exactly, "but that's the situation".

To summarize, the TC is meant to break ties when solving the problem by consensus has been tried and failed; it can also offer advice. It is a "self-nominated, DPL-appointed, last-resort conflict resolution and advice-providing body".

Decisions

Whitton went through the items that the TC has considered, or decided since the last DebConf. Two of those were procedural in nature: recommending Craig Small for the TC and re-electing Whitton as TC chair. Whenever the TC changes, as it did when it added Small, the chair resigns and a vote is taken on a new one. As can be seen in the TC-chair "bug" thread, there are plans to transition to a new chair sometime well before Whitton's term expires at the end of the year.

The TC bug filed that asked for a reconsideration of merged-/usr was considered and rejected. In another procedural decision, the committee repealed its moratorium on moving files to /usr. Another bug was a request for advice from the TC on a "pretty complicated issue about a couple of interacting packages"; the committee discussed it and supplied advice. From the bug, it would seem that the packages supply different versions of the same file.

There were also three bugs that were closed without a resolution; two of those were closed "because we thought they were premature", since there were no options to choose from and the process of finding consensus had "not been tried hard enough" (1052460 and 1052697). Another, 1065170, "resolved itself while we were still talking about it, so that's always nice".

There is one open issue, 1065416, which is a "classic overlapping-responsibility issue", Whitton said, where the owners of two packages disagree about "who has ownership over an API". The committee is trying to mediate and is hopeful that the problem can be solved without a formal resolution by the TC.

New blood

He turned back to term expiration, noting that the algorithm for who expires and when is a little complicated in order to ensure that no more than two people can expire at once, he thinks. The slides and WebM video of the session will show the list of expirations that he created; the upshot is that there is a need for one or two new people for the committee every year.

In fact, the committee spends a lot of its time recruiting new members, Whitton said, because a lot of people do not want to do that work. "We wanted to say that it is not as bad as people sometimes think it is". For one thing, not every issue is like the init system decision (which was resolved by the TC in 2014, but has cropped up multiple times since then). For another, given the existing structure, there is not all that much that the TC can actually do, which may be a different problem, he said, but it does keep the workload manageable.

The TC meets once a month for an hour, generally on IRC; "we look at each open issue and see if we can move it forward". Based on that, assignments of various tasks are made to members to try to take those next steps. The work gets divided pretty evenly between the committee members and the intent is to make steady progress. Sometimes there is a lot of email to read, but since the TC works as a group, the focus of the work is in the meetings; that may mean that the committee moves too slowly, but "that's the reality of it". There are a number of skills needed, "given that it is mostly about resolving disputes", but they mostly boil down to dealing with the people involved in good faith.

There is a need for more diversity on the TC, he said. On the train that morning, he was chatting with someone who had turned down a nomination to the committee, in part because of the amount of English-language writing that gets produced for disputes that the TC is handling. While that's true, Whitton said, it means that only people from English-speaking cultures end up on the committee, "which sucks". In truth, he thinks that "all of Debian should probably write less", but if the volume was reduced for TC disputes, in particular, that might lead to more people being willing to join the committee.

Debian members can nominate themselves or someone else for the TC. The self-nominations are preferred, mostly because it is clear that the person is willing to serve; nominating others means the committee has to check with them and "they often say 'no'". Normally, the committee members are deciding between three people or so each time, but it would be better if it could be five or six candidates, say. Those who are not recommended in a given year are often asked to return the following year, which works out well.

Questions

DPL Andreas Tille opened up the Q&A section by thanking the TC members for their hard work; given the technical-social overlap, he wondered if it made sense to have a "Debian Social Committee", which had been brought up in the debian-private mailing list along the way. Whitton said that, in theory, the Community team is meant to fill that role, though its former name, the Anti-Harassment team, points to the focus of the team. He reiterated that he thought Debian should look at what other projects are doing to try to learn from their experiences. Rivera added that the issues that reach the TC are really about a lack of ability to communicate between the parties; there is never a situation where there are two perfect solutions, so it should be about finding some middle ground, but that is obviously failing in those cases. All of that is inherently social.

An Ubuntu member in the audience offered to explain that distribution's ways of handling conflicts of this nature. There is a community council and a community team at Canonical, with some structures and procedures about how they operate, he said. Whitton and Rivera agreed that a discussion of that sort would be helpful.

Junichi Uekawa said that he might be interested in volunteering, but wondered how much work it entailed. Once a month sounds reasonable, but "do you have to read like 1000 pages" every month as well? Whitton said that the committee "asks people to summarize, we're generally not willing to read the entire dispute"; each side is expected to provide a summary of the dispute from their perspective, "so it's not that bad". What is needed is people who can jump into a discussion quickly when it arises, which might require "a few hours in any given week".

Rivera said that TC bugs "tend to come to the committee with a lot of text"; the submitter typically tries to explain why they are bringing the issue to the committee "and that means a long read". There are usually relevant mailing-list threads, as well; waiting for the members to read all of that would significantly slow things down. Fortunately, he said, it is often the case that one member researches the problem for the committee, and then condenses out the potential solutions for the others; they become something of a subject-matter expert for a given dispute. "No one on the committee expects anyone else on the committee to read everything", Whitton said; "it would be impossible", Rivera finished.

Former DPL Jonathan Carter noted that Whitton had mentioned some areas of difficulty for the TC and, perhaps, indicated some level of unhappiness with how the committee, and the distribution as a whole, are working. Given the opportunity, would Whitton "fix a few things in TC or, rather, start the discussion to redesign what our top-level problem-handling methods are?" Whitton said that "no one is happy with Debian's governance, right? We're really bad at making changes and we all know that we need to make changes."

He said that he is no expert in governance; if making changes was up to him, he would try to become an expert by looking "at what everybody else does and see what works and what doesn't work and try to integrate it with our stuff". He would not be comfortable making any recommendations before doing that research.

Rivera said that Debian is different than other distributions in that it does not have a corporation providing overall direction, so "we are largely rudderless". Occasionally, a DPL candidate will have a technical goal as part of their platform, but "a lot of the project looks down on that" and prefers a more administrative DPL. Without some kind of constitutional change, the TC cannot provide technical leadership for Debian and "I'm not sure that having some graybeards drive the direction of the project is what Debian wants".

With that, the session ran out of questions and/or time. The talk definitely gave a nice look inside the TC and some of the challenges it faces. In keeping with Whitton's interest in seeing what other projects are doing, this presentation can perhaps return the favor; one of the most obvious benefits of governing in the open is in sharing the successes and failures between projects. It is, effectively, applying the principles of FOSS to project governance, so projects can learn from each other—both for code and organizational matters.

[I would like to thank the Linux Foundation, LWN's travel sponsor, for its assistance in visiting Busan for DebConf24.]

Comments (4 posted)

Distinguishing Debian testing from unstable

Sometimes, the smallest changes create the longest discussions. As a case in point, a proposal to make a one-line change in an informational text file on systems running the Debian unstable distribution has blown up into an interminable and sometimes unfriendly debate. At its core, though, this discussion comes down to a seemingly simple question: should a program be able to determine whether it is running on a Debian testing or unstable system?

Like most distributors, Debian maintains a few versions of its distribution, some of which are more stable than others. The Debian project creates a major stable release approximately every two years; this release will receive three years of support from the project, followed by two years of reduced long-term support. The current stable Debian release is Debian 12, called "bookworm". Development of future releases is done in the unstable distribution, also known as "sid". While it is possible to run an unstable system for real-world use (and many people do), one has to be prepared for things to break on occasion. A somewhat gentler experience can be had from Debian testing, which receives packages that have lived in unstable for a period without causing serious problems.

The testing distribution is essentially the next major Debian stable release — it's just not quite done yet. It already carries the next release's code name ("trixie" at the moment); a user who installs trixie now will get a system that will evolve and stabilize into the stable release.

These distributions, like almost all others in recent times, include a file called /etc/os-release that tells interested software what kind of system it is running on. This file has a standard format that all distributions are expected to follow. The full contents of /etc/os-release on a Debian bookworm system are:

    PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
    NAME="Debian GNU/Linux"
    VERSION_ID="12"
    VERSION="12 (bookworm)"
    VERSION_CODENAME=bookworm
    ID=debian
    HOME_URL="https://www.debian.org/"
    SUPPORT_URL="https://www.debian.org/support"
    BUG_REPORT_URL="https://bugs.debian.org/"

The idea is that any program that needs to change its behavior depending on the type of system it is running on can consult this file and act accordingly. In Debian systems, /etc/os-release is shipped as part of the base-files package.

Bringing in the Technical Committee

On August 1, Luca Boccassi, frustrated with how a bug discussion was going, approached the Debian Technical Committee with a long-simmering disagreement; he requested a decision that would override the base-files maintainer and remove /etc/os-release from that package. The Debian implementation of that file, he asserted (as the maintainer of the os-release specification), is buggy. Specifically, it does not properly identify the testing and unstable distributions, making it impossible for software to distinguish between the two using only /etc/os-release. The problem, he said, is real:

These issues are not just theoretical, and do not concern mere personal preferences or cleanliness or code quality or ugliness. They cause very real, very actual and very painful grief for Debian users, as evidenced by the multiple independent bugs with multiple independent reporters chiming in, and the multiple ugly hacks that have to be implemented as Debian-specific workarounds.

He complained specifically that the code name in unstable is currently set to trixie, which he sees as being misleading at best. The testing distribution will indeed become the next stable release, he said, at which point it will gain security support, then eventually proceed toward its end of life. It is essentially trixie, so use of that code name there is appropriate. Unstable, instead, will never become stable, will never have security support, and will never reach end-of-life. Software running on these systems should be able to tell what the expected life cycle for the distribution is; using the trixie code name there makes that impossible.

He suggested that unstable should have the following lines in /etc/os-release:

    PRETTY_NAME="Debian GNU/Linux sid"
    NAME="Debian GNU/Linux"
    VERSION_CODENAME=sid

While testing should have:

    PRETTY_NAME="Debian GNU/Linux 13 (trixie)"
    NAME="Debian GNU/Linux"
    VERSION_ID="13"
    VERSION="13 (trixie)"
    VERSION_CODENAME=trixie
    RELEASE_TYPE=pre-release

The RELEASE_TYPE would change to lts once the trixie stable release happens. The key change, he said, is the VERSION_CODENAME line, which tells software what kind of Debian system they are running on.

The number of voices supporting this change was relatively small. Gioele Barabucci, the maintainer of the lsb_release implementation, said that similar issues had come up with that package. As of the bookworm release, lsb_release obtains all of its information from /etc/os-release, so it, too, cannot distinguish between testing and unstable. Users have requested that ability, he said, and may well bring the issue to the Technical Committee if the current discussion does not give them what they want. Marc Haber said that he "would not cry" if he were able to remove his current workarounds for this problem.

Disagreement

Needless to say, there is opposition to this change, or the discussion would have never reached the Technical Committee. Numerous participants said that they simply did not see the benefit of this change, and that Boccassi had not provided examples of where the distinction was truly important (a position that Boccassi disagreed with). Russ Allbery said: "I admit to not finding this use case compelling", adding that the proposed use cases come from "misunderstandings of how Debian works". The packages that are in unstable today will be in testing tomorrow, perhaps with no change at all; what was actually of interest, he suggested, was which repository new packages for a given system would come from. That information can be found by parsing the package-management configuration.

As Simon McVittie described, many Debian developers see unstable as "a tool to be used as part of the process of making our next stable release" rather than a separate distribution in its own right. To treat unstable differently in its /etc/os-release file would challenge that view of how things work; that is a step that many developers would rather not take.

In more technical terms, McVittie laid out how the proposed change could lead to trouble:

That also breaks some of the design principles of the unstable -> testing -> stable flow, which are that testing should be approximately equivalent to a previous state of unstable (one where everything worked acceptably together), the next stable should be functionally equivalent to a late-freeze version of testing, and as little as possible should change as we cross release milestones, to avoid perturbing packages' behaviour.

He later gave a specific example of how changing /etc/os-release could create regressions as packages move into testing. In short, a package that changes its behavior depending on the release code name might work in unstable, but misbehave once it transitions to testing and sees a different code name.

Allbery also described the idea of testing and unstable being different distributions as poorly defined. A system installed from the trixie repository will evolve into the next stable release and go through the life cycle described by Boccassi. A system installed from the testing repository, instead, will begin with the exact same set of packages (since trixie and testing are currently synonyms), but will move forward to the next release (to be called forky) once trixie is released and will have a different life cycle. It is not really possible for a single /etc/os-release file to describe both scenarios:

This sort of ambiguity is, I think, part of why this proposal generates so much discussion. Debian simply doesn't currently have clean semantics for testing. It exists in a sort of quantum superposition where it is multiple things simultaneously for different people, and this proposal is trying to label it in a way that collapses that state to match the mental model of one group of people, invalidating the mental model of a different group of people.

Boccassi disagreed with all of these positions, of course; he summarized the situation as "a trivial technical problem with a hard social conflict at its core".

The trial

What Boccassi did not do was change many (if any) minds over the course of this sprawling discussion. On August 6, Technical Committee chair Sean Whitton called for a vote on whether the committee should simply close the request; the operative language was: "The Technical Committee declines to overrule the maintainer of base-files, or issue any advice on issues concerning /etc/os-release". Boccassi, evidently surprised by this move, described it as "like something out of a Kafka novel". All committee members who voted did so in favor of the resolution, so the issue was closed with the committee declining to weigh in on the topic.

So it seems that this lengthy discussion is likely to result in no changes to how Debian maintains its /etc/os-release file. The Debian project's view of its testing and unstable repositories is apparently not compatible with treating them as independent distributions with their own identity, and most Debian developers do not see a valid reason why software should be able to tell the difference between them. While the discussion may wind down for now, though, this has the look of one of those topics that never completely go away. The proposed change is small, but it is a big deal to the people involved.

Comments (95 posted)

Standards for use of unsafe Rust in the kernel

Rust is intended to let programmers write safer code. But compilers are not omniscient, and writing Rust code that interfaces with hardware (or that works with memory outside of Rust's lifetime paradigm) requires, at some point, the programmer's assurance that some operations are permissible. Benno Lossin suggested adding some more documentation to the Rust-for-Linux project clarifying the standards for commenting uses of unsafe in kernel code. There's general agreement that such standards are necessary, but less agreement on exactly when it is appropriate to use unsafe.

Lossin's proposed documentation starts by describing the purpose of the unsafe keyword in Rust: to explicitly signal when code relies on guarantees that are not checked by the compiler. Since programmers writing unsafe code are relying on properties that the compiler cannot help check, it is especially important to make sure that the properties are documented: both what they are, and why they hold.

These two different types of information about program safety actually correspond to two subtly different uses of the unsafe keyword in Rust, Lossin points out. Sometimes unsafe is used to signify that the programmer knows that they're using an operation the compiler can't guarantee is safe, and sometimes it is used to inform the compiler that it cannot fully understand when a newly-defined function is safe to call. Each kind of use comes with different things that should be documented; the former should have an explanation of why the operation is safe in this case, while the latter should explain what the requirements are to use the function safely.

The examples given in the patch set are somewhat sparse, however. Alice Ryhl asked for the documentation to show how to do it right:

I think it is worth explicitly pointing out that the safety comment must explain why the preconditions are satisfied, *not* what the preconditions are. It's a really really common mistake to mix up these, and it probably even makes sense to include two examples showing the difference.

Lossin agreed, noting that if his proposed documentation standards are acceptable to the other kernel developers, he plans to go through the tree and try to improve some existing safety comments, which should provide a good source of examples of what to do or not do.

The patches highlight several important aspects of writing safety documentation for a Rust project — such as the difference between safety and soundness. In Rust, safe code has several specific guarantees. It won't access uninitialized memory, have data races, mutate immutable constants, or engage in any of the other behaviors considered undefined in Rust — provided that all unsafe code used by the program is "sound". Unsafe code is considered sound when no combination of values passed through a safe abstraction (like a function call that is not marked unsafe) can make the unsafe code engage in undefined behavior.

Lossin's documentation notes: "The term undefined behavior in Rust has a lot stricter meaning than in C or C++: UB in Rust is totally forbidden." To be able to truly rely on Rust's safety guarantees, all unsafe code must be sound, not merely rely on the current behavior of the compiler happening not to cause problems. Boqun Feng worried about how that requirement would interact with the Linux kernel memory model (LKMM):

Basically, since LKMM relies on a few things that C standard doesn't say, e.g. volatile accesses on certain types are atomic, behaviors of asm blocks, dependencies. Let alone we have data_race() where for example, the diagnostic code accesses the shared variable out of the core synchronization design.

Ultimately, Feng said, it would be better to teach Rust about some of the special behaviors that the Linux kernel relies on. But right now, that's simply not feasible, and sometimes the developers will need the freedom to act outside of Rust's constraints, he continued. Lossin disagreed, saying that the Rust code in the kernel was an opportunity to start with a clean slate, and not to need to rely on any compiler-specific behaviors. He did suggest it might be reasonable to talk about exceptions on a case-by-case basis, but "'sometimes UB is actually ok' is something that I don't want to accept in Rust as a general statement."

Daniel Almeida approved of adding additional documentation, but thought that just documentation was not enough, suggesting that the project should use a linter to ensure that there are comments in the appropriate places. Miguel Ojeda pointed out that Clippy already supports such a check, so it would be easy to add to the kernel's build. A later message said that Ojeda had gotten consensus on it from other developers and would be posting a patch set to do so.

Almeida also raised concerns over a section of Lossin's documentation that exhorted developers to centralize the use of unsafe blocks, in order to make reasoning about their behavior easier. "By all means, let's strive to push as much of the unsafe bits into the kernel crate. But, IMHO, we shouldn't treat Rust drivers as some unprivileged entity, they're also kernel code, after all."

Ryhl disagreed with Almeida's interpretation, saying that Lossin's documentation is not as strict as Almeida makes it out to be. Danilo Krummrich likewise thought that there was no real problem with Lossin's recommendation, since Rust makes it possible to build abstractions that cover the most obvious use cases for unsafe in drivers. Everyone did agree, however, that this would likely make a good topic of discussion at Kangrejos, the upcoming Rust-for-Linux conference.

In any case, there is more work to be done before this documentation change is likely to be accepted. Lossin has already stated that he plans to incorporate some suggestions where the documentation is unclear, as well as adding more examples, and bringing the existing code up to the standard set in the documentation. All of those tasks are likely to be easier, however, than getting the Rust-for-Linux developers to agree on exact standards for unsafe code — a task that will certainly require further discussion.

Comments (97 posted)

Endless OS aimed at educational and offline environments

Endless OS is a Linux distribution with a focus on improving access to educational tools by providing a simple-to-manage, full-featured desktop for educators and students — one that works offline, with minimal maintenance. The distribution also aims to be suitable for older devices, in order to promote access to computers by ensuring those systems remain usable. In pursuit of those goals, it makes some unusual technical choices. But what makes the distribution really shine is its curated collection of software and educational resources.

Endless OS is developed by the Endless OS Foundation, a non-profit organization that also provides laptops to people who would otherwise be unable to afford them, in support of its mission to ensure widespread access to educational tools. Unlike Linux distributions that grew up around a group of volunteers, Endless OS actually started as the product of a VC-funded startup. The foundation only became a non-profit in 2020. This gives the project a noticeably different culture and approach. People are welcome to participate in the Endless OS community, but most of the operating system's development is driven by employees of the foundation. While Endless OS is free for personal and non-commercial use — and the core operating system is Debian-based, made available under its normal licenses — the foundation makes some of its money by selling licenses and support (including management software for large fleets of devices) to companies that want to use the OS.

Unfortunately, many of the applications that are built on top of the base OS have proprietary licenses. Installing the OS involves agreeing to an end-user license agreement (EULA) covering the proprietary software distributed with the OS (among other terms). The project also doesn't permit modifications to the install media. Its redistribution policy specifically allows making unmodified copies, or building completely free images with the project's image builder, but not modifying the official distribution.

Endless OS 6 was released in May 2024. Unlike many other Linux distributions, which usually recommend the use of generic installation-media creation programs, Endless OS provides a customized Windows installer. Although I lack a Windows computer on which to verify this, the installer purports to be able to create bootable USB drives, replace Windows as the computer's operating system, or install Endless OS alongside Windows. This attention to ensuring that it is possible for non-technical users to try out the operating system with a minimum of friction is a good example of the project's dedication to making Endless OS easily available.

For those who are already familiar with installing other Linux distributions, the project also provides plain ISOs, virtual-machine images, and some alternative images tuned for single-board computers such as the Raspberry Pi. There are also multiple versions available that are tuned for different installation environments and languages. The "basic" variant, intended for installation on computers with consistent internet access, comes without too many additional applications installed. The "full" variant, which comes in four flavors for English, French, Brazilian Portuguese, and Spanish, comes pre-loaded with applications and educational materials so it is suitable for use in areas where internet connections may be sporadic or nonexistent.

If those images are not sufficient for a particular application, the project also provides the Endless OS image builder, which lets users create their own installation media with whatever customizations they need. All of Endless OS's images are built using libostree to provide an immutable root filesystem. The image builder builds the root filesystem in a container out of custom Debian packages and Flatpak applications. Once the filesystem is finalized, the builder converts it to a bootable ISO.

The choice to use an immutable root filesystem also makes automatic updates smooth. Updates, including security updates, are downloaded automatically by default, and applied atomically to the root filesystem when the computer is rebooted. Flatpak applications running on top of the core OS are updated separately, but also automatically.

Installation

Most people, however, will find that the default images built by the project meet their needs. Because the ISOs come with a large number of applications pre-installed, the "full" variants are quite large. The English flavor comes to 24GB. The setup process is fairly simple, with a custom graphical installer. Upon first booting the installer — after displaying an animated logo as a boot screen — it asks the user whether they would like to try Endless OS live, or install it. Choosing the latter option has the installer scan the computer for any other operating systems to set up dual booting, before offering the choice of which disk to install to. Once a disk is chosen, the installer copies the full OS image to the disk. Configuration of language, timezone, etc. are done once the user has booted the installed system as part of a first-time setup wizard.

During installation, the EULA also warns that Endless OS periodically reports the computer's location, how Endless OS was installed, and how long it has been installed — and that this data collection is "required". The OS may also collect and report other information. The EULA may be out of date, however; the metrics daemon itself — and the less elaborate script that sends daily pings to measure retention — are open source (GPLv2 or later) and do not appear to upload anything if the user opts out. [Update: The daily ping script doesn't respect the opt-out setting.] Will Thompson, an employee of the Endless OS Foundation, wrote a blog post explaining their metrics system in more detail.

Generally, the installation process is smooth, with fewer options for advanced users than it is usual to see in a typical Linux distribution. Part of this is made possible by the fact that Endless OS's immutable root filesystem greatly simplifies deployment — the installer has no need to copy or set up packages individually, instead it simply copies the image wholesale and sets up some configuration files.

The software

Endless OS distinguishes itself by offering a variety of pre-installed software. The basic desktop environment is GNOME 43.9. Upon first logging in, the user is presented with a phone-like full-screen menu of applications. The project supplies Chromium 125 as the default browser, alongside typical software such as LibreOffice 24, Shotwell 0.32, GNOME Terminal 3.46, but also less typical software, such as "Encyclopedia", a dedicated application for accessing Wikipedia.

The OS also comes with a number of educational applications that package content from places like Wikipedia, wikiHow, and TED-ed into easily searchable themed bundles on different topics. Examples include "How To", "Cooking", "Sanitation", "Health", "Myths & Legends", "Travel", "Math", and more. These themed bundles (and the unthemed Encyclopedia application) are the primary proprietary components of the operating system. There is also more interactive educational software in the form of coding environments for Scratch, Arduino, and basic web programming.

Programming is not the only skill that Endless OS supports acquiring; it also bundles drawing programs (including Inkscape, GIMP, and Tux Paint), 3D-modeling software (Blender), and audio recording and editing software (Decibels, Audacity). The flagship application in this category, however, is certainly Endless Key, which provides articles, videos, and tutorials about different skills and hobbies curated for the user's interests.

Finally, the OS also comes with a selection of games. In case the included software does not prove sufficient, the OS has GNOME Software, which allows installing additional applications using Flatpak. While Flatpak applications are often larger than native packages, separating out the applications and keeping the core OS immutable lets Endless OS installations be made much more robust, simple to maintain, and to deploy at scale. Layering Flatpak applications on top lets users enjoy the freedom to install the software they need without compromising the stability of the base OS.

Despite the obvious educational bent, however, Endless OS does have some features catering to advanced users. The OS includes Toolbx, a tool that makes setting up interactive containers that can run software from other distributions trivial. Toolbx containers use the same Linux kernel, but allow the user to run Debian, Ubuntu, Fedora, or other user spaces on top of it — including graphical applications. While this is not perfectly seamless, it does make it possible to comfortably use Endless OS for software development, which would otherwise be difficult, since the base OS does not include normal development tools or compilers.

Overall, Endless OS occupies an interesting niche compared to traditional Linux systems. Its simple installation and immutable base make it a good choice for computers that receive minimal maintenance. Its large amount of bundled educational content, including tools for artistic expression, make it well-suited to letting children explore. Since it exposes containerized environments, it should be capable of running almost any necessary Linux software. Whether these features balance out the presence of proprietary software and its other non-free components and policies will depend on individual people's priorities. While I doubt that many LWN readers will be interested in switching to Endless OS as their main operating system, it presents an attractive option for less-technical people who want a capable and well-curated Linux desktop.

Comments (13 posted)

Zettlr: note-taking and publishing with Markdown

Markdown editors are a dime a dozen. Cheaper than that, actually, since many of them are open‑source software. Despite the sheer number of options, finding an editor that has all of the features that one might want can be tricky. For some users, Zettlr might be the right tool. It is a What You See is What You Mean (WYSIWYM) editor that stores its work locally as plain Markdown files. The project is billed as a "one-stop publication workbench", and is suitable for writing anything from blog posts to academic papers, maintaining a personal journal, or keeping notes in a Zettlekasten. It is simple to get started with, but rewards deeper exploration and customization.

Zettlr history

The project was started in 2017, by Hendrik Erz, and its name comes from the German word for note, Zettel. According to the about page for Zettlr, Erz was unhappy with existing tools for academic writing, so he began work on the editor, which developed a following among academics and students. Since its inception it has had two major updates, version 2.0 released in 2021, and 3.0 released in 2023. The most recent version, 3.2.0, was released on June 11.

The project's web site has a feature table that compares Zettlr to other popular open-source and proprietary note-taking tools such as Obsidian, Logseq, Typora, and The Archive. The table might be a bit biased, seeing as the project hand-picked the features for comparison, but it does give a clear picture of which features the project sees as important and how Zettlr stacks up.

Zettlr is a cross-platform Electron application written in JavaScript/TypeScript, and uses CodeMirror as its embedded editor. It's licensed under the GPLv3 with downloads for Linux, Windows, and macOS. The project offers native packages for Fedora, Debian/Ubuntu, as well as an AppImage bundle, and a Flatpak version. Arch Linux also has a Zettlr package available in its repositories, though the Zettlr project has a mild disclaimer in its documentation that it is not responsible for Arch's package. Note that the Flatpak version does not have access to the full filesystem by default, so it may be necessary to use Flatseal or another Flatpak permission manager to provide access.

Electron tends to be a popular option for creating cross-platform applications, but it also has a bad reputation for poor performance, and for its applications having a non-native "feel". However, the Zettlr developers have taken pains to ensure that the application appears native on each platform. ("Native" on Linux means that it's designed to fit in with GNOME's human interface guidelines, so mileage may vary on other desktop environments.) Its performance is also good, at least in my experience, using native packages on Fedora 40 and Linux Mint 22 (using the Ubuntu package).

Getting started

There is a tutorial that's worth exploring before getting started on any real work. While it is possible to be productive right away without it, the tutorial will likely help save time in the long run. For example, it will guide users to install LaTeX to create PDFs using XeLaTeX. That step is not obvious without reading the documentation or following the tutorial, and attempts to export using XeLaTeX will error out instead. When using Simple PDF, Zettlr first exports to HTML and converts that to PDF. XeLaTeX has more options, including setting the page size and setting document classes.

Zettlr ships with a built-in version of Pandoc for importing, converting, and exporting files to or from Markdown so that users don't have to install it separately. (As of this writing, Zettlr includes Pandoc 3.1.13, while the most recent version is 3.3.)

If Pandoc can convert a file correctly to Markdown, it can be edited in Zettlr. Its export function is a bit more limited, however, as it only exposes a handful of formats that Pandoc supports. For example, Pandoc happily supports converting Markdown files to the EPUB format, but that option is not available by default via Zettlr's export menu. Options that are available include HTML, rich text format (RTF), OpenDocument Format (ODF), LaTeX, plain text, Microsoft Word (docx), simple PDF, and LaTeX PDF. Users can even use Zettlr for creating presentations by exporting from Markdown to the reveal.js presentation format.

If the default options are insufficient, all of Zettlr's import/export profiles are stored as YAML files and exposed through its Assets Manager. Users can tweak the settings for each format, or create new ones. Finally, it is also possible to add custom export commands, so users are not limited to Pandoc for exporting Markdown to other formats.

It is possible to simply open or create a text file and start editing, but Zettlr's design is centered around workspaces and folders (directories). The distinction is a bit fuzzy, but the short of it is that workspaces are meant to organize one or more folders where work is stored. Zettlr allows opening multiple workspaces at one time, so a user might have one for a book project, another for notes, and so on. Folders and workspaces can be converted to projects. The benefit of a project in Zettlr is that all of the files in a project can be exported to a single document, such as a PDF. For instance, one might work on each chapter of a book as a single Markdown file and then export the entire book to PDF or another format with a single click.

The project export implementation is a bit limited, however. It only supports lumping all of the files in a workspace together as a single piece of work. It would be nice if Zettlr also supported, for example, rendering content in a workspace as separate HTML files for a web site or knowledge base. But, since everything is stored on disk as plain Markdown files, it should not be difficult to use something like Jekyll for that last step if desired. This is unlike some other open-source note-taking applications, such as Joplin, that will allow users to work in Markdown but store files in an obfuscated format that makes it difficult to use with other tools.

Editing

As a WYSIWYM editor, Zettlr does not try to provide an exact preview of the final document in real time, but it does provide options for real-time Markdown rendering that make it more like working in a word processor than a plain-text editor. For example, users can choose to render links, bold text, italicized text, headers, images, and Mermaid diagrams inline. It also has a quick preview feature, which will bring up a print preview of the current document.

Zettlr's user interface is quite flexible. Users can open multiple documents in tabs and/or split windows. (The screenshot above shows Zettlr with several documents open in split windows and tabs.) Naturally, it has a full-screen mode, which expands the editor to fit the entire screen and hides most of the editor interface. The distraction-free mode hides sidebars, tabs, split windows, and grays out all text except the current paragraph. It also has a typewriter mode, which highlights and centers the current paragraph in the middle of the editor. This is extremely useful, as it keeps the cursor in the middle of the screen instead of it creeping further and further down as one writes more text. All of those modes can be combined, if desired, which writers may find helpful to stay in the flow while working.

No doubt some readers will want to know whether Zettlr has the all-important dark mode. The answer is "yes". It has five default themes that change the color scheme and font face. Each of the themes has a corresponding dark mode as well. Users can even set a schedule to switch from light mode to dark mode, and back again. Its interface is also customizable via CSS, so users can tweak the interface to their liking if the default themes are not quite right.

For those who feel most at home in Emacs or Vim, the application has the option of Emacs or Vim modes. These are implemented by CodeMirror plugins and provide only a subset of each editor's keybindings. For example, Emacs users are used to control-k deleting everything from the cursor to the end of a line. However, in Zettlr, control-k is bound to "insert link" instead. Even an imperfect implementation may still help users who have developed muscle-memory for one or the other editor, though. See the Emacs and Vim documentation on the CodeMirror site to learn more.

Speaking of keybindings, Zettlr's developers deserve kudos for having shortcuts for almost every menu option. The only disappointment is that the shortcuts are not configurable, though they have explored making them configurable. Apparently this is a major limitation of using Electron, and users should not hold their breath for this feature.

Prose tools

As a writer's workbench, Zettlr provides tools for tracking writing progress, text replacement, citations, spell checking, syntax linting for Markdown, and optional grammar/style checking using LanguageTool. Its status bar, at the bottom of the editor, shows the cursor position, how many words and characters are in the document, the editor mode (if in Vim or Emacs mode), and has a tally of possible errors in the right-hand diagnostics tab. Clicking the diagnostics tab in the status bar opens up a small panel at the bottom of the editor with a list of spelling mistakes, grammar errors, and so forth. Grammar and style checking require an account with the LanguageTool service or a self-hosted instance of the LanguageTool open-source project.

For some reason, Zettlr really cares about smart quotes, which it also refers to as "magic quotes". Its auto-correct settings tab calls out the quote settings alone at the top of the page, separate from all of the other text-replacement options. It even has status-bar buttons for enabling, disabling, and customizing smart quotes. Aside from smart quotes, it also provides a number of text-replacement options for things like converting double hyphens to an en dash (–) or inserting a Euro currency symbol (€) by typing :eur.

Writers will find a few tools for tracking productivity in Zettlr as well. The statistics button in the top toolbar brings up a quick view of how many words have been written in the past month, the daily average, and a count of how many words have been written that day. Clicking "More statistics" gives several views for word-count statistics and a graph view to show links between files. Zettlr provides a Pomodoro timer in the toolbar as well, to help break down writing sessions into manageable chunks.

The graph view is particularly interesting if one is using Zettlr as a personal knowledge database or Zettlkasten. As one might expect from the name, Zettlr has a few features specifically geared at managing a Zettlekasten or similar note-keeping scheme, and has a settings page dedicated to that purpose. Specifically, it has support for creating unique IDs for Zettel notes, for cross-referencing between files, and for tags. For example, one might add a #blog tag to every document that contains a blog post.

Tables

Markdown is a decent markup language that handles basic text formatting well, but editing tables in Markdown becomes tedious quickly. The formatting for tables in Markdown requires separating cells using a vertical bar (|) or ASCII art. Tables with more than a few rows and cells are no fun at all to wrangle using many text editors. Zettlr, though, has a table editor that makes working with basic tables easy.

The "Insert Table" button lets users specify the desired number of columns and rows, and then displays a graphical table editor with simple controls to manage text alignment, and create or delete columns and rows. The table editor is only capable of the basics, though. More complex layouts, such as splitting a single cell into multiple columns, are not supported. It is, of course, still possible to represent more complicated layouts using Markdown or HTML.

Unfortunately, and somewhat surprisingly, Zettlr is less helpful when it comes to inserting images into a document. A user might expect that the image button would bring up a file dialog to pick an image and insert the proper Markdown markup to include the image, but it does not. Instead, it simply inserts the raw code for an image (e.g., ) and whatever link might be in the clipboard as the URL for an image (whether it is an image link or not). If the clipboard does not have an image link, then it is left as an exercise to the writer to insert it manually. Zettlr does, however, support dragging and dropping images into a document. In fact, while it does not have a web-clipper, as many other note-taking applications do, Zettlr does handle copy and paste from a web browser (including formatting) reasonably well.

A handy workbench

At first glance, Zettlr seems like a fairly simple Markdown editor, but it's capable of doing a great deal more. It really depends entirely on what a user wants to do, and how deep they're willing to dig to customize Zettlr for their use case(s).

If a user is motivated to bend Zettlr to their will, they will find that the documentation for the project is well-written and comprehensive. Some documentation and screenshots are either outdated or platform-specific (e.g., the settings reference features macOS screenshots), but it is generally quite good. Each page has a link to edit the documentation on GitHub, should one encounter errors or outdated text.

The editor is mature and full-featured, but the community governance is still in a formative stage. In July, Erz announced the formation of a three-person (including himself) steering committee to "better plan and guide the development process". This is described as a first step toward opening up the project and sharing responsibility but Erz says that "I still carry the entire responsibility and accountability for the project", albeit now with help in decision-making. He also committed to transparency through frequent communication via the project's Discord server. It will be interesting to see how the project's governance and community evolves along with the project itself.

Overall, Zettlr offers a comprehensive set of features for working with prose in Markdown format, and for keeping a personal notes database. Editor preferences are highly subjective, of course, but I've found it more usable than some of its counterparts like Obsidian and Logseq. It is well worth taking for a spin for anyone looking for a good, open-source Markdown tool.

Comments (9 posted)