LWN: Comments on "Debating ifupdown replacements for Debian trixie"

ifupdown at least follows the principle of least surprise

taladar — Tue, 29 Oct 2024 10:42:08 +0000

That seems like a good thing from the perspective of having a single source of truth instead of a big mess of half-manually, half-automatically configured parts of the system.

ifupdown at least follows the principle of least surprise

zdzichu — Fri, 25 Oct 2024 05:41:33 +0000

If you have NM-managed interface, you should use NM to add addresses to it. `set ipv.addresses …` is longer than `ip a a …`, but works reliably.

NM *may* have some setting to ignore guerilla IP addresses (other tools, like systemd-network, have such settings), but I leave finding it as an excercise for the reader.

ifupdown at least follows the principle of least surprise

intelfx — Fri, 25 Oct 2024 04:32:22 +0000

> It's akin to municipal residents going to the city park, setting up a badminton net and playing a match, only to have the park manager walk by, see the net and supports and remove them in the middle of the match because he did not set them up

I thought that’s exactly how it worked :-)

ifupdown at least follows the principle of least surprise

fest3er — Fri, 25 Oct 2024 02:56:19 +0000

«(One of my pet peeves about NM is that if the interface flaps, it'll remove all IPs that were not added by it on the interface)»

In my experience, the interface doesn't need to bounce. It seems that every time NM periodically wakes up, it 'deletes' everything it did not configure. (WTF!?! I just added those addresses! Where the eff dif they go?!?) It's akin to municipal residents going to the city park, setting up a badminton net and playing a match, only to have the park manager walk by, see the net and supports and remove them in the middle of the match because he did not set them up. (Kind-of foolhardy since the badminton players might start using *him* as the birdie, just as I have to kill NM when I'm using multiple IP addrs on an IF—and then figure out how to handle WiFi myself.)

NetworkManager or networkd

mathstuf — Sat, 05 Oct 2024 12:03:23 +0000

> the report to sit unread in an issue tracker somewhere and then get closed automatically after some aggressive automated nagging that I was wasting everyone’s time by not re-checking every day till someone had the time and will to read the report.

Ugh, I hate those bots. I wish they keyed on "did a developer even look at it" before asking the user to do anything about it.

NetworkManager or networkd

NYKevin — Sat, 21 Sep 2024 22:19:38 +0000

> That two speakers /can/ establish consensus on the set of "definitely received and we both know it" bytes is quite useful and makes a lot of things possible.

This is true, but I think it is also misleading. Let's use SMTP as an example:

1. You can get to a point where both sides agree that the email has been transmitted from the client to the server.
2. You can also get to a point where both sides agree that the server has assumed responsibility for delivering the email. This is the primary state transition that SMTP is designed to accomplish, so it is important that we can get to this point.
3. You cannot get (2) without first passing through a state where it is ambiguous who is currently responsible for delivering the email (and it is mathematically provable that no extension or modification of SMTP can fix this shortcoming - it is inherent in TCP, and for that matter in IP). If the connection breaks while we are in this state, we either duplicate the email or drop it. SMTP as written specifies that the email is duplicated, but I do not know the extent to which this has been tested on real implementations.

The purpose of Paxos and Raft is not to accomplish (2). It is to prevent (3).

NetworkManager or networkd

nim-nim — Fri, 20 Sep 2024 09:47:36 +0000

> iwd really is key here for wifi, it's such a vividly better tool for managing wifi access point connections because it's designed to be, versus wpa_supplicant which is primarily a research tool that got drafted into being the AP management tool for Linux.

Note that really depends on your hardware, I enabled iwd for a time and then it started triggering kernel panics, so I returned to wpa_supplicant (less cool but tested and supported by my distro).

If I had the time and energy I would have investigated but I didn’t have the time and not interested in making it just for the report to sit unread in an issue tracker somewhere and then get closed automatically after some aggressive automated nagging that I was wasting everyone’s time by not re-checking every day till someone had the time and will to read the report.

ifupdown at least follows the principle of least surprise

maniax — Thu, 19 Sep 2024 12:54:13 +0000

I have to deal with network-scripts (ifcfg) in RHEL-based distros, netplan, networkd, NetworkManager - all this on debian and rhel-based distros, and honestly, ifupdown is one of the least painful. It does NOT react to interface state changes, which is pretty much a must for most servers, and via post/pre-up commands gives the option to have any custom config that's not really implementable otherwise (or would be a pain).

network-scripts (ifcfg) is the second best.

All the rest also happen to be non-deterministic in some situations, which might be a problem if you're trying to work around some weird bug that requires interfaces to go in specific order in a bond
(One of my pet peeves about NM is that if the interface flaps, it'll remove all IPs that were not added by it on the interface)

At some point you just say "screw this, I'll write a large and idempotent iproute2 script", but most people seem to be scared of those, and it's not trivial to convince customers that they should drop whatever the distro has and they have already configured.

No technical comparaisons

wtarreau — Mon, 16 Sep 2024 12:07:43 +0000

Maybe the "it works automatically and everywhere" approach of most modern and complex tools participates to the lack of attractivity as well ? I mean, it takes significant efforts to join a complex existing project, and many new talents prefer to reimplement their own "nicer, lighter, cleaner" equivalent of something that already exists. Sticking to reusable basics on top of which anything can be built does have some virtues in this regard.

NetworkManager or networkd

paulj — Mon, 16 Sep 2024 09:44:36 +0000

Right. This is the clarification I was trying to establish.

That two speakers /can/ establish consensus on the set of "definitely received and we both know it" bytes is quite useful and makes a lot of things possible. There is a set of bytes there-after in a grey zone, where the sender can't tell if they've been received, and the recipient can't know if the sender knows they've been received is an issue, but it's not catastrophic. That the first set can be built and (as the network permits) extended allows for useful and solid primitives to be created on top.

It's not quite as pessimistic as the point in your post above could have been read, perhaps. Distributed systems are still difficult, and it's very easy and very common for authors of them to fail to pay attention to the distinction above (leading to weird fail...), course. ;)

NetworkManager or networkd

NYKevin — Sun, 15 Sep 2024 18:37:43 +0000

(To clarify: It is possible for a party to know the consensus set contains *at least* the first N bytes. It is not possible for either party to know that the consensus set contains *exactly* the first N bytes.)

NetworkManager or networkd

NYKevin — Sun, 15 Sep 2024 18:37:05 +0000

My point is not that there is no set of bytes the parties agree on. My point is that it is not possible for either party to know exactly which bytes are in the consensus set.

NetworkManager or networkd

Sesse — Sat, 14 Sep 2024 21:11:47 +0000

16. I don't need to know anything about congestion control (a sub-category of this one is “If I don't get the speed I want, I should open multiple TCP connections”)

Any solution that isn't a wrapper posing as a tool

jkingweb — Sat, 14 Sep 2024 21:07:26 +0000

The FreeBSD kernel is no longer supported in Debian, FYI.

https://lists.debian.org/debian-devel/2023/07/msg00176.html

Hurd does remain a concern for such efforts, though.

NetworkManager or networkd

josh — Sat, 14 Sep 2024 21:06:29 +0000

Or networks that block ICMP, or networks that drop anything they don't understand...

NetworkManager or networkd

Wol — Sat, 14 Sep 2024 13:59:41 +0000

I think 14 has bitten lots of people lots of times. Usually down to router manufacturers believing a similar set of falsehoods?

Buffer bloat, of course, being a perfect example of routers breaking the congestion mechanism.

Cheers,
Wol

NetworkManager or networkd

paulj — Sat, 14 Sep 2024 12:24:46 +0000

Nice post.

One thing, unless I've misunderstood, I think you're being a bit /too/ strict and pessimistic about the consensus that is possible for 2 parties to reach over an unreliable (but non-malicious) link. It certainly is possible for 2 parties to reach a shared consensus that some /subset/ of the bytes have been received. I.e., I think your wording of 3 is too strict. There is a set of sent bytes (>= 0) where both sides can be sure that /both know/ those bytes were received, and a set of bytes following that set where they can not know. Your claim applies to the second set, but not the first set (with the caveat the first set may be 0 and remain at 0, but the size of the first set can only increase).

I.e, there /is/ a hard consensus set, and an ambiguous set. Your 3 is more that it is impossible to guarantee the consensus set will > 0, and that the ambiguous set will eventually reach 0. However, both sides can know what the consensus set is (including if it is 0).

It's a subtle difference, but you don't need Paxos for 2 parties to come to an agreement on a set of bytes having been transferred. You can't guarantee they reach a consensus on the success or failure of an attempted transfer (distinct from the ambiguity), but then I'm not sure Paxos/Raft can guarantee it either, at least in the sense of establishing that knowledge in the 2 parties (e.g., the sender may lose messages from the Paxos/Raft consensus protocol too).

NetworkManager or networkd

Wol — Sat, 14 Sep 2024 09:23:16 +0000

> This is not theoretical, within $day_job's distributed organization, most users have to juggle at least 3 VPNs on a daily basis.

As far as I'm aware, I only have two vpns to mess with at work - the work one managed by IT, and the works guest one also managed by IT. And as far as I'm aware, the rule is - even with a desktop plugged into an RJ45 - ALL pcs must come in via a vpn. This is part of the "defence in depth all pcs could be compromised" strategy.

The only exceptions that MIGHT be made are the robots. They *might* have a hard-coded MAC-IP mapping but that's over wi-fi so they might not ... (and being embedded, chances are they are trusted not to be compromised).

Cheers,
Wol

Any solution that isn't a wrapper posing as a tool

wsy — Sat, 14 Sep 2024 05:50:53 +0000

Agreed. I write my own script to invoke iproute2 to configure networks on none-trivial hosts so those wrappers won't mess with my config.

NetworkManager or networkd

NYKevin — Fri, 13 Sep 2024 22:42:04 +0000

> FWIW, I dropped NetworkManager years ago for `wpa_supplicant`-based management because I had flaky wireless situations (thick concrete walls in the dorms, roaming across campus, etc.) and any whiff of packet loss would announce to the whole machine "no network" and apps would start to freak out and react. However, it was likely to be back Real Soon™ and normal TCP recovery would make it "transparent" (if with a spike in latency).

Somebody ought to write one of those "falsehoods programmers believe" articles for TCP, because this is just reflective of a broader trend of software that thinks it knows better than TCP, and usually does not. Here, I'll even get the ball rolling (remember, all of the following statements are *false* at least some of the time, but for some of these, perhaps not very often):

1. TCP is reliable, so everything I send will be received by the other end.
2. OK, mostly reliable.
3. OK, fine, it's not reliable (in the above sense of the word), but the sender and recipient will always eventually agree on exactly which bytes made it over the transport.
4. It is possible to create a guarantee analogous to (3) by building some message-oriented application-level protocol on top of TCP, such as HTTP or SMTP.
5. There is a such thing as a TCP packet.
6. There is no such thing as a TCP packet.
7. If we fail to connect to a well-known remote host, then we must be offline.
8. Nagle's algorithm is good.
9. Nagle's algorithm is bad.
10. I don't have to care about Nagle's algorithm.
11. This is all low-level pedantry. I can think of TCP like a two-way Unix pipe that goes over the network, and completely ignore how it is implemented.
12. If the network is transparent to TCP, then it must be transparent to IP.
13. If the network is transparent to HTTP/1.1, then it must be transparent to TCP.
14. Weird networks that are not transparent to standard protocols are an aberration. I can safely ignore them.
15. TCP is implemented in terms of IP.

Explainer for 1-4: https://en.wikipedia.org/wiki/Two_Generals%27_Problem. TL;DR: If the connection breaks while an ACK is outstanding, the sender will have no way of knowing whether the segment was received, and this turns out to be an insoluble problem no matter how much complexity you pile on top of it. You need something resembling Paxos or Raft to get a guarantee like that, and that always requires a minimum of three nodes, so it can't be built on top of a single two-party TCP stream. See RFC 1047 for an SMTP-specific discussion of this problem (which still applies to modern SMTP, since RFC 2821 says that implementations MUST follow 1047's core advice), but note that some variation of this problem applies to literally every two-party TCP service (and for that matter, every UDP or IP service as well), regardless of how it works or what abstractions it introduces. SMTP is only special in that both sides are explicitly required to care about whether the message was received or not, which is marginally unusual for TCP services (compare and contrast: FTP file uploads, HTTP POST and PUT, etc., most of which omit significant discussion of client retry logic in favor of leaving it up to the application or end user).

15 is left as an exercise for the reader (hint: it is primarily of historical interest, but I'm not sure it's possible to entirely rule out modern counterexamples, since we don't know what weird stuff is going on in [any large organization]'s private network).

Any solution that isn't a wrapper posing as a tool

Hobart — Fri, 13 Sep 2024 21:43:58 +0000

Netplan, to my understanding, is trying to be Swiss-Army knives to accomplish their tasks.

Since the Debian OS can run a Linux, FreeBSD, or HURD kernel, if the system's using Linux, I'd like to see it using the kernel's own, documented, iproute2 tools for configuration.

More practically, the tools need to interact properly with the distro's boot process, and indicate exactly which system interfaces / files they're working with.

I shouldn't need to break out strace to figure out what's happening. I want a userspace whose behaviors are expected - the NetworkManager delivered by Ubuntu and Red Hat's "surprise, we've swapped your DNS resolver!" toolsets heave been less than great to work with.

NetworkManager or networkd

jhoblitt — Fri, 13 Sep 2024 20:49:30 +0000

Attaching to WIFI in offices, coffee shops, airports, eduroam, etc. is a common use case for dynamic networking but it isn't the only one. VPNs need to be handled even for relatively static desktops. It is common for VPNs to hand out conflicting address space (grrr! please use routable space to avoid conflicts) and internal nameservers. This is not theoretical, within $day_job's distributed organization, most users have to juggle at least 3 VPNs on a daily basis.

NetworkManager or networkd

WolfWings — Fri, 13 Sep 2024 19:56:19 +0000

I feel like because for the majority the only situation where you're setting things up often enough 'dynamic' is needed is nomadic wifi, and there's extremely good tools that handle that particular layer well so there's no 'itch to scratch' to integrate that directly into networkd.

No technical comparaisons

ballombe — Fri, 13 Sep 2024 19:41:00 +0000

In any case, Debian needs a tool with minimal dependencies to set up the network when networkmanager is not yet installed or for embedded systems.

Personnally, I will pick whatever system let me configure my laptop as an access point when plugging additional
USB wifi dongles with the less fuss. So far it is ifupdown.

No technical comparaisons

ebee_matteo — Fri, 13 Sep 2024 19:08:51 +0000

>> He said that providing users with a more modern, better-maintained tool was more important than retaining Debian's identity through its choice of networking tool.

> Maybe ifupdown-ng is the more modern, better-maintained tool ? Maybe Debian's identity is better.

I think the obvious implication that the original writer was making, is that ifupdown-ng is definitely NOT the more modern and better-maintained tool, and upholding Debian's identity is not a reason good enough to keep it alive.

NetworkManager or networkd

ebee_matteo — Fri, 13 Sep 2024 19:06:38 +0000

I also think systemd-networkd is more geared towards headless / server setups, while networkmanager towards a more desktop user setup. It matches the original goals and reasons those projects were started.

anything but ifupdown

rweikusat2 — Fri, 13 Sep 2024 18:00:10 +0000

> As an operator I feel ifupdown is the worst part of Debian.
> The reason is that it does not track real state of interface, but instead rely on own opinion about the
> current state. Every time I try to automate network configuration, the serious one (e.g. bond over two eth,
> vxlans, some dummy for bgp), I walk on a mine field. Any mistake on config leaves system in undefined state.
> Every config change can fail, because 'it's already configured'.

As documented in the ifup/ifdown manpage, ifupdown uses /etc/network/run/ifstate to record the state (configured or not yet configured) of interfaces managed by it. By default, it will not attempt to configure interface recorded as already configured in this file or deconfigure them if they're not recorded as already configured there. The --force command-line option can be used to override this. And it's obviously also possible to change the file itself in order to reflect the running configuration accurately.

NetworkManager or networkd

WolfWings — Fri, 13 Sep 2024 16:47:22 +0000

Yup, this nailed it. More specifics:

iwd replaces outright wpa_supplicant, and exclusively handles the wifi card physical layer for attaching to SSIDs and credentials at that layer. SystemD has no knowledge of this layer of auth.

iwd really is key here for wifi, it's such a vividly better tool for managing wifi access point connections because it's designed to be, versus wpa_supplicant which is primarily a research tool that got drafted into being the AP management tool for Linux.

networkd then matches against what SSID I'm connected to too apply any specific configurations (my 'home' network enabling LLMNR + mDNS which is disabled by default for example), or defaults to 'DHCP + DHCPv6' with name resolution restricted as my fallback 'I guess you're connecting at an airport' settings.

No technical comparaisons

intelfx — Fri, 13 Sep 2024 15:40:34 +0000

> Lack of mainteners is a worldwide issue that afflicts almost all projects

Well, clearly some projects are affected more and some are less. Otherwise the present article wouldn't exist.

No technical comparaisons

ju3Ceemi — Fri, 13 Sep 2024 15:39:39 +0000

Lack of mainteners is a worldwide issue that afflicts almost all projects

No technical comparaisons

intelfx — Fri, 13 Sep 2024 15:37:44 +0000

> The question is, popular among whom? The broader Linux community? Debian users? Old-timey Debian users who grew up editing a particular set of files in /etc/ and don't want to switch to editing another set of files in /etc/, for whatever reason?

Well, considering the density of recent articles talking about efforts to attract new blood to Debian, I'd hazard a guess that the first two demographics are _at least_ as important as the last one.

NetworkManager or networkd

mussell — Fri, 13 Sep 2024 14:56:49 +0000

I (probably) have a similar setup. iwd handles authenticating to networks similar to wpa-supplicant, networkd does all other configuration such as assigning IP addresses through DHCP/SLAAC. networkd can match on an (B)SSID to apply settings for specific networks (see systemd.network(5)), I use it to enable mDNS only on my home network and can be used as a lightweight firewalld by using NFTSet to add the wireless interface to a set only on trusted networks. resolved is both a caching DNS resolver and an mDNS resolver. networkd and resolved are tightly coupled (networkd needs to tell resolved about DNS servers from DHCP), but as far as I can tell, iwd has no integration with networkd (or anything from systemd) specifically. iwd will put the wireless interface up after authentication and networkd will know through netlink.

NetworkManager or networkd

josh — Fri, 13 Sep 2024 12:26:16 +0000

Could you say more about how you use iwd and networkd together in your setup? How much does networkd know about wireless interfaces, and what part of the setup does each tool do in your configuration?

NetworkManager or networkd

mathstuf — Fri, 13 Sep 2024 12:22:34 +0000

This is also my setup (though I tend to bypass `systemd-resolved` on various machines as it is not network namespace aware). FWIW, I dropped NetworkManager years ago for `wpa_supplicant`-based management because I had flaky wireless situations (thick concrete walls in the dorms, roaming across campus, etc.) and any whiff of packet loss would announce to the whole machine "no network" and apps would start to freak out and react. However, it was likely to be back Real Soon™ and normal TCP recovery would make it "transparent" (if with a spike in latency).

NetworkManager or networkd

bluca — Fri, 13 Sep 2024 11:44:29 +0000

Right, yes the driver of configuration is either protocols like DHCP or the config files. If someone wants to have IPC-driven configuration, they would need to work on it, and it hasn't happened so far.

Defaults are hard ...

detiste — Fri, 13 Sep 2024 10:59:15 +0000

I personaly think that systemd-cron would be a such better _default_ than Vixie cron.

But I'm not willing to try soooo hard to sell it.

cron can continue running all those "[ ! -d /run/systemd/system ] || exit 0" until the end of the world...

Too many choices

garyvdm — Fri, 13 Sep 2024 10:36:51 +0000

They way you worded that make me think you don't realize that networkd and NetworkManager are 2 separate things.

NetworkManager or networkd

garyvdm — Fri, 13 Sep 2024 10:35:17 +0000

* Ways to provide custom gui configuration screens, e.g. NetworkManager-openvpn-gnome
* Ways to provide custom gui login screens, e.g. NetworkManager-openvpn-gnome
* Ways to handle public wifi portals

NetworkManager or networkd

WolfWings — Fri, 13 Sep 2024 10:22:35 +0000

What dynamic functionality is missing?

I'm asking because I've had zero issues relying on iwd + systemd-networkd + systemd-resolved as my sole networking configuration and generally found it much simpler to setup than other network tools on both Debian laptops I have so I'm wondering what use-cases I'm not thinking of?

NetworkManager or networkd

evad — Fri, 13 Sep 2024 10:12:02 +0000

There is also this RFE for dynamic changes: https://github.com/systemd/systemd/issues/27469