LWN: Comments on "Domsch: Consistent Network Device Naming coming to Fedora 15"

Shell metacharacters

nix — Wed, 23 Feb 2011 22:34:16 +0000

Honestly, it's much easier to write bad Perl than bad Python. I'm a structure fanatic: my C, Python and Lua is structured as all hell. But I can't make my Perl look neat or structured no matter how hard I try: it disintegrates into a mess regardless.

And I am not the only person this is true of. Perl is a very messy language.

Shell metacharacters

jrockway — Wed, 23 Feb 2011 18:53:29 +0000

What's wrong with Perl for new projects? New modern object systems like Moose? 20,000 free distributions on CPAN?

Honestly, people have written bad Perl in the past, but now they are writing bad Python and Ruby too :)

Domsch: Consistent Network Device Naming coming to Fedora 15

cesarb — Mon, 31 Jan 2011 21:43:59 +0000

I just thought of yet another advantage of my scheme: with it, "iptables ... -i e+" matches everything Ethernet (ethX, emX, and esXpY...), in a single rule.

Domsch: Consistent Network Device Naming coming to Fedora 15

cesarb — Mon, 31 Jan 2011 21:40:32 +0000

> And if the catchall 's' is unweildy, you can have 'sp' for PCI, 'si' for ISA, etc.

I believe that is not the idea. The idea is, as far as I understood, to match the *labels* on the outside of the server. So the slot marked as "1" on the outside of the server should have a "1" on its network interface name, indifferent as to whether it is a PCI slot, a PCI-X slot, a PCIE slot, an ExpressCard slot some crazy hardware designer decided to add to a server, or some even crazier stuff (SDIO perhaps?).

Putting it in another way, what is in the inside (the bus used) does not matter, it is what is on the outside (the onboard port numbered Y or the slot numbered Z) that matters.

Shell metacharacters

lakeland — Mon, 31 Jan 2011 21:21:42 +0000

Personally I like that. I use spaces in my file names - can you believe how many shell scripts break because they don't escape file and folder names? You start having default system installs with spaces and hashes in them and maybe shell script writers will include them in their testing.

Domsch: Consistent Network Device Naming coming to Fedora 15

bronson — Mon, 31 Jan 2011 16:50:02 +0000

Love it. And if the catchall 's' is unweildy, you can have 'sp' for PCI, 'si' for ISA, etc.

As an ex-NOCmonkey, I really like the attention paid to phone readability. When the servers are on fire and the whole team is conferenced together, it's not fun to spend a minute trying to agree on what '#' and '_' are called.

Domsch: Consistent Network Device Naming coming to Fedora 15

cesarb — Mon, 31 Jan 2011 15:36:10 +0000

> I'm open to other suggestions for scheme

Since you asked for it...

es<slot>p<port>v<vf>

This gets rid of all non-alphanumeric characters, uses a generic "s" for "slot" instead of "pci" (which implies PCI, even if other technology is used in the future - or in the past as in ISA and MCA), and adds an "e" prefix (for Ethernet, becoming similar to your "em"). So you have:

em1
es4p1
es4p1v62

You could also omit the port number for single-port add-on cards, so for instance if the card on slot #4 only has one port, you could have "es4" and
"es4v62".

I think this scheme is shorter, prettier, and easier to pronounce over the phone.

Shell metacharacters

mab — Mon, 31 Jan 2011 05:39:32 +0000

Why is there a need for a separator at all?

Domsch: Consistent Network Device Naming coming to Fedora 15

nix — Sun, 30 Jan 2011 16:59:45 +0000

Agreed. It feels like a horrible and gratuitously nonportable layering violation to use PCI bus IDs for anything related to networking... I suppose in this case they *want* ambiguity, so that they'll get the same name on *every* machine with a card plugged into the same slot. Seems really risky to me, but, hell, I'm not the target market.

Domsch: Consistent Network Device Naming coming to Fedora 15

rwmj — Sun, 30 Jan 2011 15:47:15 +0000

Welcome to the world of virtualization :-)

70-persistent-net.rules is a constant source of trouble when V2V-ing, migrating and cloning virtual machines.

In any case, the change mentioned in this article has nothing to do with udev's persistent naming, and all to do with how the network interfaces are named when you first install an OS. It's a reasonably sensible change, although given that this driven from the BIOS, there is plenty of room for BIOS vendors to fsck things up as they normally do.

Rich.

Domsch: Consistent Network Device Naming coming to Fedora 15

mdomsch — Sat, 29 Jan 2011 14:34:31 +0000

Unfortunately, the kernel->userspace ABI for network ioctl() calls defines the interface name to be IFNAM size, or 15 characters usable. It would be impossible to change every userspace program ever produced and in use to allow for a longer device name.

Domsch: Consistent Network Device Naming coming to Fedora 15

butlerm — Sat, 29 Jan 2011 07:16:52 +0000

Mind you, there are only 15 characters of IFNAM size

Isn't that like, so 1983? Surely 30 characters or more would be a much more reasonable limit for interface names, given all the things that can be tacked on these days.

Domsch: Consistent Network Device Naming coming to Fedora 15

Tet — Fri, 28 Jan 2011 22:30:13 +0000

It's always bothered me that from my point of view the direction and development of Linux is driven by the server use case.

You're kidding, right? For me, by far the biggest problem with modern Linux is that the direction is nearly all driven by the desktop use case.

Domsch: Consistent Network Device Naming coming to Fedora 15

bpearlmutter — Fri, 28 Jan 2011 22:28:17 +0000

That isn't really a kernel issue, or anything deep. You can make more symbolic links, named as you please, with appropriate rules in the hal configuration files. In fact, names like you suggest would be a great addition, maybe /dev/disk/by-bus/pci/* and /dev/disk/by-bus/usb/* and such. Not sure if they'd see much use by scripts or automatic mechanisms, but they would certainly be useful to ls -lR and see what's what.

Domsch: Consistent Network Device Naming coming to Fedora 15

mdomsch — Fri, 28 Jan 2011 20:25:51 +0000

Thanks everyone for your insightful comments. I appreciate the votes in favor, and for helpful suggestions when you're not quite convinced yet...

It's fairly likely that we'll change from using '#' to something else (perhaps 'p' for "port") to avoid the parsing problems which are sure to appear. That's still under investigation.

Mind you, there are only 15 characters of IFNAM size, and 5 of them are already "claimed" by vconfig when creating VLANs on an interface (suffix of .1234). That realistically leaves only 10 characters for all other purposes, including :alias usage (though that should be deprecated - patches welcome to remove that from Fedora initscripts). In general we'll use three ("em1") or six ("pci4#1"), or nine ("pci4#1_62") in the SR-IOV case, so we're right up on that limit. The initial "pci" bit is to identify PCI sockets as seen on the back of the chassis, apart from embedded NICs that aren't removable. I'm open to other suggestions for scheme, but a) it can't be longer than this; b) it can't start with eth(:digit:)+.

Shell metacharacters

smurf — Fri, 28 Jan 2011 19:48:17 +0000

No script I know of is actually re-interpreting command lines.

Hashes in variable values don't even need quoting.

Domsch: Consistent Network Device Naming coming to Fedora 15

jmm82 — Fri, 28 Jan 2011 17:34:27 +0000

@nix

Thank you for the clarification. That makes perfectly good sense. I guess I still feel that this could have been done more cleanly. Relaying on the network device name for bus location seems like a conflict of interests. I just hope people don't start depending on the names not changing.

Domsch: Consistent Network Device Naming coming to Fedora 15

nix — Fri, 28 Jan 2011 12:57:06 +0000

Desktop? I have servers with twelve NICs here, with 'To Be Filled By OEM' in every single one. I'd change them if I could, but, of course, I can't.

It is generally a mistake to trust system assemblers to get *anything* right that isn't needed to boot Windows with the absolute minimum set of drivers installed. Half the time they can't even get the right amount of RAM installed: what's the likelihood that they'll name network interfaces correctly in obscure BIOS tables?

(Some BIOSes aren't that bad, if the BIOS is something that comes *with* a specific peripheral and is made by the manufacturer of that peripheral. e.g. AtomBIOS on ATI cards. For anything else, the BIOS is liable to be rife with bugs, but even then less rife than the parts of BIOSes meant to be filled out by system assemblers.)

Domsch: Consistent Network Device Naming coming to Fedora 15

nix — Fri, 28 Jan 2011 12:51:31 +0000

Well, you can rename them to whatever you like, so anything other than the stuff in very early boot that renames them (pretty much, anything after udev runs) that relies on specific names is broken already.

(FWIW, I've been running Fedora systems with network interfaces udev-renamed to descriptive names for many years with no problems at all.)

Domsch: Consistent Network Device Naming coming to Fedora 15

nix — Fri, 28 Jan 2011 12:37:55 +0000

What problem has tying to a MAC address given you? It should only cause problems if you change MACs all the time, and who does that?

Domsch: Consistent Network Device Naming coming to Fedora 15

hmh — Fri, 28 Jan 2011 01:46:36 +0000

Hmm? Anyone taking care of large datacenters has a deployment and configuration management tool. Such tools take care of updating udev rules (and anything else) at installation time easily. Also, you deploy a server type *once* by hand, write down the cabling sequence needed for use by the physical assets management team, upload the variable data management scripts to the deployment tool database, and everything else is done based on these templates. You *KNOW* what cable needs to go in what NIC to get a deterministic installer to select the right NIC.

When changing a NIC requires updating a config file for a given operating system, this IS known beforehand, and it IS written in the hardware config change checklist. It may be annoying, but it doesn't result on servers that don't boot.

So no, whatever this is about, it is certainly not about datacenters, and certainly not about first boot (when keying to MAC doesn't hurt).

Keying to PCI slots is actually less troublesome when a NIC needs to be replaced, but that does assume the user will not shuffle PCI boards around. Datacenters don't, but they don't actually need this whole stuff, anyway. Desktop users do shuffle boards around, and I am not sure what happens in a small shop with a few servers, and a single junior sysadmin.

It will also subject people to the usual negligence and incompetence found on desktop BIOS teams. You *WILL* get desktop boards with "To Be Filled By OEM" in the slot names, or with repeated slot names, or with control characters inside of ASCII in the slot names, etc. I sure hope that biosdevname tool is extremely paranoid and has a fallback to do something smart when it comes across such insanity.

Domsch: Consistent Network Device Naming coming to Fedora 15

hmh — Fri, 28 Jan 2011 01:03:29 +0000

Hmm, it is a good idea, but the implementation suggested so far is not.

Naming the embedded port em<number> isn't so bad, other than the fact that it will break a lot of half-wit crap that needs the "eth" as the prefix to know something is an ethernet NIC. Stuff like iptraf, and a great many number of scripts. This is basically unavoidable, and acceptable collateral damage.

But a "pci" prefix for PCI/PCIe-attached NICs? You KNOW it is an Ethernet NIC, at least use a prefix that is not a BUS NAME...

iptables matching is done on prefix (e.g. em+, eth+). You WANT to be able to separate ethernet NICs from other devices by their prefix. Why not use a common prefix for all ethernet-like nics, like, say, "eth"? Sure, give us ethem<number> for the embedded ones, and ethpci<something> for the pci slot named ones. Or ethe<number> and ethp<number>, whatever.

The less said about the idea of using "#", the better. Just drop it, it is a Bad Idea. It adds no advantages and it is potentially hazardous. If you need something at all, use "_" or a letter.

There is no way we'd be able to take that crap naming convention in Debian, but we could certainly adopt the whole idea if the naming was more palatable. And THIS is something we'd all benefit if every distro decided to do in the same way.

Domsch: Consistent Network Device Naming coming to Fedora 15

Wol — Thu, 27 Jan 2011 23:35:36 +0000

"Order is fixed - after first boot".

Problem is, from reading other posts, the system won't boot until after the order is fixed ... the *install* CD needs to know which port is which so the box can find the screen/keyboard to let the administrator in.

Cheers,
Wol

Shell metacharacters

cmccabe — Thu, 27 Jan 2011 23:06:09 +0000

> One thing I do not like in this... Why use a shell metacharacter? Isn't
> it asking for trouble?

Matt Domsch wrote:

> Yes. We did uncover one or two trivial-to-fix bugs in the Fedora
> initscripts due to this, but that was to be expected. There are no other
> ASCII characters that don't already have several meanings that would make
> for a better separator. In configuration files, the # is usually only
> special at the beginning of a line, not as part of another word, and they
> can always be quoted if necessary.

Shell metacharacters

cmccabe — Thu, 27 Jan 2011 23:03:06 +0000

Perl is not a shell. Therefore Perl can never replace the bourne shell.

Bourne shell scripts are still the right choice for SHORT (one or two page length) glue scripts. Once you start needing more, something like Ruby or Python is probably the right choice. Perl was really innovative in its day, but starting new projects in Perl is a terrible idea.

Domsch: Consistent Network Device Naming coming to Fedora 15

aliguori — Thu, 27 Jan 2011 20:28:21 +0000

When we introduced paravirtual block devices for KVM, we used the naming convention /dev/vdX because it seemed like a nice thing to do verses just exposing it like a SCSI device.

Over the years, I've been startled about the number of scripts (some in large projects) that assume all block devices are either /dev/hd* or /dev/sd* and subsequently broke.

I understand the motivation here, and agree that it's a useful thing to do, but lots of stuff will in break in subtle ways after this.

Domsch: Consistent Network Device Naming coming to Fedora 15

aleXXX — Thu, 27 Jan 2011 20:19:06 +0000

I fully agree, I thought the same when reading it.
Well, actually I thought that "emX" would be used only as a suffix to "eth".

Alex

Domsch: Consistent Network Device Naming coming to Fedora 15

jeremiah — Thu, 27 Jan 2011 17:39:36 +0000

My understanding of this issue, is that exists BEFORE you have the chance to get puppet or cfengine up and running. If you're doing a new image of 100 machines, your image doesn't necessarily have access to the cfengine or puppet server. Esp if the ethX might have different names. After things are up and running and you know which net dev to connect through then you can apply cfengine or whatnot to the problem, and get the 'details' of the system setup. In a lot of ways this problem probably has more to do with initrd, BusyBox, and nash more than anything else. As someone who has to do this, building complex initrd scripts is a pain in the butt, esp when it's figuring out which fracking net dev is the right one to connect through. It's not the script that's hard, it's the image->cpio->take a guess at fix->cpio->image->boot->rinse repeat that's a pain. And the less guess work taken out of that the better.

</rant></ramble>

Domsch: Consistent Network Device Naming coming to Fedora 15

butlerm — Thu, 27 Jan 2011 16:58:35 +0000

/dev/disk/by-path is useful, but surely there is something deterministic that could be assigned that is simpler than "/dev/disk/pci-0000:00:1f.2-scsi-0:0:0:0" for a simple onboard SATA or SCSI disk.

Thirty years ago, on an Apple II one could issue commands like "CATALOG,S6,D2" and it would list the contents of the second drive connected to the sixth slot. Would it be so unreasonable to expect the OS to use information from the BIOS to determine that "this PCI / bus id connects to the first onboard disk interface, the second onboard disk interface", and so on, and allocate simple static names accordingly?

SMBIOS provides this information for onboard disk controllers and other devices in addition to Ethernet interfaces, I understand.

Domsch: Consistent Network Device Naming coming to Fedora 15

mebrown — Thu, 27 Jan 2011 16:37:50 +0000

sorry, incorrect.

There are several standard ways to get information about PCI slot numbering. The $PIR table being as old as the PCI standard, itself. The $PIR table will give you a mapping of how each PCI bus/dev is physically labelled. There are also newer standards using ACPI and SMBIOS tables for the same. (The newer standards also give a mechanism to label embedded devices with their physical label, which the older $PIR lacked.)

You are probably thinking of the older problem where NIC enumeration changed from depth-first search to breadth-first, changing the order in which ethX's were labelled.

This new mechanism completely solves the problem by relying on long-standing standards.

Shell metacharacters

jmm82 — Thu, 27 Jan 2011 16:19:06 +0000

Hence auto-tools, what do you think "./configure" is doing?

Perl is not as easy to pipe multiple commands. I wish it was, but it just isn't. Also, in the embedded space microperl might reach the system, but not perl. Also, Perl is not any better to maintain than sh, maybe Python. With BusyBox you can run a lot of scripts with a few changes.

Funny you quote Larry Wall and say Bourne should be replaced by Perl.

Domsch: Consistent Network Device Naming coming to Fedora 15

butlerm — Thu, 27 Jan 2011 16:11:48 +0000

Apparently, the ethX namespace is verboten because it races with the kernel detected devices.

So why not change the kernel so that it gives non-conflicting temporary names?

Shell metacharacters

HelloWorld — Thu, 27 Jan 2011 13:45:52 +0000

That probably depends on your definition of portability. While it is theoretically possible to write bourne shell scripts that work on many platforms, almost nobody does because it's just too hard due to all kinds of differences both in the shell implementations and the tools that are available. Larry Wall was right when he said It is easier to port a shell than a shell script.

So, if you want to write a portable script, use perl. If your unix system doesn't have perl, it's dead anyway. And while perl isn't the be-all and end-all of scripting languages, it is a huge improvement over bourne style shells.

Domsch: Consistent Network Device Naming coming to Fedora 15

nim-nim — Thu, 27 Jan 2011 12:15:49 +0000

Big Enterprise customers (the ones Dell, HP and Red Hat sell to) have complex organisations where the team that racks a server in a datacenter is usually not the one that installs the system on it. That creates all kinds of problems on modern servers with multiple nic ports if the team that racks the server and the team that installs software on it have no common reference on what a particular nic port is (The racking team only knows the physical port labels it sees on the server, and the software team only knows the system port labels reported by the OS as displayed on its remote console. What Dell did is try to have both match. It will save lots of awkward phone call to its customers "if I plug it there, what interface do you see in the system, is it the right one this time" "Oops, the non-firewalled nic was plugged on the insecure public switch, etc"

Domsch: Consistent Network Device Naming coming to Fedora 15

Wout — Thu, 27 Jan 2011 11:48:45 +0000

Currently many distributions create a MAC address to ethX mapping during installation. This ensures eth3 remains the same interface after every reboot.

Why not implement that same approach using PCI slots instead of MAC address?

So instead of associating a MAC address with an eth device, associate a PCI slot with each eth device.

That way we'd still have the ethX names we like, order is fixed (after first boot) and we can determine which eth interface corresponds with which physical interface. If the order is important to the user (eth1 must be pci1#2 on every node) then modify the configuration during install.

I really don't like the # in the new names by the way. It will cause a lot of problems.

Shell metacharacters

djzort — Thu, 27 Jan 2011 11:23:50 +0000

"It is, and that's a Good Thing. Bourne-style shell scripting needs to die, and the sooner people realize that, the better."

How about something that looks like a Makefile? How about hmmm...Python?

Domsch: Consistent Network Device Naming coming to Fedora 15

djzort — Thu, 27 Jan 2011 11:22:19 +0000

"Sure, that works if you are talking about one machine. Now imagine that you have 100 identical servers, each with 4 network ports. Think of the administrative overhead (every single one of those 100 machines will require a different 70-persistent-net-rules.net) and the chance of making silly mistakes.
With Fedora's approach, you will only need to write one config per hardware platform, which is vastly more manageable for large organizations."

Youve heard of scripts right? Tools like puppet or cfengine? Why are you managing 100 hand and custom built servers?

Shell metacharacters

mpr22 — Thu, 27 Jan 2011 10:25:21 +0000

Feel free to propose a ubiquitous, reliable alternative that makes every useful thing that is simple in Bourne-style shells no more complex than it is in Bourne-style shells.

Domsch: Consistent Network Device Naming coming to Fedora 15

bpearlmutter — Thu, 27 Jan 2011 09:29:10 +0000

This problem is completely solved with storage devices and their partitions by having multiple names for each device or partition, so they can be referred to by hardware interface id, by uuid, etc.

$ ls -d /dev/disk/*
/dev/disk/by-id /dev/disk/by-path /dev/disk/by-uuid
$ ls -d /dev/[hs]d*
/dev/sda /dev/sda1 /dev/sda2 /dev/sda5 /dev/sdb

The only reason the same proper solution does not work for network devices is that their namespace lives outside the filesystem. So the *right* fix for the problem is to put their namespace back inside the filesystem, where it belongs. The hack under discussion above makes some things easier and others harder: it cannot fully solve the problem.

This is awesome

joib — Thu, 27 Jan 2011 08:26:47 +0000

Congratulations to Mr. Domsch and his team for fixing this long-standing problem!