Fedora: RFC: X.Org X11 modularization project - rpm package driver naming
From: | Rahul Sundaram <sundaram-AT-redhat.com> | |
To: | lwn-AT-lwn.net | |
Subject: | Fedora: RFC: X.Org X11 modularization project - rpm package driver naming | |
Date: | Sat, 27 Aug 2005 19:08:08 +0530 |
https://www.redhat.com/archives/fedora-devel-list/2005-Au... Overview: ~~~~~~~~ We have begun work on X.Org X11 modularization, and are in the process of packaging the video and input drivers. Upstream, each driver has its own individual tarball, which are available at: http://xorg.freedesktop.org/X11R7.0-RC0/driver One of the benefits of the modularized X.Org X11R7, is that it makes it much easier for us to provide individual driver updates without having to release the entire 150Mb monolithic X release. The ability for us to update a single driver, and release that single driver to users is an important thing for a modern OS, in particular for desktop systems. As such, we have decided to package each driver individually in its own src.rpm package. It has now come to the time where we must make a decision as to how the driver src.rpm packages will be named, so we can begin packaging them, and also let the installer team and other groups know what they're called. As such we are soliciting feedback from the Fedora community, including Red Hat developers and subsystem maintainers. Proposal: ~~~~~~~~ Here is my initial proposal for naming the src.rpms, along with brief rationale, and the real (or perceived) advantages and disadvantages: xorg-x11-driver-<type>-<name> The prefix of "xorg-x11" identifies the driver package as being an official part of the upstream X.Org project. This distinguishes the driver from one that might be provided by the "dri" project, the "gatos" project, or any other project. It makes it easier to substitute alternative driver packages that provide a driver of the same name. It also makes it clear to the user, the developer, and anyone else looking at the package, that the source code contained inside came from X.Org directly. It also makes it clearer where bug reports should be filed upstream. As such, I propose all driver packages start with the "xorg-x11-" prefix. The "driver" portion of the proposed name, indicates that it is a driver for the X server, much like "module" in kernel-module packages. It namespaces all drivers, so that they all appear together in directory listings, and are easy to group together from scripts using globs, etc. ie: # Install all of the xorg drivers: rpm -ivh xorg-x11-driver-*.rpm The "<type>" attribute is either "video" or "input", as used in the upstream tarball names, and further namespaces things, so that all input drivers appear together, and all video drivers appear together. This makes it easy to handle all input drivers or all video drivers from a script as well. Finally, the driver <name> field, is the official name of the driver from the upstream tarballs, which generally is the name of the driver binary that gets installed as well. Using this naming mechanism, I believe gives us the most flexibility with driver packages, and makes life a lot easier down the line as far as maintenance goes. It also makes the packages very obvious as to what their contents are. The only slight disadvantage to this naming scheme that comes to mind which someone might point out, is that the package names are semi-lengthy. I don't see this as a problem however, as all modern shells have filename completion, and it works quite well. The benefits of clarity of contents, directory listing grouping, CVS repository grouping, bugzilla grouping, etc., IMHO far outweigh any perceived disadvantages of lenthy names. Request for comments: ~~~~~~~~~~~~~~~~~~~~ Interested Fedora Core, Fedora Extras, or community developers who have an opinion about the X.Org modular package naming conventions, or who just want to provide feedback concerning the above proposal, are encouraged to respond to this RFC on or before Monday August 29th if possible. We look forward to hearing everyone's feedback, and incorporating the collective concious of the community into our decision making efforts. Thanks for your considerations. -- Mike A. Harris Red Hat Canada, Ltd. X Devel Team ------------ regards Rahul
Posted Aug 29, 2005 17:47 UTC (Mon)
by stevenj (guest, #421)
[Link] (7 responses)
Note that X.org recently made Version 7.0 Release Candidate Zero available for their modular tree, which currently contains almost 500 individual packages
Posted Aug 30, 2005 16:54 UTC (Tue)
by stef70 (guest, #14813)
[Link] (6 responses)
That seems a lot but in fact the modularization will make most of the packages optionnal:
Posted Aug 30, 2005 17:03 UTC (Tue)
by corbet (editor, #1)
[Link] (3 responses)
Posted Aug 30, 2005 17:25 UTC (Tue)
by stef70 (guest, #14813)
[Link]
Nevertheless, most of the new X11 users are now running a desktop like KDE or Gnome and can probably live without ever starting any of those applications.
Posted Sep 1, 2005 0:35 UTC (Thu)
by darktjm (guest, #19598)
[Link]
Posted Sep 1, 2005 14:37 UTC (Thu)
by bkw1a (subscriber, #4101)
[Link]
http://www.eecs.umich.edu/~pelzlpj/orpie/
RPN. Text. Mmmmmmmm!
Posted Sep 1, 2005 21:30 UTC (Thu)
by tzafrir (subscriber, #11501)
[Link]
As for video cards: distros will probably install all of them by default, because it's much more convinient: you don't want to start installing an RPM the moment somebody plugs in a card.
Posted Aug 29, 2005 18:27 UTC (Mon)
by jwb (guest, #15467)
[Link] (7 responses)
Oh, and not to forget the worst part, which is dozens of copies of the DEC copyright statement polluting your /usr/share/doc and package database.
To summarize: modularization good, pathological modularization bad.
Posted Aug 29, 2005 18:53 UTC (Mon)
by arcticwolf (guest, #8341)
[Link] (5 responses)
Posted Aug 29, 2005 19:35 UTC (Mon)
by elanthis (guest, #6227)
[Link] (4 responses)
Choice is all well and dandy, but it isn't always practical, nor is it always useful. It needs to be weighed against its costs in each case.
Besides, even with a monolithic RPM of some large software chunk like X.org, you always have the choice of modifying the package or installing from source. ;-)
Posted Aug 29, 2005 19:52 UTC (Mon)
by maney (subscriber, #12630)
[Link] (3 responses)
Posted Aug 29, 2005 21:16 UTC (Mon)
by tzafrir (subscriber, #11501)
[Link] (1 responses)
Compare that to libc6 and to coreutils (that were actually merged from three packages to 1 a couple of years ago). Each of the two is larger.
Posted Sep 1, 2005 14:44 UTC (Thu)
by evgeny (subscriber, #774)
[Link]
Posted Aug 30, 2005 7:59 UTC (Tue)
by niner (subscriber, #26151)
[Link]
In case of Fedora the package system would be rpm. And as I see on my SuSE system, rpm can handle patch rpms and binary deltas, so even after a complete reinstall of SuSE 9.3 (which is soon half a year old) I still do not have to download more than 10MB of security updates, including kernel and kernel source.
This is the way and not uselessly modularizing every big package and thus slowing the system down with huge package databases.
Posted Aug 30, 2005 2:02 UTC (Tue)
by daniels (subscriber, #16193)
[Link]
Posted Aug 29, 2005 19:55 UTC (Mon)
by job (guest, #670)
[Link] (6 responses)
Posted Aug 29, 2005 20:07 UTC (Mon)
by proski (subscriber, #104)
[Link] (4 responses)
Posted Aug 30, 2005 1:54 UTC (Tue)
by brouhaha (subscriber, #1698)
[Link] (2 responses)
At the time, a few vocal people shouted me down, with ridiculous complaints that it would make the system less secure. That was (and is) complete BS, because any process that runs SUID root can wreak just as much havoc using /dev/mem and other means as it could with my proposed /proc/pci device files.
Anyhow, what I was proposing back then was that the /proc/pci device files would normally have permissions like 600, but nowdays there's no reason that more sophisticated access control couldn't be used, in order to allow a non-SUID X server to access the video card.
This also would remove the redundant PCI bus scanning code from the X server. The kernel already has done it, so there's no reason for the X server to do it again.
Posted Aug 31, 2005 0:54 UTC (Wed)
by proski (subscriber, #104)
[Link] (1 responses)
I hope you would agree that permitting raw PCI access for non-privileged applications would mean a much bigger security problem than running X server suid root.
Posted Sep 1, 2005 10:37 UTC (Thu)
by farnz (subscriber, #17727)
[Link]
Posted Sep 1, 2005 9:56 UTC (Thu)
by job (guest, #670)
[Link]
And a most people with modern PCs probably already have a video driver in the kernel, either the nvidia/ati proprietary ones or DRI. That API could be extended with some protection/separation and 2D acceleration and used by all the graphics subsystems for Linux.
My main point is that I would like the privileged part of X11 to become smaller by orders of magnitude. There is a lot of code there running with full privileges that doesn't really need them. And root access is for all practical purposes just as bad as kernel level access.
Posted Aug 29, 2005 21:40 UTC (Mon)
by tzafrir (subscriber, #11501)
[Link]
The "normal" X server requires some priviliges to run efficiently.
Not related to modularity.
Posted Aug 29, 2005 20:01 UTC (Mon)
by proski (subscriber, #104)
[Link]
More elaborate solution would be providing deltas between released binaries. This way, a one-line fix in the source could result in a relatively short download.
Posted Aug 29, 2005 20:31 UTC (Mon)
by mgb (guest, #3226)
[Link] (7 responses)
If only the kernel hackers could keep the driver API stable for
Posted Aug 29, 2005 20:50 UTC (Mon)
by proski (subscriber, #104)
[Link] (5 responses)
Posted Aug 29, 2005 20:59 UTC (Mon)
by corbet (editor, #1)
[Link] (4 responses)
See LWN'a 2.6 API changes page. For starters, the original poster, given the comment, is probably unhappy with changes which break binary modules, and there's been quite a few of those. Source-incompatible changes include the removal of devfs, the USB timeout value units change, the kobject_add() and kobject_del() semantic change (no longer generate hotplug events), the prototype change for kobj_map(), the removal of bcopy(), and the whole pm_message_t change in the PCI subsystem. That's just looking back to 2.6.11, released six months ago.
Posted Aug 29, 2005 21:45 UTC (Mon)
by proski (subscriber, #104)
[Link]
You make a good point about binary compatibility. Indeed, it was never promised by kernel developers, and it can be broken even in 2.0 series. I cannot imagine rpm allowing to upgrade some drivers from 2.4.30 with drivers from 2.4.31 without upgrading the kernel. On the other hand, upgrading some drivers from 2.4.31-1 to 2.4.31-2 should be OK, and thats up to the distributor to ensure.
I can imagine that binary compatibility will be promised for certain kernel series in the future (e.g. 2.4.x, 2.6.x.y) provided that .config is unchanged. But I'm not so optimistic about the main development branch.
After all, changing headers and removing obsolete API is a way to force drivers to change and to adopt a new way of doing things. It's a way to keep the code safe and working the way it should. If a driver doesn't compile, it needs updating.
Posted Aug 29, 2005 21:46 UTC (Mon)
by vonbrand (subscriber, #4458)
[Link] (2 responses)
There is (userland) API and (in-kernel) API. I understood the former in the grandparent...
Posted Aug 29, 2005 21:51 UTC (Mon)
by proski (subscriber, #104)
[Link] (1 responses)
Posted Aug 30, 2005 16:03 UTC (Tue)
by obi (guest, #5784)
[Link]
Drivers _are_ userland, except for some that use the DRM (mostly for 3D functionality), which isn't distributed/integrated with Xorg but with the kernel.
Which drivers are you referring to exactly?
Posted Aug 29, 2005 21:51 UTC (Mon)
by khim (subscriber, #9252)
[Link]
If only the kernel hackers could keep the driver API stable for more than five minutes, Linux-based systems could work with a lot more widgets and Linux would be a lot easier concept to sell. This way lies madness. I'll be frank: it'll reduce number of widgets and make it no better then other OSes. Why ? Easy: the only way to keep something binary compatible is not change anything. I've seen ennnough drivers broken by Windows Service Packs and/or Solaris upgrades. And in most cases there are no way to use old hardware (and old widgets) with new versions of OS: "Oh, our year-old gadget does not work with your brand-new OS release ? Too bad: buy our super-brand-new gadget+ !". True: Windows or MacOS aften support new toys better then Linux. But I can use my old toys as long as there are enough users to keep driver in working state - not as long as hardware manufacturer is interested. From my expirience hardware manufacturer attention span are miniscule at best: sometimes even toys one-two years old are unsupported. And if you still want russian roulette with your drivers... you know where to go, right ?
The article description is a bit confusing, because it might be read as saying that Fedora is starting their own independent effort to modularize X.org, whereas it looks like they are simply figuring out how to package what is already being done.
Note that X.Org has its own modularization project
Actually, there are only about 270 packages since each one is provided twice (.gz and .bz2).Note that X.Org has its own modularization project
- 37 fonts including some for 'exotic' languages (arabic, cyrilic, ...) and old terminals (ibm, sun, dec).
- 28 drivers for input devices you probably never heard of (jamstudio, hyperpen , penmount, tek4957, ... )
- 41 video drivers (anyone using more than one or two?)
- 42 libraries (you probably want those) plus 30 packages for their prototypes (for developpers)
- The X server itself.
- The remaing 100 packages are old legacy X11 applications. Most of them are quite useless nowadays (viewres, appres, xmessage, xwd, xwud, xcalc, xclipboard, ...).
Hey! I still use xcalc! Those newfangled calculators look fancy, but xcalc supports RPN mode. Must be time for a grumpy editor's guide to on-screen calculators...
Useless apps
To be honest, I occasionally use some of those old X11 applications (xset, xdpyinfo, xhost, xrefresh, xkill, xgamma). Hardly xcalc! I would rather use 'bc' or one of the matlab clones such as octave. Useless apps
Funny, I never realized xcalc supported RPN (never bothered reading the manual for something I thought was a too-simple calculator). Then again, it says "HP10C", which implies 4-item stack limit, which is too little for me (I started RPN w/ forth and moved to HP28 & its successors, which have "infinite" stacks). There are a large number of other X calculators that support RPN, though (alas, many are very heavy, but that's modern software for you). The ones I know of from back when I wanted something for my PDA: galculator, which looks a lot like a better, slightly programmable, xcalc (complete with 4 item stack limit), pgcalc, which can look a lot like a real HP calculator, but isn't even remotely, and doesn't seem to support X paste, grpn, which is RPN-only, and of course various HP calculator emulators and lookalikes. I just use a real HP calculator on my desk when I don't need cut & paste and bc or sh $(()) (algebraic, but dirt simple) when I do need cut & paste. Plus, as noted above, there are even heavier apps for calculation, like scientific number crunchers and CASs.
xcalc
Gotta put in a plug for orpie:[offtopic] Another calculator: orpie
xmessage is used in many scripts.Note that X.Org has its own modularization project
I understand driver modularization and I wish the Fedora project success. However I hope they don't go crazy and hyper-modularize in the way Ubuntu has. In Ubuntu 5.10 Breezy, important binaries like xset, xmodmap, xrdb and so forth are in their own packages. This really increases the chances that a user will be able to install Ubuntu but not have a working X11 setup. Can you imagine all the mysterious and weird ways that X11 apps and their launcher scripts will break if you don't have xset? The surprise of a long-time X11 user when their .Xresources aren't merged?Fedora: RFC: X.Org X11 modularization project - rpm package driver naming
Are these things installed by default? If yes, then offering more choice to users who actually *do* know what they're doing is a good thing.Fedora: RFC: X.Org X11 modularization project - rpm package driver naming
If that choice is useful, sure. If it's a case of, "well, a user might know what they're doing, and might conceivably think up some wacky reason to not need this but still need that, we don't know, it sounds good though," then the modularization is wasted effort on packaging and maintenance for no good reason.Fedora: RFC: X.Org X11 modularization project - rpm package driver naming
Or, as the article says (and even the excerpt mentions this, albeit specifically about drivers), it allows a finer grained update when required to fix serious bugs or security issues. Now a buffer overflow in one program needn't cause everyone to fetch 50MB of mostly-unchanged binaries - that's worth quite a bit, actually.
Fedora: RFC: X.Org X11 modularization project - rpm package driver naming
On Debian Sarge the size of the deb xbase-clients is 2MB, and the instlled package size is 5MB . Fedora: RFC: X.Org X11 modularization project - rpm package driver naming
You missed the dependencies issue. E.g., out of the entire xbase-clients package I need only xauth for the ssh X forwarding. But some other executables from this package are linked against Mesa, so I have to install it too (another 11 MB). Instead of mere 32kb of a single exec, I end up with 16MB and a few dozens of absolutely unused files - which need to be periodically updated, backed up, checked for intrusion violations, etc. So modularization is a good thing. Of course, a dummy meta package can be defined (under the old name) that depends on all the individual components to make life easy for inexperienced users.Fedora: RFC: X.Org X11 modularization project - rpm package driver naming
If you have to download a whole 50MB package to fix a security issue in just one 20kb binary, then your package system is flawed and should be fixed, and not the packages ajusted to this broken system.Fedora: RFC: X.Org X11 modularization project - rpm package driver naming
This is how it has been done upstream -- see http://cvs.freedesktop.org/xorg/app/. In the end, we felt that keeping closer to upstream was more important than grouping huge hunks of source packages into large binary packages. I don't think that any of those will realistically change much at all anyway, so the only problem you have is the huge gobs of MIT boilerplates taking up your diskspace, at which point you have bigger problems anyway.Fedora: RFC: X.Org X11 modularization project - rpm package driver naming
Separate drivers sounds like a good thing, but I long for the day when I can get a non-SUID X11 server on my desktop. There's just too much of X running with complete access to my system. No, ACLs and SELinux doesn't solve this as long as the server process(es) aren't modular enough. At least this is a step in the right direction.Fedora: RFC: X.Org X11 modularization project - rpm package driver naming
The trade-off would be having the driver in the kernel, which is not necessarily safer than a SUID process.
Fedora: RFC: X.Org X11 modularization project - rpm package driver naming
It wouldn't have to be a kernel-level driver. There's no reason why the kernel can't make device files available that can be mmap()ed to get at the PCI card memory spaces. I experimented with this almost ten years ago just mmap()ing /dev/mem to get at the frame buffer, and proposed on lkml that /proc/pci should provide device files representing each memory space.
Fedora: RFC: X.Org X11 modularization project - rpm package driver naming
As far as I know, all new security extensions can only restrict access but not to add new permissions. You cannot start with a non-suid binary and give it an "endorsement" it to access PCI bus. You still have to start with a suid binary and restrict its permissions, which is what you can do now without any code changes.
Fedora: RFC: X.Org X11 modularization project - rpm package driver naming
It's not a security extension; it's a set of devices which expose raw PCI access to userspace. You can then chmod/chown/chgrp each PCI device suitably (so that the "graphics" group can play with the video card), then run X SGID as graphics.
Fedora: RFC: X.Org X11 modularization project - rpm package driver naming
I would like a kernel video driver, if the interface is good enough it has the opportunity to really clean up the mess that is svgalib, directfb, x11 so VC switching works properly between them. Hopefully in a better state than today so it is trivial for an app to use several VCs, several monitors etc.Fedora: RFC: X.Org X11 modularization project - rpm package driver naming
This already exists: Xnest. Has existed for quite a while, as a standard part of X11 (since when?)Fedora: RFC: X.Org X11 modularization project - rpm package driver naming
For starters, it requires write access to a virtual terminal. And this is but the least of the priviliges required.
Fix rpm/yum instead
One of the benefits of the modularized X.Org X11R7, is that it makes
it much easier for us to provide individual driver updates without
having to release the entire 150Mb monolithic X release.
I believe the fix belongs elsewhere. There should be a way to update source packages without forcing the users to update all binary packages generated from them.
Perhaps the simplest way would be to have checksums of all binary package contents and all metadata (except the revision number and perhaps changelog) available for download. This way, yum or apt-get could simply update the version, but not the contents, if they detect the same checksum.
"The ability for us to update a single driver, and release thatLinux too please
single driver to users is an important thing for a modern OS, in
particular for desktop systems."
more than five minutes, Linux-based systems could work with a
lot more widgets and Linux would be a lot easier concept to sell.
Could you please give us an example of non-backward-compatible API change in the stable kernel series during the last year? I'm not aware of any.
Linux too please
Linux too please
Could you please give us an example of non-backward-compatible API change in the stable kernel series during the last year? I'm not aware of any.
I don't think 2.6 kernels are considered stable (meaning that the code is changing a lot, not that they don't work well). I'm very well aware of the changes in 2.6 kernels, as I have to update my drivers all the time. 2.6.13 required major changes in PCMCIA drivers (actually, I have to admit partial responsibility for them). On the other hand, 2.4 kernels have stabilized long time ago.
Linux too please
Linux too please
Unfortunately, kernel-userland API is irrelevant when we are talking about being able to "update a single driver".
Linux too please
I thought we were talking about Xorg here?Linux too please
Linux too please