Seeking consensus on dh

Posted Jun 6, 2019 2:19 UTC (Thu) by filbranden (guest, #87848)
Parent article: Seeking consensus on dh

For someone coming from the RPM world, the first time I had to deal with this non-sense I was horrified.

What, 4 different build systems for a package? (debhelper, dh, CDBS and perhaps I'm wrong but I recall there might have been another?) Not to mention that most of this is built on top of a makefile (rules), a LOT of Perl, some control files using RFC822(!?!)-like headers and pedantic requirements for the changelog.

For some reason, after looking at building DEBs, RPM spec files looked a lot less ugly than they used to.

Oh, and when I actually needed to build a couple hundred of DEBs, I ended up building a tarball and turning that into a DEB, that was the only sane approach really. (https://docs.bazel.build/versions/master/be/pkg.html#pkg_deb)

Seeking consensus on dh

Posted Jun 6, 2019 2:34 UTC (Thu) by mirabilos (subscriber, #84359) [Link] (10 responses)

Ugh, no thanks, RPM files are underspecified crap,
incompatible between distros, and the syntax is horrifying.
Also, commented-out macros are still parsed… no, thanks!

Seeking consensus on dh

Posted Jun 6, 2019 7:51 UTC (Thu) by jond (subscriber, #37669) [Link] (4 responses)

Most people's complaints about RPM are at the layer below what OP is complaining about: the horrid binary format, strange uncompressed key/value store in the metadata, and all that stuff; OP is talking about the complexity of the source level, and having dh, debhelper, cdbs, etc. *is* a problem. There are some things that are nicer about the RPM source workflow (not everything, but some things)

Seeking consensus on dh

Posted Jun 6, 2019 9:17 UTC (Thu) by epa (subscriber, #39769) [Link] (3 responses)

So could one hack up a tool to build a dpkg from an rpm spec file?

Seeking consensus on dh

Posted Jun 6, 2019 10:06 UTC (Thu) by smurf (subscriber, #17840) [Link] (1 responses)

Sure you could, but given the impedance mismatch (RPM specs include some things Debian's don't, and vice versa; embedded scripts contain Redhat/RPM-isms, etc.) it'd make more sense to build a debian/ subdirectory from it.

The other way would be to process the RPM spec with, well, RPM, and the re-package the resulting RPM binary package to a .deb archive. This is reasonably simple to do, and flexible enough when you allow the user to edit the intermediate result.

I leave the question of whether getting RPM to run on Debian systems would be easier than writing a spec converter to whoever tries both and compares the effort involved. ;-)

Seeking consensus on dh

Posted Jun 6, 2019 13:10 UTC (Thu) by epa (subscriber, #39769) [Link]

RPM has been packaged as part of Debian since time immemorial, but as we know that's not really the point.

Seeking consensus on dh

Posted Jun 7, 2019 10:54 UTC (Fri) by Conan_Kudo (subscriber, #103240) [Link]

No need to hack up one, it already exists: https://github.com/ascherer/debbuild

I use it for building Debian packages all the time. It doesn't fully support everything a spec file has, but it's got a good chunk of it already.

Seeking consensus on dh

Posted Jun 14, 2019 11:27 UTC (Fri) by Wol (subscriber, #4433) [Link] (4 responses)

> Ugh, no thanks, RPM files are underspecified crap,
incompatible between distros,

It's not RPM's fault that different distros have different *POLICIES* - not helped by the fact that SUSE (the second major user of rpm) actually PRE-dates Red Hat and rpm.

So it IS possible to retro-fit rpm to a completely different set of packages, because that's what SUSE did all those years ago :-) Dunno how easy that would be with deb, I'm unaware of any example where that's been done.

Cheers,
Wol

Seeking consensus on dh

Posted Jun 14, 2019 15:23 UTC (Fri) by smurf (subscriber, #17840) [Link] (3 responses)

It's not a question of policy, it's a question of a file format that is somewhat unspecified, contains redundant fields, and is unnecessarily complex.

Contrast that with .deb which contains specific files in a specific format that's simple enough to be created and processed by generic archive packers (ar and tar) and shell scripts. (It actually was, in Debian's first iteration.)

Seeking consensus on dh

Posted Jun 14, 2019 15:37 UTC (Fri) by rahulsundaram (subscriber, #21946) [Link]

>It's not a question of policy

Certainly a question of policy too given that does differ occasionally between even Debian and its derivatives causing incompatibility

> it's a question of a file format that is somewhat unspecified, contains redundant fields, and is unnecessarily complex.

I am not sure what any of this really means. Specific examples would help. Pick current ones instead of things solved ages ago please.

Seeking consensus on dh

Posted Jun 14, 2019 15:42 UTC (Fri) by pizza (subscriber, #46) [Link] (1 responses)

So, are you claiming that extracting debs or rpms using the equivalent of a rock lashed to a stick is something that is remotely relevant in the past 15 years of bootable USB sticks or the additional 10 years of bootable rescue CDs?

Or are you claiming that someone would ever be insane enough to _create_ a deb or rpm package using that same flint axe? Talk about not valuing one's own time...

Seeking consensus on dh

Posted Jun 23, 2019 20:06 UTC (Sun) by mirabilos (subscriber, #84359) [Link]

FWIW, I use the official .deb of a program as part of the “sources” in
another, resource-constrained, operating environment; it contains the
PDF compiled from Teχ sources (while that other OE does not have Teχ
available, but a PDF viewer, so the full documentation is usable)

Seeking consensus on dh

Posted Jun 6, 2019 3:59 UTC (Thu) by luto (guest, #39314) [Link]

I feel the same way. The few times I’ve fiddled with Debian packages, I’ve wondered why there is more than one way I do it. I think there is way too much flexibility.

(I’m not saying RPM is great. But there is more or less one way to write a spec file and there is exactly one way to build it.)

Seeking consensus on dh

Posted Jun 6, 2019 4:37 UTC (Thu) by anguslees (subscriber, #7131) [Link] (5 responses)

There is a single way to package everything, and that is the `./debian/rules` (makefile) script. It is provided various standard arguments and environment variables, and expected to (eventually) produce .deb and other output files.

Naturally writing similar debian/rules boilerplate for similar upstream packages is undesirable, so various "helper" tools have been developed over the last few decades. It's easy to see why the "best" tool for a python module might not be the best tool for a kernel module, or a font.

To me (coming from the Debian world), it's horrifying that rpm mandates a centralised set of rpm functionality across such diverse needs ;) Imagine what we'd be stuck with if the the original dpkg standards had mandated whatever the most popular helper was 20+ years ago, without experience or a process for change!

Seeking consensus on dh

Posted Jun 6, 2019 9:26 UTC (Thu) by epa (subscriber, #39769) [Link] (2 responses)

This sounds like the same argument about autoconf/automake versus various other build systems. The standard way to build is with a Makefile; but of course you don't want to write that all by hand every time, so we need this helper script to generate it... Before you know where you are you're in a world of m4 macros, self-configuring shell scripts that generate other shell scripts, and generally a tower of abstraction that makes some things easy but other things very hard.

You could choose to simplify all that by having a build tool like scons or meson which works from a single configuration language and has richer built-in functionality than plain old make. And if there are certain projects where the tool doesn't work well, it might be a better investment of time to adapt it for them, rather than pay the cost of having lots of different build systems.

You are right, the best tool to generate debian/rules for a Python script might not be the best tool to generate it for a kernel module. But there is an implicit assumption: that you need to generate the debian/rules file at all. Perhaps a better build system would start with a less verbose configuration file that doesn't need to be generated, but can be handwritten without too much boilerplate.

Seeking consensus on dh

Posted Jun 6, 2019 9:56 UTC (Thu) by smurf (subscriber, #17840) [Link]

Well, "dh" basically allows you to write a less-verbose configuration file -- debian/rules now consists of five lines of fixed boilerplate. You might want to set what dh calls the build system (Python packages don't have Makefiles), but that's it, at least to start with.

Yes, building Haskell needs a "dh_haskell" helper. I'm reasonably positive somebody will write one, now that CDBS is on its way to being deprecated.

Seeking consensus on dh

Posted Jun 6, 2019 15:33 UTC (Thu) by rweikusat2 (subscriber, #117920) [Link]

Isn't applicable here at all. The 'standard' way to build a .deb-file from a debianized soucre tree is to use the debian/rules Makefile to build a target called 'binary' (unchanged since at least 1999). 'dh' is a set of tools implementing a sequence of steps for building a debian/rules target. For the most simple case, the rules will just contain

%:
dh $@

ie, whatever target is supposed to be built, build it by invoking the dh program with the target name as argument. 'dh binary' ends up executing 50 dh subcommands in a certain sequence, all of which can be overriden individually by putting Makefile targets named 'override_dh_<something>' with dh_<something> being a dh subcommand into debian/rules which will be activated instead of the default commands if they exist.

This is a pretty flexible and nevertheless easy-to-use approach: The most complicated Debian package I've created so far overrides just eight of these 50 subcommands and usually, the overrides just do something in addition to the default operation.

Seeking consensus on dh

Posted Jun 10, 2019 8:42 UTC (Mon) by LtWorf (subscriber, #124958) [Link] (1 responses)

RPM has a lot of duplication. All those postinst scripts that dh_whatever generate are not automatically generated.

Seeking consensus on dh

Posted Jun 10, 2019 11:31 UTC (Mon) by rahulsundaram (subscriber, #21946) [Link]

> RPM has a lot of duplication. All those postinst scripts that dh_whatever generate are not automatically generated.

It depends on how you write spec files. The recommended practice in distributions that use RPM like Fedora have been to move away from scripts to macros and file based triggers that avoid said duplication. Scriptlets should be reserved for a one off use cases.

Seeking consensus on dh

Posted Jun 6, 2019 6:11 UTC (Thu) by tzafrir (subscriber, #11501) [Link]

It's funny that your rpm generator function gets a spec file as a parameter, whereas your deb generator function gets description, homepage, and such rather than a control file.

Seeking consensus on dh

Posted Jun 6, 2019 8:19 UTC (Thu) by dskoll (subscriber, #1630) [Link] (9 responses)

Building Debian packages is indeed an exercise in thrashing around until it works. Debhelper and dh are sprawling messes; I have yet to find "big picture" documentation that puts everything in context so you can understand what's actually going on.

Seeking consensus on dh

Posted Jun 6, 2019 9:10 UTC (Thu) by smurf (subscriber, #17840) [Link] (2 responses)

Debian's policy manual describes how the source looks like and what the resulting package is expected to contain. The steps from A to B are well-documented; if you don't know how to do step X you run "dh_X -v". What more do you need?

This is easy stuff.

Seeking consensus on dh

Posted Jun 6, 2019 11:27 UTC (Thu) by dskoll (subscriber, #1630) [Link]

Just what I said: overview documentation rather than (or in addition to) dense reference documentation.

Seeking consensus on dh

Posted Jun 6, 2019 22:09 UTC (Thu) by samuelkarp (subscriber, #131165) [Link]

When trying to package something for Debian the first time, I found this difficult as well. I'm fortunate to work with a DD who was able to guide me, but the policy manual was not very helpful to me. The policy manual seems to focus on details of the resulting package (as is important for policy), but not how to create a package for the first time.

Seeking consensus on dh

Posted Jun 6, 2019 15:41 UTC (Thu) by rweikusat2 (subscriber, #117920) [Link] (3 responses)

I've been actively maintaining as much as 91 Debian packages in the past, being a mix of customized 'standard' Debian packages, completely original developments and specifically packaged open source packages in the past (presently only 28) and it's nothing like this.

Seeking consensus on dh

Posted Jun 6, 2019 16:55 UTC (Thu) by antiphase (subscriber, #111993) [Link] (2 responses)

It's possible you're not the target audience for the missing documentation then.

Seeking consensus on dh

Posted Jun 6, 2019 17:22 UTC (Thu) by rweikusat2 (subscriber, #117920) [Link] (1 responses)

When I started doing this (IIRC, some time in 2011), I didn't have anything but the available documentation to learn about it (I'm not a Debian developer, I'm doing this for the software of my employer).

Seeking consensus on dh

Posted Jun 6, 2019 17:34 UTC (Thu) by rahulsundaram (subscriber, #21946) [Link]

One has to recognize that different people find different approaches more effective for them. Typically you want to provide both the overview type documentation as well as the reference type documentation. Others might even learn by doing watching some videos online or doing workshops in person.

Seeking consensus on dh

Posted Jun 7, 2019 2:05 UTC (Fri) by murukesh (subscriber, #97031) [Link]

Have you read the New Maintainers' Guide? https://www.debian.org/doc/manuals/maint-guide/

Seeking consensus on dh

Posted Jun 7, 2019 17:22 UTC (Fri) by k8to (guest, #15413) [Link]

I have to agree. There's no clear entry point in the developer guide to put the pieces together, and a lot is left out.

I know enough about .deb files and practice that I've written tooling and scripts to do mass investigations of how customers have broken their systems etc, or how my former employer's release team had broken our customers etc. I've created shim packages to avoid installing things that the package graph required. But I still have no idea how to go about creating a sane debian developer style package, despite trying.

I'm sure it's possible to succeed from the published docs, but I'd bet most developers start out by cribbing from another package.

Seeking consensus on dh

Posted Jun 6, 2019 9:04 UTC (Thu) by smurf (subscriber, #17840) [Link]

Well, to be nitpicky about it, from a high-level point of view there's exactly one build system: you call "debian/rules" with one of a few documented arguments.

The fact that this is processed by /usr/bin/make is just convenience, even though that's actually mandated by Debian's policy these days. The various build helpers / "build systems" also are, or started as, simply convenience; you're free to use debhelper/dh to generate a plaintext Makefile instead, though I'd wonder why you'd want to do that.

Frankly, I don't understand your problem with mass-building DEBs. Debian packaging is little more than "check that your Makefile obeys $DESTDIR (which you also need for RPM), run "dh_make -n" (or manually create three boilerplate files and one somewhat-less-complicated-than-an-rpmspec file), run 'debuild'. Done.". Yes, you need a heap of details to make them distributable (dependencies, documentation package, debug package, post-install script, …), but you need the same details in an RPM specfile.

Alternately, in the context of your build system (and the low-level point of view you actually need for it), things are even easier. You'd create the destination tree in debian/PACKAGENAME, write a few lines to a reasonably documented file in the DEBIAN/ subdirectory thereof, create the md5sums file ("dh_md5sums -v" tells you how to do that, if you don't want to depend on debhelper behing installed), and run "dpkg-deb -b debian/PACKAGENAME". Or, if you want to run on a non-Debian system, you can assemble the .deb yourself, it's not difficult (quite in contrast to RPM).

Any reasonably competent Debian developer could have told you this in ten minutes.