Python cryptography, Rust, and Gentoo [LWN.net]

Python cryptography, Rust, and Gentoo

Posted Feb 12, 2021 11:14 UTC (Fri) by LtWorf (subscriber, #124958)
In reply to: Python cryptography, Rust, and Gentoo by roc
Parent article: Python cryptography, Rust, and Gentoo

> Adding a step before the build that makes sure the library package is installed, e.g. providing instructions *per-distro* to install it manually, making my software harder to build

You mean a README file with a list of dependencies? I'm sure people on a certain distribution know how to use their package manager.
If they don't know, they won't be compiling your software anyway, because they don't know how to install a compiler.

> For distros that don't package the library (or package a version of it that's older than I need), providing instructions to build and install that library manually, making my software even harder to build

Users of stable distributions are familiar with the issue.

> Making sure my software builds and runs with a range of library versions packaged by different distros and distro versions, potentially packaged in different ways with different directory layouts etc across distros.

That is incentive to:
1. Do not depend on amateur libraries that change API
2. Use autotools and let it figure out all this stuff

> On platforms like Windows

There are no package managers on windows. So that is a completely different situation. But anyway you won't be using the same binary on linux and windows.

to post comments

Python cryptography, Rust, and Gentoo

Posted Feb 12, 2021 13:57 UTC (Fri) by mathstuf (subscriber, #69389) [Link] (4 responses)

> There are no package managers on windows.

There are. They're not as mature as anything Linux has AFAIK, but there is at least (in no particular order):

- vcpkg (probably the most useful for the discussion at hand)
- Conan (CMake-based)
- chocolatey (binary-based, includes Visual Studio/MSVC packages)
- anaconda (scientific/Python oriented, but has other bits too)

I think there's another, but I can't remember it's name. There's also zero surprise from me if there are others I haven't heard of.

Python cryptography, Rust, and Gentoo

Posted Feb 12, 2021 21:58 UTC (Fri) by roc (subscriber, #30627) [Link] (3 responses)

You really need a single standard package manager, preferably shipped with the OS, so there is a high chance users already have it installed and "install package manager" doesn't just make your installation process longer.

Python cryptography, Rust, and Gentoo

Posted Feb 13, 2021 13:35 UTC (Sat) by mathstuf (subscriber, #69389) [Link] (2 responses)

While true, I think what I'd do is use vcpkg for developer management, bundle everything up into a single package and ship that via normal means (installer/relocatable zip) and maybe chocolatey depending on the tool target audience.

Anaconda would probably be better if you're already in that realm though.

Python cryptography, Rust, and Gentoo

Posted Feb 13, 2021 20:39 UTC (Sat) by roc (subscriber, #30627) [Link] (1 responses)

vckpg looks cool, thanks for pointing to it. But it looks more like "cargo for C++" than a distro package manager.

Python cryptography, Rust, and Gentoo

Posted Feb 13, 2021 20:41 UTC (Sat) by roc (subscriber, #30627) [Link]

Of course, as such, it may be a good answer to the problem of "how do I consume C third-party libraries" which was the original issue before we got into a discussion of the "use distro packages" non-solution.

Python cryptography, Rust, and Gentoo

Posted Feb 12, 2021 21:56 UTC (Fri) by roc (subscriber, #30627) [Link] (1 responses)

> You mean a README file with a list of dependencies? I'm sure people on a certain distribution know how to use their package manager.

Sure. One problem is, that package list changes with each distribution and sometimes within versions of each distribution. So we only have instructions for Fedora and Ubuntu, and those instructions are wrong for some versions of those distros.

Anyway, every single step makes the software harder to build.

> Users of stable distributions are familiar with the issue.

So? The issues still exist.

> That is incentive to:
> 1. Do not depend on amateur libraries that change API

Distro policies are responsible for varying file layouts and naming conventions. Distros also make varying decisions about library versions and which features they enable at build time.

> 2. Use autotools and let it figure out all this stuff

Autotools are a nightmare and writing autotools feature tests for everything I care about would be a ton of extra work.

> you won't be using the same binary on linux and windows.

Indeed, but once you've done the work to build Windows binary you can reuse that work to build a Linux binary with vendored libraries.

People arguing that the Right Way to build C/C++ software is to make it Linux only, use distro libraries, do a ton of extra work, and downgrade the performance and functionality of that software to fit the shipped libraries, are not doing C/C++ any favours.

Python cryptography, Rust, and Gentoo

Posted Feb 13, 2021 8:12 UTC (Sat) by abartlet (subscriber, #3928) [Link]

> Sure. One problem is, that package list changes with each distribution and sometimes within versions of each distribution. So we only have instructions for Fedora and Ubuntu, and those instructions are wrong for some versions of those distros.

This got so bad for Samba (both the distribution versions to cover and the Samba versions to cover) that we ended up building a massive infrastructure to:
- Create Docker images for CI
- Test every build in all the supported distributions
- Publish a 'install dependencies for samba' script.

Just looking at the table here: https://wiki.samba.org/index.php/Package_Dependencies_Req... to see where this ends up.

Even the source data for those generated scripts, for a single release is quite complex: https://gitlab.com/samba-team/samba/-/blob/master/bootstr...

So for software of any serious size, it is not just a README with a list of dependencies. Furthermore, Samba has found we have to have configure checks looking for the library (otherwise folks complain that their build failed) and to make those fail by default (not 'auto-detect' and work around) because otherwise features just go missing.

All in all it is hard to argue that this is really a good vision to match.