Python cryptography, Rust, and Gentoo

A goal of an application developer is to provide value to users. Around 99% of users use iOS/Android/Windows/macOS, not classic distros.

For all distros that any of my users might conceivably use? And then wait for several years for users to actually update to distro versions where the new package is present? No, that is completely unreasonable.

It's actually kind of breathtaking what you're suggesting here --- become a member of many different distro communities, learn all their different processes, persuade all of them to accept the library (what if they don't?), and stay engaged long term. All to avoid vendoring one library. I doubt there is a single person who has ever done this.

> Or maybe the benefits of the latest and greatest compression algorithm are outweighed by older ones due to their accessibility. Develop with library versions that are commonly available, not the latest and greatest. Wait for features to mature (and possibly help them mature) before depending on them.

Yes, creating worse performing, less capable software is definitely an option. I prefer not to.

> If you want to write brittle broken software that needs constant attention and maybe doesn't even work at all in a few years, then yes, go ahead and keep doing things this way.

Your preferred approach "needs constant attention and may not even work at all in a few years" --- you require me to pay constant attention to how distros are packaging my dependent libraries and regularly contribute to that process. In fact, because bugs are found and requirements change, any project with external dependencies requires ongoing attention.

My main project Pernosco is in Rust, has tons of dependencies (because it does a lot), and Rust+cargo have done a great job of managing those dependencies over the last five years. I am happy to keep on doing this this way.

I agree. I think distro library management incentivises working around bugs, while cargo* incentivises helping upstream libraries.

I don't see this point brought up very often when discussing cargo, but I consider it to be one of the principal advantages of cargo.

> Develop with library versions that are commonly available, not the latest and greatest.

I think this is a sensible approach if you limit yourself to libraries, dynamically linked and available in distros. However I think it demonstrates how distro package managers incentivise *not* helping the libraries you're using.

Imagine you're writing some code and you come across a bug in a library you're using. You can choose to fix the bug upstream, or you can choose to work around it in your downstream code. With cargo you clone your dependency's git repo, fix the bug, push the change to a pull request upstream and update your Cargo.toml dependency to point at your new git revision with:

mydep = { git = "https://github.com/me/mydep.git", rev = "9f35b8e" }

You can leave it pointing at that specific revision until the upstream makes a new release at which point you update your Cargo.toml back to:

mydep = "3"

Fixing the bug (or adding the feature) upstream is the path of least resistance. Once you do it others who are using the library can benefit at the time of their choosing. In my mind many small fixes like this **is** the maturation process

Now what's the process with distro package managers? You're working on your new feature for your software. You come across a bug. You fix it upstream, you wait for it to get accepted upstream, you wait for upstream to make a new release and then you wait a few years for it to get into enterprise distros. Then you upgrade your infrastructure to a new major distro version, and only then can you deploy your new software that depends on this bug-fix/feature to get it in the hands of your users.

No, waiting, waiting and waiting is not going to fly. You want to help upstream but depriving your users of the new feature in your software for years is too high a cost to pay. So you work around the bug in your software and maybe if you've got time left over you also submit a fix upstream.

> Wait for features to mature (and possibly help them mature) before depending on them

I think this is the crux of my argument. cargo makes it easy to help features mature. Limiting yourself to distro repos means you have to wait for them to mature.

* possibly other language package managers too, but I'm not sure. I think cargo is best-in-class in this regard and some of its advantages may not apply to non-compiled languages/languages that don't statically link.

I have seen both autoconf and cmake referred to as a problem more often than as a solution:-)

Watch out: The C/C++ world has tooling for dependency management incoming as well. Sooner or later you will have similar problems with those languages depending on very specific versions of libraries as well. Developers want that stuff, so they will write it. In the end each developer gets the tooling she deserves;-)

> If a library you want to use is good, and well maintained but not packaged, help them package it.

Why? The package will either be too old, or have the functionality my program needs stripped out since some packager did not like a dependency it introduced. Or its crippled or patched to no longer work properly for my application. Or the necessary files for your buildsystem of choice are not installed. Or they can't be found. Or users have crippled the libraries to prevent some imaginary issue or another. I had to deal with all of the above in bug reports already. It is such a huge pain to debug this kind of issue.

As a developer you need to vendor libraries (at thebvery least as a fallback if system libraries are not found!), even when running on distributions that have official packages of the required libraries. And you will get bug reports due to incompatible libraries because some distro packager will unvendor your libraries for you, blissfully ignoring the documented requirements.

> There is an awfully large amount of well written C software that has been written this way, has stood the test of time and will likely be around for a long time to come.

That is survivors bias... tons of poorly written crap written in C got lost and good riddance! Undoubtedly a lot of code written in any other language will not stand the test of time either.

> But the long term end result is a more sustainable ecosystem with a lot less work over the entire community.

There is less code because it takes ages to write and maintain. Is that a benefit to the eco system as a whole? I doubt it: Making something hard does make the persistent people stick around, and many good programmers I know are lazy and easy to distract:-)

> Vendoring something might make less work for you in the moment, but is more work for other people (or even your future self) down the line and doesn't solve anything for other people with the same problems as you.

It is more work for me and other people down the line. You get so many more bugs due to library incompatibilities and such. Those produce extra work for the users, the packagers and the developers. We are just used to this work load, so we ignore it.

> If you want to write brittle broken software that needs constant attention and maybe doesn't even work at all in a few years, then yes, go ahead and keep doing things this way.

If you want brittle software now, then go ahead:-)

Seriously there is great and well tested C code out there. That is wonderful! There is also some great rust code out there. Also awesome! Great code is a wonderful thing to have in any language! In my mind code quality is not related to how hard it is to use 3rd party libraries in the programming language of choice.

I also do not want to see programming as an activity where you need to recite obscure texts that got cargo-culted down to you by your elders! And it is rare to find a medium sized project in C or C++ that does not do exactly that -- at the very least in some dark corner of its build system. Rust projects at least do have less dark corners. Part of the reason is of course that rust has not accumulated so much historical baggage yet:-)

There are. They're not as mature as anything Linux has AFAIK, but there is at least (in no particular order):

- vcpkg (probably the most useful for the discussion at hand)
- Conan (CMake-based)
- chocolatey (binary-based, includes Visual Studio/MSVC packages)
- anaconda (scientific/Python oriented, but has other bits too)

I think there's another, but I can't remember it's name. There's also zero surprise from me if there are others I haven't heard of.

Sure. One problem is, that package list changes with each distribution and sometimes within versions of each distribution. So we only have instructions for Fedora and Ubuntu, and those instructions are wrong for some versions of those distros.

Anyway, every single step makes the software harder to build.

> Users of stable distributions are familiar with the issue.

So? The issues still exist.

> That is incentive to:
> 1. Do not depend on amateur libraries that change API

Distro policies are responsible for varying file layouts and naming conventions. Distros also make varying decisions about library versions and which features they enable at build time.

> 2. Use autotools and let it figure out all this stuff

Autotools are a nightmare and writing autotools feature tests for everything I care about would be a ton of extra work.

> you won't be using the same binary on linux and windows.

Indeed, but once you've done the work to build Windows binary you can reuse that work to build a Linux binary with vendored libraries.

People arguing that the Right Way to build C/C++ software is to make it Linux only, use distro libraries, do a ton of extra work, and downgrade the performance and functionality of that software to fit the shipped libraries, are not doing C/C++ any favours.