The things nobody wants to pay for
The things nobody wants to pay for
Posted Jan 26, 2024 0:20 UTC (Fri) by geofft (subscriber, #59789)Parent article: The things nobody wants to pay for
What should we do, and how do I, as someone with some amount of influence but no actual spending authority, get us to do it?
The Linux kernel contribution maturity model (https://docs.kernel.org/process/contribution-maturity-mod...) does not seem particularly relevant to us. We're not a hardware vendor or otherwise in the business of writing kernel code. We're at level one - we can and do contribute patches to the Linux kernel, in practice every couple of years. Level two suddenly talks about people being expected to contribute as part of their job and having my contributions assessed in performance reviews. That's a drastic step up! I cannot figure out what an employee who has sent in the occasional patch (like me) would possibly do to work on the Linux kernel with even 10% of my time year-round. I'm honestly not sure LKML would really want me to dive in and say "Look guys, my boss told me to contribute, so I'm just going to keep refactoring this subsystem." And the people who do my performance reviews aren't really in a spot to evaluate my contributions beyond whether they happened or not.
Besides, this article is mostly about specific existing projects and specific existing contributors talking about whether they can do them, not the need for new blood in general. (There are other articles about that.)
So, okay, let's not think about contributing more code ourselves. Greg KH points out that there isn't enough testing because companies aren't funding the work. We'd love more testing, it would make us more confident in upgrades. Where do we send a check? (Again, I have contributed to the kernel a little bit, and I have no idea what I would be asking my company to do.) The Linux Foundation gets a good bit of corporate money, as I understand ($177M from their last form 990). Does that money go to this sort of work? If we send LF a small amount more money, would it fund this work?
More importantly, how do I convince my company that this is a good use of our money? I doubt I can convince my employer to sponsor even one full-time engineer at the wages we pay our full-time engineers; we have enough problems of our own to solve, and as important as kernel testing is, it's not as important as anything else we could put one full-time engineer on. (We run at a scale large enough to see kernel regressions that weren't caught by anyone else, so it would concretely help us, but we don't run at a scale large enough that it's regular: it happens about once every few years.) We certainly benefit from Linux in general, but we're not talking about paying to make sure we can keep using Linux at all, we're talking about making the development process a little faster and a little more reliable. But the argument in the article about tooling paying off compounding rewards is an interesting one. Is there a compelling argument that it really is a good use of our money to fund the kernel by an entire engineer's salary?
Or let's say that we want to contribute a fraction of an engineer's salary. How do I convince the people who sign checks that this money is going to go somewhere useful? It won't make any of the existing contributors have more hours in the day (or would it? is part of their income coming from Linux-irrelevant private consulting work that they could stop doing?). I suppose we and many other companies like us could collectively fund one engineer, but how do we get that to happen?
(Is this a question for us the corporate users to answer? Like should we be proactively reaching out to other companies in our industry and saying, hey, all of us get value from Linux being good, let's collectively fund someone?)
Or maybe funding development time isn't the right thing to focus on. I see in the thread some talk about KernelCI. Would contributing money to KernelCI to buy more machines or more public cloud time help? If I go to the KernelCI home page, there's a "contributions welcome!" button ... that links to GitHub. A little bit below, it says, "Learn how your organization can contribute to the project now," but the way to learn is to send someone an email. What does it mean to join? And then there are logos of a couple of companies below it, at least a few of which are much larger than we are and care much more than we do about Linux development. Are they contributing so much that we'd be a drop in the bucket? I have no idea. Nothing on the website answers the question about how much money KernelCI is spending and whether they could productively spend more (but it does have some brand guidelines, great).
Or maybe I'm totally off base here, and when Greg KH and others are talking about "companies," they are actually talking about gold/platinum members of the Linux Foundation who are already contributing a lot but could still justifiably employ several more individuals full time on upstream kernel work, and non-huge companies like mine are just not part of the conversation, and articles like these aren't intended for people like me to try to get our employers to contribute more. So maybe a simpler question is - in an ideal world, where everyone in my employer's management was completely and accurately convinced of the need to fund Linux kernel development, would that accurate funding really be more than the almost-zero it is now? (One argument in this direction is we pay quite a bit of money to our hardware and cloud vendors, and a feature of their product is compatibility with upstream Linux, so we're already doing the thing that ought to fund kernel development. In that case, perhaps what I really should do is tell our account reps at those companies - all of which are already gold/platinum members of the LF with full-time kernel developers on staff - that we want to see them funding this work even more.)
Posted Jan 26, 2024 5:25 UTC (Fri)
by NYKevin (subscriber, #129325)
[Link] (2 responses)
At the same time, I must say that I find it rather disturbing that you were unable to get useful answers to your questions about certain specific areas of contribution. It is all well and good to complain that nobody wants to pay for certain things, but you can't put this sort of information at the bottom of a locked filing cabinet in a disused lavatory etc. etc., and then complain that nobody managed to find it and contribute.
Posted Jan 26, 2024 8:06 UTC (Fri)
by lunaryorn (subscriber, #111088)
[Link] (1 responses)
Posted Jan 27, 2024 1:05 UTC (Sat)
by NYKevin (subscriber, #129325)
[Link]
Posted Jan 26, 2024 7:46 UTC (Fri)
by pbonzini (subscriber, #60935)
[Link] (2 responses)
Posted Jan 26, 2024 8:26 UTC (Fri)
by Wol (subscriber, #4433)
[Link] (1 responses)
If your setup is at all out-of-the-ordinary, see if you can set up your own CI test-bot, and hammer it (as part of the general kernel testing system), and then you can sell it as buying you extra confidence when you upgrade, you know your own setup has been put through the mill on your own test-bot.
Cheers,
Posted Jan 26, 2024 12:12 UTC (Fri)
by spacefrogg (subscriber, #119608)
[Link]
For many projects it would also be helpful to:
People are motivated to build stuff that helps them(!), the aforementioned topics are chores that do not benefit the original authors and thus need monetary persuasion. This is where your company has leverage where private people have not.
Free Software is a common good like streets, utilities or parks and nature reserves. Your company, as any other company, usually does not directly invest into common good but rather pays taxes. Putting money into free software is charity, not investment (aside from rare cases in highly specialised fields). Thus, you should never try to sell it as that. So, as for convincing the bean counters? You won't. Either it is in your company's DNA to give back to common good or it is not, and it won't happen. There will be no outcome that can positively reflect in an Excel sheet. Even in those rare cases, the outcome will not be measurable and always under scrutiny of being scrubbed for short-term benefit. The will to pay has to come from outside of money making or it won't last. You might sell it as good marketing, or your boss being the cooler kid on the golf course etc. These are all human factors and not numbers.
Posted Jan 26, 2024 11:11 UTC (Fri)
by dvrabel (subscriber, #9500)
[Link]
Posted Jan 26, 2024 20:34 UTC (Fri)
by mfuzzey (subscriber, #57966)
[Link]
But there are lots of other, often critical, OSS projects that have very little funding - maybe it would make more sense to contribute to those rather than the kernel.
Posted Jan 27, 2024 11:06 UTC (Sat)
by amacater (subscriber, #790)
[Link]
If you have some in-house tooling/integration that works for you - _really_ document it well. Explain it - you'll benefit when those maintaining it move to another project. If it's specific to you, but the general knowledge you used to build it was hard-won because you had to look in 20 places - write up the experience and list the 20 places so the next person doesn't have to do that. Use that documentation experience to help upstream documentation / write technical blog posts / summarise where to find the vital information and share the 20 places. Write the missing manuals from bitter experience ...
if all else fails, Debian could do with clueful experts lurking on mailing lists and contributing real experience where it's meaningful :) Teach good practice in real-world sysadmin and security or contribute constructively to debate - but contribute.
Posted Jan 27, 2024 17:30 UTC (Sat)
by ceplm (subscriber, #41334)
[Link] (5 responses)
Posted Jan 27, 2024 18:05 UTC (Sat)
by geofft (subscriber, #59789)
[Link] (4 responses)
Also, I would need to successfully convince my company to spend money on it. I can pretty easily get the people who manage the donations and conference sponsorship budget to give, say, $10,000 to a relevant open-source nonprofit. I absolutely cannot get the purchasing team to buy a $10,000 product we won't use.
And if I tell them the motivation is "Well, the company that sells this product is going through layoffs, but maybe they'll have fewer layoffs if we support them, which might cause them to resume spending resources on the infrastructure behind the shared commons behind the product we won't use, because we also use that shared commons, and by the way their competitive differentiation is things that aren't part of that same shared commons, because the shared commons is free and we've been using it for free for twenty years," I would probably not change their minds.
I do actually think these companies would be well placed to sell a "sponsor 25% of an upstream developer" product. We would get something measurable about what our money is accomplishing. They'd be in a good spot to put that money to use, because they could reallocate an existing employee's time or use their reputation and market position to hire someone new and find three other customers. But they don't seem interested in selling that.
Posted Jan 27, 2024 18:30 UTC (Sat)
by ceplm (subscriber, #41334)
[Link] (1 responses)
We may not pay for the things (I work for SUSE, but of course I have zero influence on the corporate decisions of the company), but we spend zillion of money on upstream development. Yes, you may not be able to decide what exact part of the Linux universe you support, but comparing with supporting somebody who says they support Debian, it is the easiest and most secure way how to do it (OK, I see https://www.debian.org/donations, but I guess that goes completely on the Debian infrastructure, right? So, nothing goes upstream … that’s absolutely nothing against them, but if you want to support upstream development, this may not be the right channel).
Of course, I don’t know if you want to target your donations than you have to work your way through various pages like https://www.linuxfoundation.org/about/join, https://sfconservancy.org/, or many other organizations like that.
Posted Feb 1, 2024 13:36 UTC (Thu)
by rhertzog (subscriber, #4671)
[Link]
Indeed money donated to Debian is not used to pay for the time of Debian developers/contributors. It might be used for hardware, hosting, travel reimbursement, debconf organization, and that kind of activities.
<shameless plug>
For now, it's the margin on the various services that make this possible but we are considering to add some broader offer that would also encompass the "please take my money to contribute to Debian and make it a bit more sustainable" use-case.
I'd be happy to discuss with companies that are looking to contribute to Debian in such a way to help refine our future offer.
Posted Jan 28, 2024 10:07 UTC (Sun)
by jengelh (guest, #33263)
[Link] (1 responses)
Now, if RH/SUSE systems were running "popcon", corporate would at least have an idea that they should be spending so-and-so-many percent for the desktop.
Posted Jan 28, 2024 11:03 UTC (Sun)
by rahulsundaram (subscriber, #21946)
[Link]
They have clear stats on which packages are being used because a commercial subscription already provides that data. I would bet they are already using this to drive decisions,
Posted Jan 28, 2024 3:53 UTC (Sun)
by jmalcolm (subscriber, #8876)
[Link]
In my view, a good Open Source citizen helps promote what they use, files bug reports when they should, contributes what they can, and at least partially funds the "change" they need to see in the code ( either through contribution or cash ). If we stick to the original "Free Software" vision, just using the software is contributing to the change some people want to see in the world.
For smaller projects, the core devs may not be making a living. Those kinds of projects would most directly benefit from some cash infusion if you have the means. For projects that have already crossed a certain threshold though, I see no reason to feel bad about using software as-is without funding its development. I mean, if you are getting a bunch of direct commercial benefit from a specific project, it would be nice to share the wealth. I am not trying to dissuade anybody from funding Open Source by any means.
I guess what I am saying, as an example, is that most of us should not feel bad for using the Linux kernel "for free". My use of Linux these days places no demands on the kernel as a project. Nobody is having to spend any dev time directly on my behalf. If there were some deficiency I needed to see addressed, I would think about how to contribute to that. As you say though, for most of our use cases, the kernel is simply excellent these days already. Yes, I benefit from it getting even better but I am not the "primary" beneficiary. I am simply along for the ride and, in the case of Linux, I am a "free rider" but I am a "zero cost" one. Unlike say roads, I am not wearing the Linux kernel out by using it. I do not consume any of its resources. In fact, as above, my use of Linux "for free" can be thought of as a contribution of sorts.
Concentrate on the projects where you can and do add value. Don't worry about using Open Source for free. Just be thankful.
Although I guess I will make one request. Please do not turn around and insist that the fine people that do the work, that author all this amazing software for us, bow down to "your rights" when they do try to get some value for what they do. We have had some "community" back-lash moments over the last year that, in my view, show a massive lack of respect and appreciation for the people that write the code.
Enjoy what you use. Fund or contribute where you can but don't sweat it when you don't. At the very least, refrain from attacking the people doing the work.
Posted Jan 30, 2024 13:19 UTC (Tue)
by metan (subscriber, #74107)
[Link]
As for myself I've been maintaining the Linux Test Project for more than a decade and what we really need is contributors and maintainers who can help with the patch review. However I have no idea how to make this happen in practice, there is a more than handful of companies that use the testsuite for testing, but only a few allocate resources to contribute back or even invest into a developer becoming a maintainer.
The things nobody wants to pay for
Moving someone else's money (eg a donation or recurring fund from a company) around on behalf of someone else (like an open source project which isn't even a legal entity on its own in most cases) comes with so many legal, taxation, billing, accounting, reporting, etc. issues that it's probably not surprising that many open source projects have no infrastructure and thus no documentation around this.
The things nobody wants to pay for
The things nobody wants to pay for
The things nobody wants to pay for
The things nobody wants to pay for
Wol
The things nobody wants to pay for
- diversify their build/test infrastructure and
- do the boring jobs (writing, maintaining documentation, translations, platform-independence, packaging)
The things nobody wants to pay for
The things nobody wants to pay for
I think the kernel is one of the better off OSS projects that already has quite a lot of funded developers.
The things nobody wants to pay for
Check that it actually *has* a maintainer and that it's not being maintained by the QA team in Debian because there isn't an actual maintainer somewhere. Allow them some time to look at bug reports for that utility, check whether they apply, submit fixes (or suggest closure because "can't reproduce here") If your firm can help reduce backlogs on fixes by contributing a fix/patch - do so.
The things nobody wants to pay for
The things nobody wants to pay for
The things nobody wants to pay for
The things nobody wants to pay for
With Freexian we try to help fill that gap. Thanks to the various LTS offers, we are paying Debian developers to work on Debian. This is part of our mission and we are regularly reporting our activities:
https://www.freexian.com/about/
https://www.freexian.com/about/debian-contributions/
</shameless plug>
The things nobody wants to pay for
The things nobody wants to pay for
The things nobody wants to pay for
The things nobody wants to pay for