Clouds and VPSes [LWN.net]

Clouds and VPSes

Posted Apr 21, 2022 8:11 UTC (Thu) by rwmj (subscriber, #5474)
Parent article: Fedora considers deprecating legacy BIOS

A lot of comments so far talking about hardware, but to my mind it's really the cloud providers (especially the second tier ones) and VPS hosters where this is going to be painful. We've supported UEFI (OVMF) in qemu since the early 2010s, and the other hypervisors have also had UEFI for a long time, but BIOS boot is still very common for VMs running in clouds and often the only option supported.

to post comments

Clouds and VPSes

Posted Apr 21, 2022 10:23 UTC (Thu) by johannbg (guest, #65743) [Link] (36 responses)

I'm not seeing the relevance here I mean that this is being painful for some cloud/vps providers presumably due to them experiencing some sort of management tools issues is a self inflicted wound is it not?

Those companies should have the required means to prevent this from happening or is the (f)oss ecosystem supposed to be handholding businesses these days?

Clouds and VPSes

Posted Apr 21, 2022 10:28 UTC (Thu) by rwmj (subscriber, #5474) [Link] (24 responses)

I'm going to guess they have a system that works (BIOS) and has done for 15+ years, they have management tooling all around that, probably custom stuff written a decade ago, so why would they see any need to change?

There is a technical issue too since UEFI is considerably slower to boot and has much greater complexity.

Clouds and VPSes

Posted Apr 21, 2022 11:13 UTC (Thu) by johannbg (guest, #65743) [Link] (18 responses)

> I'm going to guess they have a system that works (BIOS) and has done for 15+ years, they have management tooling all around that, probably custom stuff written a decade ago, so why would they see any need to change?

UEFI is just one step short from becoming mandatory requirements in government contracts and usually if for example the U.S government says jumps, the vendors jump and businesses better be in shape when that call to jump comes.

Anyways still not seeing why *we* ( and by we I mean the entire (f)oss ecosystem ) should care I mean it's companies own responsibility to keep themselves relevant and competitive on the market, we do not exist to do that for them.

> There is a technical issue too since UEFI is considerably slower to boot and has much greater complexity.

This is the direction the industry is taking ( and has been for quite a while, as you yourself are very much aware of ), so it's not like there are any other choice other than to put in work to mitigate any downsides it brings.

People might not like it and businesses might choose to stick their head in the sand and ignore it but it is what it is.

Clouds and VPSes

Posted Apr 22, 2022 3:13 UTC (Fri) by dvdeug (guest, #10998) [Link] (17 responses)

> This is the direction the industry is taking ... so it's not like there are any other choice other than to put in work to mitigate any downsides it brings.

Or the FOSS community can put its weight where it is most useful. There's no value to forcing cloud computing to move from BIOS to UEFI, and as long as Linux supports it, these cloud providers can continue using it. Not permitting AWS to run away with everything is of value for the FOSS community; it reduces the leverage Amazon has over the community, and more generally the FOSS community is opposed to monopolies.

Clouds and VPSes

Posted Apr 22, 2022 7:21 UTC (Fri) by johannbg (guest, #65743) [Link] (16 responses)

Yes and no the fact is we live in an interconnected world were devices need to be protected not just at the application and OS levels, but all the way down to the firmware and hardware levels.

And now the industry is faced with deep, firmware- and hardware-level attacks and the reason for that being is there is a major firmware security gap with zero visibility in it.

In this interconnected world we (now?) live in, organazations and businesses need to be able verify that the internal components of their purchased computing devices are genuine and have not been altered with during the manufacturing and distribution processes ( for example manufactures swapped out components during the chip shortage for different ones, from different suppliers ) or otherwise modified throughout the device´s life cycles in their infrastructure or during the warranty of the device as a part of their business solution(s).

That the firmware that the every day device relies on, be it the "smart" coffee machine, corporate laptops, the network equipment, the servers or the even the personal/corporate EV's ( you dont want your Tesla's to run rampage, plowing down pedestrians like an elephant tramping over ants do you, or turn left when it should be turning right after the vehicle recieved altered firmware update over the air right? ) has not been tampered with.

All these devices have firmware on them so it's vital that organazations and businesses have their cybersecurity supply chain in check.

The unified extensible firmware interfaces is an critical components in the cybersecurity supply chain since it's an integrated part of infrastructures that runs this fairly newly founded interconnected world and I most certainly would think that the FOSS community would want to be integrated part in helping pushing, validate and securing and otherwise help strengthen the cybersecurity supply chain as well as phase out solutions like legacy bios's that are no longer being maintained and are considered obsolete in the industry since more often than not Linux is the underlying technology running this new world.

I'm not too overly concerned with cloud vendors or other buisnesses that do not have their cybersecurity supply chain in check since the free market will take care of those that wont ( they break the root of trust and will just be stuck with financially starved clients ).

For example if I was a government entity or a car manufacturer like Toyota,Volvo or Tesla I would not be doing any business with companies that dont have their cybersecurity supply chain in check and one of the things I would be checking for would be if the relevant vendor supported uefi and which hw manufacture it ran, in it's infrastructure on ( all the way down to component level ) etc. so I could validate and ensure that *my* cybersecurity supply chain was in check and *my* root of trust was secure.

I would exclude IBM/RH/Amazone from every government/business contracts since it's quite apparent based on the feedback here and on fedora-devel that they are not ready for this.

Their cybersecurity supply chain is not in order thus cannot be trusted and wont be for years ( from the looks of it ) thus I would choose a different vendor that provided me with a secure root of trust. ( There are business opportunity for those that can, to get ahead of large vendors like Amazon for example )

Fedora will not be allowed to make the required change to prepare RHEL due to the legacy trolls that come crawling out of last century, screaming ME ME ME, MY USECASE, FOREVER!!!! while riding on their roating,rusting, steam powered technology devices demanding that the technology universe remains frozen in time and be supported forever, as is ( another example is Adam trying to deprecate the legacy xorg driver ) as opposed to be thinking a bit further away from their own noses, ahead into the future, to the state that the industry is in now and evolving into.

The Fedora solutions MOAR Community Burecracy ( because that as worked so well! ) by forming a "Legacy SIG" to deal with the problem. So RHEL 10, nope propably more like RHEL 11...

Clouds and VPSes

Posted Apr 22, 2022 8:22 UTC (Fri) by LtWorf (subscriber, #124958) [Link]

> the free market will take care

Have you seen the world? The free market doesn't take care of putting bad products out of business.

Clouds and VPSes

Posted Apr 22, 2022 8:23 UTC (Fri) by LtWorf (subscriber, #124958) [Link]

Also USA federal government and armed forces are quite happy with using ubuntu. I guess they are equally as happy to use red hat.

Clouds and VPSes

Posted Apr 25, 2022 3:41 UTC (Mon) by wtarreau (subscriber, #51152) [Link] (13 responses)

> Fedora will not be allowed to make the required change to prepare RHEL due to the legacy trolls that come crawling out of last century, screaming ME ME ME, MY USECASE, FOREVER!!!! while riding on their roating,rusting, steam powered technology devices demanding that the technology universe remains frozen in time and be supported forever, as is ( another example is Adam trying to deprecate the legacy xorg driver ) as opposed to be thinking a bit further away from their own noses, ahead into the future, to the state that the industry is in now and evolving into.

This paragraph is very interesting because I've long been interpreting the situation exactly the other way. What if those unnecessary changes were only pushed hard by a few vendors who need to force their customers to regularly buy new hardware, and by software developers who see it as a guarantee to get a lifetime job ? I mean, I'm fine with changes that bring improvements, but there are many changes we're forced to swallow that significantly degrade our user experience. Sometimes you're even forced to abandon hardware by lack of support from new software, and for what justification, beyond "look how great the new version is" ?

I used to have machines that took 3 seconds to start to boot from power-on in the past. At work in the lab we have a UEFI machine that takes more than one minute doing whatever in your back before trying to boot, and you have just 2 seconds to decide what device to boot from so you're forced to stay in front counting in your head. That's one of the machines I run test kernels on... It's a good example of crap I don't need and that significantly degrades my experience by preventing me from remotely booting test kernels.

It's not about wanting to stay in the previous century, it's a concrete example of "improvements" that I didn't need and that makes users suffer for no reason except some vendors forcefully pushing that down their customers' throat.

It would be nice if software developers could sometimes try to argument their improvements as benefits perceived by their *users* and not only by themselves as software maintainers. Just claiming that "new feature X is much better and if you don't want to adopt it we'll remove the previous one and you'll have no other choice" isn't exactly how the free software movement started, quite the opposite in fact. I remember the time when we were proud to recycle old machines to make powerful Linux servers. Nowadays some linux distros force you to trash powerful machines. There must be something really wrong with that policy. The only reason I'm thinking about is the distro's policy possibly being dictated by too powerful companies whose business does not benefit from small systems.

But surely I'm wrong on all that line and it's normal to trash perfectly working hardware, I'm the only one concerned about e-waste and with purposely spending my money to buy less pleasant replacement hardware...

Clouds and VPSes

Posted Apr 25, 2022 7:30 UTC (Mon) by johannbg (guest, #65743) [Link] (9 responses)

> But surely I'm wrong on all that line and it's normal to trash perfectly working hardware, I'm the only one concerned about e-waste

That would be insecure working hardware, connected to the network that requires more "power" than current solutions, which adds to the environmental problem.

The people/companies that live by a concept called "carbon footprint" installed by the oil companies as part of an deceptive PR campaigns [1] on their behalf, the biggest one in history and claim that they care about the environment should not be buying/using anything involving computers,solar panels, tv's etc since in the devices manufacturing process is an chemical ( that is among few that was conveniently left out of the Kyoto Protocol international climate change agreement ) called Nitrogen Trifluoride(NF3) is being used.

"The gas is 17,000 times more potent as a global warming agent than a similar mass of carbon dioxide. It survives in the atmosphere about five times longer than carbon dioxide" [2]
It's use increases as the world is being deliberately pushed into adopting more technology as can be clearly seen in the market projections for the chemical [3].

People wont like what I'm about to say but the fact is we are way beyond point of no return for our planet at this point so if people genuenly care about the environment then they should disconnect, find another line of work and go and live like an Amish in the literal sense and figuring out solution that a) change the worlds economy in an instant and b) reduce the human population because *none* of the solution out there fix anything ( but they do produce profit ), at best they just delay it.

This whole act of people pretenting to be "green" to ease their environmental guilt which has been installed by their government in conjunction with the oil companies and an trillion dollar environmental industry is just ridiculous.

1. https://mashable.com/feature/carbon-footprint-pr-campaign...
2. https://scripps.ucsd.edu/news/potent-greenhouse-gas-more-...
3. https://www.marketquest.biz/report/108079/global-nitrogen...

Clouds and VPSes

Posted Apr 25, 2022 8:49 UTC (Mon) by kleptog (subscriber, #1183) [Link] (6 responses)

While I agree that them term carbon footprint can be misleading, some of your other points are off the mark. NF3 has been regulated under the Kyoto Protocol since 2012, largely as a result of the the research you're referring to. Its total radiative forcing compared to CO2 is rounding error. It's something to monitored, but hardly a major issue.

Sources:
https://en.wikipedia.org/wiki/Nitrogen_trifluoride#Greenh...
https://unfccc.int/process-and-meetings/transparency-and-...

Perfect is the enemy of good. If we only accept perfect solutions we'll never get anywhere.

I don't disagree with your sentiment that we're already basically screwed, but I don't think that's an excuse to just give up completely.

Clouds and VPSes

Posted Apr 25, 2022 17:36 UTC (Mon) by johannbg (guest, #65743) [Link] (5 responses)

> Perfect is the enemy of good. If we only accept perfect solutions we'll never get anywhere.

It's not like we are getting anywhere without the solutions being perfect now is it.

And it's not like the real/perfect solutions aren't known as in changing the world's economy and reduce the human population, they just cant be realistically implemented, well technically they can the former will never be allowed to happen and the latter never be socially accepted.

> I don't disagree with your sentiment that we're already basically screwed,

Biodiversity, polution in earth, water and air yup we are pretty fucked.

> but I don't think that's an excuse to just give up completely.

True we can always hope that good size meteor hits the planet.

Clouds and VPSes

Posted Apr 25, 2022 18:22 UTC (Mon) by Wol (subscriber, #4433) [Link] (4 responses)

Well, iirc the prediction (in 1980) for the millenium world population was 12Bn, and if we were unbelievably lucky it might be only 8Bn. What's the current figure?

To the best of my knowledge, we've undershot pretty much every prediction since, and the forecast for 2100 or 2150 is only 3Bn ...

Populations naturally boom and bust, and it looks like we might be hitting bust, luckily for Earth ...

Cheers,
Wol

Clouds and VPSes

Posted Apr 25, 2022 19:08 UTC (Mon) by johannbg (guest, #65743) [Link] (1 responses)

Well it's not like the human population will stop breeding tomorrow despite the genderless generation's attempt to put a dent in the population growth rate and the fact is we kinda need a meteor like right now, not 50+ years from now.

And who does not want to go out in a blaze of glory of kinetic energy = mass/2 * velocity^2

It sure beats dying of old age doesn't it...

Clouds and VPSes

Posted Apr 25, 2022 22:13 UTC (Mon) by Wol (subscriber, #4433) [Link]

Well, for a start a repeat of Tunguska over Stal (sorry, Put)'s palace might be a good idea ... let's hope the universe obliges ...

Cheers,
Wol

Clouds and VPSes

Posted Apr 26, 2022 12:50 UTC (Tue) by nix (subscriber, #2304) [Link] (1 responses)

This is not a population bust in the biological boom/bust sense. These are caused by starvation or predator pressure. What we are seeing almost worldwide is the voluntary reduction of breeding rates in order to focus more resources on fewer offspring. This is largely because this is what women want, but also because there is an economic incentive: in poverty-stricken rural areas children become a source of free labour fairly fast as they grow up, so you can use one round of children to help fund the next, and having *lots* is to a certain degree beneficial to all of them. In cities they are and remain a substantial cost to the whole family, including any older children. So it's beneficial to have fewer.

(There is probably also something related to falling death rates: if you know that most of your children will survive, you'll have fewer. But the economic incentive is substantial even in the absence of that.)

Clouds and VPSes

Posted Apr 27, 2022 1:33 UTC (Wed) by JanC_ (subscriber, #34940) [Link]

There is also the concept of children taking care of the elderly in poor areas, which is less needed in places where we have pensions.

Clouds and VPSes

Posted Apr 25, 2022 14:20 UTC (Mon) by wtarreau (subscriber, #51152) [Link] (1 responses)

> reduce the human population because *none* of the solution out there fix anything ( but they do produce profit ), at best they just delay it. This whole act of people pretenting to be "green" to ease their environmental guilt which has been installed by their government in conjunction with the oil companies and an trillion dollar environmental industry is just ridiculous.

This actually is the part I agree with :-)

Note that I wasn't doing some greenwashing, just explaining that I'm not seeing any reason to trash perfectly working hardware (which comes with costs and waste) just for the sake of making some developers' life easier when they claim that may hardware is dead.

Clouds and VPSes

Posted Apr 25, 2022 15:31 UTC (Mon) by Wol (subscriber, #4433) [Link]

Yup.

Running costs are only part of the equation. How much environmental damage is done and making new and disposing of obsolete hardware? Surely the extra costs of running old hardware may well be cheaper ...

Much as I dislike the waste-to-power plant just down the road (and even more so because we're a poor East London borough having all of posh West London's waste dumped on us), the fact is that is very environmentally friendly - it means a load of waste is converted efficiently to energy and CO2 rather than inefficiently converted to methane ... and it leaves a load of oil/coal unmined and in the ground.

If you're being green, you need to look at the big picture, not just minimise your costs at the expense of creating even bigger costs elsewhere (cough cough the government pricing our steel industry out of existence cough cough).

Cheers,
Wol

Clouds and VPSes

Posted Apr 25, 2022 12:15 UTC (Mon) by james (guest, #1325) [Link] (2 responses)

... a UEFI machine that takes more than one minute doing whatever in your back before trying to boot, and you have just 2 seconds to decide what device to boot from so you're forced to stay in front counting in your head.

Just in case you haven't heard of it, does sudo grub2-reboot work for you? You run it from the Linux command line, specifying a grub menu entry, then reboot: Grub2 uses that menu entry just that once.

Works for me on Fedora.

Clouds and VPSes

Posted Apr 25, 2022 14:23 UTC (Mon) by wtarreau (subscriber, #51152) [Link]

Yep it does and I discovered this one very recently, thank you for sharing anyway because it's possible it'll help others!

However it only works when the machine reaches grub, and that one is particularly stubborn and decides to boot again from the first disk because boot priorities are reset every second boot or so... I simply gave up doing kernel testing on that one.

Clouds and VPSes

Posted Apr 25, 2022 15:45 UTC (Mon) by Wol (subscriber, #4433) [Link]

Does it work if you can't move the default from entry 1?

Last time I read the grub documentation, it told you how to changed the default boot entry from entry one, and then had the caveat "this only works if ...". Of course, my system breaks that "if"...

Cheers,
Wol

Clouds and VPSes

Posted Apr 22, 2022 1:06 UTC (Fri) by bartoc (guest, #124262) [Link] (4 responses)

I have never, ever, experienced a UEFI system that was slower to boot in UEFI mode than legacy mode. Things are usually faster because, for example, the firmware won't initialize my SAS expander unless it's actually going to boot off it. I've had several systems where some add-in card with legacy OpRoms would hang the system for over _30 minutes_ on boot if you turned on legacy support.

This applies to laptops, desktops, and server hardware.

I haven't used a system that doesn't support EFI only mode (i.e. no oproms or anything required) for at least 10 years.

In any event I would like fedora to stop silently installing in BIOS mode, I've made that mistake several times and you need to reinstall the whole OS to fix it.

Clouds and VPSes

Posted Apr 22, 2022 9:35 UTC (Fri) by rwmj (subscriber, #5474) [Link] (3 responses)

We're talking about VMs, and UEFI is noticeably many times slower to boot.

Clouds and VPSes

Posted Apr 22, 2022 10:52 UTC (Fri) by amacater (subscriber, #790) [Link]

Everyone will have different experiences: mine is that UEFI on a VM is still quicker than BIOS - Tianocore and kvm work perfectly well.

BIOS is _dead_ in the sense that new hardware doesn't support it, it's not being developed - and when Linux eventually is forced to move to attestation by market pressure it'll be dead anyway.

Although I've a 25 year old laptop somewhere in my junkpile - I'm hardly expecting to resurrect it - and it's been some years since I've _needed_ BIOS (except for a couple of HP Microservers which are annoying in that firmware is a paid for support contract upgrade).

Clouds and VPSes

Posted Apr 22, 2022 11:30 UTC (Fri) by mjg59 (subscriber, #23239) [Link] (1 responses)

Which environments are you testing this in? It doesn't match my experience at all - we transitioned all of Google's internal cloud desktop environments to UEFI without anyone noticing.

Clouds and VPSes

Posted Apr 24, 2022 11:55 UTC (Sun) by johannbg (guest, #65743) [Link]

Google also ( almost to the date ) 2 years ago made Unified Extensible Firmware Interface (UEFI) and Shielded VM the default for everyone using Google Compute Engine and provided migration path from on premis uefi based vm images into that environment.

Clouds and VPSes

Posted Apr 21, 2022 10:30 UTC (Thu) by leoluk (guest, #97665) [Link] (10 responses)

As long as distros keep supporting legacy boot, cloud providers have little incentive to switch to EFI boot.

Clouds and VPSes

Posted Apr 21, 2022 10:41 UTC (Thu) by johannbg (guest, #65743) [Link] (6 responses)

Right because people are like water, they always take the shortest route.

Clouds and VPSes

Posted Apr 21, 2022 12:23 UTC (Thu) by dullfire (guest, #111432) [Link] (5 responses)

> Right because people are like water, they always take the shortest route.

I would suggest looking at some maps with rivers on them.

Clouds and VPSes

Posted Apr 21, 2022 13:41 UTC (Thu) by smoogen (subscriber, #97) [Link] (4 responses)

There is a common language translation where a phrase gets switched by one or two words and looks the same to the non-native speaker. If we go with 'people like water always take the easiest route', does that make the original phrase better for you?

Clouds and VPSes

Posted Apr 21, 2022 14:34 UTC (Thu) by Wol (subscriber, #4433) [Link]

Time flies like an arrow, fruit flies like a banana ...

Cheers,
Wol

Clouds and VPSes

Posted Apr 21, 2022 16:08 UTC (Thu) by rgmoore (✭ supporter ✭, #75) [Link] (2 responses)

The criticism isn't of the wording, it's of the content. Water frequently does not take the shortest route, as you can determine by looking at a map. Instead, rivers meander all over the place. A river can even erode through the soil into the underlying rock, causing its old meandering path to be literally set in stone.

There may be some truth to the idea of people behaving like water, but it has more to do with their path being heavily determined by history. Both will usually take an established route even when it is long and convoluted. It's only the occasional dramatic event that causes rivers- or people- to change their course.

Clouds and VPSes

Posted Apr 21, 2022 16:16 UTC (Thu) by sfeam (subscriber, #2841) [Link]

People are like water: both will stagnate in a local minimum rather than finding the energy to overcome a barrier that constrains them.

Clouds and VPSes

Posted Apr 21, 2022 17:38 UTC (Thu) by johannbg (guest, #65743) [Link]

I propably should have said people are like water, they always take the path of least resistance.

Those that are bothered by the inaccuracy of that can replaces "least resistant" with the "distribution of flow that will lead to the least "total" resistance" but most people should have gotten the gist of what I meant.

Clouds and VPSes

Posted Apr 21, 2022 13:42 UTC (Thu) by rwmj (subscriber, #5474) [Link] (2 responses)

If every distro switched this might be the case, but it's unlikely if only Fedora does it. However big things which might make this happen would be Windows 11 and RHEL 10, although one is not very relevant on cloud and the other is some years in the future.

Clouds and VPSes

Posted Apr 21, 2022 18:18 UTC (Thu) by johannbg (guest, #65743) [Link] (1 responses)

Well it all starts with one distribution taking the lead, usually it's Arch these days but Fedora is better positioned to take the lead for this particular change.

That said why should UEFI miracles start happening when RHEL 10 get's released?

Clouds and VPSes

Posted Apr 22, 2022 0:52 UTC (Fri) by bartoc (guest, #124262) [Link]

One could argue that arch supports neither firmware standard, since they don't have an installer. At best they just package bootloaders.

Clouds and VPSes

Posted Apr 21, 2022 14:37 UTC (Thu) by cortana (subscriber, #24596) [Link]

I think there are still issues about snapshotting VMs that were booted with UEFI (though perhaps this is a limitation in libvirt rather than QEMU?)

Clouds and VPSes

Posted Apr 21, 2022 21:10 UTC (Thu) by dmoulding (subscriber, #95171) [Link]

Yes, exactly this. I'd bet the number of Linux installations on BIOS *today* outnumbers the number of installations on UEFI ten fold. And they are even *thinking* about deprecating BIOS now?