Eclipse Foundation announces collaboration for CRA compliance
The Eclipse Foundation, the organization behind the Eclipse IDE and many other software projects, announced a collaboration between several different open-source-software foundations to create a specification describing secure software development best practices. This work is motivated by the European Union's Cyber Resilience Act (CRA).
The leading open source communities and foundations have for years developed and practised secure software development processes. These are processes that have often defined or set industry best practices around things such as coordinated disclosure, peer review, and release processes. These processes have been documented by each of these communities, albeit sometimes using different terminology and approaches. We hypothesise that the cybersecurity process technical documentation that already exists amongst the open source communities can provide a useful starting point for developing the cybersecurity processes required for regulatory compliance.
(Thanks to Martin Michlmayr.)
Posted Apr 5, 2024 16:30 UTC (Fri)
by kleptog (subscriber, #1183)
[Link]
Posted Apr 6, 2024 12:54 UTC (Sat)
by tialaramex (subscriber, #21167)
[Link] (7 responses)
That sounds like a claim without evidence. Worse, it sounds like _exactly the same claim_ we see from C++ proponents. "We don't write software with these bugs" - a claim which is simply counterfactual on its face. Typically this will be rephrased as a No True Scotsman, "Ah, the software we developed with bugs, that wasn't using the proper process, using the process does work though". In C++ this idea is _so old_ that the previous No True Scotsman, "Modern C++" has been retired increasingly in favour of "Contemporary C++" as a way to dismiss all the "Modern C++" which nevertheless is a bug ridden nightmare because of course it is.
If you _have_ a process which works, you need evidence, and given how much the software doesn't work I'd suggest it needs to be very strong evidence, to make a case for the process being effective, as the default necessarily must be to assume that whatever it is you're doing isn't working.
If you actually _don't_ have a process which works then you need to be explicit that the goal is to develop one.
Posted Apr 6, 2024 13:28 UTC (Sat)
by dvrabel (subscriber, #9500)
[Link]
Posted Apr 6, 2024 21:06 UTC (Sat)
by kleptog (subscriber, #1183)
[Link] (5 responses)
> That sounds like a claim without evidence.
"Secure development process" doesn't imply "bug-free software". It means you have a process in place in an attempt to reduce security bugs, e.g. code review, automated testing, documentation covering the security aspects of the application, etc. A process to fix the bugs that that are reported and to release those fixes in a timely manner.
Quite a lot of open-source projects meet those goals easily. Better than a lot of companies do. Look at how long it took from when the xz issue was reported to when updates were released. Clearly the system can work quickly when needed.
Bug-free software doesn't exist, and we should not pretend it does. The CRA doesn't require that either. It's about the process, not the end result. There are always improvements possible and it about ensuring we keep looking for improvements and not giving up.
There is the point that a true secure developement process requires starting at the design phase but at least for existing open-source software that's just not going to happen. Although some security sensitive components like SSH probably did do that. At least for a lot of core internet infrastructure it's about maintenance, not building from scratch.
Posted Apr 6, 2024 23:45 UTC (Sat)
by tialaramex (subscriber, #21167)
[Link] (4 responses)
Secondly it implies this process has some security properties. I say you couldn't know that if you've never measured, and I suspect most and maybe even all projects which do have a process have never measured whether it achieves any such properties. They're hoping.
Why would we want a process? And measurements? Because then we can improve. If I can see that the outcomes are indistinguishable between project A which has security bugs notified to a dozen people by pager in under an hour, and project B which uses a mailing list with three people on it, we know "Require pagers for security bugs" and "Have more people alerted to security bugs" isn't helpful and we shouldn't waste on time on those ideas, meanwhile if we can measure (as Google did) that using Rust to write new components instead of C++ significantly reduces the rate of security bugs in the components then that's actionable.
And no, I doubt SSH was informed by a "truly secure development process" during the design phase. Tatu Ylönen was annoyed by password sniffing attacks in 1995, and so he wrote SSH (and gave it away) to solve this problem. It was huge, and Tatu took it (commercial and) proprietary soon after, but OpenSSH is based on Tatu's code, so you're basically imagining that a researcher, trying to scratch a personal itch would use a "truly secure development process" to invent a categorically new tool and that's unrealistic.
Although SecSH (the IETF working group to standardize what we'd now call SSHv2) came up with something a little different from Tatu's original protocol, it's not so different that you'd write the software from scratch, and for a considerable period OpenSSH supported both protocols.
Posted Apr 7, 2024 11:26 UTC (Sun)
by farnz (subscriber, #17727)
[Link]
One example of a process that has some (limited) security properties is the one Graydon Hoare calls the "The Not Rocket Science Rule Of Software Engineering"; automatically maintain a repository of code that always passes all the tests.
In its simplest-to-implement form, code is not ready for review if any test case is failing on the latest version; changes to code after review cause a requirement for re-review if they cause a test case to fail (even if that test case is flaky). This is a process, and it has some security properties; notably that if you merge a test case that will fail if a certain security-relevant bug is introduced, that security-relevant bug cannot be introduced into the code without someone accepting a change to the test case in code review.
Posted Apr 7, 2024 15:17 UTC (Sun)
by kleptog (subscriber, #1183)
[Link] (2 responses)
However, just because most projects don't document their security development process doesn't mean there isn't one. Sure, it's probably an ad-hoc process, that largely relies on a handful people willing to spend their free time monitoring their mail and pushing out updates. But it's there, and can be pretty effective.
I think we mostly disagree on whether the phrase 'ad-hoc process' mean anything.
Yes, obviously we want better documented processes but I think that's exactly what the Eclipse foundation is trying to do here. It's way easier to come up with "standard open-source project maturity levels" that projects can measure themselves against, then requiring everyone to reinvent the wheel. In fact, we should have done it years ago but evidently it took the passing of regulations to make it happen.
My point was that Tatu did write the first SSH using his experience to build a secure shell. A single developer writing a new tool is part of a "security development process", just not a very good one but better than nothing. Later it was done better, and surely in the future there will be an even better process. You have to start somewhere.
Posted Apr 7, 2024 16:49 UTC (Sun)
by pizza (subscriber, #46)
[Link] (1 responses)
Using this definition of "process", pretty much *everything* qualities as a "process", which effectively renders the term meaningless without some sort of additional qualifier.
A former colleague of mine quipped "Process is how you get consistent results out of mediocre people". Corporate cynicism aside, the entire point of process is consistency, which in turn requires some measure of formalization.
Posted Apr 8, 2024 7:09 UTC (Mon)
by roc (subscriber, #30627)
[Link]
Posted Apr 6, 2024 21:54 UTC (Sat)
by calumapplepie (guest, #143655)
[Link]
Posted Apr 7, 2024 16:57 UTC (Sun)
by eharris (guest, #144549)
[Link] (24 responses)
(1) Recently (well in the last twenty years) we have had "The Agile Manifesto" position - Quote: "Individuals and interactions over processes and tools"
(2) In the 1990's various large consulting organisations did actually build end-to-end process descriptions. In the 1990's these were called "methodologies:
(3) Between then and now there have been multiple boosters for the somewhat disconnected approach provided by UML.
I'm interested in the opinions of LWN readers (and commentators) about "process". Is "process" item #1, #2, #3.....or something else entirely????
Posted Apr 8, 2024 12:02 UTC (Mon)
by kleptog (subscriber, #1183)
[Link] (23 responses)
Well, a process can be something like a program, with instructions being executed. But a state machine is also a process. A list of "when X happens, do Y" statements is also a process, an event-driven one. The only requirement is there is some kind of goal defined.
It may be my mathematical background showing through, but for me there's also the "trivial process" which does nothing at all.
What I think differs between disciplines like construction engineering and software development is that with engineering you have the laws of physics as your backstop. Anybody can take a proposed design for a bridge/road/building/railway, run it through modelling software and come up with a conclusion if it will work or not with very high confidence. You can create processes that produce this result with high reliability.
Software development is not like that at all. There are many layers of abstraction, all of which shift like quicksand over time. The entire stack from CPU to desktop is so complicated nobody can even hope to understand even a small fraction of it all. Every layer is riddled with bugs. Bugs get fixed, which then trigger bugs elsewhere. It's a miracle your system even boots most days.
As such, it's very important how you deal with the inevitable breakage. Yes, a good design and development process up front will reduce the effort later on. But no amount of careful planning and design upfront can save you from something like log4j. So you need a process to deal with shit like that. All these systems that assume that if you just build it right in the first place it will work perfectly forever are completely misguided. It just doesn't work like that.
So Agile switches the focus to dealing with issues and changes as they come up. It's closer to real software development than the classic waterfall. And for better security we need to come up with processes to improve that. Better design up front is good, but for the bulk of software out there it's about dealing with the inevitable breakage.
The bitrot is real.
Posted Apr 8, 2024 12:35 UTC (Mon)
by Wol (subscriber, #4433)
[Link] (22 responses)
> Software development is not like that at all. There are many layers of abstraction, all of which shift like quicksand over time. The entire stack from CPU to desktop is so complicated nobody can even hope to understand even a small fraction of it all. Every layer is riddled with bugs. Bugs get fixed, which then trigger bugs elsewhere. It's a miracle your system even boots most days.
Software development IS like construction engineering. You have the laws of logic (maths) as your backstop. Anybody can take a proposed program and run it through an Arithmetic Logic Unit, and come up with a conclusion if it will work or not. You can create processes that produce this result with high reliability.
> As such, it's very important how you deal with the inevitable breakage. Yes, a good design and development process up front will reduce the effort later on. But no amount of careful planning and design upfront can save you from something like log4j. So you need a process to deal with shit like that. All these systems that assume that if you just build it right in the first place it will work perfectly forever are completely misguided. It just doesn't work like that.
But no amount of careful planning and design upfront can save you from something like aerated concrete. So you need a process to deal with shit like that. All these systems that assume that if you just build it right in the first place it will work perfectly forever are completely misguided. It just doesn't work like that.
The similarities between the birth of Civil Engineering, and now with the birth of Software Engineering, are immense. Back then they were experimenting with all these new wonder materials, and shit happened on a daily basis (it still is happening). The only difference with Software Engineering, is that instead of trying all these new wonder materials, and improving them, we are now trying all these wonder components, and instead of improving them when they break, we are replacing them with equally shitty alternatives. We are refusing to learn from history.
Cheers,
Posted Apr 8, 2024 20:43 UTC (Mon)
by kleptog (subscriber, #1183)
[Link] (21 responses)
Well, not really. Besides the Halting Problem, you cannot actually prove a program does what its supposed to just by running it. You'd need to run it under all possible inputs and possible environments. Mathematical proofs are only as good as their assumptions.
You can demonstrate a building will stay up by actually building it. You don't need to try it out on every location on Earth.
> You can create processes that produce this result with high reliability.
This is true, but not really the problem. Secure development would be the equivalent of an engineer designing a building/bridge/etc while there's a daemon out there who can change the laws of physics in localised areas when they feel like it. While construction engineering could certainly evolve in such a situation, I think it would look a look more like software development: less focus on perfect construction up front and more on holding it together once it's built.
(I guess you cold continue the analogy by including weather, wear-and-tear and all that, but the point is your adversary is *intelligent* and that makes all the difference.)
> and instead of improving them when they break, we are replacing them with equally shitty alternatives. We are refusing to learn from history.
I disagree. The progression from assembly to C to scripting languages and Rust/Go/C++/etc is undeniably an improvement at every step. C is just 50 years old. We have a long way to go.
Posted Apr 8, 2024 22:52 UTC (Mon)
by Wol (subscriber, #4433)
[Link] (20 responses)
Really? Wasn't the problem with the Boston Bridge disaster the fact that they DIDN'T focus on holding it together once it was built?
And I chose aerated concrete deliberately - that's a classic example of cost cutting - where civil engineering and software engineering are extremely similar ...
> > and instead of improving them when they break, we are replacing them with equally shitty alternatives. We are refusing to learn from history.
> I disagree. The progression from assembly to C to scripting languages and Rust/Go/C++/etc is undeniably an improvement at every step. C is just 50 years old. We have a long way to go.
And how many people are determined to stick with C? Far too many, I think. But I think log4j is a very good example of all that's wrong with software engineering. Am I correct in thinking it was ripped out and replaced? With an equally badly designed replacement?
There's absolutely no reason we can't design our software components properly, do the job properly, and to a large extent fix a huge amount of problems. Just apply the exact same techniques we use for civil engineering! The problem is (a) programmers all too often consider "design" a dirty word (or their managers do), and (b) it's cheaper and more commercially viable to do a shoddy job.
Cheers,
Posted Apr 8, 2024 23:11 UTC (Mon)
by pizza (subscriber, #46)
[Link] (1 responses)
You can't "design" something to "do the job" without the "job" being properly defined to begin with.
...And that won't stop the next person from taking something perfectly "designed to do job X properly" and proceed to apply it to job Y, and have something [not-so-]subtly go wrong. [1]
> (b) it's cheaper and more commercially viable to do a shoddy job.
No, the programmers are doing precisely the job they're being paid to do. If they're being paid at all. Culture rots from the top down.
[1] A former colleague of mine witnessed a Professional Engineer lose his license because a structure he designed (or at least signed off on) failed -- A reasonable outcome until you consider that unbeknownst to him the builder put up an identical copy of the designed structure on different site, which failed.
Posted Apr 9, 2024 7:53 UTC (Tue)
by Wol (subscriber, #4433)
[Link]
Agreed. But equally, the use of a small amount of brain would remove a large chunk of logical fallacy from our components. It doesn't take much to design obvious failure modes out of components. My favourite example from gentoo - a two-way truth table with states "does an unmounted boot partition exist?", and "may I mount an unmounted boot partition before updating it?"
Obviously if the first is false, the second is meaningless. But if the first is true and the second is false, the update process aborted! Surely the correct behaviour is to update the boot *directory* as the flags tell you? It gets worse - setting the flag to "don't update target directory" causes the search location for the source files to change, and a working update now fails with "can't find source files" ...
(The reason I had that slightly odd setup was because letting the default installer tamper with grub screwed up grub.conf and made the system unbootable!)
And I do a lot of work in VBA :-( Surely a simple line like "if not object is nothing" should return false if the object doesn't exist (like an open Excel spreadsheet). Instead, it crashes VBA! So I have to iterate through a list of objects looking for the one I want. Okay, that example isn't quite as egregious as the gentoo one - there might be good reasons for the object code not returning nothing, I don't know - but it pushes a LOT more work onto the end user and is a nasty papercut.
Likewise, the number of two-way truth tables with only three handled results ...
> ...And that won't stop the next person from taking something perfectly "designed to do job X properly" and proceed to apply it to job Y, and have something [not-so-]subtly go wrong. [1]
But that is harder if the component is properly designed and specified in the first place - and no I don't mean specified as in "this is what it's supposed to do" I mean "this is what it actually does".
It's the difference between writing a component to solve a problem, and defining a problem the component is intended to solve. You don't need much outside input to do the latter (which is the correct behaviour), but far too many programmers do the former.
And if you need the former (like we all do), then doing the latter will result in a far better and more robust program. Not least because you'll pick up corner cases the guy who defined the problem missed - usually sharp edges that bite you in production!
> A reasonable outcome until you consider that unbeknownst to him the builder put up an identical copy of the designed structure on different site, which failed.
I think you've mentioned that before ... and yes I agree it's a travesty ...
Cheers,
Posted Apr 9, 2024 9:16 UTC (Tue)
by farnz (subscriber, #17727)
[Link] (17 responses)
A serious issue here is that we can't apply the exact same techniques we use for civil engineering, since one component of those techniques is a feedback loop from the humans who build structures to the humans who design structures. But in programming, there is no human doing the build work; that is done by automation, called a "compiler", and thus we don't get human feedback on the issues with the design that make it sub-optimal according to the builder.
And we have a worse problem than civil engineering with the gap between the end-user specification and the design; by its nature, civil engineering is working in a lower-dimension problem space than software but even with that, civil engineers get awfully weak specifications and have to refine them into a design using rules of thumb and similar (how else do you go from "I want a bridge over that river" to "I need a bridge that can handle this dynamic load, this static load, this wind load etc"?).
The thing that protects civil engineers here is that they get to push back on bad management, since someone signs off and says "I will be personally liable if this design is faulty to the extent of harming human life". There's currently no equivalent liability in software, and until there is, there's no presssure to do better, since we can just disclaim the bad outcomes as "PROVIDED AS-IS WITH NO WARRANTY".
Posted Apr 9, 2024 9:45 UTC (Tue)
by Wol (subscriber, #4433)
[Link] (16 responses)
Except the compiler has nothing to do with the spec or the design. But I think the problem has been touched on elsewhere - someone mentioned kitchen appliances. Where the buyer far too often is not the end user, so the feedback loop to improving quality is broken. The same with software - the "buyer" is generally management, and the users are, well, considered two-a-penny and their opinion isn't worth consulting.
So Sales talk to Management, and Production and Users never meet, and no wonder the result is a mess. But if Production (aka the analysts and programmers) were encouraged to ask "does this spec even make sense?", and design components *as*components*, not as solutions, it would fix a lot of problems.
Civil Engineering says "I need these components, to these specs", and then manufacturing pushes back and says "a girder this thin won't take that load" or whatever. Analysts should say "I need these components to these specs", and then programmers push back and say "this component is logically incomplete, or inconsistent" or whatever.
The end result, be it Civil or Software, is a system that is far more likely to perform as designed, AND as required.
Cheers,
Posted Apr 9, 2024 10:14 UTC (Tue)
by farnz (subscriber, #17727)
[Link] (12 responses)
In civil engineering, the builder has nothing to do with the spec or the design, either. They are responsible for taking the design and constructing it as given, asking for changes to the design if, in their professional opinion, the design would result in a sub-optimal final structure (in terms of quality or cost). The civil engineer remains responsible for the design, and is supposed to ask for spec changes if the design is going to result in a sub-optimal final structure (in terms of quality or cost).
Note that in civil engineering, there are several phases of design; you'll start with rough drawings, and refine the design until you get to a complete construction plan that includes things like required materials, calculations that show that you have adequate supports and so on.
In programming, the source code is the construction plan equivalent; but the compiler takes the place of the builder, and the compiler is not nearly as good at providing feedback to the designer as a builder is at providing feedback to the engineer. A builder can ask for reassurance based on "vibes", given that they see a risk in the structure; a compiler doesn't have vibes to begin with, and won't ask you to reassure it based on a gut feeling.
This is a misrepresentation of programming; the programmer is not manufacturing, they're component design. In programming, manufacturing is done by the compiler, and in civil engineering, the manufacturing is done by the builder. And in civil engineering, the manufacturer is in a position to say "given where we're putting it, this girder is going to be under loads it can't handle because you've not got an annotation for wind load here", whereas in programming, you have a girder that meets specs, but is unsuitable for where it's being used, because the compiler isn't clever enough to say "you can't use that here and have a reliable program".
Posted Apr 9, 2024 13:55 UTC (Tue)
by Wol (subscriber, #4433)
[Link] (2 responses)
But a programmer should have vibes, given that they see a risk in the component. If you have a two-by-two truth table, and your component only has three states, then something is fucked up. THAT attitude alone would result in a sharp rise in quality.
That's why, when I program, I try to account for ALL POSSIBLE states. Yeah, I might handle certain states with a comment saying "this is not relevant to me, I haven't implemented it", but it's documented, the possibility someone else might want to handle it is recognised, and I DON'T sabotage it by accident because I was oblivious to the possibility ...
I'm currently getting far too much of that at work - the professional programmers designing our new system are ploughing ahead oblivious to the real world problems. "A driver has just rung in sick and won't be in for a week" - deleting the driver's shifts deletes all his runs, deleting 12 customer vans per run, deleting 20 customer orders per van! What's that going to do to our reputation - AND profit margin!!! 2000 customers pissed off ...
Don't laugh - this is real world :-( In the old days you'd just get another driver, or offer overtime, or whatever. In this Brave New World "I'm sorry Dave, I can't let you do that".
Cheers,
Posted Apr 9, 2024 14:17 UTC (Tue)
by pizza (subscriber, #46)
[Link]
One of the printer drivers I care about has 4096 potential _input_ state bits, and that's ignoring the lower-level USB stuff its protocol runs over (and possible timing-related issues [1]) Care to construct a truth table covering 2^4096 elements?
And this is relatively _simple_ stuff. There are over 500 billion state bits in the computer I'm using to type this -- Which works out to more possible states than there are atoms in the universe.
If you're going to say "oh, you can collapse large portions of that" then congratulations, you've just recreated the problem.
[1] This particular printer family infamously breaks when running on the first three Raspberry Pi generations due to the latter's USB controller effectively DoSing the printer through non-rate-limited flow control messages. It can be worked around by plugging in a low-speed device that slows down the overall message rate to the printer can keep up.
Posted Apr 9, 2024 14:45 UTC (Tue)
by farnz (subscriber, #17727)
[Link]
Sure, and in civil engineering, the architect (if any), the structural engineer, the planner, the surveyor and all the other sub-disciplines that go into producing an artefact also have vibes. But in software, the final sanity check of the design during conversion of the construction plan to a usable artefact is done by a computer program (the compiler), not a human.
This was not always the case; back in the day, when programming was a young discipline, the compiler was a human themselves, and they could do a vibe check, too. But now that the compiler is a computer program, we omit that final sanity check, because while a builder will say "hey, you've said this is a 10 storey building; why can't I find any escalators or elevators on the plans", a compiler will merrily assume that the reason you don't have things is that you don't need them.
And that makes direct comparisons to civil engineering problematic; most real structures have had at least one issue identified between the construction plans being handed over (where the programming world stops having humans involved) and the construction project being finished, as a builder looks at the plans and says "I'd expect to see this, why can't I see it?".
Posted Apr 9, 2024 13:59 UTC (Tue)
by Wol (subscriber, #4433)
[Link] (8 responses)
No. The programmer is the builder - he builds the source code. The compiler is the equivalent of the cranes, JCBs etc.
In programming, the compiler is a TOOL. Look at it *as*a*tool*, and the comparison makes sense.
Cheers,
Posted Apr 9, 2024 14:21 UTC (Tue)
by farnz (subscriber, #17727)
[Link] (7 responses)
The source code is a design, not a final artefact. The compiler builds the final thing that people use - the binary.
The equivalent of source code in civil engineering is the construction plans; these are the plans you create when you have all the information you need to build something, and where every decision that can be made is contained in the construction plans.
And yes, the compiler is a tool, but it replaces the builder, who in civil engineering is the one who takes the construction plans (source code for a bridge, a road, a building, whatever) and turns them into the final artefact that people use.
Posted Apr 9, 2024 16:03 UTC (Tue)
by Wol (subscriber, #4433)
[Link] (6 responses)
Programming is a mathematical discipline. Mathematically, the source and the binary (absent bugs in the compiler) ARE THE SAME THING.
> The equivalent of source code in civil engineering is the construction plans; these are the plans you create when you have all the information you need to build something, and where every decision that can be made is contained in the construction plans.
Are you saying the construction plans, and the building itself, are the same thing?
At the end of the day, what's wrong with programmers using their brains to say "this spec is wrong", just like you apparently expect the builders to say "these plans are wrong".
THAT is my point - the final stage where where human input goes into a program is the conversion from spec to code. The final stage where human input goes into a building is the construction. And be it civil engineering or software engineering, there are far too many screw-ups where it is blatantly clear the programmer/builder couldn't be arsed to understand the design he was implementing.
THAT is what I'm railing against - there are far too many jobbing idiots who can't read a blueprint! For f's sake, rub your braincells together!
And we should be using the exact same disciplines in software engineering, as they use in civil engineering, to try and make sure either (a) the design as implemented is the design as specified, or (b) the design as specified is modified because it's formally recognised it's wrong! We need two-way communication that makes sure the final blueprints and implementation match, not jobbing idiots who bodge a flawed mess together.
Cheers,
Posted Apr 9, 2024 16:27 UTC (Tue)
by farnz (subscriber, #17727)
[Link] (2 responses)
No amount of capital letters makes you more convincing, nor does your use of truncated swear words and insults. Programming is not just a mathematical discipline, since we run programs on real machines, and the final output is not a mathematical proof, but a program running on real hardware, causing real physical effects.
For the specific case of a program that has no physical effect at all (no I/O), programming is purely mathematical; once you introduce I/O, it's not just mathematics, it's also electronics and physics.
And mathematically, the construction plans and the final building are the same thing. What I'm claiming is that there's one human last step in construction that is not present in software, of converting the construction plans to a real building where faults in the plans can be corrected by the person who converts the plans into an artefact with real physical implications; that doesn't exist in the software world, because it's a computer program that converts the plans into an artefact with real physical implications.
I get your point, but it summarises down to "I think everyone else is an idiot, and you are too dumb to argue with me because I am right and use capital letters and offensive phrasing to make it clear how stupid everyone else is".
In civil engineering, there's more than one person between the original specification ("build me a bridge over this river to take traffic") and the final construction plans, just as in software. The problem is that we get from specification to design (which in civil engineering gets you from customer to structural engineer), and then have no further involvement from humans from design (source code) to final artefact (running binary doing a task), whereas in civil engineering, there's one last human step that can identify problems.
And the various civil engineers are all supposed to use their brains to say "this spec is wrong", and yet even with that, lots of plans with (in some cases serious) faults make it to the builders; there's "far too many jobbing idiots" in civil engineering who can't do the job, and rely on the builders correcting them when they come up with designs that can't even be built, let alone used.
If you actually talk to civil engineers, you'll find that a lot of problems in civil engineering are exactly the same as in software - indeed, they look to us as exemplars of how to do better, because they have a serious problem with people designing to specification regardless of whether the specification makes sense, whereas we are perceived to have better discipline that means that what we design tends to be usable by the time they see it.
It's only after long conversations with people who do civil engineering for a living that you realise that there is precisely one significant divergence between us and them; with us, if I put together some software and say "it's not yet ready to be shipped to users", someone else can ship it anyway with zero consequences. With them, if they put together a design and say "it's not ready to be built yet", anyone who authorizes building it is taking on criminal and civil liability if the design turns out flawed. Every other difference you see is simply because you very rarely deal with civil engineering as a peer, but only ever as a customer - and they see us as doing far better, precisely because they never see the messes that get cleaned up behind closed doors.
Posted Apr 9, 2024 18:24 UTC (Tue)
by Wol (subscriber, #4433)
[Link] (1 responses)
And here is the disagreement between us. To me, the construction plans are the blueprint for the building, the specification is the blueprint for the code. So where you argue that the step of converting (source) code to (object) code has no equivalent in the civil world, I would argue that converting code to equivalent code is a no-op so that's irrelevant.
The fact that we find problems with our blueprint by implementing it repeatedly until it works actually has great similarities with Civil Engineering too ... buildings / bridges / tunnels regularly collapse because we botched the "blueprint to reality" conversion. Less so nowadays but even today (the Millenium Bridge over the Thames, anyone?) Civil engineering cock-ups aren't a rare occurrence. How many arches collapsed before the medieval builders discovered the parabola? And how many collapsed after because medieval journeymen didn't UNDERSTAND the parabola?
It's just that with software it's much easier to sweep the cockups under the carpet :-) (provided they don't escape, of course).
> It's only after long conversations with people who do civil engineering for a living that you realise that there is precisely one significant divergence between us and them; with us, if I put together some software and say "it's not yet ready to be shipped to users", someone else can ship it anyway with zero consequences. With them, if they put together a design and say "it's not ready to be built yet", anyone who authorizes building it is taking on criminal and civil liability if the design turns out flawed.
Thank God for the CE mark ... okay it may not involve criminal liability, but it is certainly enforcing some degree of civil liability. And this goes back to me comparing the *birth* of Civil Engineering with Software. If the biggest difference is primarily the lack of consequences for shipping broken software, then Civil and Software Engineering are clearly converging ...
And I stand by my comment that the biggest problem with both Civil Engineering and Software Engineering, is people failing to use their brains to look for OBVIOUS logical fallacies in the design - be it Civil blueprints, or Software specifications. (Or simply asking questions like "why are we doing it *this* way?")
"This doesn't make sense" is a comment that should be said a lot more than it is.
Cheers,
Posted Apr 9, 2024 19:12 UTC (Tue)
by farnz (subscriber, #17727)
[Link]
A civil engineer would argue that the step of converting a set of constructions plans (of which a blueprint is one component, there's a lot more in there) to a building is converting plans to equivalent building, and is as much a no-op as running an optimizing compiler over the code is.
And note that both computerised compilers and hand compilers don't do a completely literal conversion - they apply "optimizations" in the computer world, where they create object code that is not a 1:1 representation of the source code, but instead (assuming the optimizations are sound) encodes the same allowed behaviours in a very different fashion. In general, while the object code should function the same as the source code, it's not equivalent, and converting the object code back to equivalent source code is rarely possible with an optimizing compiler (since the compiler removes chunks of code that have no effect on the outcome, changes the operations in use, and otherwise creates a very different output to that set out in the design document).
Similar applies in construction - the builder does not literally convert the thing in the construction plans into an output artefact; they make small changes that give you a better outcome (for example, substituting materials according to known rules that guarantee improvements at worst).
Posted Apr 9, 2024 18:09 UTC (Tue)
by daroc (editor, #160859)
[Link] (2 responses)
But "For f's sake, rub your braincells together" seems like a pretty clear personal attack, which we don't permit here. Please refrain from attacking other commenters like that.
Posted Apr 9, 2024 19:37 UTC (Tue)
by Wol (subscriber, #4433)
[Link]
It was, however, a cry of frustration against the world in general - it's just the amount of grief that's caused by people who don't rub their brain cells together ...
Cheers,
Posted Apr 10, 2024 16:48 UTC (Wed)
by metalheart (guest, #89328)
[Link]
Posted Apr 9, 2024 20:38 UTC (Tue)
by mathstuf (subscriber, #69389)
[Link] (2 responses)
Posted Apr 10, 2024 21:29 UTC (Wed)
by kleptog (subscriber, #1183)
[Link] (1 responses)
Posted Apr 11, 2024 10:20 UTC (Thu)
by farnz (subscriber, #17727)
[Link]
Part of the problem is that people don't know what's easy and what's hard (XKCD comic) in software, so when we say things that people think are hard are easy for software, and things people think are easy are hard for software, they lose track of what we can and cannot do, and just assume that anything they ask for is possible with enough demand.
This is where I see the value in teaching everyone to code; it's not that most people "should" write code, it's that by being able to do some coding, you get an understanding of why we say some things are easy and other things are hard. And with that in hand, you're less likely to ask for the impossible, because you don't see us as living in a confusing world where hard things are easy, but instead as having a set of rules for what's hard that aren't the same as those you normally live with.
Eclipse Foundation announces collaboration for CRA compliance
Eclipse Foundation announces collaboration for CRA compliance
Eclipse Foundation announces collaboration for CRA compliance
Eclipse Foundation announces collaboration for CRA compliance
Eclipse Foundation announces collaboration for CRA compliance
Eclipse Foundation announces collaboration for CRA compliance
Eclipse Foundation announces collaboration for CRA compliance
Eclipse Foundation announces collaboration for CRA compliance
Eclipse Foundation announces collaboration for CRA compliance
Eclipse Foundation announces collaboration for CRA compliance
Eclipse Foundation announces collaboration for CRA compliance
- Ernst & Young: Navigator
- Deloitte: 4FRONT
- Arthur Anderson: Method One
- IBM: A/D-Cycle
- .....and so on.....
Eclipse Foundation announces collaboration for CRA compliance
Eclipse Foundation announces collaboration for CRA compliance
Wol
Eclipse Foundation announces collaboration for CRA compliance
Eclipse Foundation announces collaboration for CRA compliance
Wol
Eclipse Foundation announces collaboration for CRA compliance
Eclipse Foundation announces collaboration for CRA compliance
Wol
Eclipse Foundation announces collaboration for CRA compliance
There's absolutely no reason we can't design our software components properly, do the job properly, and to a large extent fix a huge amount of problems. Just apply the exact same techniques we use for civil engineering! The problem is (a) programmers all too often consider "design" a dirty word (or their managers do), and (b) it's cheaper and more commercially viable to do a shoddy job.
Eclipse Foundation announces collaboration for CRA compliance
Wol
Eclipse Foundation announces collaboration for CRA compliance
Except the compiler has nothing to do with the spec or the design. But I think the problem has been touched on elsewhere - someone mentioned kitchen appliances. Where the buyer far too often is not the end user, so the feedback loop to improving quality is broken. The same with software - the "buyer" is generally management, and the users are, well, considered two-a-penny and their opinion isn't worth consulting.
Civil Engineering says "I need these components, to these specs", and then manufacturing pushes back and says "a girder this thin won't take that load" or whatever. Analysts should say "I need these components to these specs", and then programmers push back and say "this component is logically incomplete, or inconsistent" or whatever.
Eclipse Foundation announces collaboration for CRA compliance
Wol
Eclipse Foundation announces collaboration for CRA compliance
Eclipse Foundation announces collaboration for CRA compliance
Eclipse Foundation announces collaboration for CRA compliance
Wol
Eclipse Foundation announces collaboration for CRA compliance
Eclipse Foundation announces collaboration for CRA compliance
Wol
Eclipse Foundation announces collaboration for CRA compliance
Eclipse Foundation announces collaboration for CRA compliance
Wol
Eclipse Foundation announces collaboration for CRA compliance
And here is the disagreement between us. To me, the construction plans are the blueprint for the building, the specification is the blueprint for the code. So where you argue that the step of converting (source) code to (object) code has no equivalent in the civil world, I would argue that converting code to equivalent code is a no-op so that's irrelevant.
Eclipse Foundation announces collaboration for CRA compliance
Eclipse Foundation announces collaboration for CRA compliance
Wol
Eclipse Foundation announces collaboration for CRA compliance
Eclipse Foundation announces collaboration for CRA compliance
Eclipse Foundation announces collaboration for CRA compliance
Eclipse Foundation announces collaboration for CRA compliance