Security quotes of the week [LWN.net]

I have manually reported several compromised shops and got some curious responses:

We don’t care, our payments are handled by a 3rd party payment provider

If someone can inject Javascript into your site, your database is most likely also hacked.

Thanks for your suggestion, but our shop is totally safe. There is just an annoying javascript error.

Or, even better:

Our shop is safe because we use https

— Willem de Groot on reporting online skimming to merchants (Thanks to Paul Wise.)

The need for what we have heretofore called cybersecurity is now so varied that it is no longer a single field but many. There are over 800, perhaps over 1000, cybersecurity startups in some stage of the funding game, a fair fraction of them spinouts from highly focused university research projects. Generalists such as myself cannot be replaced -- there is too much for the novitiate to learn. The core knowledge base has reached the point where new recruits can no longer hope to someday become competent generalists, serial specialization is the only broad option available to them.

As I say often, cybersecurity is perhaps the most difficult intellectual occupation on the planet. Note that I said "occupation" rather than "profession." Three Septembers ago, the U.S. National Academy of Sciences concluded that cyber security should be seen as an occupation and not a profession because the rate of change is simply too great to consider professionalization. Ray Kurzweil is beyond all doubt correct; within the career lifetime of nearly everyone in this room, algorithms will be smarter than we are, and they will therefore be called upon to do what we cannot -- to protect us from other algorithms, and to ask no permission in so doing. Do we, like Ulysses, lash ourselves to the mast or do we, as the some would say, relax and enjoy the inevitable? What would we have science do? What are the possible futures you will tolerate? What horses do you want not let out of the barn? Where do we put our intelligence budget? US CYBERCOM's budget is $500 million, JPMorganChase, alone, is spending $600 million. Is that surprising or is that as it should be?

— Dan Geer

to post comments

Security quotes of the week

Posted Oct 20, 2016 0:55 UTC (Thu) by Garak (guest, #99377) [Link] (21 responses)

algorithms will be smarter than we are, and they will therefore be called upon to do what we cannot -- to protect us from other algorithms, and to ask no permission in so doing. Do we, like Ulysses, lash ourselves to the mast or do we, as the some would say, relax and enjoy the inevitable?

Just because some kid can recite more digits of pi than I can doesn't make them smarter than me. And just as volutarily selling oneself into slavery is a clearly bad idea, so too would be voluntarily selling oneself into slavery to algorithms.

Security quotes of the week

Posted Oct 20, 2016 10:05 UTC (Thu) by farnz (subscriber, #17727) [Link] (20 responses)

Against that, the baseline human error rate is somewhere around 0.5% for purely mechanical tasks (but can be trained to be much lower for repetitive mechanical tasks), and closer to 5% for tasks involving complex reasoning. If you go with an error rate of 0.1% for handling problems, you expect about 1 error in every 1,000 problems that people handle; most of these errors will be insignificant (e.g. driving at 36 MPH when the speed limit is 35 MPH), but some will be significant.

Where algorithms shine is when they act as a filter to remove the obvious and trivial from human attention; if you have an algorithm with 100% precision (i.e. no false positives - if it detects a problem at all, there is a problem) taking action when it detects a problem, and passing the remaining data to an algorithm with 100% recall (i.e. no false negatives - if it says there isn't a problem, there isn't one) to pick out cases for human attention, you can take a firehose of millions of reports per hour, and reduce it down to a manageable size for a small team of humans.

You can't rely on the algorithms alone, but you can use them to reduce human involvement to the difficult cases only, where a judgement call is required.

Security quotes of the week

Posted Oct 20, 2016 17:27 UTC (Thu) by mgb (guest, #3226) [Link] (19 responses)

> You can't rely on the algorithms alone, but you can use them to reduce human involvement to the difficult cases only, where a judgement call is required.

And instead of staying in practice the humans are suddenly dumped into an unexpected situation which they fail to analyze before the airliner hits the ocean.

Security quotes of the week

Posted Oct 20, 2016 17:30 UTC (Thu) by farnz (subscriber, #17727) [Link]

That's what regular training is for. The algorithm handles the easy cases, and your training sessions are entirely about the judgement calls.

In any case, in the context of Internet abuse handling, I don't think there will be a shortage of unexpected situations - aircraft are a completely different field.

Security quotes of the week

Posted Oct 21, 2016 11:53 UTC (Fri) by jwarnica (subscriber, #27492) [Link] (17 responses)

Flying is quite easy. For that matter, so is landing and taking off. Assuming, that is, that the weather is reasonable and that the equipment is operating well. One does not need a lot of regular practice to maintain that skill.

The situations where you do need a human to take over from the computers are not really helped at all by being able to do the normal thing. You fly - and crash - simulators to learn how to handle the unusual.

Security quotes of the week

Posted Oct 21, 2016 18:11 UTC (Fri) by raven667 (guest, #5198) [Link] (16 responses)

I think this is a round-a-bout reference to Air France Flight 447 which crashed into the Atlantic on the way back from Brazil to Paris, where all of the redundant pitot tubes iced up, depriving the autopilot, and human pilots, of critical airspeed information, causing the autopilot to give up and rendering inoperative many of the automatic warnings and safety controls that rely on valid sensor data. The pilots never figured out what was going on and kept applying throttle and pitch up, as well as going around in circles, stalling the aircraft until it crashed.

Security quotes of the week

Posted Oct 21, 2016 18:14 UTC (Fri) by farnz (subscriber, #17727) [Link]

Mmm, and based on the actual crash investigation report, the pilots probably wouldn't have figured out what was going on even if they were used to flying manually.

The proximate cause of the crash was one pilot pulling up, even while the other tried to pitch down. Nothing about normal flight would help there - the pilot pulling up either did not realise that they were doing it, or would have attempted it in a "no airspeed indicator" situation anyway.

Security quotes of the week

Posted Oct 27, 2016 3:13 UTC (Thu) by paulj (subscriber, #341) [Link] (14 responses)

A lack of basic airmanship from the junior co-pilot by, combined with the junior co-pilot (presumably due to some kind of panic) then going on to frustrating the efforts of the more experienced co-pilot (who was acting as pilot-in-command while the captain was resting) to recover were the immediate causes of the crash.

The junior co-pilot responded to the lack of speed information by pitching up, bizarrely. As stall speed increases with altitude, when flying high the difference between stall speed and max speed can be small. It is therefore imperative to not lose speed. This is super-basic knowledge about flying a jet at altitude. Should speed information be lost, the correct thing to do is pitch *down* slightly to ensure speed can not decrease below stall speed. It is better to risk slight over-speed than stalling.

The senior co-pilot / PIC knew exactly what was going on and what the correct control inputs to give were - at least least in terms of the aircraft's systems, its initial behaviour and correct airmanship. The PIC gave those correct control inputs, once he took over controls from the junior co-pilot - who had taken the wrong actions. What the PIC did not realise is that the junior co-pilot would continue to give control commands to the aircraft, despite having been told to relinquish control, and what the PIC did not figure out was how the contradictory inputs of the junior co-pilot were being averaged with his own by the aircraft to result in non-useful inputs. (There were some audible warnings given by the aircraft about the control being taken and conflicts IIRC, but obviously the PIC - and later the captain - didn't immediately process those). The captain and senior co-pilot did eventually figure out the junior co-pilot was the immediate source of the problem, but by then it was too late - they were going to hit the sea.

Basically, the junior co-pilot crashed that aircraft, with some "basic 101 of flying high" major fail, and then - for some terrible reason - counter-acting the efforts of the other pilot to recover. The contributing factor was how the aircraft's control system responded to conflicting inputs, and how it gave feedback (or didn't).

If not for the junior co-pilot's continuing actions, the senior co-pilot likely would have recovered the aircraft - after the initial poor response by the junior co-pilot. If the junior co-pilot hadn't been there at all, if it had been just the captain and the senior co-pilot in the cockpit, likely there'd have been no incident at all on that flight - other than just a relatively nonchalant, slow, descent to less cold air, which would have de-iced the pitots. A report to maintenance and to the authorities, and at worst a divert.

Sources for the above: The crash report, and discussion I've had on AF447 with a pilot who has experienced similar pitot icing at altitude.

Security quotes of the week

Posted Oct 27, 2016 17:39 UTC (Thu) by Wol (subscriber, #4433) [Link] (8 responses)

> The PIC gave those correct control inputs, once he took over controls from the junior co-pilot - who had taken the wrong actions. What the PIC did not realise is that the junior co-pilot would continue to give control commands to the aircraft, despite having been told to relinquish control, and what the PIC did not figure out was how the contradictory inputs of the junior co-pilot were being averaged with his own by the aircraft to result in non-useful inputs. (There were some audible warnings given by the aircraft about the control being taken and conflicts IIRC, but obviously the PIC - and later the captain - didn't immediately process those).

Problem is, those audible warnings were the sensors shouting "Nose up! Nose up!". I can't remember why. So the junior pilot was doing exactly what the automated warning systems were telling him to.

> Basically, the junior co-pilot crashed that aircraft, with some "basic 101 of flying high" major fail, and then - for some terrible reason - counter-acting the efforts of the other pilot to recover. The contributing factor was how the aircraft's control system responded to conflicting inputs, and how it gave feedback (or didn't).

And the fact that the warning systems, iirc, were programmed on the assumption that a stall was impossible (the pitot tubes would force the nose down if a stall was likely), so the whole automated system, and the warnings given, bore no resemblance to the reality in the air. As I say, the warning systems were telling the pilots "Nose up!" when the aircraft was in a stall. And the pilots took a while to recognise the problem because they had precious little experience of what a stall actually was!

(imho all commercial pilots should have at least some, regularly refreshed, aerobatic experience even if only on a "let's have fun" basis :-)

Cheers,
Wol

Security quotes of the week

Posted Oct 27, 2016 18:27 UTC (Thu) by paulj (subscriber, #341) [Link] (6 responses)

That's not right. The audible warnings were stall warnings - spurious when speed information was lost initially with the pitot freeze, but for real later when they warmed and speed information was regained - and IIRC a warning for the junior co-pilot pressing the "I have control" button on the stick (can't find that back right now).

Where do you get that "Nose up" warnings sounded? I've never read that in the transcripts or the final report?

Also "(the pitot tubes would force the nose down if a stall was likely)" does not make sense. The pitot tubes are metal tubes, that siphon and direct air to pressure sensors, from which airspeed information is derived. The aircraft's flight director systems went into "alternate law" mode when speed information became unreliable. I.e., the pilots had pretty much full control over the aircraft, with no computer control assistance or overrides, other than some very basic damping and limiting of extreme inputs. Note that even without speed information, other instruments could have helped them ascertain the failure was in the speed readings and hence they weren't stalled - and the senior co-pilot knew it. They were _not_ in a stall initially. It was the junior co-pilot's _reactions_ that then stalled the aircraft. Even then, the senior co-pilot knew and would have recovered, if not for the continued counter-actions of Bonin.

The problem with AF447 was the junior co-pilot, Bonin, and basic flying fail. Exacerbated by the lack of cross-stick feedback of input, and the lack of 'real' flying experience of many modern comercial pilots, due to the highly automated flight management systems of today. As per the official report:

"Current training practices do not fill the gap left by the non-existence of manual flying
at high altitude, or the lack of experience on conventional aeroplanes. Furthermore,
they limit the pilots’ abilities to acquire or maintain basic airmanship skills."

Security quotes of the week

Posted Oct 31, 2016 23:30 UTC (Mon) by Wol (subscriber, #4433) [Link] (3 responses)

> Where do you get that "Nose up" warnings sounded? I've never read that in the transcripts or the final report?

Probably an analysis of the crash on Risks, from the person who regularly analyses all that stuff.

From what I remember, the aircraft "could not stall", a bit like the Titanic could not sink. But because the sensors froze, it did stall, and everything stemmed from that. The warning system got confused, the junior pilot panicked, etc etc.

The thing that sticks in my mind was that the senior pilot recognised the stall and was trying to put the aircraft nose down to gain speed, while the sensors recognised that the aircraft was losing height or something, and were emitting audible warnings to "pull up" to gain height. So the junior pilot, unbeknownst to the senior pilot, was trying to pull up and frustrating the senior pilot's attempts to recover the aircraft.

(Note for mjg, I'm fairly certain I did, at some point, read the official report, although quite a long time ago ...)

Cheers,
Wol

Security quotes of the week

Posted Oct 31, 2016 23:41 UTC (Mon) by mjg59 (subscriber, #23239) [Link]

> The thing that sticks in my mind was that the senior pilot recognised the stall and was trying to put the aircraft nose down to gain speed, while the sensors recognised that the aircraft was losing height or something, and were emitting audible warnings to "pull up" to gain height. So the junior pilot, unbeknownst to the senior pilot, was trying to pull up and frustrating the senior pilot's attempts to recover the aircraft.

That isn't what happened. When people indicate that you're making unsubstantiated claims, making further unsubstantiated claims rather than checking to see whether you're correct is probably not a great approach.

Security quotes of the week

Posted Oct 31, 2016 23:52 UTC (Mon) by paulj (subscriber, #341) [Link] (1 responses)

Again, that's not correct.

- There was never any "pull up" warning.

- The aircraft did not stall because the pitot tubes froze.

- The aircraft stalled because of the actions of the junior co-pilot (who was the pilot flying, the senior co-pilot was not flying but was in command) in response to the loss of speed information

- The aircraft should never stalled and would never have stalled but for the actions of the junior co-pilot - the senior co-pilot clearly knew what was required (which, again, is really basic airmanship wrt flying at altitude).

Again, I know a pilot who had a pitot freeze up at altitude. The correct response is to descend at a relaxed rate (the altitude and vertical airspeed indicators were still fully functional).

Security quotes of the week

Posted Nov 1, 2016 0:36 UTC (Tue) by paulj (subscriber, #341) [Link]

Oh, if there was a "pull up" warning it would have been towards the end, as the radio altimeter information would have started to show too-imminent approach of the ground (and outside of landing mode). That had about as much to do with the /cause/ of the crash as the impact with the sea did.

Here's an english translation of the transcript of the CVR btw:

http://tailstrike.com/010609.pdf

No pull up warnings till they were at 4k feet or below, stalled - not flying but basically just free falling fast, almost belly first, with a bit of forward momentum too.

It also shows the "Dual-input" warnings, which the Airbus sounds, but which I couldn't find in the other well-known web article with an english transcript (the popular mechanics one). As both Roberts and Bonin are giving inputs - which the Airbus flight director helpfully averages out, with no other feedback but the "Dual-input" warnings.

There was an earlier "priority right" warning, when Bonin pressed the button on his stick to indicate he intended to take control. By that point, Roberts has also become confused, presumably by the combination of the incoherence of Bonin and the attitude the aircraft is now in due to Bonin's action (earlier, at around 2h10m31s, so 27s odd into the event, Robert was giving the correct instructions).

Security quotes of the week

Posted Nov 1, 2016 8:17 UTC (Tue) by farnz (subscriber, #17727) [Link] (1 responses)

Part of the issue from my perspective is that we count all flying time as roughly equivalent for judging competence, and with the rise in automation, it's not. So, a junior pilot can have 5,000 hours of flying time, where 4,000 hours of it was depending on the automation to keep the plane safe, while another pilot has a mere 2,000 hours, but all of that in planes without automation, or in situations where the automation is not enabled. In the current models of competence, the first pilot is "more experienced", but that's no longer true - the second pilot has more experience of "hard" flying.

Combine that with a junior pilot who didn't release control to the senior pilot (probably wasn't even aware that the senior pilot was trying to override him), and you have a recipe for disaster.

Security quotes of the week

Posted Nov 1, 2016 18:09 UTC (Tue) by paulj (subscriber, #341) [Link]

That is part of the problem. Another part of the problem is a culture that focuses heavily on process and frowns on any deviation from process - for very good safety reasons - but where the processes do not really allow pilots to practice actual flying, except for a few days a year in a simulator (landings excepted, but those are becoming more and more automated too).

Especially at major airlines, departure from process is heavily frowned upon - career limiting. A pilot who tries to practice his manual flying when the process says the plane must be flown by the management systems (as is generally always the case for cruise) will be admonished if found out. As happened to the pilot I spoke to, when he was captain at a major airline.

Security quotes of the week

Posted Oct 27, 2016 18:29 UTC (Thu) by mjg59 (subscriber, #23239) [Link]

> those audible warnings were the sensors shouting "Nose up! Nose up!".

No they weren't. You keep making factual assertions on various topics that are, well, not actually factual.

Security quotes of the week

Posted Oct 28, 2016 16:29 UTC (Fri) by raven667 (guest, #5198) [Link] (4 responses)

> A lack of basic airmanship from the junior co-pilot by, combined with the junior co-pilot (presumably due to some kind of panic) then going on to frustrating the efforts of the more experienced co-pilot (who was acting as pilot-in-command while the captain was resting) to recover were the immediate causes of the crash.

This is relevant to the original discussion of self-driving cars in that instead of highly trained pilots, who can still panic and make poor choices, you will have untrained people who are going to be expected to start making complex split-second decisions when the autopilot gives up if the situation is too much for it to handle. The risk for each mile driven by a human in a self-drive car is much much higher than now, because the only miles driven will the the riskiest ones that the autopilot can't handle, without the driver having the benefit of experience to help handle it. It'll be a miracle if the driver could get the car off the road and stopped without hitting anything in that case.

Security quotes of the week

Posted Oct 28, 2016 16:41 UTC (Fri) by farnz (subscriber, #17727) [Link] (3 responses)

On the other hand, a self-driving car has a safe option that a self-flying aircraft does not - come to a halt, hazards on, and call for help. You can then make overriding the car something that requires separate authorisation to using the car - in other words, instead of simply going "I can't cope - you drive!", the car can go "I can't cope - I'm stopping now, and the dealer will send a trained driver out to get you to safety".

Security quotes of the week

Posted Oct 28, 2016 17:52 UTC (Fri) by mgb (guest, #3226) [Link] (2 responses)

Quite often when there is a serious accident on a freeway/motorway there are two or three fender benders in the resulting traffic backup/jam.

A confused AI stopping and putting on the hazard lights would be a hazard, a danger, and a liability.

But then today's AI is really not suited to anything much more serious than uncrowded residential streets in good weather.

Security quotes of the week

Posted Oct 28, 2016 17:57 UTC (Fri) by farnz (subscriber, #17727) [Link]

Sure, but a confused human driving on or stopping is also a hazard, a danger, and a liability.

The question to answer is not "when will a computer drive perfectly in all conditions", but "when will a computer drive better than the human currently driving the car"; we already accept a degree of costs due to human error, and if the computer can shave those costs, then we'll take it. After all, we've already (in my driving career) accepted automatic gearboxes, TC, ESC, ABS, emergency automatic braking and other setups where the automation takes over from a human (and usually does a better job than a human would); this is just one step on.

Security quotes of the week

Posted Oct 31, 2016 23:34 UTC (Mon) by Wol (subscriber, #4433) [Link]

A fender bender isn't a problem. I've been in a crash like that (the car behind me took off at a junction faster than I did - bigger engine, lighter car). I stopped, he stopped, car behind stopped, car behind him didn't ...

But the thing is, if the cars are travelling in the same direction, a fender-bender smash - even a big one - is not likely to leave anyone seriously hurt. Nearly all serious casualties are caused when something heavy - an artic, a loaded truck, or the like - then smashes into the fender-bender.

Cheers,
Wol