The Internet of criminal things

Posted Oct 5, 2015 19:08 UTC (Mon) by przemek.klosowski (guest, #100907)
Parent article: The Internet of criminal things

The 'defeat device' is not ill-defined media contraption---to the contrary, it is very precisely defined by US law, https://www.law.cornell.edu/cfr/text/40/86.1809-10 .

Herein lies a problem, though: IANAL but my understanding is that the manufacturer is deemed to have installed the 'defeat device' if certain set of measurements satisfies the criteria. Those results can happen either because the code has a sinister detectEPAtestAndCheat() procedure written at the request of a pointy-haired boss, or because the algorithm parameters were optimized to death and resulted in such operation. I don't mean to find excuses for the manufacturer---I just hope the investigation will determine the exact sequence of things that lead to the problem, and shine light on how much it was the perfidy of the management versus engineering corner-cutting. I think there's a difference there.

One way of looking at this is: would you judge the situation differently if the algorithm was really a neural network whose learning set consisted of both the EPA test constraints, and a maximum power/torque constraint?

The Internet of criminal things

Posted Oct 5, 2015 19:32 UTC (Mon) by dlang (guest, #313) [Link] (11 responses)

along similar lines, over the weekend news broke that Samsung is being accused of installing a 'defeat device' in their TVs to detect that they are under test and use less power.

Samsung is responding that the test patterns are static, and trigger power savings optimizations that are part of normal operation.

so "defeat device" in software depends a lot on the parameters that trigger it and the intent of the programmers.

In the case of VW, is the different fueling used because the throttle is constant and so it can satisfy the power requirements more efficiently? (a valid optimization, even on the open road) or because it detects something that's actually specific to the test?

The Internet of criminal things

Posted Oct 5, 2015 19:45 UTC (Mon) by bronson (subscriber, #4806) [Link] (2 responses)

> In the case of VW, is the different fueling used because the throttle is constant and so it can satisfy the power requirements more efficiently? (a valid optimization, even on the open road) or because it detects something that's actually specific to the test?

It actually detected the test: "The 'switch' senses whether the vehicle is being tested or not based on various inputs including the position of the steering wheel, vehicle speed, the duration of the engine's operation, and barometric pressure. These inputs precisely track the parameters of the federal test procedure..."

http://www3.epa.gov/otaq/cert/documents/vw-nov-caa-09-18-...

The Internet of criminal things

Posted Oct 5, 2015 21:53 UTC (Mon) by dlang (guest, #313) [Link] (1 responses)

note that you are pointing at the accusation document, not the result of any investigation that actually see the code

If there is (as is alleged) a "dyno mode" vs a "road mode", the dyno mode would also kick in when doing other dyno tests (performance and economy come to mind)

unless the dyno mode is _very_ specific to the exact details of the EPA test, which seems incredibly unlikely (the test changes over time)

They have a number of fuel maps in the ECU that are used under different conditions, the accusation lumps them all together into 'road mode', but it's very possible that there is very little difference between some of the 'road mode' maps and the 'dyno mode' map. it all depends on exactly what the conditions are for switching.

The EPA thinks they have evidence that it's malicious, and at this point I don't think we will ever find out the real details. The new VW management wants to get this behind them and survive. Proving that this wasn't necessarily malicious would by a Pyrrhic victory at this point.

The Internet of criminal things

Posted Oct 5, 2015 23:46 UTC (Mon) by bronson (subscriber, #4806) [Link]

The Notice of Violation is all we have right now. The investigation, as you know, could take years.

You can choose to consider the EPA incompetent or liars if you want. That seems a strange stance to take because even VW publicly stated in a conference call on Sept 3 that the Notice of Violation is basically correct.

And, even if you don't want to take the EPA and VW's word for it, why would VW's fuel maps look so radically different depending on steering input? There's simply no physics-based explanation for why their engine computer would demonstrate such strange behavior.

I'm also looking forward to reading the report. I expect it will describe a series of small and expedient decisions that resulted in this ECU mode. An evil boss telling his engineers, "write me a defeat device from scratch" seems implausible and difficult to keep under wraps. :)

The Internet of criminal things

Posted Oct 5, 2015 19:48 UTC (Mon) by bronson (subscriber, #4806) [Link] (5 responses)

> a valid optimization, even on the open road

No, if it increases NOx output 40X above the limit, it is obviously not a valid optimization. Did you mean something else?

The Internet of criminal things

Posted Oct 5, 2015 21:43 UTC (Mon) by dlang (guest, #313) [Link] (4 responses)

cars put out a lot more pollution on the open road driving at real speeds, climbing real hills than they do in the EPA simulated test.

They are going faster, carrying more weight, almost always driven with a heavier foot on the skinny-pedel, etc.

Go read up on what the test actually consists of and you will be horrified at how little resemblance it has to real-world driving.

The Internet of criminal things

Posted Oct 5, 2015 23:21 UTC (Mon) by bronson (subscriber, #4806) [Link] (3 responses)

Where did I say the test resembled real-world driving? And, what does it matter? My statement: if the VW is putting out 20-40X more NOx than it should over a particular driving profile, then that is simply not a valid optimization to make.

At first i thought your reply was saying: since real world driving doesn't match the test very closely, we should just chuck the test out the window and give up. But that's both defeatist and not in reply to anything I said... so I'm guessing I'm misinterpreting?

The Internet of criminal things

Posted Oct 5, 2015 23:24 UTC (Mon) by dlang (guest, #313) [Link] (2 responses)

I think we disagree that it's the same driving profile

The Internet of criminal things

Posted Oct 5, 2015 23:51 UTC (Mon) by bronson (subscriber, #4806) [Link] (1 responses)

I only know of one driving profile that we're talking about: the one the EPA tested and VW optimized. I'm not aware of any others that matter for this discussion...?

The Internet of criminal things

Posted Oct 6, 2015 20:46 UTC (Tue) by dlang (guest, #313) [Link]

the accusation by the EPA is that there are multiple profiles in the ECU and that the car detects that it's being tested by the EPA and switches to one that pollutes less than the ones that are used for normal driving.

The EPA test profile is very strict and not something that will exactly match any on-the-road test. It was people doing on-the-road emissions tests and looking at differences that raised the concerns and started the investigation.

how close the EPA dyno test is to the on-the-road test that showed issues is part of what ends up confusing the issue (see the samsung TV power consumption issue), it all depends on what the exact triggers are to change the fuel profiles are and what the justification and logic for them are.

The Internet of criminal things

Posted Oct 7, 2015 20:49 UTC (Wed) by mathstuf (subscriber, #69389) [Link] (1 responses)

> In the case of VW, is the different fueling used because the throttle is constant and so it can satisfy the power requirements more efficiently?

Even if it were, the excess NOx makes the "on-the-road" profile invalid to choose in the US under any circumstances.

The Internet of criminal things

Posted Oct 7, 2015 21:01 UTC (Wed) by dlang (guest, #313) [Link]

> Even if it were, the excess NOx makes the "on-the-road" profile invalid to choose in the US under any circumstances.

That is where you misunderstand the regulations.

The requirements for the amount of NOx produced are under specific conditions. Under other conditions (full load climbing a hill), the vehicle is not expected to work the same way it does under light load.

This is what I was referring to when I talked about how unrealistic the tests are compared to real-world driving earlier.