Weekly Edition
Return to the Security page
| |||||||||||||
Deep packet inspection engine goes open source (ars technica)
Ars technica looks at a free software release of deep packet inspection (DPI) code from ipoque. At least part of the motivation for releasing the code is to allay fears that ipoque's DPI hardware is digging into the actual content, rather than the packet formats and timing, of encrypted traffic, but this release may not succeed in doing that:
"The OpenDPI engine, released under the LGPL license, differs from ipoque's commercial scanning engine in its high-priced DPI hardware. The open-source version is much slower and (more importantly) doesn't reveal ipoque's methods for identifying encrypted transmissions. DPI vendors all claim high levels of success at identifying such traffic based on the flow patterns and handshake signatures common to protocols like BitTorrent and Skype, even if they cannot crack the encryption and examine the content of those transmissions."
(Log in to post comments)
Deep packet inspection engine goes open source (ars technica) Posted Sep 9, 2009 14:48 UTC (Wed) by ajb (subscriber, #9694) [Link]
Even if it doesn't read the contents of the packets, why would we want our ISPs to discriminate based on protocol?
A more useful approach would seem to be the re-ecn system, currently attempting to enter the IETF standards process: (http://trac.tools.ietf.org/area/tsv/trac/wiki/re-ECN). This is intended to expose the minimum necessary for ISPs to cleanly arbitrate between users: information about congestion. Currently only endpoints can easily observe congestion (well, it's slightly more complicated than that, but I'm not sure I understand the details).
Deep packet inspection engine goes open source (ars technica) Posted Sep 9, 2009 19:28 UTC (Wed) by MattPerry (guest, #46341) [Link]
> Even if it doesn't read the contents of the packets, why would we want
> our ISPs to discriminate based on protocol?
What makes you think on ISPs will find this tool to be useful? I could see it being used to enforce company computing and network policies.
Deep packet inspection engine goes open source (ars technica) Posted Sep 9, 2009 19:43 UTC (Wed) by ajb (subscriber, #9694) [Link]
The whole DPI controversy is about its use in ISPs. No-one cares what corporates do on their own networks.
From IPOQUE's press release: "A general problem is the lack of transparency from the vendors' side, which makes Internet users afraid of this technology".
Deep packet inspection engine goes open source (ars technica) Posted Sep 9, 2009 19:52 UTC (Wed) by drag (subscriber, #31333) [Link]
The real solution to the ISP problem is to force ISPs to publish their network policies. At least it would be a first good step. Then customers can see what is really going on. Similar to how people are forced to publish the contents of food items sold in the stores.
DPI is just a tool, and a potentionally very useful one for lots of networks... even ISPs. It does not even mean throttle or blocking. Tools like Wireshark and Snort do similar things, and I am thinking (just now) that quite possibly this code could be used to improve those products!
Deet packet inspection engine goes open source (ars technica) Posted Sep 10, 2009 1:24 UTC (Thu) by ras (subscriber, #33059) [Link]
> Even if it doesn't read the contents of the packets, why would we
> want our ISPs to discriminate based on protocol?
Simple. ISP's have different classes of users. The vast majority are casual users who just do webby type things, but there are a few heavy users who doing P2P type stuff. But in the US there is a problem: both types of users get charged the same amount. Ergo, the US ISP's make their money off the webby users, and have to put up with the P2P users are have come along for the "all you can eat" ride. The US ISP's solution: when resources become tight keep the people you make money off happy by throttling the people who came along for the ride.
This is another solution: usage based charging. With usage based charging the P2P users are your highest spending users. This has a remarkable effect on the ISP's attitude to them. It literally does a 180 degree switch: P2P users going from "our worst customers" to "our highest spending".
How do I know? Because I live in Australia, and that is what happens here. Australian ISP's positively encourage you to use P2P. They literally throw away DCMA notices, and when taken to court about that by the Music/Movie companies fight it all the way to our highest court. When comparing their actions to their US (or indeed European) ISP cousins, the contrast could not be more stark.
This has an interesting side effect. There is no "network neutrality" debate here in Australia. Oddly enough, it turns out provided ISP's get paid for roughly the amount of service they provide they are perfectly happy to treat all bytes equally. It is simpler for them. No need for DPI gear, no need to tell lies about there being an unlimited account, no need to think of weasel words to put into contracts and Acceptable Usage Policies, no need to fire employees who mistakenly tell the truth. It is so easy when all you have to do is count the bytes and charge accordingly.
Deet packet inspection engine goes open source (ars technica) Posted Sep 10, 2009 2:16 UTC (Thu) by dlang (✭ supporter ✭, #313) [Link]
ISPs in the US used to charge more for more useage.
after some companies started offering unlimited usage accounts (for more money than others were charging for the limited usage accounts by the way) people started buying the unlimited accounts instead and over time just about all accounts are now unlimited.
so it's not a matter of people being reluctant to offer tiered accounts, it's that users are unwilling to put up with it.
similar things are happening in the cell phone space where per-min accounts are moving to unlimited time or unlimited rollover accounts (and per text SMS accounts are being replaced by unlimited texting accounts)
Deet packet inspection engine goes open source (ars technica) Posted Sep 10, 2009 2:49 UTC (Thu) by ras (subscriber, #33059) [Link]
Yes, I am aware the US ISP's would love to move to usage based charging. It would be better for everyone if they could find some way to do it. The net neutrality issue will disappear for a start.
In Australia, we got lucky. The US carriers had a monopoly in the early days, and so in the traditional US way they stung us with unbelievably high peering fees. The bastards. We tried the "we will pretend this is an unlimited account" thing for a while, but the limits had to be set so low it didn't work out. So now we are one of the few countries where sanity prevails.
I, like you, can't see any easy path to creating a saner charging model now. Please excuse my obvious gloating. I can't help but enjoy watching those same US carries flail around in the mess their "take no prisoners" approach to competition created.
Deet packet inspection engine goes open source (ars technica) Posted Sep 10, 2009 5:53 UTC (Thu) by TRS-80 (subscriber, #1804) [Link] No, the real reason Australia has usage-based plans is because of the cost of getting traffic from the US. Undersea cables cost a lot of money and this is reflected in Australian ISP costs. ISPs in the US just peer with other ISPs for free and maybe pay a little in actual transit to a Tier 1 or 2 network, since most of the content is located in the US.
Deet packet inspection engine goes open source (ars technica) Posted Sep 11, 2009 7:18 UTC (Fri) by Wol (guest, #4433) [Link]
I'm thinking of moving to an ISP that has a hybrid approach.
The standard account is both capped and unlimited - between 0600 and 2359 there are usage based limits for which you get charged if you go over. Between 0000 and 0559 there's no limit, but if it gets congested then tough.
Cheers,
Deet packet inspection engine goes open source (ars technica) Posted Sep 14, 2009 14:37 UTC (Mon) by nye (guest, #51576) [Link]
Most of the better, more expensive ISPs in the UK work this way[0]. I've been very happy with this model, especially in comparison to the cheaper 'unlimited' accounts which come with arbitrary throttling, blocking, shaping, and caps - all of which are kept secret so you can never know exactly what you're paying for, and whether you're getting it.
[0] When I say 'more' expensive, I'm excluding 'most' expensive - it's possible to get a truly unlimited account if you're willing to pay enough - like something which probably resembles my total salary.
Deet packet inspection engine goes open source (ars technica) Posted Sep 10, 2009 6:04 UTC (Thu) by dlang (✭ supporter ✭, #313) [Link]
instead of the net nutrality debate you have the great firewall debate
Deet packet inspection engine goes open source (ars technica) Posted Sep 10, 2009 6:14 UTC (Thu) by ras (subscriber, #33059) [Link]
> instead of the net nutrality debate you have the great firewall debate
Yeah, go on, just rub my nose in it.
Deep packet inspection engine goes open source (ars technica) Posted Sep 9, 2009 20:01 UTC (Wed) by busterb (subscriber, #560) [Link]
Company releases "Deep packet inspection" engine to convince people that it
doesn't actually look very deeply into packets. That's the truth: for many protocols, its just checking a few bits and a port number. So, why call it deep packet inspection? It doesn't even appear to reassemble TCP or understand IP fragmentation.
Deep packet inspection engine goes open source (ars technica) Posted Sep 9, 2009 23:29 UTC (Wed) by Los__D (guest, #15263) [Link] Deep Packet Inspection, not Deep Stream Inspection.
If it did TCP reassembly, it would be working on the streams.
Deep packet inspection engine goes open source (ars technica) Posted Sep 10, 2009 13:28 UTC (Thu) by busterb (subscriber, #560) [Link]
I know a number of vendors that tout deep packet inspection imply stream reassembly. I looked up
'deep stream inspection' on google and this comment is the second hit, which may be telling. Citrix seems to think the two things are separate, while SonicWall treats them the same, as the first page of hits shows.
In the case of IPS/IDS systems, this is a must, since masking malicious traffic by exploiting quirks in
It appears you could fool this 'DPI' system by breaking its assumption that the application headers
Deep packet inspection engine goes open source (ars technica) Posted Sep 10, 2009 13:38 UTC (Thu) by Los__D (guest, #15263) [Link]
Cool, I'm now officially a SEO master. ;)
I think we can agree that this release is pretty unusable for IDS, it is just a simple protocol detection utility.
The "deep" part is just that it looks at the data, not (only) the headers.
Deep packet inspection engine goes open source (ars technica) Posted Sep 10, 2009 5:38 UTC (Thu) by djm (subscriber, #11651) [Link]
From the code snippet posted to Ars, it looks like OpenDPI's detection engines for the protocols that it supports are written in C. Apart from being retro and poorly extensible, this approach begs for security problems. Consider the attack surface presented by parsers for _every protocol_ that it supports. An approach like Snort or Wireshark's, where inspectors are written in a custom packet parsing language, is safer and more flexible.
Deep packet inspection engine goes open source (ars technica) Posted Sep 10, 2009 5:43 UTC (Thu) by dlang (✭ supporter ✭, #313) [Link]
wireshark is just about the poster child for how _not_ to do parsers. the steady stream of vulnerabilities in their parsers is horrific.
Deep packet inspection engine goes open source (ars technica) Posted Sep 10, 2009 10:17 UTC (Thu) by nix (subscriber, #2304) [Link]
Yeah, but thanks to their privilege separation, most of these attacks no longer give root, at least. (All they give is, uh, access to the account running wireshark. Still not very good.)
Deep packet inspection engine goes open source (ars technica) Posted Sep 10, 2009 19:14 UTC (Thu) by dlang (✭ supporter ✭, #313) [Link]
and if you are running wireshark to actually sniff traffic (as opposed to replaying a capture made independently) you have to run it as root
Deep packet inspection engine goes open source (ars technica) Posted Sep 10, 2009 19:35 UTC (Thu) by samroberts (subscriber, #46749) [Link]
> and if you are running wireshark to actually sniff traffic (as opposed to replaying a capture made independently) you have to run it as root
You haven't run wireshark in a while, apparently:
% sudo tshark
Wireshark does not have to be run as root in order to live capture.
It implements the classic unix privilege separation strategy, a separate privileged executable that does nothing but capture packets and forward them to the non-privileged wireshark/tshark process which does the decoding.
Also, as you can see, it disables lua when running as root.
Deep packet inspection engine goes open source (ars technica) Posted Sep 11, 2009 16:27 UTC (Fri) by nix (subscriber, #2304) [Link]
That's not been true for a long time, as samroberts has pointed out.
The 'only' privilege escalation that can be carried out is to your regular user account. (This is hardly ideal. An extra layer of privsep, with protocol decoding being done by another binary, setuid to an account dedicated only to running wireshark decoders, seems best to me. But still it's better than the all-as-root-all-the-time approach Wireshark used to use.)
Deep packet inspection engine goes open source (ars technica) Posted Sep 10, 2009 19:28 UTC (Thu) by samroberts (subscriber, #46749) [Link]
Wireshark's inspectors are written in C.
It is possible to write them in Lua, but the ones you get when you download wireshark are not.
Deep packet inspection engine goes open source (ars technica) Posted Sep 17, 2009 7:53 UTC (Thu) by djm (subscriber, #11651) [Link]
oh, I stand corrected then :)
|
|||||||||||||