Intel's zero-day problem

Posted May 3, 2017 20:07 UTC (Wed) by mikemol (guest, #83507)
Parent article: Intel's zero-day problem

> for people who've been setting up systems with internet-accessible out-of-band management capability

I know there are people who've put their management ports on public IPs.

But still. *Jeeze*. Don't Do That!

Your management interfaces belong on private ports (or VLANs, if you must) on private broadcast domains reachable *only* via authenticated, secured VPN.

*All* code has bugs, so you hide away access to sensitive execution environments behind as many security layers as you reasonably can, to mitigate the impact in the eventuality that that one of those layers is flawed.

Intel's zero-day problem

Posted May 3, 2017 20:56 UTC (Wed) by sionescu (subscriber, #59410) [Link] (22 responses)

If it's a corporate laptop and it's on an IPv6 network, the machine may be exposed.

Intel's zero-day problem

Posted May 3, 2017 22:55 UTC (Wed) by mikemol (guest, #83507) [Link] (21 responses)

It shouldn't be accessible to the open internet, however, even if it's on an IPv6-enabled network with a globally-routable IP; firewalls for IPv6 aren't difficult in the slightest; at their simplest, you permit outbound NEW,RELATED,ESTABLISHED, permit inbound RELATED,ESTABLISHED, and have a policy of DROP.

This was an inherent technical necessity with the masquerade-style NAT that's ubiquitous on IPv4, but it's by no means unique to it.

Intel's zero-day problem

Posted May 3, 2017 23:06 UTC (Wed) by raven667 (subscriber, #5198) [Link] (7 responses)

I would expand this point to make the general assertion that devices modern enough to have IPv6 stacks, esp. if running Linux or *BSD, are also modern enough to have built in packet filters as you describe, and so should be built with the capability of handling these kind of problem all by themselves. The problems that firewalls were originally designed for, by DEC back in the early IPv4 days, were a lot of deployed devices with no packet filters, non-random TCP initial sequence numbers, no SYN-cookies, no cryptographic authentication where TCPWrappers was a major innovation, that's not the world we have today, we have relatively more robust TCP, widely deployed cryptographic authentication/transport, built-in high quality packet filters and should raise the bar correspondingly to our current security capabilities.

Intel's zero-day problem

Posted May 3, 2017 23:15 UTC (Wed) by mikemol (guest, #83507) [Link] (2 responses)

No host-local firewall will protect you from your hardware siphoning off and diverting packets.

The firewall I described belongs on the network gateway. (In addition to the host.)

Intel's zero-day problem

Posted May 5, 2017 15:55 UTC (Fri) by raven667 (subscriber, #5198) [Link] (1 responses)

I was thinking local to the embedded host, in this case the OS running on the AMT CPU that is exposed to the Internet, not the OS running on the main CPU. If you don't control security policy on the BMC/AMT device then sure, you need some external device to enforce policy, but we shouldn't be building devices that don't allow the owner to control basic access and auth. The point I'm trying to get across is that whatever embedded software is used in the network thing, it probably is capable of supporting basic access and auth control on its own, if we choose to enable the feature and build the interface to control it. How hard would it really be to provide an interface for the device owner to put in a few stateless netfilter rules on the AMT/BMC, and use the same authentication protocols that defend our main systems from Internet-scale attackers.

Intel's zero-day problem

Posted May 9, 2017 5:30 UTC (Tue) by ringerc (subscriber, #3071) [Link]

Right. It's more like some appliance or IoT device than a real host, and should be treated with the same extreme dubiousness about its security and trustworthiness.

Intel's zero-day problem

Posted May 9, 2017 5:26 UTC (Tue) by ringerc (subscriber, #3071) [Link] (3 responses)

Actually, I think with the IoT "revolution" we're right back there again.

Pathetically insecure devices that never get security audits or updates. That have no user or admin configurability or documentation. Devices hacked together by a rushed team adapting a related product's code in a single 36-hour sprint to try to beat a competitor to market, before completely forgetting about it.

Devices where if you're lucky you might be able to update or fix them with a vendor-pushed remote firmware update (assuming an attacker doesn't abuse the inevitably insecure mechanism instead). Or with a soldering iron to access pads for USB-serial, RS232, or even JTAG. Occasionally you'll have a webui to upload a firmware blob. But your chances of actually fixing anything without the vendor supplying a canned update image are nearly nil.

The more that these seething piles of crap can be hidden from the rest of the Internet, the better.

Intel's zero-day problem

Posted May 10, 2017 16:17 UTC (Wed) by raven667 (subscriber, #5198) [Link]

> Actually, I think with the IoT "revolution" we're right back there again.
> Pathetically insecure devices that never get security audits or updates

I think you are right and its frustrating because unlike in the past, where the technology to secure devices wasn't built yet, people building IoT devices are taking tools which have access control, auth, etc. available and turning those features off, making by their action something reasonably secure, pathetic. In the same way that SSH raised the bar for default remote access/admin we need the same kind of effort to make the defaults for these kind of embedded devices reasonably decent. Basic netfilter rules and key-based auth using widely audited tools, could make these kinds of devices secure against the kind of wholesale attacks we are seeing.

Intel's zero-day problem

Posted May 12, 2017 6:37 UTC (Fri) by if.gnu.linux (guest, #88877) [Link] (1 responses)

> Devices hacked together by a rushed team adapting a related product's code in a single 36-hour sprint to try to beat a competitor to market, before completely forgetting about it.

I think that there are two other sides we have to consider.

The first one is we as consumers are 'guilty' for always demanding new products which are more speedy, have more RAM etc. and for replacing one year old devices with new ones.

The second one is softwares need more RAM, more GPU power etc. to run. I think software developers think that "Nowadays a device has 4 GB RAM so what is wrong if my application use 1 GB of that RAM."

Intel's zero-day problem

Posted May 12, 2017 11:16 UTC (Fri) by pizza (subscriber, #46) [Link]

> The second one is softwares need more RAM, more GPU power etc. to run. I think software developers think that "Nowadays a device has 4 GB RAM so what is wrong if my application use 1 GB of that RAM."

Note that it is often *cheaper* to put more RAM in the device than less.

Intel's zero-day problem

Posted May 4, 2017 6:38 UTC (Thu) by Gollum (guest, #25237) [Link] (4 responses)

The whole point of this is that the Operating System, whatever it is, DOES NOT SEE THE PACKETS.

The ME steals the packets off the NIC before the OS gets any opportunity to know about them. Which means that an OS based firewall will be oblivious to these connections. Otherwise, you could lock yourself out of your device by installing the firewall, and forgetting your creds. That would defeat the point of being able to manage even a powered off device (obviously, standby power is required).

Intel's zero-day problem

Posted May 4, 2017 10:51 UTC (Thu) by mikemol (guest, #83507) [Link] (3 responses)

Which is why you need said firewall *on your network gateway*, as I've already explained.

You're still at risk from your local "trusted" network, but then you always will be without things like isolated private VLANS or firewall rules applied at your switching layer and in your APs.

Intel's zero-day problem

Posted May 4, 2017 11:19 UTC (Thu) by Gollum (guest, #25237) [Link]

Yeah, I saw that after posting my comment.

Intel's zero-day problem

Posted May 4, 2017 15:10 UTC (Thu) by felixfix (subscriber, #242) [Link] (1 responses)

Assuming your firewall isn't similarly vulnerable....

Intel's zero-day problem

Posted May 4, 2017 20:35 UTC (Thu) by mikemol (guest, #83507) [Link]

> Assuming your firewall isn't similarly vulnerable....

Hence a component of my remark:

> as many security layers as you reasonably can,

Keywords being *reasonably* and *layers*.

If you can't trust your network gateway, you can't trust your network gateway. If you find it a reasonable precaution, you might further segment your network such that the device you don't trust has an additional firewalled router between your clients and the untrusted device.

I do exactly this in one of my networks; I have to deal with routers on my premises owned and managed by AT&T and Comcast for upstream ISPs. Any traffic coming in on those first passes through a network segment for unfiltered traffic before they reach a router I own and control. And even on the *filtered* side of that router, there are routing nodes with stateful firewalls that segment, e.g. office activity traffic from server activity, server activity from various device management interfaces, etc. In the eventuality of a laptop or desktop infection, I don't need the infected code having easy access to nodes that the laptop or desktop shouldn't already have access to.

It's honestly not a difficult architecture to maintain, though it'd be overkill for home users.

Intel's zero-day problem

Posted May 4, 2017 19:38 UTC (Thu) by JanC_ (guest, #34940) [Link] (7 responses)

That's all not really useful if you are on the road (or even at the employee's home) with that laptop; even if you are not on a public IP, you have no control over the local network, and it might have any number of affected devices on it that try to attack the laptop.

Intel's zero-day problem

Posted May 4, 2017 20:47 UTC (Thu) by mikemol (guest, #83507) [Link] (6 responses)

Sure, it's possible for an attacker to get a foothold on a vulnerable device and pivot inward. But that's not what I was cautioning against; I was cautioning against management interfaces being on the *open internet* which would necessarily be unfirewalled, publicly-routable networks.

For every countermeasure, it can be assumed there's a counter-countermeasure; even DPI firewalls all the way down at the network's layer 2 could be worked around, depend on how the NIC views locally-destined packets in the outbound buffer.

But the countermeasure that defeats the vast majority of attacks is simply having a firewall between your node and the Internet that blocks unsolicited inbound connections.

Intel's zero-day problem

Posted May 6, 2017 0:23 UTC (Sat) by dlang (guest, #313) [Link] (5 responses)

with this problem, if you use an Intel motherboard as your firewall, you are by definition putting it out on the open Internet.

It's unreasonable for Intel or anyone else to ship something like this that can be accessed on any network interface, underneath the OS.

I fully understand the value of network management interfaces, but they should be on a separate interface that cannot mix with the traffic on the 'normal' interfaces.

Intel's zero-day problem

Posted May 6, 2017 8:23 UTC (Sat) by nix (subscriber, #2304) [Link] (4 responses)

Of course, this is what Intel does with its server-class machines: there is a separate management LAN that only the BMC can talk to. This is negated by the fact that the BMC can *still* talk out of the other LANs as well -- but you can turn that feature off (if you trust the BMC enough to believe it will turn itself off when you do that).

(It's probably really useful in the things' intended habitat, in the datacentre, where all the management LANs from a bunch of machines can end up on a physically distinct subnet from the actual traffic. It's not such a useful feature for the machine sitting by my desk :) I can understand why machines intended to be used in non-datacentre environments don't spend the extra on separate hardware for another physical network interface that will almost never be used.)

Intel do at least make timely security updates for the BMC and ME on machines they made the motherboard and chipset for available on the open Internet, making them the only vendors I have ever had experience with who will actually provide this sort of thing for people paying less than $BIGBUCKS for a support contract. (Or... as timely as you can expect for software that can brick large numbers of very expensive machines if it goes wrong enough.)

Intel's zero-day problem

Posted May 10, 2017 7:24 UTC (Wed) by joib (subscriber, #8541) [Link] (3 responses)

AFAIK it's actually fairly common to not have a separate RJ45 port for the BMC, but rather the BMC is connected to one or more of the onboard RJ45 ports and shared with the host OS. The BMC does have it's own MAC though, and standard practice is to use a different VLAN for the management network.

E.g. Dell is one such biggish vendor that typically doesn't provide dedicated BMC ports.

So it's nice in that you don't need separate cables and switch ports for the management network, but I guess there's a security trade-off as well. Though I haven't heard of attacks that manage to cross to the "wrong" VLAN (which of course doesn't prove that such doesn't exist).

Intel's zero-day problem

Posted May 10, 2017 15:03 UTC (Wed) by nix (subscriber, #2304) [Link] (2 responses)

AFAIK it's actually fairly common to not have a separate RJ45 port for the BMC, but rather the BMC is connected to one or more of the onboard RJ45 ports and shared with the host OS. The BMC does have it's own MAC though, and standard practice is to use a different VLAN for the management network.

Oh indeed: I've only ever owned one machine with a dedicated BMC RJ45 out of several with BMCs. I'm not aware of any machines on which the BMC is *only* connected to a dedicated RJ45, though -- they all seem to be able to share NICs with the main CPU (and indeed are usually configured that way by default).

Intel's zero-day problem

Posted May 10, 2017 15:09 UTC (Wed) by pizza (subscriber, #46) [Link] (1 responses)

> I'm not aware of any machines on which the BMC is *only* connected to a dedicated RJ45, though -- they all seem to be able to share NICs with the main CPU (and indeed are usually configured that way by default).

I have several SuperMicro BigTwin and FatTwin 4-node boxes that have dedicated ports for each node's BMC.

Intel's zero-day problem

Posted May 12, 2017 12:44 UTC (Fri) by nix (subscriber, #2304) [Link]

They don't listen on any other RJ45 ports? Interesting to learn that such machines exist in the vast domains beyond my price range :) I suppose if you have multi-node boxes that sort of sharing might become actively confusing, which is probably why they moved to purely dedicated ports...

Intel's zero-day problem

Posted May 3, 2017 22:50 UTC (Wed) by raven667 (subscriber, #5198) [Link] (3 responses)

>> for people who've been setting up systems with internet-accessible out-of-band management capability
> I know there are people who've put their management ports on public IPs.
> But still. *Jeeze*. Don't Do That!
> Your management interfaces belong on private ports (or VLANs, if you must) on private broadcast domains reachable *only* via authenticated, secured VPN.

That's going in the wrong direction, we have a mobile devices, IoT, heading to a more connected future, these devices have to have the security tools builtin to protect themselves against the big bad internet without the crutch of VPNs and VLANs and NAT44. Anything running Linux should be modern enough to at least have a packet filter if not modern crypto, so we should demand simple robust security tools for these kinds of embedded long-lived devices, rather than just assuming that they are always on a "secure" network where there are never any malicious agents. If anything use IPv6 link-local addresses only for initial zero-touch provisioning, enroll key-pairs with your management systems, then you can drop the device anywhere on the internet with mutual authentication, safely.

Intel's zero-day problem

Posted May 3, 2017 23:01 UTC (Wed) by mikemol (guest, #83507) [Link]

It's going in every direction, not just one. When you're dealing with security, you deal with layers. You can't rely on just one layer to be secure, so you hedge and harden everywhere you can.

A host packet filter isn't going to help if your NIC is siphoning off packets for consumption by a management engine before it reaches the host network stack. You should certainly have such a filter, but you can't rely on it exclusively.

Intel's zero-day problem

Posted May 9, 2017 5:48 UTC (Tue) by ringerc (subscriber, #3071) [Link] (1 responses)

> these devices have to have the security tools builtin to protect themselves against the big bad internet without the crutch of VPNs and VLANs and NAT44

That'd be nice. But the opposite is happening. There are no economic incentives to make secure consumer routers, IoT devices, and appliances. There are huge incentives to save development costs, and entirely forego software support for the lifetime of the product. In the world of the 30 day warranty (oh, how glad I am for Australia's legally mandated 1 year minimum warrenty), security holes in your fridge aren't even on the radar.

These things need to be treated like the plague-bearing insecure and un-secureable monstrosities they are, until or unless the economic incentives applied by buyers to vendors change that.

This becomes a problem when their functionality requires them to:

* Accept input, or even new code to run, from "trusted" Internet hosts, which often mean "whatever updates.mycompany.com resolves to and replies on port 80"
* Use UPnP to accept inbound network requests from arbitrary Internet hosts
* Uncritically communicate with anything on the local network segment or anything directly routable and addressible from it

or, often as not, when their functionality doesn't require that, but they do it anyway.

I honestly won't be surprised if we start seeing household firewall products (both discrete and home-router-embedded) that proxy your IoT crap behind a secondary wifi network, filtering out the exploits and limiting their access.

The real worry is the ones that talk directly to public WiFi, to 3G/4G cellular networks, to hotel networks, etc, where there is no trust. You could make an Ethernet/wifi condom (proxy/filter) device for ones that use WiFi or Ethernet, but not really for cellular.

I have no idea how to help change the situation to create meaningful incentives for secure device development and deployment. But without it, the whole Internet's going to be in trouble.

Intel's zero-day problem

Posted May 11, 2017 10:52 UTC (Thu) by Wol (subscriber, #4433) [Link]

Maybe brickbot will give manufacturers that incentive.

For those who haven't heard, a ?white?hat wrote a worm. It targeted several common IoT devices, and did NOT try very hard to break in. When it did, it tried to switch on as many security features as it could before self-destructing.

If it *couldn't* secure the device, it overwrote the firmware with /dev/zero and rebooted...

Apparently it's made a massive dent in the current round of DDos attacks :-)

Cheers,
Wol