Don't give out ssh access

Posted Sep 1, 2011 9:16 UTC (Thu) by ebirdie (guest, #512)
In reply to: Don't give out ssh access by slashdot
Parent article: kernel.org compromised

>Well, it's probably not a good idea to give ssh access to 448 people.

Does it change much, if the intrusion was through one of the admins?

"a trojan was found on the personal machine of kernel developer H Peter Anvin"
The Register: Kernel.org Linux repository rooted in hack attack
http://www.theregister.co.uk/2011/08/31/linux_kernel_secu...

If I remember correctly, H Peter Anvin was, at least, an admin for kernel.org services.

That would give easy answer, how root access was possibly gained. I find two possibilities. First if sudo authentication was passwordless on utilities with low security and/or sudo authentication had shortcircuit to ssh-login, a single-sign-on, then the intruder did not need to use any vulnerabilities in software code but just found weak spots in methods to use root priveledges. Secondly, if the ssh-client binary were trojaned at H Peter Anvin's use, typed password for sudo were logged as well.

According to the above article the trojan logged pretty much information, which suggests the above might be, what have happened. Ie. the intruder has found from the logs, how to gain root access and rest is just keeping yourself hidden.

This scenario closes your claims, but does not exclude the possibility that root access has had been available through some vulnerability in some software somewhere in the system, but finding the vulnerability to use had put the intruder to higher exposure to be found sooner.

to post comments

Two-factor authentication

Posted Sep 1, 2011 15:01 UTC (Thu) by Cato (guest, #7643) [Link] (17 responses)

Protecting against a targetted attack on an administrator's system is tough, but it might have helped to require two-factor authentication for all access to the kernel.org servers. Something like Yubikey is not expensive ($25 a key), works with a range of open source software, and can make even a keylogger less useful.

The Fedora project seems to have switched to using this for its project infrastructure, so there is some precedent: http://www.yubico.com/fedora-uses-yubikey-for-strong-two-...

There are many other things that should be done (e.g. intrusion detection to discover injected trojans, ideally on client systems as well) but this is a simple action to take that helps protect against compromised client systems to some degree.

I have no connection with Yubikey, but I have just ordered a couple for use with the excellent LastPass, so I don't feel so paranoid when using its passwords on a Windows PC. Though perhaps I should enable it on Linux as well...

Two-factor authentication

Posted Sep 1, 2011 15:30 UTC (Thu) by yokem_55 (subscriber, #10498) [Link]

Google's 2-factor authentication (the android app for it is open source and straight forward) also can be used in an SSH context using the google_authenticator pam module. Sadly though the regular SSH key-authentication overrides the pam auth methods and so the two can't be combined.....

Two-factor authentication

Posted Sep 1, 2011 16:37 UTC (Thu) by endecotp (guest, #36428) [Link] (4 responses)

> Something like Yubikey is not expensive ($25 a key)

$25 * 448 = $11,200

Two-factor authentication

Posted Sep 1, 2011 17:31 UTC (Thu) by rgmoore (✭ supporter ✭, #75) [Link]

Now compare that $11,200 to the cleanup cost of having kernel.org hacked. Plus the opportunity cost of having the system down while you respond. And the reputational cost of having such a major system compromised. Good security only looks expensive until it's compared to the cost of insecurity.

Two-factor authentication

Posted Sep 1, 2011 17:32 UTC (Thu) by jonabbey (guest, #2736) [Link]

And cheap at the price.

Two-factor authentication

Posted Sep 1, 2011 21:51 UTC (Thu) by Cato (guest, #7643) [Link]

The cost would be a good motivation to have far fewer SSH logins on kernel.org. Non-SSH services wouldn't really need 2-factor authentication, as the risk of privilege escalation is much lower.

Two-factor authentication

Posted Sep 2, 2011 6:51 UTC (Fri) by job (guest, #670) [Link]

There are volume rebates, list price in their online shop is $6750 USD for 450. Put it in perspective, what are the costs of hosting and administering the machines? If that is still to steep I might be a good idea to contact them, I believe there are some free software enthusiasts associated with the company.

Two-factor authentication

Posted Sep 1, 2011 17:45 UTC (Thu) by skvidal (guest, #3094) [Link] (3 responses)

To be fair we've had some.... complications implementing yubikeys in Fedora Infrastructure.

The pam integration is... kinda clunky.
It is also vulnerable to reply attacks in some cases which is not fantastic, either.

personally I've found the google 2-factor auth easier to use.

Two-factor authentication

Posted Sep 2, 2011 7:12 UTC (Fri) by job (guest, #670) [Link] (1 responses)

Replay attacks? That's serious. I believe they are supposed to use some session identifier against that. Do you have more details?

Two-factor authentication

Posted Sep 3, 2011 11:00 UTC (Sat) by Cato (guest, #7643) [Link]

I found this from 2 years ago - replay attack due to a bug in the Yubico authentication server, since fixed by Yubico: http://www.grennan.com/?p=113

It's up to the authentication server to do the right checks, so perhaps some authentication servers have bugs.

I'd like to see more on this claimed replay attack, too.

Two-factor authentication

Posted Sep 10, 2011 6:51 UTC (Sat) by Cato (guest, #7643) [Link]

I think this is a reference to the fact that Yubikey's model is event-based one-time passwords, as with HOTP which it does support as an option. These comments include a response from the vendor explaining more and linking to a third party security analysis: http://www.mnxsolutions.com/security/secure-ssh-and-wordp... - the article talks about using Yubikey to secure SSH and WordPress logins.

Two-factor authentication

Posted Sep 1, 2011 20:15 UTC (Thu) by ebirdie (guest, #512) [Link] (6 responses)

In case of a trojaned ssh-client does any authentication really help?

The trojan may grap unlocked ssh-key from ssh-agent and grap the key password, if ssh-key is unlocked with -i switch.

In the case PAM and a two-factor authentication is used the trojan may quietly wait until all the connection authentication hoopla is done. The hoopla does not reveal any passwords, OTP or whatever to the attacker yet, but all the attacker needs through the trojaned ssh-client is an authenticated ssh-connection to the target system.

At the target system, while it has a spooky terminal connection via ssh to it, all the attacker need is to wait for terminal with root privileges. From there it is only couple seconds that the target system has rootkit. If all goes well, the rootkit does not crash the system, reveal itself and lets the attacker in.

One can't trust the terminal attached to the ssh-connection, no matter how well the connection was authenticated. Sudo at the target machine can't come to client machine to verify that the calling terminal weren't spooky.

Does this make sense?

Two-factor authentication

Posted Sep 1, 2011 23:16 UTC (Thu) by slashdot (guest, #22014) [Link] (5 responses)

Indeed, that method only foils a passive keylogger.

The only truly secure and somewhat practical method is to acquire a cheap low-end machine (preferably with hardware trusted execution path verification) which is exclusively used as a ssh terminal, and blocks all incoming traffic not related to an outgoing tcp connection.

Of course compromise of the server through the services it exposes is still possible.

Two-factor authentication

Posted Sep 2, 2011 7:59 UTC (Fri) by dugsong (guest, #79624) [Link] (3 responses)

Nothing's perfect, but a practical defense is to use your smartphone for out-of-band verification.

Disclaimer: Our company, Duo Security, provides this for free to open-source projects. Because it's the right thing to do!

As an aside, PAM and pubkey auth in OpenSSH are mutually exclusive. We work around this with a simple login_duo utility that doesn't require sshd restart, or even root access.

Two-factor authentication

Posted Sep 2, 2011 18:12 UTC (Fri) by slashdot (guest, #22014) [Link] (2 responses)

As far as I can tell ebirdie's objections apply to your solution as well.

In short, the issue is that a compromised client has full control of the connection after the authentication is done, regardless of whatever fancy mechanism you use to authenticate.

If you don't care about detection, it doesn't even require a compromised client: just software that detects authentication being successfully completed and simulates some keyboard/mouse input that gives the attacker full control of the server and shuts out the administrator from it.

Two-factor authentication

Posted Sep 2, 2011 23:12 UTC (Fri) by jonoberheide (guest, #71029) [Link]

Actually, our Duo Push authentication allows you to approve/deny individual transactions as you see fit, preventing the sort of session-riding attack that you're referring to.

For example, in the follow screenshot, you can see the exact command that an attacker is attempting to execute:

http://blog.duosecurity.com/wp-content/uploads/2011/04/pu...

Obviously, you would deny the attacker's attempted "rm -rf" here. ;-)

Regards,
Jon Oberheide

Two-factor authentication

Posted Sep 5, 2011 10:36 UTC (Mon) by sitaram (guest, #5959) [Link]

Disclaimer: I am the author and maintainer of gitolite.

If we make the assumption that all 448 users really do not need an actual *shell*, and that they will be mostly doing git push or putting files in some designated area using rsync, you can actually use gitolite to limit what they can do quite handily.

They don't get a shell, their access are limited to whatever repos they've been given access to, and even the rsync command can be access controlled using the same software, limiting users write access to specific directories only.

I've kinda lost track if they found the actual *escalation* vector involved but I'll bet it needed shell on the server.

Two-factor authentication

Posted Sep 2, 2011 8:22 UTC (Fri) by Cato (guest, #7643) [Link]

A lower cost way of having a "separate PC as SSH terminal" might be Qubes, which uses multiple VMs with differing levels of trust - could use one VM for general web browsing, one for shopping/banking, and one for SSH access.
https://lwn.net/Articles/385213/

The problem is that this requires significant work to set up for the owner of the client PC, and it's hard to mandate this setup. However, cutting the number of SSH users from 448 to a handful would make this more practical for kernel.org.