Security Certification - The Open Source Way [LWN.net]

November 12, 2003

This article was contributed by Jake Edge.

An open approach was used in the first ever security certification for Linux, as befits the open source nature of the operating system. IBM and SuSE teamed up to certify SuSE Linux Enterprise Server 8 (SLES 8) on IBM eServer xSeries hardware and achieved Common Criteria Evaluation Assurance Level 2+ (EAL2+) in July. Much of the documentation that was done to accomplish this is available from the SuSE and IBM Linux Technology Center web sites.

"This very open approach is unusual for a certification," according to Klaus Weidner, senior IT security consultant for atsec, the German firm responsible for the evaluation. "The overall effort for another distribution is significantly lower if they re-use the material that has been released to the Open Source community from the evaluation of SLES 8," he said. The material that has been released includes a high-level design, a security guide, the security target, test plans, and the certification report. In addition, bugs found during the process have been fixed and the resulting patches fed back to the developers for inclusion in upcoming releases.

Common Criteria security certification consists of two elements: the "security target" (or "protection profile") that specifies the security features of the product to be evaluated and the "assurance level" which provides a level of confidence that the security functions perform as documented. For the EAL2 certification, the security target was created by IBM and SuSE. The evaluation process looked at SuSE's "configuration management, acceptance procedures and development security," Weidner said, and SuSE was "found to meet and exceed all requirements for this evaluation." A few bugs were found in the testing process, particularly in PAM authentication, and they were fixed and funneled back to the development community.

Looking forward, the evaluation and testing for an EAL3 certification is currently under way using the Controlled Access Protection Profile (PDF format) (CAPP), which is a standardized security target created by the NSA. CAPP is the target that was used by Microsoft to achieve an EAL4 certification for Windows 2000. These certifications are widely seen by companies and government agencies as a seal of approval for the security functions of a product.

The main areas that need work for the EAL3 certification are adding an auditing subsystem and documenting what Weidner called "security-relevant subsystem interfaces". As part of that process, any undocumented Linux system calls need to have man pages written for them; the resulting pages will, of course, be provided back to the Linux community. The audit subsystem has been completed and is undergoing tests, the kernel portion is based on the systrace patch along with a set of user-space utilities that were developed by IBM and SuSE. These too will be open source.

EAL4 certification (should IBM and SuSE take that step) will require even more documentation, including internal interfaces inside the kernel. "Kernel hackers may be happy with using the source code as a reference, but EAL4 requires a descriptive low-level design document," Weidner said. This effort would be huge and it is not known whether it will be done, but it would obviously serve as a great reference to kernel internals.

One of the bigger questions surrounding these certifications is what they really mean for the security of the system. Unfortunately, the answer seems to be: not much. Professor Jonathan Shapiro of Johns Hopkins University has an analysis of the Windows 2000 EAL4 certification and much of what he says can be applied to the EAL2 (and presumably upcoming EAL3) certification of SLES 8. In summary the CAPP (and the target used for EAL2) both define away most of the "real world" security problems that operating systems face. From the CAPP document:

The CAPP provides for a level of protection which is appropriate for an assumed non-hostile and well-managed user community requiring protection against threats of inadvertent or casual attempts to breach the system security. The profile is not intended to be applicable to circumstances in which protection is required against determined attempts by hostile and well funded attackers to breach system security.

which Shapiro translates into:

Don't hook this to the internet, don't run email, don't install software unless you can 100% trust the developer, and if anybody who works for you turns out to be out to get you you are toast.

While CAPP is the "standard", it really does not provide requirements that would make a system secure from the biggest security threats that exist today. It seems somewhat unlikely that the cracker community is particularly well funded, but they certainly are hostile, clever, and persistent. Given the volume of exploits against the CAPP/EAL4 certified Windows 2000, it seems clear that certification is mostly a marketing bullet point to make purchasers more comfortable without actually providing a secure system.

(Log in to post comments)

Security Certification - Does It really mean not much?

Posted Nov 13, 2003 9:33 UTC (Thu) by noxxi (subscriber, #4994) [Link]

working for a company who did ITSec certification (german security
certification) a few years ago and does it again I must say that
the certification process itself can make a product a lot more secure.

For ITSec Level 3 you have (among other things) to:
- document all tools and interfaces
- document the design and not only document it, but
also explain all the security relevant decisions and
show potential weaknesses
- provide a lot of tests for *fully* testing the certified functionality

If you've done this you
- rethought a lot of design decisions (because you need to
explain it to someone who knows, not some marketing guy)
- have a test suite which you can rerun again and again while
you continue to develop the product, thus making sure that none
of the important stuff breaks

This way you don't get a 100% secure product, but the product is
way better than before.

Security Certification - Does It really mean not much?

Posted Nov 13, 2003 10:53 UTC (Thu) by dd9jn (subscriber, #4459) [Link]

We could start a long thread on certification but well, I have to do other things. So just one hint: Get Peter Gutman's book and read the relevant chapters to see how useless certification of software is. A few of the other chapters are online.

Werner

Security Certification - Does It really mean not much?

Posted Nov 13, 2003 16:51 UTC (Thu) by kweidner (subscriber, #6483) [Link]

I know, you're not supposed to feed the trolls...

Chapter four (PDF) in Peter Gutman's book is about software verification techniques and criticizes the Orange Book (the CC predecessor) formal system verification methodology, suggesting a cryptography-based approach instead.

That is rather beside the point here, because formal design specifications and verification are only done at high assurance levels (>= EAL6), involving a complexity and amount of work that are far beyond what would be reasonable for a general-purpose operating system.

If you disagree, please let me know where I can get an operating system that is based on a cryptographic security architecture that could replace a Linux server in real-world use. Similarily, Shapiro claims in his article that the capability-based EROS-OS will be secure enough for EAL7 verification. It's an intriguing concept, but then again EROS-OS also looked very intriguing when I first heard of it a couple of years ago, and I'm not holding my breath waiting for it to be useful. If you want a working capability-based OS, take a look at OS/400, which is an elegant system but not famous for being user-friendly.

I'm not saying that the research Shapiro and Gutman are doing isn't interesting and potentially valuable, but in the meantime there are people who need to get real work done and want to use what is actually available and works.