Security Certification - The Open Source Way

"This very open approach is unusual for a certification," according to Klaus Weidner, senior IT security consultant for atsec, the German firm responsible for the evaluation. "The overall effort for another distribution is significantly lower if they re-use the material that has been released to the Open Source community from the evaluation of SLES 8," he said. The material that has been released includes a high-level design, a security guide, the security target, test plans, and the certification report. In addition, bugs found during the process have been fixed and the resulting patches fed back to the developers for inclusion in upcoming releases.

Common Criteria security certification consists of two elements: the "security target" (or "protection profile") that specifies the security features of the product to be evaluated and the "assurance level" which provides a level of confidence that the security functions perform as documented. For the EAL2 certification, the security target was created by IBM and SuSE. The evaluation process looked at SuSE's "configuration management, acceptance procedures and development security," Weidner said, and SuSE was "found to meet and exceed all requirements for this evaluation." A few bugs were found in the testing process, particularly in PAM authentication, and they were fixed and funneled back to the development community.

Looking forward, the evaluation and testing for an EAL3 certification is currently under way using the Controlled Access Protection Profile (PDF format) (CAPP), which is a standardized security target created by the NSA. CAPP is the target that was used by Microsoft to achieve an EAL4 certification for Windows 2000. These certifications are widely seen by companies and government agencies as a seal of approval for the security functions of a product.

The main areas that need work for the EAL3 certification are adding an auditing subsystem and documenting what Weidner called "security-relevant subsystem interfaces". As part of that process, any undocumented Linux system calls need to have man pages written for them; the resulting pages will, of course, be provided back to the Linux community. The audit subsystem has been completed and is undergoing tests, the kernel portion is based on the systrace patch along with a set of user-space utilities that were developed by IBM and SuSE. These too will be open source.

EAL4 certification (should IBM and SuSE take that step) will require even more documentation, including internal interfaces inside the kernel. "Kernel hackers may be happy with using the source code as a reference, but EAL4 requires a descriptive low-level design document," Weidner said. This effort would be huge and it is not known whether it will be done, but it would obviously serve as a great reference to kernel internals.

One of the bigger questions surrounding these certifications is what they really mean for the security of the system. Unfortunately, the answer seems to be: not much. Professor Jonathan Shapiro of Johns Hopkins University has an analysis of the Windows 2000 EAL4 certification and much of what he says can be applied to the EAL2 (and presumably upcoming EAL3) certification of SLES 8. In summary the CAPP (and the target used for EAL2) both define away most of the "real world" security problems that operating systems face. From the CAPP document:

which Shapiro translates into:

While CAPP is the "standard", it really does not provide requirements that would make a system secure from the biggest security threats that exist today. It seems somewhat unlikely that the cracker community is particularly well funded, but they certainly are hostile, clever, and persistent. Given the volume of exploits against the CAPP/EAL4 certified Windows 2000, it seems clear that certification is mostly a marketing bullet point to make purchasers more comfortable without actually providing a secure system.