Security testing tools for Fedora

Red Hat's Steve Grubb recently unveiled a suite of lightweight Linux security-testing tools. Grubb's tools are not a post-installation system audit; rather, they help the distribution team find and catalog common problems with binary packages, scripts, and applications. Although he announced them on a Fedora project mailing list, they are not inherently distribution-specific, so other distributions may want to look into them and adapt them for their needs.

The security assessment tools

The announcement came in an August 3 email to the Fedora testers' list. Grubb described the collection as "for assessing security of the distribution. It is by no means a comprehensive auditing tool, but the scripts definitely find problems." The collection, which is hosted at Grubb's personal Red Hat page, is introduced with:

The page currently hosts thirteen utilities, all of them shell scripts. The first script's comment block describes it as being available under GPLv2. Most of the others simply mention "the GNU public license," although a few extremely short scripts (of one to three lines each) do not mention licensing at all. The longer scripts are commented with the purpose and usage instructions, although Grubb also includes a plain English description of each script on the main page.

The scripts are designed to iterate through the filesystem looking for specific security-setting, permissions, or behavioral problems that might not get caught by the normal QA process. Grubb said in his introductory email that he has found problems in Fedora 15 with all of the scripts.

The first two on the page tackle compatibility with particular policies. The first, stig-file-test.sh, is the longest, and tests for compatibility with the US Defense Information Systems Agency's (DISA's) Security Technical Implementation Guide (STIG) for Unix systems. The STIGs contain "technical guidance to 'lock down' information systems/software that might otherwise be vulnerable to a malicious computer attack," including basic guidelines on things like file and directory ownership and permissions. The script incorporates 35 individual tests for log, library, configuration, system command files, and miscellaneous directories. Each test includes a comment that references a section of the most recent Unix STIG.

The second script, rpm-chksec, tests for compatibility with Fedora's recommendations for ELF executables. It checks RPMs to see if the binaries it contains were compiled with the Position Independent Executables (PIE) and Relocation Read-Only (RELRO) compiler flags. It can either be run against a single RPM or against all packages installed on the target system.

The next five scripts each deal with detecting an individual risk factor with executables. The find-nodrop-groups and rpm-nodrop-groups scripts look for programs (installed programs in the former script; RPM packages in the latter) that use setgid without using either initgroups or setgroups. Without such safeguards, the program inherits all of the groups of the calling user, and can thus unintentionally run with elevated privileges.

The find-chroot and find-chroot-py scripts look for programs and Python scripts that call chroot but do not call chdir, and thus may have the current working directory outside the chroot sandbox. The find-execstack script looks for programs that have marked the stack as executable (which opens the door to an attacker using a stack buffer overflow to execute arbitrary code).

The find-hidden-exec script looks for executables that are "hidden" — in the sense that either their name or the name of one of their parent directories begins with a ".". That is not intrinsically a security problem (particularly on a production machine, post-installation), but from the distribution's perspective it is highly suspect for a fresh install and warrants investigation. The find-sh4errors script simply attempts to parse all shell scripts with sh's -n flag, noting any that fail. As Grubb put it on the mailing list, it is most likely to catch accidentally-broken scripts, but is still important.

There are two SELinux scripts, one that checks for devices in /dev that are incorrectly labeled, and one that checks the running processes to find daemons that do not have an SELinux policy. In both cases the results depend on the hardware and the daemons that happen to be in use on the test system, rather than reporting all possible problems in the distribution.

Finally, there are two scripts to check the behavior of programs with regard to temporary file creation. One checks all shell scripts to look for any that are using /tmp as a temporary storage location but have not called the mktemp utility. The other checks binary executables to find programs that are using /tmp but do not appear to be using a good pseudo-random name generator to create file names. Both of these scripts are called works-in-progress that may generate false positives, but they are helpful nonetheless. A program that uses /tmp but employs predictable file names may be vulnerable to attacks that replace the real temporary file with a malicious payload. There are also problems with privileged programs that use predictable file names that could be symbolically linked to another file by an attacker.

Fedora QA

Adam Williamson of the Fedora QA team replied to Grubb's post asking whether or not the tests could be integrated into the distribution's AutoQA test framework, which watches for new package builds (among other events) and executes tests against them. Grubb cautioned that not all of the scripts were robust enough yet to be relied on in automated tests, but that most were good enough for the general QA process.

AutoQA developer Kamil Paral expressed interest in adding the tests to the system, with a few changes. The scripts would need AutoQA wrappers to be written, in order to be managed by AutoQA's test "harnesses." The scripts would also need to be added to the AutoQA Git repository, as the infrastructure does not handle outside tests, and would need to be modified to report results via email to package maintainers, which is AutoQA's only notification method. As they are now, the scripts report the security faults they discover to stdout, most with color-coding to highlight the severity of problems found.

"That said," Paral concluded, "we would love to execute more tests for Fedora. But until the proper support is ready, it takes quite some effort. The first approach is go through the tests, select some appropriate ones and do that now." An alternative suggested by Paral would be to wait until AutoQA could be adapted to handle third-party tests, but there was no time frame for that feature enhancement.

In the meantime, Grubb asked Fedora testers to run the scripts manually, particularly the rpm-chksec script that looks for the recommended compiler flags. Fedora 16 has adopted a new policy with regard to RELRO, recommending that all packages be compiled with at least "partial" RELRO support, and important programs use RELRO in full.

For other distributions

Most other distributions have started moving towards RELRO and PIE in recent years, and although SELinux is still not the default in several major distributions, the majority of Grubb's other scripts could prove to be useful QA tools on non-Fedora systems as well. The chief obstacle is that many of them rely on the rpm command-line tool to inspect packages (and not just those that look for RPM package problems).

For example, the find-chroot and find-chroot-py scripts both use rpm to detect and then report which package owns any problematic file. When you execute it on a Debian-based system, it flags the packages but tacks on the semantically-valid-sounding (but incorrect) message "file FOO is not owned by any package." The same is true of find-sh4errors, find-execstack, and the /tmp usage scripts. Apt offers the same functionality, so adapting the rpm-dependent scripts to work in an Apt-based distribution would not be difficult.

Still, the other scripts are useful today even on Debian-based systems. I ran several of them on my normal Ubuntu desktop machine, and was surprised to find (for example) how many hidden executables there were tucked away. The majority were not actually ELF binaries or scripts of any kind — although there were a handful of oddly-named PHP files from some packages I have tested — instead they were backup data files, such as OS X resource files of raw photos from the last time I traveled with an Apple laptop and a camera. They were named .IMG_NNNN and were of no value — but they definitely did not need the execute bit set.

That example is still a good illustration of the value of Grubb's QA tools, however: the permissions were wrong, even if (as far as I could tell) the contents were not dangerous. From a security standpoint, finding seemingly-innocuous permissions or ownership problems is just as important for a distribution as verifying patches that fix known CVEs. Although no one would intentionally ship a broken shell script or choose a guessable temporary file name, their presence on the installer image still constitutes a risk to users.