Automating hinting for every script

At TypeCon 2016 in Seattle, Dave Crossland presented a look at the recent advancements in the free-software utility ttfautohint, which has rapidly grown from being a stop-gap measure with plenty of critics into a core part of many font-development workflows—including commercial foundries. The program automatically adds hinting information to fonts by analyzing the shapes of the included glyphs, alleviating the need to add hints manually. Challenges still remain for ttfautohint, though, including expanding the set of writing systems it can handle.

Crossland began with a disclaimer that, although he is known largely for his work with Google Fonts in recent years, ttfautohint is a distinct side project. That said, one of the early challenges faced when building the Google Fonts library was that font hinting is a notoriously complicated, labor-intensive process. The library was growing by more than a font every week in 2011, which provides considerably less time than a typical type foundry would have allotted for tasks like hinting.

So the team needed a different solution. Hinting is vital for reading text on screen, and particularly on Windows machines (which, unlike other platforms, never adopted a text-rendering model that would produce eye-pleasing results without relying on hinted fonts). The essence of hinting is the addition of embedded instructions to be read by the font renderer; instructions that keep similar letters the same size (in Latin alphabets, such alignment efforts keep lower-case letters aligned to the x-height, capitals to the capital height, and so on) and that keep glyph strokes aligned with pixel boundaries. PostScript fonts and their successors, Compact Font Format (CFF)-flavored OpenType fonts, rarely use any hinting because the PostScript rendering model takes care of such alignments internally. But TrueType fonts rely on hints that are embedded in the font and use a particular instruction set for the TrueType virtual machine.

But, many years ago, FreeType developer David Turner had developed a workaround for making those fonts without hints look good: he wrote an "autofit" module for FreeType that used many of the same concepts from the CFF renderer and applied them to TrueType fonts. It was developed, initially, because FreeType had been threatened over patent-infringement claims about the use of built-in TrueType hints. Autofit let FreeType make TrueType fonts look good on screen, essentially, through its own means.

The patents in question eventually expired, but the autofit workaround gave Crossland an idea: why not take the instructions generated and turn them into actual TrueType hints that could be merged into the font file? In 2011, he posed the question to then FreeType maintainer Werner Lemberg, who took the project on. In a few months there was a working prototype called ttfautohint.

Awkward fit

Although the "Inception-style" approach used by ttfautohint (using CFF rendering analysis of a font to inject TrueType rendering hints that will then be used by a TrueType renderer) seemed like it might be workable, Crossland said, there were some practical problems. First, hinting is usually done as an interactive process: the engineer adds hints, examines the rendered result immediately, then adjusts them. But ttfautohint was a command-line-only program with no interactive features. Second, hinting is ideally done during the design process, starting with a few core glyphs, then branching out as the font grows. But ttfautohint was a one-and-done program run only on a completed font file.

So the project set up a crowdfunding campaign to fund Lemberg's time, specifically to add a GUI and to further instrument ttfautohint with the controls needed to fine-tune the results. Other new features were targeted in the campaign, too, including hinting families of related fonts and supporting scripts other than Latin. In 2012, a few months after that campaign ended successfully, Google Fonts began using ttfautohint on fonts in its library.

But even then, the reaction from other font projects was not entirely enthusiastic. Göran Söderström famously criticized ttfautohint's output in a 2012 blog post [Wayback link], calling some of the results borderline unreadable and saying "I can’t really see the point of using ttfautohint at all." He also made a point of lamenting the fact that the hints generated by ttfautohint cannot be edited or adjusted; they are simply generated and inserted into the font file.

Work, however, continued. In 2012, Frederik Berlaen of RoboFont designed a Python-based GUI that would fit in with Apple's Mac OS X user-interface guidelines, although Crossland noted that the GUI's complexity has continued to grow and may have "gotten a little bit out of control." Other contributions began to come in from the developers of various font libraries and editors. In 2013, he noted, the project ran into a nasty bug encountered on certain laser printers, where rounding errors in the printer were causing wildly distorted results. The ttfautohint team released a fix.

Stability and extension

In 2014, ttfautohint version 1.0 was released. That version added support for a control file, which designers could use to tweak hints produced by the program—thus answering one of Söderström's criticisms. By the end of 2014, ttfautohint was in use at the commercial foundry ProductionType and was included with the proprietary Glyphs font editor.

In 2015, Lemberg began to look at how to support additional writing systems. He started by taking the Noto fonts by Google, which are open source and were extensively hand-hinted, stripping out all of the hinting, then working to reproduce the original results with ttfautohint.

There are now 22 scripts supported by ttfautohint. Crossland referred the audience to the online manual for a better description of how the hinting process works. In a nutshell, ttfautohint first analyzes each font file to automatically determine the width of vertical and horizontal strokes. It does this by examining characters that resemble the Latin letter "o" (there are similarly round characters in most, although not all, writing systems). Each glyph in the font is then examined to locate the stems and each stroke segment that can be adjusted as a single unit, using an approach described in a 2003 paper [PDF] by Turner and Lemberg.

Next, ttfautohint identifies what are called "blue zones" (after terminology used in PostScript), which roughly correspond to the vertical alignment areas of the script—in Latin, as mentioned above, those are the baseline, the x-height, the capital height, and so on. Naturally, each writing system has its own intrinsic set of these blue zones; ttfautohint looks at specific character codes in each script to assess where the blue zones of the font are.

Subsequently, ttfautohint will use this information to make sure that hinting adjustments do not push glyph components out of the zones, in theory keeping the entire font well-aligned. Sets of hints are generated to align edges to the blue zones, then to align components to pixel boundaries, then to align serifs. For now, all of ttfautohint's hinting is done only to change vertical alignments, because altering horizontal alignments usually has the unwanted side effect of changing the number of lines of text.

In practice, as one might expect, there is more to the process than this: there are many special cases, and matters get more complicated once composite glyphs (like accented characters) are added to the mix. Adding support for a new writing system, in most cases, begins with the question of identifying the key characters for ttfautohint to examine and the blue zones that are important to the writing system—followed by a great deal of testing.

Acceptance among designers has increased in recent years, too, Crossland said. In March, Ramiro Espinoza announced on the TypeDrawers discussion forum that he found ttfautohint's results to now be comparable to those from Microsoft's interactive hinting program Visual TrueType—and, in some cases, ttfautohint's to be better. Jasper de Waard discovered ttfautohint and used it to build a large font family called Proza Libre, a process he wrote about online.

Wide world of scripts

Looking forward, Crossland identified two areas where ttfautohint is needing to improve next. The first is it support for hinting font families. Although a family with a lot of weights will hopefully look harmonious when the weights are used together in a page, a bolder glyph is almost always taller than a lightweight version of the same glyph, because the strokes are thicker. Or, at least, that is what happens with lowercase glyphs; capitals tend to stay the same height for a given point size.

This presents a challenge to ttfautohint's analysis, which tries to align all of the glyphs to a single height. The approach currently being explored is to provide separate controls for uppercase and lowercase blue zones.

The other big issue is adding support for Chinese, Japanese and Korean (CJK) fonts, which do not follow the patterns used by the writing systems currently supported in ttfautohint. Thus, new algorithms will need to added. Crossland noted that there is some existing work in this area already, specifically Belleve Invis's sfdhanautohint.

There will, no doubt, be additional challenges in the years to come. But ttfautohint has already come a long way from its genesis as a workaround, so it may not be out of tricks just yet.

Comments (2 posted)

Backports and long-term stable kernels

Anybody wanting to use the kernel in a production setting tends to be torn between two conflicting needs. On one hand, the desire for stability and a lack of surprises argues for the use of an older kernel that has been maintained under a fixes-only policy for some time. But such kernels tend to lack features and hardware support; one needs to run a recent kernel for those. The answer that results from that conflict is backporting: pick a long-term support (LTS) stable kernel as a base, then port back the few things from newer kernels that one simply cannot do without. With luck, this process will produce a kernel that is the best of both worlds. In practice, the results are rather more mixed.

The problems with backporting are numerous. It will never be possible to pick out all of the important patches to port, so these kernels will always lack important fixes, some of which could leave open severe security holes. The mainline kernel benefits from widespread testing, but a backported kernel is a unique beast that certainly contains bugs introduced by the backporting process itself. The effort that goes into the creation of backport-heavy kernels is unavailable for the task of getting vendor changes upstream, with costs to both the vendor and the community as a whole. Users of highly divergent kernels are dependent on their vendor for support and updates; the community lacks the ability to help them. And so on.

Backports in the embedded industry

Alex Shi started the discussion with a mention of Linaro's LSK backporting effort and asking if there were ways that groups doing this sort of backporting could collaborate. The development community wasn't much interested in discussing backporting collaboration, though; the conversation turned quickly to the value of backporting efforts in general instead. Sasha Levin got there first with a statement that "what LSK does is just crazy" and suggesting that, if vendors want the latest features and fixes, the best way to get them is to run mainline kernels. He was not alone in this opinion.

James Bottomley pointed out that the backporting happening in the embedded industry looks a lot like what the enterprise vendors did in the 2.4 kernel era. They ended up with patch sets that were, in some cases, larger than the kernel itself and were a nightmare to maintain. To get away from these issues, the kernel's development model was changed in 2.6 and the distributors focused on getting features upstream prior to shipping them. That has greatly reduced the load of patches they have to carry, allowed them to run newer kernels, and reduced fragmentation in the kernel community. Why, he asked, can't embedded vendors do the same?

From the discussion, it would seem that, while there are many reasons cited for shipping backported kernels, there is one overwhelming issue that keeps vendors stuck on that model: out-of-tree code. A typical kernel found in an embedded or consumer electronics setting has a vast pile of patches applied, the bulk of which have never made their way into a mainline kernel. Every time that a vendor moves to a new base kernel, this out-of-tree code, perhaps millions of lines of it, must be forward-ported. That is a huge effort with risks of its own. It is unsurprising that vendors will tend to delay doing that work as long as possible; if an older kernel can support a new device through the addition of a few more backported drivers, that is what they will do.

The longstanding "upstream first" policy says that these vendors should have pushed their code into the mainline before shipping it in their devices; then they would have no forward-porting issues when moving to a newer kernel. But "upstream first" has never been the rule in this part of the market. These products are designed, built, shipped, and obsoleted on an accelerated schedule; there is no space in that schedule for the process of getting code upstream, even if the process goes quickly — which is not always the case. Upstreaming after shipping can be done, but the vendor's attention has probably moved on to the next product and the business case for getting the code upstream is not always clear. Five or ten years in, when vendors find themselves struggling under millions of lines of out-of-tree code, they might have cause to wish they had worked more with the development community, but the people making these decisions rarely look that far ahead.

There was some talk about how the out-of-tree code problem could be addressed, but with few new solutions. As Linus Walleij noted, the only reliable solution seems to be customers demanding upstream support from their suppliers. He suggested that if Google would ever make such a requirement for its Nexus Android devices, "then things will happen". Until then, the best that can be done is to continue to talk to and pressure companies and help them to change slowly. Some of this pressure could yet take the form of changes in how stable kernels are managed.

How stable kernels fit in

While much of the conversation talked about the evils of backporting, another branch was focused on the role that stable kernels play in that ecosystem. Vendors naturally gravitate toward kernels with long-term support, including LSK, the Long-Term Support Initiative (LTSI), or the mainline LTS kernels, as the base for their backports, though, as it turns out, they don't use those kernels as their maintainers might wish.

As Tim Bird described, the kernels shipped in devices are often subject to more than one level of backporting. The system-on-chip (SoC) within the device will have been provided with a kernel containing plenty of backported code, but then the integrator who is building a product from that system will have another set of patches to add. The value of initiatives like LSK and LTSI is that they have reduced the number of kernel versions being provided by SoC vendors, making life easier for those doing backports at the integrator level. Projects like LTSI also list upstreaming of vendor code among their goals, and some of that has been done, but their most important role seems to be to serve as a common base for vendor-specific kernel forks.

There was a certain amount of unhappiness with how these long-term-supported kernels are used, though. An LTS kernel like 4.4 will be supported for at least two years; the LSK and LTSI kernels, based on LTS kernels, will have a similar support period. But SoC vendors are not actually making use of that support. Instead, they grab whatever version of the kernel is available at the time and simply stick with it going forward, ignoring any further updates to that kernel. Should a fix land in the kernel they started from that cannot be done without (a highly publicized security fix, for example), the vendors will, naturally, backport it. Product vendors then take a snapshot of the SoC vendor's kernel and ignore any updates from the SoC vendor in a similar manner. This pattern has led developers like Ted Ts'o to question the value of the entire stable-kernel process and suggest, once again, that vendors would be better off just using mainline kernels:

Or, he said, SoC vendors could just start with a mainline release and pick their patches from subsequent releases rather than from the LTS kernel.

Time for a change?

Greg Kroah-Hartman, the maintainer of the long-term support kernels, agreed with this assessment of the situation, noting that even serious security fixes don't find their way through to the kernels shipped by vendors despite being promptly included in the LTS kernels. So he is mulling the idea of stopping the the maintenance of the LTS kernels entirely:

Getting to the point where companies might actually see the wisdom of that approach will take some time, he acknowledged, and there will be a great deal of education required. But, he said, he has been talking to people at some vendors in the hope of improving the situation. He closed by saying there might not be a long-term support kernel next year, since it wouldn't be needed. Or, at least, "one has to dream".

In this context, it's interesting to look at this recent post from Mel Gorman, which talks about the problem of performance regressions in newer kernels. The performance hit caused by moving to a newer kernel can often be surprisingly large. It can also be difficult to fix, since it is usually the result of many patches adding a 0.1% cost rather than one or two big mistakes. The work required to get that performance back is significant, and it helps him to understand why vendors in general might be reluctant to move to newer kernels:

If embedded-systems vendors were to come to a similar conclusion, the result could be significant changes in how that part of the market works. The benefits could be huge. The upstream kernel would, hopefully, gain the best of the work that those vendors are carrying out of tree for now; the rest would be replaced with more general solutions that would better serve all users. Kernels shipped with devices would have more features and fewer bugs while being more secure than what is shipped now. It might actually become possible to run mainline kernels on these devices, opening up a range of possibilities from third-party support to simply enabling hobbyist developers to do interesting hacking on them. The considerable energy that goes into backporting now could be directed toward testing and improving the mainline kernel. And so on.

All of this seems like a distant dream at the moment, but our community has made a number of dreams come true over the years. It has also been quite successful in convincing companies that working with the community is the best approach for long-term success with Linux. Perhaps we are getting closer to a point where embedded-systems vendors will be willing to rethink their approach to the Linux kernel and find ways to work more closely with the development community that, in the end, they depend on to produce the software they ship. One does indeed have to dream.

Comments (25 posted)

Filesystem images and unprivileged containers

At the 2016 Linux Security Summit, James Bottomley presented some problems that users of unprivileged containers have encountered with regard to the user and group IDs (UIDs and GIDs) stored in filesystem images. Because of the way that user namespaces remap these IDs, there is a need for privileges in the initial namespace to access some files, which is exactly what unprivileged containers are trying to avoid. Bottomley also described some in-progress potential solutions to the problem.

He began by noting that his experience in container technology predates Docker. His interest lies in the kernel's primitive interfaces for containers, rather than the higher-level view of containers that Docker and other container orchestration tools have. Every orchestration tool uses those same kernel interfaces, so any work that is done on the kernel API is automatically made available to them—as long as they turn the feature on, that is.

One of the advantages of the kernel API is that it provides granular virtualization options so that container implementations can pick and choose. Container orchestration systems can enable the virtualization of various kinds of resources via namespaces—or not. That is what allows Docker to not use user namespaces for its containers, since it is not forced to do so, he said.

On the other hand, that granularity makes for an infinite variety of configurations for containers. The Open Container Initiative (OCI) standardization effort does not address this problem at all, Bottomley said. It is concerned with the packaging format for containers and says nothing about the kernel and container configuration. But there is also a subset of those configurations (which is also infinite) that are "completely insecure from the get-go".

Control groups (cgroups) and namespaces are the building blocks of containers, but he noted that most of the interesting development for containers—particularly security-related work—in the kernel right now is happening in the namespace code.

One of the more important namespaces for containers is the mount namespace, which allows containers to have their own view of the mounted filesystems. It is somewhat complex to set up securely, however, he said. The problems for unprivileged containers are primarily found in the mount namespace.

The process ID (PID) namespace is used by many container managers, but Bottomley said he has never quite understood why. There are both system containers, which are those that are meant to contain a full running system, and application containers (like Docker) that exist only to run a single application. The PID namespace exists partly so that there can be multiple init processes all running as PID 1 in separate system containers. The main advantage of having application containers in their own PID namespace is that it virtualizes the PIDs in the system so that containers cannot see the processes in other containers.

User namespaces

User namespaces were dissed in an earlier talk, he said, but he has a different take. "Instead of telling you why you should fear user namespaces, I'd like to tell you why you should love user namespaces."

The kernel API for containers is often described as "completely toxic", Bottomley said. Docker will proudly proclaim that the interface is too hard to be used, which is why everyone should use Docker. But unprivileged containers, which are containers that have been set up without relying on an orchestration system, also provide the "backbone of all security in container subsystems".

As the name implies, unprivileged containers are those that don't have a privileged root user. But that means different things to different people. The idea is to have a container where there is a root user ID inside it, but that ID maps to the unprivileged "nobody" user outside of the container. One way to do that is to have a container that doesn't map the root user (UID 0) at all, which is something that "a lot of people should do, but don't", he said with a chuckle. But some applications may need some "root-y privileges", so there needs to be a UID 0 inside the container that has some amount of elevated privileges with respect to the other UIDs in the container.

User namespaces can be used to implement both cases, but the "contentious bit" is having a root user with some privileges inside what is supposed to be an unprivileged container. In the ideal case, root inside the container would not have any privileges outside of it, but there are lots of actions that require privileges—including setting up namespaces. Many of the container primitives in Linux (e.g. unshare()) need root privileges.

The current state (as of Linux 4.8-rc3) is that user namespaces work well for unprivileged containers. But "cgroups don't work at all" for them. Thus, his talk is all about namespaces because he can't get cgroups to work for him.

Effectively, user namespaces give enhanced privileges to a user. Any time there is a mechanism to upgrade user privileges, it has the potential to be a security problem. But, he said, user namespaces do allow giving users enhanced privileges such that they believe they are root in a container, though they cannot damage the rest of the system using those privileges. That is the ideal that is being sought.

The allegation that he has heard is that "we are not there yet", but he disagrees. The IBM Bluemix container cloud is running in a bare-metal environment that employs user namespaces as the sole mechanism to protect the host from the guests, Bottomley said. The company is demonstrating that user namespaces are sufficient for security separation in a publicly accessible cloud. It has effectively bet its cloud business on user namespaces.

At its core, a user namespace is a mapping between IDs inside and outside the namespace. It is controlled by /proc files (uid_map and gid_map) that describe ranges of UIDs and GIDs to map from and to. There is also a projid_map file that is for group quotas, which can largely be ignored since it is only available for XFS, though ext4 support is coming. Finally, there is a setgroups file that can be used to deny processes in the namespace the ability to drop groups, which could actually grant privileges in some (believed to be fairly uncommon) configurations.

The user that creates a user namespace only has privileges to map their own UID into the new namespace. There are privileged utilities (newuidmap and newgidmap) that will allow additional mappings. The UID that creates the namespace is considered the owner of that namespace, so only it and root can actually enter the namespace (using nsenter()). In addition, unmapped UIDs are inaccessible even to root inside the namespace.

Filesystems

The kernel maps between the uid_t that represents the UID in the namespace to a kuid_t that is the real UID. For filesystems mounted in a namespace, that mapping is still done. So a container filesystem image gets handled with the real kuid_t values that have been stored as a file's owner and group.

So, if you try to run a standard Docker image in an unprivileged container, "it will fall over" because it has the wrong UIDs. Filesystem images can be shifted to have UIDs that get mapped into the container, but managing multiple shifted filesystem images is problematic.

What is really wanted is that IDs would be changed to their real counterparts within the kernel (so that any escape from the container would not be done using the container-specific UIDs), but that accesses to the filesystem would be translated back to the container UIDs—but only for filesystems mounted within the container. This is an unsolved problem in Linux right now, and one that is currently being worked on.

An old solution is bindfs, which is a FUSE filesystem that remounts a subtree with UID and GID changes. It requires root to do the mount. One problem is that the mappings are done one by one on the FUSE command line, so handling a thousand remappings is problematic. That is a solvable problem, but container developers are leery of depending on FUSE because of performance concerns.

Two other solutions were proposed for 4.6: Bottomley's shiftfs and portable root filesystems from Djalal Harouni. Shiftfs is effectively an in-kernel bindfs that uses ID ranges to avoid that problem in bindfs. It also requires root to set up a remapped subtree, which can then be bind mounted into the container. It "works reasonably well", he said.

Portable root filesystems allow any mounted filesystem to be marked as "shiftable", which means that it can be bind mounted into a container using the existing user namespace remapping. It requires changes to the VFS and underlying filesystems to add the mark, however. Both shiftfs and portable root filesystems allow bind mounting a subtree into a container, which solves most of the problem.

In addition, Seth Forshee is working on unprivileged FUSE mounting, which is part of what has held up either of the other two solutions from getting merged—beyond the fact that no one is quite sure which of the two should be chosen. Being root inside a user namespace does not provide enough privileges to mount a FUSE filesystem, so Forshee and namespaces maintainer Eric Biederman are looking to add filesystem ownership to user namespaces.

Effectively, the superblock of a filesystem would have a mapping for UIDs and GIDs that would be applied whenever a write was done to the filesystem. That would mean there would be a "double shift" of IDs: once from the namespace to kernel and then from the kernel to the filesystem view. But, to him, it looks like a good solution since it would move any security problems in shiftfs from being his responsibility to Forshee and Biederman, he said with a grin. That might not make for a particularly good argument from a security perspective, however.

The challenge now is to integrate the various pieces. Instead of two solutions (shiftfs, portable roots), where one needs to be chosen, there are those two solutions plus a "radically different" approach (superblock mapping). Both shiftfs and portable roots would trivially be able to use the superblock mapping (since they both have superblocks), but it all needs to be put together in a form that's ready for merging. He doesn't expect that to happen for a few more kernel development cycles, so there is still time for security folks to weigh in with concerns if they have them.

In conclusion, Bottomley said that the problem of accessing container images in unprivileged containers is unsolved at this point, but the broad outlines of potential solutions are taking shape. If there are security concerns with those, and "the squeaking in the room" seemed to indicate that there are, now would be the right time to bring them up. Either a solution is found or containers will always have a root user in them, which is more of a security threat than providing a sort of "fake root" for accessing container images.

[I would like to thank the Linux Foundation for travel support to attend the Linux Security Summit in Toronto.]

Comments (5 posted)

Minijail

Sandboxing services and applications running on the Linux kernel is a way to help mitigate problems they can cause when they have bugs or are compromised. While there are multiple technologies in the kernel to help with creating the sandbox, it is easy for programmers to get it wrong. Jorge Lucangeli Obes gave a presentation on minijail, which is a tool and library that Google uses in multiple systems for sandboxing. In fact, he said, Google uses it everywhere: on Android, Chrome OS, on its servers, and beyond.

He started the talk by showing a portion of a ps listing from his laptop that showed multiple root-owned processes running. Each of those processes is "one bug away" from an attacker getting root privileges. For example, the Bluetooth daemon is running as root and listening on the air even on a "super modern kernel". He could have set up a Bluetooth beacon in the room to try to exploit the Bluetooth stacks in the laptops present, which would have given him complete control of them if it was successful; he didn't do that, but it is certainly possible.

Part of the reason that so many processes run as root is that there are misaligned incentives, Lucangeli said. Administrators don't know what permissions are needed by the software and developers don't know where their software is running. Even when the developers do try to reduce the privileges their programs need, they make mistakes as there are a lot of pitfalls in doing so correctly.

So instead of reinventing the wheel for each program and expecting the developers to be experts in security hardening, Google developed minijail. That way, those who are writing Android or Chrome OS system programs do not have be security experts; there is simply a library they can use to handle these sandboxing chores. That library will be regularly tested to ensure that it always works and there will be one place to fix bugs when it doesn't.

Minijail is also part of what allows Android apps to run on Chrome OS, he said. It is effectively creating a container for programs that use it. So minijail is a "containment helper" for Android, Chrome OS, Brillo, and more.

The goal is to eliminate as many of the services running as root as possible. For one thing, minijail uses Linux capabilities to reduce the privileges a process needs. For example, the Bluetooth daemon needs the ability to administrate network interfaces and to open sockets, but it does not need to be able to remount filesystems or reboot the system. So it is given the appropriate capabilities that allow it to do its job—and no others.

In Chrome OS, for example, no network-facing services are running as root. They are not completely unprivileged, of course, but instead try to follow the principle of least privilege.

There's more to minijail than just capabilities, though. Processes with a restricted set of capabilities can still access the entire kernel API. It really doesn't make sense for a process that doesn't have the capability needed to mount a filesystem to still have access to the mount() system call, Lucangeli said.

So minijail uses seccomp to restrict the system calls that processes can make. For example, cat needs only nine system calls to function, instead of the 350 or so that are available in the kernel API. The idea is that even if the process gets subverted, it can't really do anything more than it is meant to do. The Chrome rendering process only needs around half of the available system calls to do its job; with seccomp protections, malicious content still can't cause it to make any of those other calls.

Minijail uses LD_PRELOAD to ensure that the mini-jail is entered before the program's main() function is called. This has the advantage that the system calls used by glibc initialization do not have to be added to the seccomp rules, since glibc is loaded and initialized before the jail.

There is another reason that LD_PRELOAD is needed, he said. Ostensibly, capabilities are inherited over execve(), so you can have a launcher that sets up the sandbox and runs the program in it, but there is a hitch. Unless filesystem capabilities are enabled, it is impossible to actually pass the capabilities on to the new program. There are good reasons not to enable the file-based capabilities, however, because they allow processes to gain capabilities at runtime, which makes reasoning about them more difficult. "Everyone who tried to use capabilities to do something useful" has seen the problem, he said. The solution was ambient capabilities, which allow processes to pass their capabilities across an execve() call without using filesystem capabilities.

Sometimes code is not prepared to deal with the errors returned from a capability check or a seccomp rule, so there is another option in that case: return a dummy object. That is the way he thinks of namespaces in some contexts. They allow the kernel to return "fake" objects for some resources. Namespaces make it easier to port code from elsewhere without having to do major surgery on the code, Lucangeli said.

All seven of the Linux namespaces are supported in minijail at this point. He showed an example using process ID (PID) namespaces, which can be used to prevent "exploiting horizontally"—attacking other processes rather than the kernel. Separating processes into their own PID namespace prevents compromised programs from even seeing the other processes. Over the years, there have been several bugs in the code checking for ptrace() access, but they can't be exploited if the target PID cannot even be seen.

The minijail0 binary wraps all of these techniques up together into a single program that can start and enter namespaces, apply seccomp rules, manage capabilities, and so on. It provides access to all of the Linux sandbox features in that one binary. When starting a PID namespace, it will launch a small program that knows how to act like init in the namespace. It will also use a mount namespace to remount /proc inside the mini-jail.

While there may be security concerns about user namespaces, they are the thing that "ties everything together" for minijail. Up until user namespace support was added to minijail, minijail0 had to be run as the root user. The team got requests from within Google to be able to run minijail on systems where root access was not available. Now it can be run as a regular user, which has opened up new applications for minijail, such as on build systems or in the fuzzing infrastructure.

There are some processes that need to run as root, such as the Android init process. So, for the Android container on Chrome OS, the team put the Android system into a user namespace where it was root; some parts of the filesystem were bind-mounted into the container so that init could find things where it expected them. Everything "pretty much just worked". Input events were plumbed into the container and graphics textures are sent out to Chrome OS over a file descriptor; those were the two main changes to Android to make it work. Minijail allowed most of Android to run unmodified on Chrome OS and it also solved many other problems in Chrome OS, Lucangeli said.

Many people were involved in developing minijail. It is used in Chrome OS and will be in Android 7.0 (Nougat), mostly for the seccomp support. It is available under the BSD license in the Android repositories.

[I would like to thank the Linux Foundation for travel support to attend the Linux Security Summit in Toronto.]

Comments (5 posted)

Security quotes of the week

Comments (10 posted)

curl: certificate reuse

Comments (none posted)

elog: unauthorized posts

Comments (none posted)

file-roller: file deletion

Comments (none posted)

gdk-pixbuf: denial of service

Comments (none posted)

gnutls: certificate verification bypass

Comments (none posted)

inspircd: user impersonation

Comments (none posted)

libarchive: two vulnerabilities

Comments (none posted)

libarchive: file overwrite

Comments (none posted)

libtorrent-rasterbar: denial of service

Comments (none posted)

mysql: SQL injection/privilege escalation

Comments (none posted)

openjpeg2: two vulnerabilities

Comments (none posted)

pdns: denial of service

Comments (none posted)

php: multiple vulnerabilities

Comments (none posted)

python-jwcrypto: information disclosure

Comments (none posted)

qemu: directory/path traversal

Comments (none posted)

webkit2gtk: multiple vulnerabilities

Comments (none posted)

wget: race condition

Comments (none posted)

wordpress: multiple vulnerabilities

Comments (none posted)

xen: multiple vulnerabilities

Comments (none posted)

xen: privilege escalation

Comments (none posted)

Kernel release status

The known regression list for 4.8 has ten entries as of September 11.

Stable updates: 3.14.79, the last of the 3.14.x series, was released on September 11.

The 4.7.4 (announcement missing) and 4.4.21 stable updates are in the review process as of this writing; they can be expected at any time.

Comments (none posted)

Quotes of the week

     yell_WTF(nr_wtf_moments);

Comments (2 posted)

Lindholm: UEFI Driver Development - Part 1

Comments (none posted)

Exclusive page-frame ownership

An attacker wanting to get the kernel to run arbitrary code faces a problem: where can that code be put so that the kernel might run it? If the kernel can be convinced to run code found in user space, that problem becomes much easier to solve, since placing code in user-space memory is something that anybody can do. Since user-space memory remains mapped while the processor is running in kernel mode, all that needs to be done is to convince the kernel to jump to a user-space address. Years ago, it was possible to simply map the page at address 0 and find a bug that would cause the kernel to jump to a null pointer. Such simple attacks have been headed off, but more complex exploits are still common.

Obviously, the best solution is to ensure that the kernel will never try to jump to a user-space address. If one accepts that there will always be bugs, though, it makes sense to add other defenses, such as simply preventing the execution of user-space memory by the kernel. The PaX KERNEXEC and UDEREF mechanisms are designed to prevent this kind of user-space access. More recently, the processor manufacturers have gotten into the game as well; Intel now has supervisor mode access prevention and supervisor mode execute protection, while ARM has added privileged execute-never. On systems where these mechanisms are fully implemented, it should be impossible for the kernel to execute code found in user-space memory.

Except, as this paper from Vasileios P. Kemerlis et al. [PDF] points out, there's a loophole. User-space memory is accessed via a process's page tables, and the various access-prevention mechanisms work to block kernel access via those page tables. But the kernel also maintains a linear mapping of the entire range of physical memory (on 64-bit systems; the situation on 32-bit systems is a bit more complicated). This mapping has many uses within the kernel, with page-level memory management being near the top of the list. It provides a separate address for every physical page in the system. Importantly, it's a kernel-space address and, on some systems (x86 before 3.9 and all ARM), this memory range is executable by the kernel.

If an attacker can cause the kernel to jump into the direct mapping, none of the user-space access-prevention mechanisms will apply, even if the target address corresponds to a user-space page. So the direct mapping offers a convenient way to bypass these protections, with only one little catch: an attacker must be able to determine the physical address of the page containing the exploit code. As the paper points out, the pagemap files under /proc will provide that information, and, while these files can be disabled, distributions tend not to do that. So, on most systems, everything is in place to enable an attacker to exploit a bug that can cause a jump to an arbitrary address and the existing access-prevention mechanisms are powerless to stop it.

(Life gets a little harder on current x86 kernels, where it is no longer possible to directly execute code via the direct mapping. In such cases, the attacker must resort to return-oriented programming instead — not a huge obstacle for many attackers.)

The solution, as described in the paper and implemented in the exclusive page frame ownership (XPFO) patch set posted by Juerg Haefliger, is to take away the back-door access to user-space pages via the direct mapping. The mechanism is fairly simple in concept. Whenever a page is allocated for user-space use (something the kernel already indicates with the GFP flags in the allocation request), the direct mapping for that page is removed. Thus, even if an attacker can generate the directly mapped address for the page and get the kernel to jump there, the kernel will fault due to lack of access permissions to that page. When user space frees a page, it will be zeroed (to prevent attacks via hostile code left in the page) and returned to the direct map.

There are times when the kernel must access user-space memory, of course; the copy_to_user() and copy_from_user() functions are obvious examples. In such cases, the direct mapping is restored for the duration of the operation.

Naturally, there is a performance cost to this. The mapping and unmapping of pages in the kernel's address space will slow things down somewhat, as will the zeroing of returned user-space pages. Perhaps more significant, though, is a change in how the direct mapping is implemented. Normally, the kernel creates this mapping with huge pages; that, among other things, greatly reduces the pressure on the processor's translation lookaside buffer (TLB) when the direct mapping is accessed. But use of huge pages is incompatible with adding and removing mappings for individual (small) pages in that range, so, with XPFO, the huge-page mappings have to go. There is also some increased memory overhead resulting from the need to store more per-page information. All told, enabling XPFO has a performance cost up to about 3% in the worst case, though most of the benchmarks reported in the paper suffered much less than that.

The patch set needs some completion work before it can be seriously considered for merging into the mainline. Once that point comes, one can assume that the conversation will hinge on how effective it is at preventing exploits and whether it is worth the performance cost. The fact that the slowdown for kernel builds is 2.5% could prove to be a bit of an obstacle in this discussion. A performance hit on that scale is a hard thing to swallow, but so are successful exploits. Which pill will prove to be the bitterest will have to be seen as the patch set progresses.

Comments (19 posted)

The need for TTY slave devices

A typical computer system has numerous "buses" to carry data and control from the CPU out to various peripheral devices and back. Linux generally supports these buses by having "master" drivers to manage the hardware at the CPU end of the bus, and "slave" drivers to manage the peripheral. There is one particular bus for which there are no slave drivers, at least not in the normal sense, but for which there is recurring interest in supporting the creation of such drivers. The asynchronous character-oriented serial bus, one of the oldest bus types that is still in wide use today, is managed quite differently from other buses, but might soon be enhanced to better meet current needs.

One difficulty I have in discussing this bus is that there does not seem to be a suitably generic name. Once upon a time I would have called it a "serial connection", but today most connections are serial, whether SATA, SAS (serial attached SCSI), Ethernet, or I2C. So that name doesn't work. RS-232 was once a popular name, but that specifies higher voltage levels and more wires than are normally found on the intra-board connections that we will be interested in. The name UART, standing for Universal Asynchronous Receiver/Transmitter, is at about the right level of generality, but really refers to the controlling hardware rather than the bus itself. TTY, an abbreviation for "teletype", is the name I will largely use, not because there are any teletypewriters connected to any computers I have used in a long time, but because it is a name that is widely used in Unix and Linux history and in present implementations, and it is nice and short.

When a computer system has some TTY ports, Linux will discover these ports and create devices like /dev/ttyS0 to allow them to be managed. In general, Linux knows nothing about what might be connected to the port. One exception is that a "console" might be known to be attached to one of the ports, and Linux will then send kernel messages to that port. In other cases, Linux needs to be explicitly told what is attached if it is expected to handle it in any particular way.

Line disciplines

Linux doesn't always need to know what is attached to a TTY port — a program in user space can open the /dev/ttyXX device and read or write as appropriate. Sometimes, though, it can be helpful for the kernel to take a larger role; for those times there are "line disciplines", which is really just another name for "protocol handlers". As an example: dial-up networking uses a modem to connect a TTY port on one machine to a similar port on another machine. Once a connection is established over the modem, the PPP protocol is often used to allow Internet traffic to flow between the computers. As this requires tight integration with the networking subsystem in the kernel, it is easiest if the PPP protocol itself is handled directly by Linux. To this end, there is an N_PPP line discipline. Once the connection is established, pppd (the user-space daemon for managing the connection) sets the line discipline to N_PPP and all further traffic is handled directly by the kernel.

Another line discipline that was once more widely used than it is now is the N_MOUSE protocol for communicating with serial-attached mice. N_MOUSE passes data from the TTY port though to the "input" subsystem so it appears on /dev/input/mouse0 or similar and can be easily used by your windowing system. There are a collection of other line disciplines for various different serial protocols. Each one needs to be explicitly activated by a program like pppd for N_PPP, inputattach for N_MOUSE, and hciattach for N_HCI (the protocol for communicating with Bluetooth transceivers). The line discipline only remains active for as long as that program keeps the TTY device open.

If line disciplines were being invented today, they would almost certainly be drivers on a bus that would get bound to the hardware either automatically, or by writing to a bind file in sysfs.

Problematic use cases

Though the mechanism for attaching a line discipline to a TTY port allows a lot of serial-attached devices to be used quite effectively, there are two areas where the current solution is not entirely satisfactory thus motivating various people to seek improvements. These areas involve transparent discovery and sideband controls such as power management.

If I have a computer system, such as a mobile device, which has, for example, a Bluetooth transceiver permanently attached to a UART, then I shouldn't have to tell the software running on that device about the hardware arrangement. The firmware on the device should know about the Bluetooth device, possibly from nodes in a device-tree description of the hardware, or possibly from information in the ACPI tables, and something should read that description and configure the TTY port appropriately. It might be possible for a user-space program to extract the information and run hciattach, but as firmware tables are normally interpreted by the kernel, and as hciattach does little more than request the N_HCI line discipline, it seems sensible to have the kernel set everything up transparently. The "little more" that hciattach does might involve setting a bit rate, performing some initialization, or uploading firmware. All of these are the sorts of things the kernel already does, so it would be no extra burden.

Even in cases where the device can be managed without a dedicated line discipline, there might be a need to do more than just send and receive bytes. Power management is important in all computer systems these days and, while some serial-attached devices can power themselves up or down in response to commands over the serial connection, this capability is not universal. Were we using RS-232, the DTR (data terminal ready) line would probably be used to manage power, but many UARTs do not have a DTR line, and asserting a signal is not necessarily the best way to control power for an integrated device. Device power management in Linux is generally handled by the device driver for the particular device, since it knows the needs and is able to assert a GPIO output, activate a regulator, or whatever else is needed. But, with TTY ports, there is no slave device driver to perform these tasks.

Both of these difficulties could be solved if a TTY were treated more like a bus that could have slave devices attached as children. The configuring of child devices is the normal way that device information from device tree or ACPI tables is handled, and these devices would be well placed to select a non-default line discipline or to control the power management of the device when it is opened or activated.

Where to put the device

Though I was not involved in the most recent discussions on this topic, I have attempted to make progress in this problem space in the past; a recurring problem is that it wasn't obvious, to me at least, what shape the solution should take. Above, I have described the need as being for a "TTY bus" with "slave devices" but that understanding only came about after several failures, and there is not yet a certainty that it is best solution.

Linux has a concept of a "platform bus", which is a "pseudo-bus" that is described more by examples than by a concrete purpose. It is contrasted with "large formally specified [buses] like PCI or USB." A driver to control a GPIO line to manage the power of a GPS device attached to a TTY could easily be seen as part of the "platform" rather than part of a genuine bus, particularly if you didn't think of a TTY as a "bus", which I certainly didn't. So an early attempt created a platform device to handle power management and taught the TTY driver to tell the attached platform device when it was opened or closed. This didn't address the auto-detection need, which did not concern me at the time. The patch was vetoed by Greg Kroah-Hartman, both when I proposed it and when it was recently re-proposed by Sebastian Reichel, who is trying to make the Bluetooth transceiver on the Nokia N950 work correctly. As Kroah-Hartman put it: "I've said before that a "serial" bus should be created".

Rob Herring responded to this challenge and proposed a "UART slave device bus" that is not entirely unlike something I proposed last year. Linux contains a "serial core" subsystem that supports a wide range of serial character devices and which provides a uart_port abstraction. This is separate from the "tty" subsystem, which provides a tty_port, handles all the peculiarities of Posix TTY devices, and manages the line disciplines. As all the devices that anyone wanted to create a slave device for were UARTs, it seemed natural to both Herring and myself to make changes at the uart_port level.

Alan Cox vetoed this one. In his view, the UART isn't the right place to attach slaves because not all serial devices use the UART code, or not in the same way. In particular, USB-attached serial ports do not use the UART code at all. Cox recalled that: "As I told you over six months ago uart_port is not the correct abstraction. You need to be working at the tty_port layer," and again: "This again is why it needs to be at the tty_port layer." The tty_port interface, provided by the TTY layer, is clearly the more general interface for serial devices .... or is it?

The serio bus

There are some serial character devices that don't use UARTs and don't even interface with the TTY layer. The most common example is the PS/2 mouse. The over-the-wire protocol used by a PS/2 mouse is similar to that used by serial-port mice, but is more constrained and so can be supported with simpler hardware than a UART. In Linux, the driver for PS/2 mouse hardware (and PS/2 keyboards as well) is attached to the serio (serial I/O) bus, which feeds into the input subsystem.

The N_MOUSE TTY line discipline mentioned earlier is really a generic gateway from TTY devices into the serio bus. It was designed for use with serial mice, but could be used with any device with a similar character-based interface. Herring, with a little prompting from Arnd Bergmann, wondered if the serio bus could become the place to attach the slave devices that we seem to want. To this end, he prepared some patches that allow device tree configuration to direct a serio device to attach to the HCI interface for the Bluetooth subsystem. With these patches it is still necessary to run inputattach to gateway the TTY to the serio bus using the N_MOUSE line discipline. Herring claims: "Once a tty_port based serio port driver is in place, this step will not be needed". In some ways, this seems like an step in the right direction, in others it seems like it might just be moving the problem to a new location.

While this serio approach could work well for auto-configuration of Bluetooth devices, it isn't obvious that it works well for power management of GPS receivers using sideband signaling. For a GPS receiver we really still need the TTY device, /dev/ttyXX, to exist much as it does now. We don't want to attach an alternate line discipline, because the kernel doesn't understand the protocols (such as NMEA and various binary protocols) that GPS devices use. The current solution of running gpsd to interpret these protocols is quite effective. Though Marcel Holtmann attested that he is "not convinced that GPS should be represented as /dev/ttyS0 or similar TTY" and Kroah-Hartman expressed support for this position, the creation of a GPS device type seems to be a separate need than allowing a device to be powered on when a TTY is opened, and powered off when it is closed.

Ideas for forward progress

Though this recent conversation does not seem to have produced any code that is likely to get merged, it did bring up a lot of ideas and a lot of constructive criticism for why some of the proposals were not satisfactory. One of the most discouraging responses one can get when submitting a patch is to have it rejected with no clear explanation of why it was rejected. That didn't happen here. Of all the feedback that was provided, possibly the most concretely useful was Cox's insistence that tty_port should be the basis of a new bus ("The basic problem is that the bus should be tty_ports not uart, fix that and the rest starts to make sense.") and his explanation of the role of the tty_port as the item in the device model which "has the lifetime of the hardware".

When serial hardware is discovered, whether it is a UART, a USB-attached serial port, or something else, a tty_port is created. It is currently registered as a character device so that an entry appears in /dev, which can then be opened. When it is opened, a tty_struct is attached, and line disciplines can be attached to that. The right approach seems to be to insert a bus abstraction under the tty_port so that different drivers can be bound to the port. The default driver would register a character device that would attach a tty_struct when it was opened. Other drivers might connect through to the Bluetooth subsystem, or might interpose some power management controls and then register a TTY character device.

One reason this hasn't been done already is that the TTY layer is a little complicated. tty_port and tty_struct are closely intertwined and separating them, as seems to be required, is not a task for the timid. Cox has posted an RFC patch that takes a step in this direction by allowing a tty_port to be used without an open file handle. There is a lot more that would need to be done, but this is a valuable start, particularly as it comes from someone with a deep knowledge of the TTY layer who can probably see the end game more clearly than the rest of us.

The conversation has died down for the moment. That might mean that people have been distracted by more urgent issues, or it could mean that now is a time for coding rather than discussion. This is a topic that has arisen several times in the past and while it is generally met with enthusiastic in-principle agreement, it does not seem have been quite important enough to anyone to push through the various barriers to find a solution that is broadly acceptable. Maybe this time will be different.

In a conversation on the Kernel Summit email list concerning the different sorts of "stable" kernels that vendors use and how much is being backported to them, Tim Bird lamented that "there are still significant areas where the mainline kernel just doesn't have the support needed for shipping product." The appropriate emphasis, in the mainline kernel community, to require well designed and fully general solutions inevitably means that some functionality takes a while to land. This means that vendors with tight deadlines need to choose between staying close to mainline or having all the functionality they want. It is understandable that they will often choose the latter. Finding ways to minimize the need for this choice is one of the ongoing challenges for the kernel community and one that we might see playing out, in a small way, with the need for TTY slave devices.

Comments (5 posted)

BlackArch: a distribution for pen testing

BlackArch Linux is a specialized distribution for security and forensic testing. Its simple interface and array of tools makes it suitable for the home user who wants to check the security of their router's firewall, as well as the seasoned professional hacker who has been hired to discover weaknesses in their client's computer system. At the end of August, the BlackArch team announced the release of version 2016.08.31.

BlackArch is designed specifically for penetration testing or, more colloquially, "pen testing". This involves using various tools to access a system by exploiting security holes. "Pen testing", like the term "hacking", is often misunderstood to mean criminals trying to access unauthorized information. But competent system and network administrators routinely carry out pen tests on their own systems to ensure they are safe to use, either by using a distribution like BlackArch or by authorizing a security professional to do the same. BlackArch also contains tools that go beyond simple pen-testing, including some to thwart forensic analysis or to exploit compromised systems. The distribution was initially created for the developers' own use but now is available to the public.

When it comes to pen-testing distributions, comparisons to other more established players, such as Kali or BackBox, are inevitable. But the sheer range of tools (1,532 at the time of writing) as well as the care that has gone into choosing them makes for a very favorable comparison.

According to DistroWatch, there have been three stable releases of BlackArch since 2014, released on roughly an annual basis. BlackArch is based on Arch Linux, which follows a rolling release model. This means that if BlackArch is installed on top of Arch itself, then running a system update via the package manager pacman is all that's required to get security and other updates from Arch. For tools that are not packaged for Arch directly, the system can be updated using the BlackArch repository in pacman to pick up the latest security and other updates that have been released by the team. Alternatively, users can download new versions of BlackArch whenever an ISO image is released. As BlackArch is based on Arch Linux it uses the Arch User Repository (AUR), so it can take advantage of those unofficial, user-contributed packages as well.

BlackArch was initially founded by Evan Teitelman, also known as "paraxor", who continues to develop it along with a team of developers and contributors. It boasts a number of download mirrors, most recently including Princeton University. Although the default language and keyboard layout is US English, its user base is multinational and the BlackArch Guide has been translated into French, Turkish, and Brazilian Portuguese.

BlackArch 2016.08.31 is based on the 4.7.1 kernel and comes with an updated installer as well as a number of additional tools. Tools are arranged logically according to category, such as DoS (Denial of Service), backdoor, networking, and much more. These can all be installed at once, in groups, or individually.

Tools

By default, BlackArch contains many of the most well-known security tools. This includes Nmap — a tool used for network discovery and security auditing. The program scans the network to determine which hosts and services are available, as well as which OS they are running.

The Aircrack-ng suite is preinstalled — along with Airoscript-ng and gerix-wifi-cracker, which act as GUIs for the Aircrack-ng. The tools are capable of both "sniffing" and creating packets to inject into a network in order to crack WEP/WPA wireless network keys. Another bundled application is Kismet — along with Kismon which is a GUI for it. Kismet is a network detector and packet sniffer similar to Aircrack-ng that works entirely passively. This allows Kismet to detect not only rogue wireless access points but even to analyze packets to detect wireless sniffing programs or network attacks.

No pen-testing distribution would be complete without Wireshark. The self-proclaimed "foremost network protocol analyzer" is capable of deep inspection of a number of network protocols. It can also capture data packets live for offline analysis and displays the results in a color-coded GUI.

Another vital bundled tool is the Metasploit Framework.The program is part of the Metasploit project, which exists to provide information about security vulnerabilities, or in its own words: "Knowing the adversary's moves helps you better prepare your defenses." The main purpose of the tool is to develop and execute exploit code on a target system. BlackArch comes with the free version of Burp Suite, which is an integrated platform for performing security testing of web applications. It does this using various tools such as an intercepting proxy that allows an admin to inspect and modify traffic between browsers and target applications.

Another well-known security tool that comes with BlackArch is John the Ripper, along with its GUI Johnny. This tool can autodetect password hashes and crack them. One attack mode it uses is a dictionary attack that works in conjunction with wordlists to encrypt common passwords as well as words from a dictionary, then compare them to the password hash on a target system. In addition, there's a "brute force" mode whereby the program simply tries every combination of password, although this can potentially take years. The version bundled with BlackArch is free. There is a Pro version that supports more hashes.

The latest version of BlackArch includes over 100 new tools. These include anti-XSS, a cross-site scripting (XSS) vulnerability scanner that doesn't seem to be installed by default in Kali (although Kali has XSSer). BlackArch also now includes Scamper — a handy tool for probing the internet to analyze its topology and performance. Another bundled favorite is BoNeSi — a tool to simulate botnet traffic in a test-bed environment. Another BlackArch-specific tool — introduced in 2014 — is netcon.sh, which is designed to establish network connections with the option to spoof MAC addresses, hostnames, peer IDs for BitTorrent, and so on.

Despite the vast collection of tools, care seems to have gone into those that are selected, with an emphasis on any that automate common pen-testing tasks. One example is Medusa, a massively parallel login brute-force attacker for networks that comes pre-installed in the full ISO.

The "anti-forensics" category is particularly worthy of mention. Anti-forensics is a broad term for tools and techniques to counter forensic analysis of computers. In this vein, BlackArch includes the now defunct TrueCrypt as well as its more modern successor, VeraCrypt, both of which can be used to create encrypted file containers or partitions for sensitive data. The forensic tools aesfix and aeskeyfind are included to allow scanning of memory dumps for encryption keys as part of a "cold boot attack". Provided an adversary can get to the target device in time to perform a memory dump, the attack can be used to retrieve passwords and decrypt sensitive data.

Devices or partitions encrypted by TrueCrypt or VeraCrypt have plausible deniability in that it's difficult to determine the difference between encrypted and random data. Encrypted file containers can potentially be detected quickly by TrID, which identifies files based on their binary signatures. TrueCrypt and VeraCrypt containers appear as blocks of random data with no file header. Naturally, the fact that a file has no header and contains only seemingly random data is not absolute proof that it contains encrypted information. However, there are few plausible reasons to have blocks of purely random information on a device. TrID will easily locate all such files, to allow for further analysis. Once they are located, the tool thc-keyfinder [Safe Browsing alert] can be used to measure the entropy (randomness) of a file to determine if it might contain encrypted data.

One of the more terrifying backdoor tools that is included is Backdoorme, which is similar to Joshua Pitt's The Backdoor Factory that is part of the Kali tool set. Backdoorme automates inserting key-loggers and creating new users. The software requires an active SSH connection for the time being. The Backdoor Factory is also installed and, unlike backdoorme, can be used to patch win32/64 binaries with shell code.

A full list of the tools is available from the BlackArch website. Alternatively, you can install all available tools from the BlackArch repository by using pacman.

Installation options

Like Kali, BlackArch is available as ISOs for 32-bit or 64-bit systems, though they are 4.3GB and 4.2GB in size, respectively. They weigh rather heavily against Kali's 2.6GB ISO. BlackArch redeems itself here, however, with a minimal netinstall ISO of around 400MB.

Attempting to boot in a virtual machine with both ISOs was successful, but it may be necessary to set the boot flag edd=off to ensure the "Probing EDD" message doesn't display for several minutes prior to the OS loading.

For Arch users, BlackArch can be installed on top of the existing system. One way to do this is by executing the strap.sh Bash script as root, which will add BlackArch as an unofficial user repository along with its corresponding keyring. A full version of this script is available for review on GitHub.

You can then install all BlackArch tools with a simple terminal command. BlackArch's issue tracker on GitHub has noted that sometimes running strap.sh results in an error involving an invalid keyring signature. A full discussion of the issue and a possible resolution can be found on the issue tracker itself.

Alternatively, you can retrieve the PKGBUILDs (build scripts) from GitHub and build the blackarch packages from source using the blackman package manager. More information on both install methods is available on the BlackArch downloads page.

Unlike Kali, BlackArch has yet to release ARM images with BlackArch pre-installed. Nevertheless, BlackArch supports all the ARMv6 and ARMv7 platforms listed on the Arch Linux ARM website including the Raspberry Pi. In order to install BlackArch on an ARM platform, follow the install instructions for your device on archlinuxarm.org.

For those unready to commit fully to the command line, BlackArch Version 2016.08.31 also comes with new menu entries for the various window managers available such as awesome, Fluxbox, Openbox, and spectrwm. The default is Fluxbox.

While on the subject of window managers, those used to the relatively lush GUI of Kali or BackBox Linux may find BlackArch to be rather spartan. The emphasis seems to be on listing as many tools as it can, as clearly as possible. BlackArch nevertheless has its own wallpaper and logos, namely a stylized sword running through a red Arch logo.

The team of volunteers behind BlackArch is keen to point out on the home page that the OS is relatively new when compared to other pen-testing distributions. They request that bugs be reported through the issue tracker on GitHub.

The team also asks that anyone with comments stop by the official BlackArch IRC channel [IRC] or follow BlackArch on Twitter.. BlackArch's IRC channel is particularly useful as the team has a handy bot that notifies the channel of Git commits and package updates/additions. Although a placeholder page exists, BlackArch doesn't have its own wiki at the time of writing. Any developers or translators interested in contributing should follow the steps in the BlackArch Guide before forking the repository.

Given the extremely simplified GUI and the veritable Swiss Army knife of pre-installed tools for those willing to download the hefty ISO, the overall impression of BlackArch is that it's a distribution that appeals more to the white-hat hacker than to the casual pen-tester. This said, the tools have clearly been well-considered and are neatly categorized. Anyone familiar with other pen-testing distributions should be able to navigate BlackArch in minutes. GUIs for popular programs as well as a choice of window managers make sure that this OS isn't only for lovers of the command line. The only slight criticism would be the fact that ARM images are still not available at the time of writing, which make setting up such a system a bit more tedious than for Kali.

Comments (none posted)

Distribution quotes of the week

Comments (none posted)

Distribution newsletters

• DistroWatch Weekly, Issue 678 (September 12)
• Lunar Linux weekly news (September 9)
• openSUSE news (September 14)
• openSUSE Tumbleweed – Review of the Week (September 10)
• Ubuntu Kernel Team newsletter (September 6)
• Ubuntu Weekly Newsletter, Issue 481 (September 11)

Comments (none posted)

Elementary OS Loki Has Arrived (Linux.com)

Comments (none posted)

Network access by Debian package builds

The Debian project is known for strictly adhering to its various internal rules and guidelines; a package that some developers feel is in violation of the Debian Free Software Guidelines, for example, is sure to spawn considerable debate, regardless of its popularity or its origin. Recently, a Debian package maintainer realized that a particular package violated a guideline prohibiting network access during the build process, thus sparking a discussion about that guideline and what its ultimate purpose is.

On September 7, Vincent Bernat wrote to the debian-devel list, noting that the python-asyncssh package, when building, runs a unit test that attempts a DNS lookup. That would seem to violate section 4.9 of the Debian packaging policy, which states "For packages in the main archive, no required targets may attempt network access."

The issue was reported by Chris Lamb in a bug tagged as "serious." Fixing that violation, Bernat said, is simple enough—just disable the test in question—but Bernat wondered whether or not the policy rule is genuinely useful. Since the test originates in the upstream project, Debian would have to carry the patch indefinitely, adding what might be deemed considerable overhead for little real gain.

Furthermore, the test in question performs a lookup for a host named fail, which is expected not to succeed, and the purpose of the test is to ensure that the package handles the lookup failure gracefully. The build, Bernat said, works in an isolated network namespace and the package builds reproducibly with or without network access. Consequently, he asked for feedback on ignoring the violation in this case:

Various responses to Bernat's question suggested alternatives to removing the test, but much of the discussion focused on the purpose of the no-network-access rule. Christian Seiler, among others, said the rule was intended to prevent information leaks, which a DNS lookup would certainly cause.

Paul Wise, however, felt that the rule was intended to ensure that nothing outside of the local build environment had any impact on the result of the build process. Steve Langasek concurred with that viewpoint, noting that:

It would appear that there are more than a few packages in the archive that do violate the no-network-access rule. Christoph Biedl said "a certain package (name withheld) did a *lot* of DNS traffic in the test suite, so far nobody has shown concerns". And the test in python-asyncssh, as written, is problematic: if there is a host named fail in the local DNS zone, the lookup will succeed. While unlikely, this is possible; a better test would be to look up a hostname that is guaranteed to be nonexistant by IANA rules (such as .invalid). In addition to being a better test of lookup-failure handling, that change would avoid the risk of an information leak through the lookup request (or, at least, reduce the risk, depending on the behavior of the nameserver).

Russ Allbery contended, though, that there is no genuinely important information leak anyway. He made that comment in reply to Thomas Goirand, who suggested that the issue of attempting network access was valid, but questioned whether "serious" was an appropriate severity level:

Others on the list, starting with Gregor Hermann, suggested revising the wording of the rule itself. Allbery proposed two rules, one saying that the package build "must not fail when it doesn't have network access" and another that warns against leaking privacy-related information; several similar variations arose from other participants in the thread. But Adam Borowski replied that attempts to distinguish between different types of network usage are, ultimately, doomed to fail, making such an effort pointless:

Borowski reiterated the privacy-leak angle, saying that even innocent-looking DNS lookups violate "the Dissident Test"—that is, a user performing the build in some location where state-sponsored surveillance is a threat could put themselves at risk of investigation.

But not everyone found the "Dissident Test" argument persuasive. Allbery, in particular, contended that the lookup of a well-known hostname did not reveal significant personally identifiable data, saying:

Moreover, Allbery continued, the entire issue is somewhat overblown:

Goirand concurred with that sentiment, noting that Debian already has a contentious relationship with some upstream projects. Zlatan Todoric also agreed, saying "I also feel that we are losing too much energy on this and this is not sustainable long term, nor fun."

The discussion eventually tapered off without a firm conclusion as to whether or not Debian policy should be amended and with no guidelines for a broad approach to assessing future network accesses that occur during package builds. It seems that the status quo will remain in place, then. Each package maintainer will have to individually assess any problematic network-access attempts in build targets, and some of those access attempts may survive for some time if they are determined to pose neither a serious privacy risk nor to impact the result of the build.

On the surface, it might seem like the Debian project has decided to let a minor rules violation slip through the cracks. Bernat had closed the bug as wontfix prior to raising the issue on the discussion list and has not revisited it. But, at another level, the project could be said to have taken the underlying issues—privacy and reproducible builds—seriously and determined that those principles were being upheld. And that would surely be considered Debian behaving as it usually does, with a strict attention to detail.

Comments (14 posted)

Quotes of the week

Comments (2 posted)

Vim 8.0 released

Comments (16 posted)

python-can 1.5 released

Version 1.5 of python-can, the CAN (Controller Area Network) Bus support package for Python, has been released. "This has been cooking in a development branch for a few months with plenty of changes and bug fixes. Documentation has been updated quite substantially, a virtual interface has been added and a consistent filtering api has been added. " Also noteworthy is that the next release, 1.6, is slated to drop support for the J1939 automotive message protocol.

Full Story (comments: none)

Kügler: LTS releases align neatly for Plasma 5.8

Comments (37 posted)

U-Boot v2016.09 is available

U-Boot version 2016.9 is now available. The announcement notes that "lots and lots" of new architecture, SoC, and platform updates are included in the release. The update also adds support for device tree overlays and for Secure Boot on ARM and PowerPC.

Full Story (comments: none)

Development newsletters from the past week

• What's cooking in git.git (September 8)
• What's cooking in git.git (September 9)
• Git Rev News (September 14)
• This Week in GTK+ (September 12)
• OCaml Weekly News (September 13)
• Perl Weekly (September 12)
• PostgreSQL Weekly News (September 11)
• This Week in Rust (September 13)
• Wikimedia Tech News (September 12)

Comments (none posted)

Apache NetBeans Incubator Proposal

Comments (27 posted)

Abbott: Success with Interns

Laura Abbott marks the end of the latest round of open-source internships at Outreachy with a blog post reflecting on "what makes an internship successful", especially as seen in the kernel team's internships. Among Abbott's lessons: "Choose your tasks carefully. Tasks with a specific goal but multiple ways to solve are best. Too open ended tasks can be frustrating for all involved but there should be some chance for decision making. Just giving a list of tasks and exactly how they should be completed isn't good for learning. Give your intern a chance to propose a solution and then review it together." Also: "Speaking of review, code review is a skill. Model how to respond to code review comments. Encourage interns to practice reviewing others code and ask questions as well." That is just a sampling; in total, Abbott lists well over a dozen take-aways from the experience, all worth reading.

Comments (2 posted)

FSFE Newsletter - September 2016

Full Story (comments: none)

Tridgell: ArduPilot and DroneCode

Comments (17 posted)

CFP Deadlines: September 15, 2016 to November 14, 2016

If the CFP deadline for your event does not appear here, please tell us about it.

netdev 1.2 weekly update

Full Story (comments: none)

Git microconference accepted into LPC 2016

Full Story (comments: none)

Events: September 15, 2016 to November 14, 2016

If your event does not appear here, please tell us about it.