|
|
Subscribe / Log in / New account

Development

A unified TLS API for Python

By Jake Edge
January 19, 2017

Back at the 2016 Python Language Summit, Cory Benfield and Christian Heimes gave a presentation on the future of the language's ssl module. It provides the standard library's TLS support, but suffers from a number of problems, much of it baggage that has accreted over the years. Now Benfield has proposed a PEP to, essentially, move on from ssl, and to create a set of abstract base classes (ABCs) that would define the API for TLS going forward.

Benfield posted the PEP (as yet without a number) to the Python Security SIG mailing list, but it has been in progress since last October in a GitHub repository. Since ssl is used by the pip installation tool, it is important that it works for all of the systems that Python gets installed on, Benfield said. But it is reliant on OpenSSL, which is not well supported on macOS and Windows:

This problem ultimately stems from the fact that Python has never exposed the ability to use TLS backends other than OpenSSL in the standard library. As Christian and I discussed back at PyCon US 2015, the ssl module would more properly be called the openssl module, due to exposing many OpenSSL-specific concepts and behaviours in its API. This has meant that the Python ecosystem is overwhelmingly also an OpenSSL ecosystem, which is problematic on Windows and macOS (and Unices that for whatever reason aren't interested in shipping an OpenSSL), as it has meant that Python needs to bring its own OpenSSL, and that it is troublesome to interact with the system trust database.

To address that, a more generic API for TLS is needed, he said. There are two implementations that he considered:

The first is to have this module be a generic concrete implementation that can be compiled against multiple TLS backends (like curl). This would require that all the relevant bindings be built into the standard library from the get-go, which provides a substantial maintenance burden on a team of people that are already understaffed maintaining the ssl module alone. The second approach is to define a generic high-level TLS interface that provides a minimal usable interface that can be implemented by both first- and third-party modules. This would allow the standard library to continue to ship with only exactly what it needs (for example, OpenSSL, SecureTransport and SChannel), but would give those who want to use more esoteric TLS choices (NSS, GnuTLS, mbedTLS are some examples) an API that they can implement that will guarantee that complying modules can use the appropriate TLS backend.

As might be guessed based on that description, Benfield has chosen the latter approach. The in-progress PEP proposes a set of ABCs that are meant to allow any underlying TLS library provide a compliant implementation. That would allow users, distributors, and others to swap in other TLS libraries as needed—something that is not really possible with today's ssl.

The PEP outlines the various pieces that need attention and where they are currently implemented in ssl (if they are):

  1. Configuring TLS, currently implemented by the SSLContext class in the ssl module.
  2. Wrapping a socket object, currently implemented by the SSLSocket class in the ssl module.
  3. Providing an in-memory buffer for doing in-memory encryption or decryption with no actual I/O (necessary for asynchronous I/O models), currently implemented by the SSLObject class in the ssl module.
  4. Specifying TLS cipher suites. There is currently no code for doing this in the standard library: instead, the standard library uses OpenSSL cipher suite strings.
  5. Specifying application-layer protocols that can be negotiated during the TLS handshake.
  6. Specifying TLS versions.
  7. Reporting errors to the caller, currently implemented by the SSLError class in the ssl module.
  8. Specifying certificates to load, either as client or server certificates.
  9. Specifying which trust database should be used to validate certificates presented by a remote peer.

The PEP then proposes ABCs to support each of those (though the TLS cipher suite specification is still listed as "Todo"). It also briefly looks at the other standard library modules that will need to be revised to use the interfaces. Benfield lists seven different modules (such as asyncio, http.client, imaplib, and smtplib) that would need to change.

The reaction to the posting was largely favorable, though there were plenty of technical concerns mentioned, many of which have been addressed with changes to the PEP. It is already a large specification, but there were questions about additional features. Benfield, Heimes, and others were resistant to calls for more features in the first round, though Heimes brought up the need to support SRV-ID (used to identify different service types) eventually. Benfield wanted to take a wait-and-see attitude for features like that:

The advantage of this API is that it would be extensible. We can extend it as needed over time, we don't need to shove everything in at once. So I'm inclined to want to defer this until we see what the implementations actually do.

Heimes agreed and, in another sub-thread, pushed back on some somewhat exotic features that were being proposed. Though Benfield sees the server-side TLS support as mandatory, Heimes would even be willing to leave that behind in order to get something out there more quickly.

There is, it seems, a deadline of sorts at hand. As Donald Stufft reported to the Distutils SIG mailing list recently, the content delivery network (CDN) that PyPI and other Python infrastructure uses will be phasing out TLS 1.0 and 1.1 support over the next year and a half. Some solution needs to be in place before June 2018 or macOS pip clients will not be able to run; others may well be affected too.

So Heimes is interested in a bare-bones solution, but one that gets the job done: "Personally I would rather remove half of the PEP than add new things." He is concerned that more features will make it that much harder for implementers in the time frame available. Stufft agreed as well: "Getting too lost in the weeds over advanced features like hot-config-reload I agree is a bad use of resources."

But Wes Turner was concerned that adding a TLS configuration object later, as was being advocated, would be hard to do. Nick Coghlan, on the other hand, thought that was the proper approach:

The appropriate time to define that lowest-common-denominator configuration format is *after* there is a working programmatic API that covers at least the 3 major implementations of interest (OpenSSL, SecureTransport, SChannel), and hopefully a few other implementations as well (e.g. NSS, BoringSSL).

It turns out that Turner was actually looking beyond simply configuring the underlying library and was instead concerned with adding an interface to determine whether a given configuration was consistent with a particular security policy. It is something that will need to be addressed, eventually, but once again is "vastly beyond the scope of the problem I'm trying to solve here", Benfield said. It is tempting to add all of the needed pieces at once, he continued, so that all of the TLS dragons can be slayed in one go, but he has a different vision moving forward:

I want us to develop a culture around TLS functionality where we are willing to continually iterate on it, and where the community feels willing-and-able to provide extensions and new functionality incrementally, rather than all at once. Let's not have the perfect be the enemy of the good here. I highly recommend that people who are interested in TLS policy workshop a separate PEP that discusses what we should build to resolve that issue. But we should not block solving problem A (how do we use something that isn't OpenSSL with the standard library) on solving problem B (how do we define and enforce TLS policy).

A more iterative culture around TLS is likely something that Python needs. The ssl module has been a problem child for some time now, at least partly because it mostly works—though some of its defaults are insecure—and lacks for developer time. But protocols change over time and older versions get deprecated. At the moment, that is providing some impetus to speed up the changes that have been needed for some time, which will help. But it would be nice to see proactive efforts at keeping up with TLS down the road.

Comments (12 posted)

Brief items

Development quotes of the week

May not be as exciting, since there are no new visible features but… damn, it’s Gtk+ being rendered with Vulkan on Wayland.
— Georges Stavracas

To start off, I’d like to address the question of “Is it just me?” It is not. I feel similarly that today we often unnecessarily over-complicate software development. My career spans back to 1999, and I got to see how software development became complicated and the very good reasons why it did. Today, however, there is no balance: you either do exactly what the crowd does or you are wrong. This is symptomatic of being only a Novice as defined by the Dreyfus Model of Skill Acquisition. To put it more plainly, if you are strictly adhering to the rules, you lack experience.
— Neil Green

Comments (1 posted)

Calligra 3.0 released

Calligra 3.0 has been released. The Calligra Suite includes office, graphics, and project management applications. "We have chosen to cut back on the number of applications. Krita has left us to be independent and although it was emotional it was also done with complete support from both sides. We are saying goodbye to Author, which never differentiated itself from Words. We also removed Brainstorm the purpose of which will be better fitted by a new application (nothing planned from our side). Flow and Stage has gone in this release but we intend to bring them back in the future." The 3.x series updates the applications to use KDE Frameworks 5 and Qt5.

Comments (none posted)

GNU ed 1.14.1 released

Version 1.14.1 of the line editor GNU ed has been released. "Version 1.14 is the largest bug hunt ever attempted in GNU ed. Other goals of version 1.14 are to complete the documentation and to remove any gratuitous incompatibilities with the POSIX standard. Thanks to Ori Avtalion for initiating all this with a couple bug reports. ;-)"

Full Story (comments: none)

The Linux Test Project has been released

The Linux Test Project test suite stable release for January 2017 is out. There are new test cases, a new shell test library and many tests rewritten to make use of it, and much more. LWN looked at LTP last December.

Full Story (comments: none)

Plasma 5.9 Beta Kicks off 2017 in Style

KDE has announced the release of Plasma 5.9 beta with many new features. This release includes productivity improvements, more streamlined visuals, global menus, a new network configuration module, improvements to Wayland support, and more.

Comments (none posted)

Python 3.4.6 and Python 3.5.3 are now available

Stable Python versions 3.4.6 and 3.5.3 have been released. "Python 3.4 is now in "security fixes only" mode. This is the final stage of support for Python 3.4. Python 3.4 now only receives security fixes, not bug fixes, and Python 3.4 releases are source code only--no more official binary installers will be produced." Python 3.5 is still in active "bug fix" mode.

Full Story (comments: none)

Newsletters and articles

Development newsletters

Comments (none posted)

Prokoudine: GIMP 2016 in review

Alexandre Prokoudine looks at user-visible changes for the GNU Image Manipulation Program (GIMP) over 2016. Changes include better handling of layers, channels, masks, and paths, remembering defaults across sessions, improved configurability, color management, and more.

Comments (6 posted)

Quantum Computing Is Real, and D-Wave Just Open-Sourced It (Wired)

Wired covers the release of Qbsolv as open-source software (under the Apache License v2) by D-Wave, which is a company that makes quantum computing hardware. Qbsolv is "designed to help developers program D-Wave machines without needing a background in quantum physics". Further:

Qbsolv joins a small but growing pool of tools for would-be quantum computer programmers. Last year Scott Pakin of Los Alamos National Laboratory–and one of Qbsolv’s first users–released another free tool called Qmasm, which also eases the burden of writing code for D-Wave machines by freeing developers from having to worry about addressing the underlying hardware. The goal, Ewald says, is to kickstart a quantum computing software tools ecosystem and foster a community of developers working on quantum computing problems. In recent years, open source software has been the best way to build communities of both independent developers and big corporate contributors.

Of course to actually run the software you create with these tools, you’ll need access to one of the very few existing D-Wave machines. In the meantime, you can download a D-Wave simulator that will let you test the software on your own computer. Obviously this won’t be the same as running it on a piece of hardware that uses real quantum particles, but it’s a start.

Comments (9 posted)

Page editor: Rebecca Sobol
Next page: Announcements>>


Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds