LWN.net Weekly Edition for July 23, 2015

Guido on Python

Guido van Rossum's EuroPython 2015 keynote was billed as part prepared remarks, part Q&A, but he changed all that when he stepped up on the stage. Instead, it would all be Q&A, but he did prime the pump with some questions (and answers) of his own before taking audience questions. Topics included Python 3 (and 3.5), why there won't be a 2.8, why there are so many open bugs, PyPy, and what he hates about Python.

Django Girls

Van Rossum's first question to himself was about what he thought of Django Girls—the subject of the previous day's keynote. It was a great talk, he said, and he loved the storytelling. There will be none of that in his talk, nor any "pretty slides". He was stunned when he "heard that Ola ... or Ola ... had drawn the squirrels and badgers" for those slides.

Another aspect that he liked was their statement that they didn't know what they were doing. It reminded him of when he started working on Python 25 years ago. He didn't know what he was doing then, either. For example, he had no idea that a programming language needed a community with lots of different roles.

He was also quite impressed by the "strong brand" they have created in one year. "I predict that Ola and Ola, and Django Girls, will go really far."

Python versions

Shifting gears, his next query was on why developers would switch to Python 3. "Why can't you just give up on Python 3?", he asked himself rhetorically. But he is not saying that people should switch. He does want them to, but it is "a lot of hard work that could be spent on other things", such as features for their application or web site. Python 2.7 is not dead yet and will be supported with security fixes and, perhaps, security features for the next five years. Porting to Python 3 is a lot of grunge work, so why bother?

Python 3 is a "much better language" than Python 2, for one thing. It is much easier to teach. For example, the Django Girls workshops are all based on Python 3. That wouldn't have been possible if the Django developers hadn't done the grunge work to port the framework. That makes for a more pleasant first experience with the language (and the framework).

It is also getting better over time. There is "lots of cool new stuff" in Python 3.5, for example. Python 2 is a fine language and will remain exactly what it is, which is kind of asymptotically approaching 2.7-perfect, he said. The only way to benefit from all of the work that the core developers are doing is by moving to Python 3.

The perennial question of why not make a Python 2.8 release was next up, though Van Rossum noted that maybe that particular question was getting somewhat dated at this point. Python 2.8 wouldn't solve any of the problems that make people want it. Either it would have no new features, which means there would be no reason not to just stay at 2.7, or the floodgates get opened for backports from Python 3. That would make porting to 2.8 nearly what is needed to port to 3.

Unicode is, of course, the big hurdle to moving to Python 3. But, "enough is enough". So Python 2 is in a state where it will get no new features. That allows the core developers to focus their energy on making Python 3 better.

He then moved on to Python 3.5, which is due in September. He would have trouble choosing a favorite feature from the release because there are "way too many cool things" in it. For example, the performance improvement coming from os.scandir() is great, but it probably won't even be noticed by most. There is a small group of Python users that will be happy to see the new matrix multiplication operator. Numeric packages like NumPy and others will be able to start using it, which will allow writing matrix multiplication in a "much more natural fashion than calling a function".

Perhaps his favorite 3.5 feature should be type hints, since it is a PEP he worked on himself. It took a lot of work for the PEP to get accepted, which is a little bizarre since he is the benevolent dictator for life (BDFL) and could accept his own PEP. But he wanted to have independent review and acceptance of the PEP, which Mark Shannon was graciously willing to provide as the BDFL delegate, he said.

If you caught him unaware, though, he probably would name the new async and await keywords for coroutines as his favorite. It was the last PEP accepted for 3.5 and it provides a more natural way of specifying coroutines.

Open bugs

Someone recently asked him about all of the open bugs in the Python bug tracker. If you pick a random open bug, you will find that it probably has patches attached to it, a bunch of discussion, and even renowned core developers saying that the patches need to go in, but the bug is still not fixed. Is it because of an old boys' network or lame core developers? What needs to be done to get those patches applied?

The situation is the same for any large project, he said. Bugs that can't be closed right away, due to things like misread documentation, tend to pile up. They can be hard to reproduce due to the hardware or environment, for example. But those kinds of bugs don't have patches attached.

There are also bugs that are feature proposals that do have patches attached, but there is a general hesitation to accept changes like that because there is concern that they aren't useful, won't mesh with other similar language features, or that they will cause backward incompatibilities. It is hard to take patches without breaking things all the time.

In addition, the core developers all have a lot on their plates and no one is paid to merge patches for core Python. So if none of the core team cares about a particular patch or feature, they may not find the time to shepherd it through the merging process.

In a company, things are a little different. People are paid to do some amount of grunge work, but with open source you have to volunteer to do unpleasant tasks. Some core developers have been doing that for so long that they want a break from that kind of grunge work. These are some of the many reasons that there are lots of open bugs with long histories in the bug tracker.

Lastly, there is a statistical effect that many overlook. If you pick any bug at random, including closed bugs, you would likely get a closed bug. Many bugs are closed quickly and bugs that are easy to fix tend to get fixed quickly. But the average lifetime of an open bug grows linearly with the age of the project, he said.

The GIL

Someone from the audience asked about the global interpreter lock (GIL), looking for more insight into the problem and how it is being addressed. Van Rossum asked back with a grin: "How much time have you got?" He gave a brief history of how the GIL came about. Well after Python was born, computers started getting more cores. When threads are running on separate cores, there are race conditions when two or more try to update the same object, especially with respect to the reference counts that are used in Python for garbage collection.

One possible solution would be for each object to have its own lock that would protect its data from multiple access. It turns out, though, that even when there is no contention for the locks, doing all of the locking and unlocking is expensive. Some experiments showed a 2x performance decrease for single-threaded programs that didn't need the locking at all. That means there are only benefits when three or more threads and cores are being used.

So, the GIL was born (though that name came about long after it was added to the interpreter). It is a single lock that effectively locks all objects at once, so that all object accesses are serialized. The problem is that now, 10 or 15 years later, there are multicore processors everywhere and people would like to take advantage of them without having to do multiprocessing (i.e. separate communicating processes rather than threads).

If you were to design a new language today, he said, you would make it without mutable (changeable) objects, or with limited mutability. From the audience, though, came: "That would not be Python." Van Rossum agreed: "You took the words out of my mouth." There are various ongoing efforts to get around the GIL, including the PyPy software transactional memory (STM) work and PyParallel. Other developers are also "banging their head against that wall until it breaks". If anyone has ideas on how to remove the GIL but still keep the language as Python, he (and others) would love to hear about it.

PyPy

He was asked about PyPy, whether he used it and whether it might someday become the default interpreter. He does not use PyPy, but he does download it once in a while, plays with it for a few minutes, and likes what he sees. He uses Python in two modes, either writing a short little script to get something done, for which he just uses one of the interpreters he already has built on his system, or as a Dropbox engineer deploying Python code to its cluster.

The Dropbox cluster runs a modified Python 2.7, he said, which elicited audience laughter. "I said it, it is no secret", he said. Some parts of Dropbox do use PyPy where it is faster, but the company is worried that some subtle incompatibility will produce weird bugs that are hard to track down. "We have enough of those already".

PyPy shows that you can execute Python faster than CPython. It also provides a testbed where interesting ideas like STM can be tried. But conservative principles lead people to only use PyPy when they know they need the speed. The problem is that by the time you find that out, you have already deployed to enough machines that it makes switching hard. In that, it is like the problem with switching to Python 3.

Dropbox has a lot of third-party dependencies, some of which cannot even be rebuilt from the sources that it has. That is generally true of any company that has millions of lines of Python in production; he also saw it at Google. That also makes switching hard.

In summary, PyPy is a "really cool project", but there are checkboxes that it needs to satisfy that "are hard to check off". He half-jokingly suggested that perhaps PyPy needed to hire Ola and Ola from Django Girls to help create a bigger community around the project.

Favorites

Five short questions about his favorites was up next. Favorite web framework? He said he only writes one web app in any framework and that the last he tried was Flask. Favorite testing library? He mostly just uses unittest and mock from the standard library. Editor? He uses Emacs, but started out with vi (both of which were greeted with applause, presumably by different sets of audience members). He still uses vi (or Vim) occasionally, but if he does that for five minutes, it takes him fifteen minutes to get used to Emacs again.

What is his favorite language besides Python? He used to say C, but "that's kind of boring". People he trusts tell him that modern C++ is a good language. He likes Go, but hasn't written anything significant in it. From talking with the designers, he likes the looks of Swift, which stole a bunch of things from Python. It is easy to steal bad stuff from languages you like and end up with an incoherent pile of features, but Swift's designers appear not to have done that. Finally, favorite exception? He chuckled and answered KeyboardInterrupt to more applause and laughter.

And what he hates

The final question was about what he hates in Python. "Anything to do with package distribution", he answered immediately. There are problems with version skew and dependencies that just make for an "endless mess". He dreads it when a colleague comes to him with a "simple Python question". Half the time it is some kind of import path problem and there is no easy solution to offer.

With that, his time was up. The EuroPython organizers had a gift for each of the keynote speakers: a Basque beret and bandana. They were presented to Van Rossum at the end of the talk (seen above at right).

Comments (13 posted)

A look at the Fotoxx image editor

Note: several of the project sites linked to below are temporarily offline, evidently due to a storage fault incident at SourceForge. Hopefully they will be back online shortly.

Image editing, like a great many tasks, tends to be dominated at any one time by a small handful of applications. For example, at the moment, the most popular programs for editing photos are probably GIMP, darktable, digiKam, and RawTherapee. A few years ago, the list might have included Rawstudio, F-Spot, Geeqie, and gThumb. Tastes change; individual projects gain and lose steam, and what counts as state-of-the-art technology shifts over time. Nevertheless, at any given time, most of the current crop of applications tend to adhere to the same general approach where user-interface and interaction decisions are concerned. So it is all the more intriguing to encounter a project like Fotoxx, an image editor that takes its own approach. The latest release recently became available, and offers an interesting editing experience.

Fotoxx has been in development since mid-2008. The latest release, version 15.07, was unveiled in early July. Packages are available in Debian format and there are the requisite source-code bundles for other distributions. The only external dependency that might cause some trouble is the raw-photo processing package UFRaw. UFRaw is most often used as a plug-in for GIMP, so it might need to be installed for users who favor some other raw processor.

In some respects, Fotoxx is a standard image-editing tool. It lets the user index their image collection (in place, so images are not needlessly duplicated), browse and search through it, and perform a wealth of corrections and transformations on individual images or in batches.

What makes Fotoxx distinct, though, is its atypical user interface, which has seemingly evolved over the years along a path completely different from that of its contemporaries. Fotoxx uses GTK+ as its GUI toolkit, but it does not sport a tool palette or control panels that resemble any of those used by GIMP, darktable, and the like. In fact, it eschews any number of standard GTK+ practices—such as having a menu bar at the top of the window.

Instead, there is a single strip of buttons down the left-hand side of the application window. Clicking on a button pops up a floating text menu of available features, and almost all of the features open their own pop-up windows filled primarily with text-labeled buttons rather than graphics. For some tools, this is not a big change. Adjusting the hue of the image, for example, requires only a few sliders: the effects of moving those sliders are visible immediately on the image, so no other UI elements are required.

For other tools, the text-label approach is more challenging. The paint/clone tool, for instance, uses text labels to indicate the radius and transparency level of the paintbrush. GIMP's approach is clearer, simply showing a circle of the appropriate size and opacity. The text-driven UI is also interesting because it has some absolute limits: there are some controls where only a graph will do, such as adjusting brightness or tone curves. For those tools, Fotoxx displays a graph.

The other manner in which the Fotoxx UI differs from its contemporaries is that almost all of the pop-up windows it uses to provide features are modal. That is, they sit on top of the main image window and take control of the interface; until you are done with one tool or filter and close its window, you cannot open a different tool. This design decision leads to some interesting side-effects—such as a tendency toward a step-by-step, linear processes to perform editing tasks, as opposed to image tools that you can open up and fiddle with haphazardly.

Some of the standard editing features certainly suffer as a result. When selecting part of an image, Fotoxx pops up a window filled with checkboxes and text buttons that correspond to the various selection modes (drawing boxes, drawing freehand lines, select-by-color, and so on). You can construct your selection area by alternating between the tools available, drawing individual selection regions that appear on the image as red outlines. When you are done, you click on the "Finish" button, then must individually click in each zone of the image to activate the selection. When selecting complex or compound shapes, this becomes tedious quickly.

On the other hand, some features benefit from the approach. For example, the perspective-correction tool is among the simplest of any image editor. You merely click on four points in the image (in any order), then click "Apply" to transform the image so that the quadrilateral you have chosen becomes a proper rectangle. The corresponding GIMP tool uses a floating grid rendered on top of a preview image—which can be difficult to see through and unresponsive to manipulate.

Apart from the novel user interface (which can certainly be deemed a plus or a minus, depending on the individual user's inclinations), Fotoxx excels in the areas of speed and simplicity. Image operations are fast and, importantly, that speed includes showing an immediate preview of the effect in question on the open image. Many of GIMP's effects, by comparison, provide only a thumbnail-sized preview of their output. That can inadvertently add time as you adjust the settings and sliders, then must pan and zoom around the thumbnail preview to see if your changes produce the desired outcome.

Naturally, simplicity is a double-edged sword. Darktable and GIMP (especially GIMP combined with the G'MIC plugin) offer a dizzying array of filters and effects. For simple editing of photos that do not suffer from major defects, darktable and GIMP can be on the sluggish side. Applications that focus on "image-collection management" (such as digiKam and GNOME Photos) generally offer quick-adjustment features, but those applications also have a tendency to try taking over one's image collection even when not asked—scanning your home directory and firing up databases and indexing processes.

Fotoxx fills in nicely for these situations. It does not try to force you into indexing your complete photo archive (or, worse yet, copying it to the application's preferred location), but it provides access to color and tone adjustments, sharpness and noise filters, and standard retouching tools. And, if the adjustments needed do go beyond what Fotoxx itself can handle, the application allows you to open the image in GIMP by clicking on the "Tools" -> "Plugins" -> "GIMP" option.

This is not to say that Fotoxx is lacking in powerful features. Recent releases have added a "World Maps" feature that lets you view all of your geotagged images on a global, zoomable map. There is also support for some specialty image-editing operations like exposure stacking (which is used, for example, to combine multiple dim astrophotography images into a single photo) and panorama stitching. Such specialized features, though, tend to require the user to make a large time investment both behind the camera and in front of the keyboard, so comparing them to the competition (e.g., Hugin) objectively can be difficult.

For everyday editing, however, Fotoxx is well worth keeping around. Its peculiar interface may occasionally cause some frustration, but it also prompts some "why haven't other projects thought of that?" moments.

Comments (6 posted)

Django Girls one year later

Though it got a bit of a late start due to some registration woes, the first day of EuroPython 2015 began with an engaging and well-received keynote. It recounted the history of a project that got its start just a year ago when the first Django Girls workshop was held at EuroPython 2014 in Berlin. The two women who started the project, Ola Sitarska and Ola Sendecka, spoke about how the workshop to teach women about Python and the Django web framework all came together—and the amazing progress that has been made by the organization in its first year.

It all started when Sitarska and Sendecka met while organizing DjangoCon 2013 that was held in their home country of Poland. They decided to hold the event in a circus tent at a horse racing track in Warsaw. They were able to make that event work, which provided a strong bond that obviously continues to this day. Sendecka said that experience is a great way to get to know someone and that they both recognized they could pull off "crazy things" together.

It is, they noted, quite a coincidence that their names are the same length and only differ by a few letters; both also come from the same region of Poland (Silesia). They now live in London and both work as Django developers for a company called Potato. Perhaps unsurprisingly, some people think there is only one "Ola", which results in a fair amount of misdirected email and other types of confusion.

A fable

As a way of introducing Django Girls, they began with a fable of sorts that was illustrated with some eye-catching drawings made by Sendecka (slides at Speaker Deck). The story was about "Liz", who is a squirrel who is fascinated by technology and programming in particular. As she makes her way into the "forest of technology", though, she finds that there are few squirrels who are interested in technology. In fact, most of those who are learning about technology in the forest were all very similar to each other, but much different from her.

In fact, the others were badgers, she discovered. She wanted to fit in, but always stood out from the badger pack. They were generally nice, and she had nothing against badgers, but she could feel the eyes of the badgers on her whenever she entered a room and she was often "complimented" with lines like: "You are not like other squirrels" or "You are pretty good at technology for a squirrel."

She realized that the badgers knew lots about technology, so she thought she should perhaps become more badger-like to fit in. But that didn't change anything, she was still an outsider. At one point she organized a technology event and was asked by a badger whether she was just there for social reasons. She constantly had to prove her knowledge of technology, while badgers who knew far less than her were able to speak up freely.

Liz started to lose confidence in herself, worried that she was too shy and reserved to ever fit into the technology crowd, even though she loved the technical aspects of the field. She was able to make some friends among the badgers, though when she looked closely, she realized they weren't really badgers. Instead, they were really various other types of animals that only just kind of looked like badgers. The support from these friends helped her push on with her studies.

After a while, Liz realized that the problem wasn't with her, it was with the technology forest itself. She had to do so much more than badgers did to get the same recognition, but when she brought that subject up, it was not well-received, or even believed, by the badgers. She asked around about the problem on the internet and got the predictable responses about squirrel's inherent inferiority and inability to ever truly master technology.

Liz despaired of the situation ever changing. She was too tired to fight every day for her place, but then, through a fortunate series of events, she met another squirrel who loved technology. This squirrel was also named "Liz" and they became fast friends. She finally had someone with whom she could share her frustrations of trying to live in the forest of technology. She realized that she is not the only squirrel who loved technology, but was as intimidated and scared by the badger's world as she was.

Liz and Liz started to think about how awesome it would be if there were more squirrels that could get involved with technology and who could learn to love programming the way they did. So they came up with a crazy plan to teach other squirrels how to code. They told all of their friends about the idea and with their support, and the help of lots of others who joined in, the idea became a reality.

In the year since then, thousands of squirrels have learned how to program. It is all because two little squirrels met at the right time and became excited about this crazy idea. Today, they said, Liz is standing on the stage at EuroPython telling the story of this journey and of a dream that became a reality. That dream is Django Girls.

Teaching Python and Django

Sitarska and Sendecka started organizing the first Django Girls workshop just two months before last year's conference. It was targeted at teaching women who were complete beginners and had "never written a line of code in their life". In the year since, they have seen over 1600 women attend these workshops from "literally all over the world".

After a meeting with one of the EuroPython organizers, Sitarska got the idea to run a workshop. She immediately sent Sendecka email asking if she wanted to help. In something of a massive understatement, in the email she said "I don't think it's gonna be a lot of work ...". Sendecka responded "she lied" with a laugh. The idea was to get twenty students and six coaches for the workshop and to use an existing Django tutorial from the internet.

So they started working on putting the workshop together that day. But, most of the time, "we had absolutely no idea what we were doing". The amount of work started to snowball. They came up with the idea of providing financial aid for attendees, which required more sponsors and figuring out how to transfer money to the students.

They also realized that they didn't like any of the tutorials that were out there. All of them assumed that the reader was already a programmer and knew what a web framework is, what a server is, and so on. They ended up writing their own tutorial, which turned out to be roughly 90 pages of text.

In addition, they promoted the workshop in all of the places they could think of and ended up spending a lot of time evaluating the 300 attendee applications. Eventually the workshop expanded to 45 attendees, which meant that 15 coaches were needed as Django Girls has a 3:1 ratio for students to coaches.

The Django Girls tutorial has now been read by over 94,000 people at this point, which is 1/4 the population of Bilbao (where EuroPython 2015 was held), they noted. In total, 1,646 women have learned Python and Django in these workshops. More than 70 have now been held in 34 countries on 6 continents. Nothing, so far, in Antarctica, but "we are working on that too", Sendecka said with a chuckle.

News

The project has grown so much that a legal entity was needed to administer it, so the Django Girls Foundation was started in June. Django Girls has been so successful because of the huge numbers of volunteers who have put in their own time as organizers (137), contributors to the tutorial (111), and as part of a "huge army of coaches" (548). It came to a point where overseeing all of that became too much, so they asked for help and got four more people who are now helping to administer the project; they "help keep us sane".

That still is not enough, they realized, so they have made the big decision to hire the first paid position for the foundation. They are looking for an "Ambassador of Awesomeness" to help ensure that the organization does the best work that it can. It was a bit of a terrifying decision, especially since "we still don't know what we are doing and are making it up as we go". But they are sure it is the right decision to make to ensure that Django Girls "is sustainable and can grow beyond us".

They believe that things are just getting started for the project and that it can have an even bigger impact down the road. They have another big project in the works that they shared for the first time at the conference. The tutorial was enjoyable to write and "to our surprise we must have done a pretty good job". They get many requests for more advanced tutorials or ones that cover other parts of computer science. The tutorial was necessarily geared toward how much can be taught in a single day, so it skimmed over lots of material to make it all fit.

But there is much more about computers and the internet that can be taught, so they are in the process of writing a book. It will try to capture the style of the workshops but cover more material in a book that will be "beautifully crafted". The book is called Yay Python and those interested can follow the progress at that web site. The announcement was met with a loud round of applause and a stage whisper that there is "no going back on that now".

The book will build on the Django Girls tradition, with "a good dose of emoji and funny little quirks". It is still a tutorial on how to build a Django web site, but there will be lots more material, including chapters on topics like "how a computer works" and a "connect the dots" exercise to explain HTTP requests and responses. It will also talk about the communities behind Python and Django, along with how open source works. The idea is to help people "fall in love with the internet as we did".

One rule

The one rule they have adhered to throughout the Django Girls process is to "always go the extra mile" and the details of the workshops show this, they said. The room is decorated with flowers and balloons, rather than being a "boring classroom" and they do fun things like cupcake tasting or yoga during the day. To ensure that organizers have all of the information they need to run the event, and that attendees have a similar experience no matter where the event is held, there is an organizer's guide. There is also a coaching manual. All of that material is open source, so that it can be repurposed as needed. It "took weeks to write down" all of that information, but they want to ensure that Django Girls is bigger than just the two of them.

Workshops are meant to be fun and friendly. Enthusiasm from the organizers, coaches, and participants helps with that. "The power of enthusiasm is huge", they said. When the two of them get excited, that gets passed on to organizers and from there to participants. That is part of what has made Django Girls so successful so quickly.

But every labor of love comes "with a huge emotional cost". It is an enormous job and one that they always feel they are not quite putting enough of themselves into. The thing that keeps them going when they are feeling that way, though, is the stories from women who have literally changed their lives by attending the workshops.

For example, there is Dori, who was sitting in the keynote audience somewhere, and was attending her second EuroPython. At the pilot workshop, she spent half of the time in the workshop trying to get the Django server to play nicely with her Hungarian keyboard. At the time, they thought she had "the worst experience ever" and would probably never want to have anything to do with Django or programming ever again. But today she is a Django developer in Budapest, has organized a monthly meeting for Python developers, and organized two Django Girls workshops there.

Another was Josie, who was only 13 when she attended the first workshop with her mother. Immediately after that, she did a lightning talk at EuroPython in front of a thousand people about her Django Girls experience and her plan to organize the Pyxie Dust project to create a similar workshop for girls her age. That event took place in Zagreb, Croatia in 2014.

Or Linda, who organized a workshop in Nairobi, Kenya just a few months after the initial workshop in Berlin. She "blazed a trail" for other women who organized workshops in Africa after that, they said. They talked about several other women who have "graduated" from attending the workshop to organizing and coaching at others, as well as getting involved in the Python and Django communities in other ways. The Django Girls blog has lots of similar stories as well as event reports and other news.

They concluded their talk with a "huge thank you" to everyone that has helped out along the way. Their dream is for Django Girls not to be needed at some point. It may take 5, 10, or 15 years, but if that can happen, they will have achieved everything they set out to do. That is the ultimate goal for the project.

One thing they learned over the last year is that "kindness is contagious". If you put a lot of energy and effort into something to help others, "the universe will find a way to pay it all back to you". They encouraged others to follow in their footsteps; to find their dream and make it happen. All that it took was "two little squirrels with a big, big dream and a generous, wonderful community" to create something like Django Girls. Remember that "you are not alone, you are among friends", they finished—to thunderous applause.

A YouTube video of the talk is also available.

Comments (10 posted)