|| ||"Eric S. Raymond" <esr-AT-snark.thyrsus.com>|
|| ||1.7 design direction and the great Python shift|
|| ||Fri, 2 Jan 2009 10:19:26 -0500 (EST)|
|| ||Article, Thread
We have been presented with what I think is a very important
A potential contributor, Ivan Illarionov, reports that he has
translated a substantial portion of Wesnoth into Python and has a
version running with several key framework classes in Python. In his
words: "It replaces parts of WML and parts of C++, and yes, game
controller, play controller are all implemented in Python, only core
game engine is C++"
This cracks open a problem I've been thinking about for months but
have not been able to make headway on because I've been too busy
fighting fires: Moving as much as possible of the Wesnoth codebase
out of C++ to Python.
== Pros and Cons of Python ==
Sirp agreed in principle on IRC with moving the Wesnoth codebase
towards Python many months ago, basically in order to attract a larger
population of developers - because, even though Sirp doesn't
particularly *like* Python, he knows it's more accessible than C++.
I will note for the record that though I personally *like* Python and
have come to dislike C++ intensely since learning it for Wesnoth,
these are not actually the reasons uppermost in my mind for wanting to
merge Illarionov's code. Nor is Sirp's reason. Here is how I view the
The main reason to hesitate before moving core code to Python is that
it will require Wesnoth developers to know two languages before
working on the code. I do not view this as a major problem, as Python
is much easier to pick up than C++; I learned it in four days.
A lesser thing to worry about is performance. Python is an
interpreted language and relatively slow. Experience with Python AIs
has shown that this is not a blocker in practice, and machines
are still getting faster.
I think Sirp's reason for endorsing a language he's not really
personally a fan of (attracting a larger developer base) is sound, but
I think there is an even more important one: defect reduction.
Most of you know that I have taken primariy responsibility for
triaging bugs and keeping the tracker bug list clean over the last
year or so; if you get a bug assigned to you, it was likely I that
did it. I have also engaged in several major bug hunts during which
I've personally dispatched a couple of hundred bugs.
Accordingly, I probably have a more detailed sense of the project's
defect patterns than anyone else now active. I know where we are
vulnerable and where we tend to screw up. And *that* is why I want to
get cracking on shifting as much of the code to a language with true
variable-extent types as possible.
Python has two huge advantages over C++ that are relevant to defect
reduction, especially in a program like Wesnoth that does a lot of
tricky stuff with data structures:
1) No more memory-allocation screwups, *ever*. Python has no pointers
and is garbage collected; Python applications cannot core-dump. The
complex tangle of standard and local custom memory allocators we
presently use, and that are the source of so many of our bugs, will
be chopped away as we move to Python -- and good riddance.
2) I have observed Python code is between 2 and 5 times more compact
than C/C++. The higher end of the range is achieved by
data-structure-intensive programs like Wesnoth. This is significant
because one of the best-established results from large-scale software
engineering is that defect-per-KLOC rates in large codebases are
*insensitive to the language used*. One of the normal effects of
moving to a higher-level language is to decrease the KLOC of the
codebase, and as a result to decrease the bug load.
I will finish my discussion of pros and cons by noting that my
personal fondness for Python is not completely irrelevant. I can
write open-source code full-time, and Wesnoth has attracted the lion's
hare of my coding hours for two years now. I am thus the closest
thing Wesnoth has to a full-time developer, and anything that
increases my productivity (like allowing me to code in Python rather
than C++) will have a correspondingly large positive effect on the
== Getting There From Here ==
It would be crazy to try an architectural change this major as we're
in the run-up to 1.6. Especially since Mordante, who is the most
reliably present-for-duty core developer on C++ other than me, has his
hands full making the new widget code work.
I can, however, review and test Ivan's code prior to merging and have
already cloned a copy of his git repository to do that. loonycyborg
is also testing.
I think integrating Ivan's code should be our main project for
early in the 1.7 cycle. I am willing to take primary responsibility
for reviewing and merging; actually, I can't imagine a better use of
my project time than that. I suspect loonycyborg will be taking an
active role in the merge as well.
We have another technical problem; GNA blocks Ivan's ISP for some
unknown reason. Somehow we need to straighten that out so he can have
<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>
A nation or civilization that continues to produce soft-minded men
purchases its own spiritual death on an installment plan.
--Martin Luther King, Jr.
to post comments)