|| ||David White <dave-AT-whitevine.net>|
|| ||dev-talk <wesnoth-dev-AT-gna.org>|
|| ||Re: 1.7 design direction and the great Python shift|
|| ||Fri, 02 Jan 2009 11:22:25 -0800|
|| ||Article, Thread
I think that moving some of our code to Python is an idea worth
considering. However, I don't think it needs to be a "big dramatic
decision" that we need some kind of consensus on and will change Wesnoth
forever. I think that making a fiat decision like this would not end
well, and would breed resentment between those preferring C++ and those
Rather, we should take an evolutionary approach to matters. Python
already exists in Wesnoth, as an AI framework. Developers who think that
Python would advantage Wesnoth should simply begin implementing
additional components in Python.
If someone is developing a new component for Wesnoth, and that person
thinks the component would work best in Python, they should do so. If
someone is one of the primary maintainers of an existing implementation
of a component, and they feel that component would be more maintainable
in Python, then they can re-implement it in Python.
Ideally, we would start by only implementing one or two major components
in Python. The network system, for instance, could be re-written in
Python. Or, perhaps even better, we could implement an interface to
generate maps in Python, and re-write our random map generators in
As components are re-written in Python, developers could note how
maintainable these components are, and then choose whether or not to
migrate additional components to Python.
This approach is simple, evolutionary, and allows a 'bottom-up' rather
than 'top-down' approach, which is how Wesnoth has always tended to
On Fri, 2009-01-02 at 10:19 -0500, Eric S. Raymond wrote:
> We have been presented with what I think is a very important
> A potential contributor, Ivan Illarionov, reports that he has
> translated a substantial portion of Wesnoth into Python and has a
> version running with several key framework classes in Python. In his
> words: "It replaces parts of WML and parts of C++, and yes, game
> controller, play controller are all implemented in Python, only core
> game engine is C++"
> This cracks open a problem I've been thinking about for months but
> have not been able to make headway on because I've been too busy
> fighting fires: Moving as much as possible of the Wesnoth codebase
> out of C++ to Python.
> == Pros and Cons of Python ==
> Sirp agreed in principle on IRC with moving the Wesnoth codebase
> towards Python many months ago, basically in order to attract a larger
> population of developers - because, even though Sirp doesn't
> particularly *like* Python, he knows it's more accessible than C++.
> I will note for the record that though I personally *like* Python and
> have come to dislike C++ intensely since learning it for Wesnoth,
> these are not actually the reasons uppermost in my mind for wanting to
> merge Illarionov's code. Nor is Sirp's reason. Here is how I view the
> The main reason to hesitate before moving core code to Python is that
> it will require Wesnoth developers to know two languages before
> working on the code. I do not view this as a major problem, as Python
> is much easier to pick up than C++; I learned it in four days.
> A lesser thing to worry about is performance. Python is an
> interpreted language and relatively slow. Experience with Python AIs
> has shown that this is not a blocker in practice, and machines
> are still getting faster.
> I think Sirp's reason for endorsing a language he's not really
> personally a fan of (attracting a larger developer base) is sound, but
> I think there is an even more important one: defect reduction.
> Most of you know that I have taken primariy responsibility for
> triaging bugs and keeping the tracker bug list clean over the last
> year or so; if you get a bug assigned to you, it was likely I that
> did it. I have also engaged in several major bug hunts during which
> I've personally dispatched a couple of hundred bugs.
> Accordingly, I probably have a more detailed sense of the project's
> defect patterns than anyone else now active. I know where we are
> vulnerable and where we tend to screw up. And *that* is why I want to
> get cracking on shifting as much of the code to a language with true
> variable-extent types as possible.
> Python has two huge advantages over C++ that are relevant to defect
> reduction, especially in a program like Wesnoth that does a lot of
> tricky stuff with data structures:
> 1) No more memory-allocation screwups, *ever*. Python has no pointers
> and is garbage collected; Python applications cannot core-dump. The
> complex tangle of standard and local custom memory allocators we
> presently use, and that are the source of so many of our bugs, will
> be chopped away as we move to Python -- and good riddance.
> 2) I have observed Python code is between 2 and 5 times more compact
> than C/C++. The higher end of the range is achieved by
> data-structure-intensive programs like Wesnoth. This is significant
> because one of the best-established results from large-scale software
> engineering is that defect-per-KLOC rates in large codebases are
> *insensitive to the language used*. One of the normal effects of
> moving to a higher-level language is to decrease the KLOC of the
> codebase, and as a result to decrease the bug load.
> I will finish my discussion of pros and cons by noting that my
> personal fondness for Python is not completely irrelevant. I can
> write open-source code full-time, and Wesnoth has attracted the lion's
> hare of my coding hours for two years now. I am thus the closest
> thing Wesnoth has to a full-time developer, and anything that
> increases my productivity (like allowing me to code in Python rather
> than C++) will have a correspondingly large positive effect on the
> == Getting There From Here ==
> It would be crazy to try an architectural change this major as we're
> in the run-up to 1.6. Especially since Mordante, who is the most
> reliably present-for-duty core developer on C++ other than me, has his
> hands full making the new widget code work.
> I can, however, review and test Ivan's code prior to merging and have
> already cloned a copy of his git repository to do that. loonycyborg
> is also testing.
> I think integrating Ivan's code should be our main project for
> early in the 1.7 cycle. I am willing to take primary responsibility
> for reviewing and merging; actually, I can't imagine a better use of
> my project time than that. I suspect loonycyborg will be taking an
> active role in the merge as well.
> We have another technical problem; GNA blocks Ivan's ISP for some
> unknown reason. Somehow we need to straighten that out so he can have
> dev access.
to post comments)