Weekly Edition
Return to the Front page
| |||||||||||||
Python slithers into WesnothProposing to change the implementation language for a large project is hardly uncontroversial, but when that proposal calls for moving from C++ to Python, one might expect an enormous flame fest. Surprisingly, a proposal to do just that with the code for the "Battle for Wesnoth" strategy game has resulted in a fairly flame-free discussion. Whether or not the project actually makes the switch—it looks unlikely that any wholesale switch is imminent—there is a great deal of value in the discussion, particularly in its tone. Eric Raymond is the Wesnoth developer proposing this shift, but it is not his "personal fondness" for Python that is behind it. Instead, he sees it as a way to reduce bugs. Raymond has been handling bug triage for the project for the last year or so, which gives him a good grasp of where the Wesnoth bugs tend to be:
I know where we are
vulnerable and where we tend to screw up. And *that* is why I want to
get cracking on shifting as much of the code to a language with true
variable-extent types as possible.
Raymond is cognizant of the downsides of moving to Python as well. First, Wesnoth developers will have to be familiar with both languages, which Raymond, at least, does not view as a problem: "Python is much easier to pick up than C++". Performance is another concern, one that he glosses over with a breezy "machines are still getting faster"—others seem less sanguine about the issue—but he does see two major benefits:
1) No more memory-allocation screwups, *ever*. Python has no pointers
and is garbage collected; Python applications cannot core-dump. The
complex tangle of standard and local custom memory allocators we
presently use, and that are the source of so many of our bugs, will
be chopped away as we move to Python -- and good riddance.
2) I have observed Python code is between 2 and 5 times more compact than C/C++. The higher end of the range is achieved by data-structure-intensive programs like Wesnoth. This is significant because one of the best-established results from large-scale software engineering is that defect-per-KLOC rates in large codebases are *insensitive to the language used*. One of the normal effects of moving to a higher-level language is to decrease the KLOC of the codebase, and as a result to decrease the bug load. Attracting more developers to the project is another reason to move to Python, one that lead developer David White—often referred to by his IRC nickname "Sirp"—is motivated by. Raymond has been thinking about how to move Wesnoth to Python for a while, without making any progress, but recently a new developer, Ivan Illarionov, has appeared on the scene having translated some portions of Wesnoth into Python. Just how much has been done is still something of an open question, but his approach is an evolutionary one. That is important to White:
Rather, we should take an evolutionary approach to matters. Python
already exists in Wesnoth, as an AI framework. Developers who think that
Python would advantage Wesnoth should simply begin implementing
additional components in Python.
If someone is developing a new component for Wesnoth, and that person thinks the component would work best in Python, they should do so. If someone is one of the primary maintainers of an existing implementation of a component, and they feel that component would be more maintainable in Python, then they can re-implement it in Python. Overall, the reaction has been fairly positive, Wesnoth developers seem to be open to the possibility that C++ is not the be-all and end-all of languages for game development. That said, they aren't necessarily willing to hear new developers obnoxiously proclaim that Wesnoth should be redone because Python is "better" than C++, without much in the way of details. Unfortunately, that is the tack that Illarionov has taken, which led White to patiently explain:
I'm going to be really honest: you're presenting yourself in entirely
the wrong way to the project. I don't think very many people care much
to hear "this code here should be in Python because it is better for it
than C++!!!" What we'd much prefer to hear is, "I implemented this
really cool feature which our users will love, and oh yeah...the
implementation is in Python."
Illarionov also ran afoul of Raymond, who publicly castigated him: "You [have] done such an inept and -- at times -- arrogant-seeming job of presenting yourself that you have already alienated some senior developers on this project in just the few days since you've shown up here." Illarionov replied contritely, but almost immediately stirred things up again by a posting with the subject "Wesnoth refactoring and future direction plan". In that thread, he also points to Linus Torvalds's rant about C++, which just gets further under the developers' skin. But, the Wesnoth developers have shown a great deal more patience than many development groups would. As White puts it: "Trust me, if you sent an email to the Linux Kernel Developers Mailing List entitled 'Linux refactoring and future direction', you would receive MUCH more hostile responses then you have received here." It is likely that much of the tone of discussion on the wesnoth-dev mailing list derives from White's leadership; the contrast between his and, for example, Torvalds's more combative tone is quite apparent. There are hints that some of the conversation has been less civil, especially on IRC but, on wesnoth-dev, even the rebukes have been relatively polite—certainly by the standards of most development mailing lists. This is a project struggling with a difficult decision without lashing out at those questioning it or outright opposing it—or for that matter, those ineptly championing it. The wesnoth-dev conversation went quiet around January 6th, but the project is known to be very active on IRC, so perhaps it moved there. Even if that turned ugly, the email conversation sets quite an example. Should it come to pass that Wesnoth starts including more Python—by Raymond and Illarionov or others—we will get an opportunity to see if the hoped-for improvements come about. Projects often consider which language to choose, either initially or for a reimplementation, but there are few examples or case studies of comparative language benefits. A year or two down the road, Wesnoth might provide just that kind of comparison. (Log in to post comments)
Python slithers into Wesnoth Posted Jan 14, 2009 21:34 UTC (Wed) by ncm (subscriber, #165) [Link]
If they're planning to add a lot of Python, I hope they're also adding a lot of automated testing apparatus, and project standards requiring comprehensive tests along with code contributions. Automated testing is good for C++ code, too, of course, but for a dynamic language it's essential, if you hope to be able to make structural changes to the code later. Without, it's just too dangerous to change anything.
Python slithers into Wesnoth Posted Jan 14, 2009 22:13 UTC (Wed) by elanthis (guest, #6227) [Link]
So true.
I particularly loved how when I proclaimed a love for static typing to Python fanatics, I was told that Python is so much better because I "don't have to type superfluous argument type names into function definitions." They conveniently leave off the fact that said typing removes the need for a ton of unit tests for each and every caller of said function to ensure they're passing in the right types.
Every single call to every single module has to be tested _in addition_ to the testing of the module's behavior itself. If you aren't testing everything, you're going to get bit in the ass when you use a fully dynamic language for large-scale application development.
It's wholly different scenario than when using such a language for tasks like serving web pages, where the modules have little interaction. In something like a game, every module interacts with almost every other module. Instead of a simple tree of dependees and dependents, you have a huge web. Unit testing all those interactions is a major PITA, and is so much worse in a dynamic language were you have to test all the things the language can't bother to enforce for you.
Python slithers into Wesnoth Posted Jan 15, 2009 12:51 UTC (Thu) by liljencrantz (subscriber, #28458) [Link]
True enough. I've tried to explain to dynamic typing lovers that static typing is just a special, very powerful meta-language used for creating unit tests that perform input validation. Very useful that; especially how you annotate the code itself with these little unit tests.
On the other hand, Python does have some nice features like lambdas, continuations, etc..
Python slithers into Wesnoth Posted Jan 15, 2009 15:25 UTC (Thu) by lysse (guest, #3190) [Link]
Funnily enough, I'm a preferrer of dynamic typing who got shouted at by fans of static typing for saying more or less the same thing... oh well.
Python slithers into Wesnoth Posted Jan 15, 2009 18:21 UTC (Thu) by cyd (guest, #4153) [Link]
When programming with a dynamical language, you can simply check the type of the argument in the function body, which gives you most of the advantages of type checking (albeit at run-time). The main advantage of dynamic typing, in my mind, is not that you don't have to enter the type of the argument in a function definition. It's that you can write functions that explicitly accept arguments that can be of several possible (carefully specified) types, without messing around with kludges like class inheritance.
Python slithers into Wesnoth Posted Jan 15, 2009 21:35 UTC (Thu) by njs (guest, #40338) [Link]
>It's that you can write functions that explicitly accept arguments that can be of several possible (carefully specified) types, without messing around with kludges like class inheritance.
NB, every statically typed language more recent than C allows for this. (Even C++, via overloading and templates, and C++ is way behind the state of the art in static type systems.)
Python slithers into Wesnoth Posted Jan 15, 2009 22:00 UTC (Thu) by boudewijn (subscriber, #14185) [Link]
Aw, and that already was a pretty polite and respectful discussion,
especially compared to some others...
Python slithers into Wesnoth Posted Jan 15, 2009 22:58 UTC (Thu) by njs (guest, #40338) [Link]
I've read your comment several times, and maybe I'm just being slow today, but I can't figure out what you're implying :-). (Was that intending criticism, compliment, random observation, posted in the wrong place...? I'm lost!)
Python slithers into Wesnoth Posted Jan 16, 2009 16:01 UTC (Fri) by boudewijn (subscriber, #14185) [Link]
I thought the qt is lgpl discussion on lwn was pretty respectful and flamefree compared to some
other discussions I've read recently.
Python slithers into Wesnoth Posted Jan 16, 2009 8:14 UTC (Fri) by liljencrantz (subscriber, #28458) [Link]
Sure it's possible to type check in dynamically typed languages, but but statically typed languages offer a much more compact syntax to do the same thing, so it's much more convenient to do it there. And doing it at compile time is a distinct advantage as well, since it means that even code paths like esoteric error handlers that are extremely rarely run, still get at least some checking.
As to duck typing, it's very convenient sometimes, sure. But as another poster said, many modern statically typed languages support it as well. And quite frankly, it's really easy to mess duck typing up, so I'd much rather do it in a language where it's easy to check that the supplied types are reasonably suitable for the task at compile time.
Python slithers into Wesnoth Posted Jan 18, 2009 12:19 UTC (Sun) by rwmj (subscriber, #5474) [Link]
They conveniently leave off the fact that with type inference you don't need to annotate all the
functions.
Such an opportunity to use a decent functional language here, especially since functional languages
Python slithers into Wesnoth Posted Jan 16, 2009 10:31 UTC (Fri) by eru (subscriber, #2753) [Link] Automated testing is good for C++ code, too, of course, but for a dynamic language it's essential, if you hope to be able to make structural changes to the code later.Does "dynamic" equate with runtime type checks? If not, consider the ML family of languages like SML and CAML, where you get the automatic memory management, pointerlessness and nice built-in dynamic data structures, but they still perform static type checking and compilation. In most cases you don't need to declare types for parameters and variables, because the compiler deduces them for you, eliminating the "less to type" argument.
Python slithers into Wesnoth Posted Jan 16, 2009 18:19 UTC (Fri) by salimma (subscriber, #34460) [Link]
Ocaml, while impressively fast, does not handle threading very well, as far as I know (and the same is true of most current functional languages).
Moving from C++ to a different language is probably harder than moving from, say, C -- most native-compiling languages have a C FFI but not C++.
Both are not insurmountable -- the first can be solved by using a multi-process design (with pipes and shared memory -- not sure how well SHM works with Ocaml though); the second is probably a deal-breaker. Python is an exception thanks to Boost::Python.
Python slithers into Wesnoth Posted Jan 17, 2009 10:18 UTC (Sat) by joe_oblivian (guest, #51538) [Link]
>Ocaml, while impressively fast, does not handle threading very well, as >far as I know (and the same is true of most current functional languages).
Erlang being a notable exception.
Python slithers into Wesnoth Posted Jan 18, 2009 12:21 UTC (Sun) by rwmj (subscriber, #5474) [Link]
No language handles concurrency well (except, maybe Erlang and some very experimental
languages involving transactional memory).
(BTW, shared memory works fine with OCaml - it's even supported by the stdlib).
Python slithers into Wesnoth Posted Jan 14, 2009 22:10 UTC (Wed) by stijn (subscriber, #570) [Link]
I appreciate the analysis made in this article about the tone of the conversation. So many times in
collaborative (open/free) software development and in technical fora or channels the discussion gets from condescending and unpleasant to inflammatory. The attitude sometimes is 'we do not do polite, join in the fray or stay away', which I think is awful. This has probably been analyzed to death in terms of geek culture, meritocracy, and the relative anonymity of online life. It has also been labeled as something that repels a certain gender and people from different cultures, but I am quite certain people in general are put off. The code of conduct is one of the things I really admire in Ubuntu, it is hard to overestimate the value of what lies beneath it.
apropos to another recent story Posted Jan 15, 2009 0:20 UTC (Thu) by sbishop (guest, #33061) [Link]
I tend to read LWN entries chronologically, working my way up in other words. I found this article to be quite refreshing after the reading the comments on the Qt-going-LGPL announcement. Remember, people: Keep it "polite, respectful, and informative"! ;)
Python slithers into Wesnoth Posted Jan 15, 2009 0:51 UTC (Thu) by qg6te2 (guest, #52587) [Link] 1) No more memory-allocation screwups, *ever*. Python has no pointers and ...While pointers are legal code in C++, one can also (and easily) write C++ code where their use is greatly minimised (if not completely eliminated). ... is garbage collected Garbage collection is really a lazy solution for poor coding in C++. If a language (such as Python) needs it due to internal organisation, then so be it. However, ascribing garbage collection as a necessary part of every language is ill informed. The complex tangle of standard and local custom memory allocators we presently use, and that are the source of so many of our bugs, will be chopped away as we move to Python -- and good riddance. One can perfectly use only one allocator in C++. Moving to Python wholesale simply due to this issue is an insufficient reason. Perhaps a more productive approach would be stringent coding standards, where introducing Yet Another Allocator is forbidden. 2) I have observed Python code is between 2 and 5 times more compact than C/C++ Lumping C and C++ together is a mistake. They are two very different beasts, with a large gap in code compactness and type safety. Performance is another concern, one that he glosses over with a breezy "machines are still getting faster" The glossing over is also ill advised, as Python is not suited for any computationally intensive work unless the heavy lifting is done by external modules. While one can use Boost libraries to interface C++ code with Python, there is some loss in terms of type safety and efficiency (as objects may need to copied and/or converted). Overall it appears that the motivation to move to Python is not due to technical reasons, but simply because Python forces a certain coding style. May I suggest that instead of converting from one language to another, the existing code is simply cleaned up ?
Python slithers into Wesnoth Posted Jan 15, 2009 2:04 UTC (Thu) by jd (guest, #26381) [Link]
I would also point out that there -are- stretchy-buffer mallocs, where if you exceed the allocated space, the malloc library automatically traps, reallocs and completes.
I'm not going to critique the choice of Python. On that score, I will only say that there is a difference between the developers not knowing how to do X, Y or Z in a given environment, and X, Y and Z being genuinely impossible in that environment.
I -am- curious, though, as to why there are so many buffer overflows. Buffer overflows can happen for many reasons, not least trying to put data into the wrong part of an array (miscalculating the offset). If the offset is incorrect, avoiding crashes merely allows you to produce garbled output rather than a crash. There is a whole school of thought that says that errors are better off fatal than non-fatal, as non-fatal bugs are a real pain to track down. Debugging them is a nightmare in any language because the errors will slowly build and the distance between the actual bug and where you detect the bug can be huge.
I am also curious as to why there are ever any core-dumps. C++ has exception handling, signals (such as segmentation fault) can be trapped, you can always isolate errors further by having unrelated code running in isolated containers. If a container crashes, it takes out that container until it gets restarted, but doesn't kill everything. In short, if you don't want uncontrolled termination in a program, you don't need to have uncontrolled termination in that program. It's up to you.
All in all, their choice to use Python is ultimately their choice and I'm sure they know what they are doing. The problem is that they do not seem to know what they are not doing.
Python slithers into Wesnoth Posted Jan 15, 2009 5:50 UTC (Thu) by drag (subscriber, #31333) [Link]
I suppose they just want to be lazy programmers and enjoy hacking on what they care about... such as game logic and or better enemy AI, rather then worrying about creating in-code sandboxes to setup and then catch exceptions as well as tracking down memory leaks and buffer overflows.
They probably could do everything that you've guys been talking about, or they could just get rid of C++ and use python. Lazy is usually not to shabby approach at getting something done... People only have so much time to devote to a project like this. It's not like they use it at work or are working on something that is highly important.. it's just a video game and is designed to be a happy pastime.
----------------------
Of course re-doing working code for the sake of just using python is probably a waste of time also.
It's all a question of balance, I guess. Many popular programs use python as a secondary programming language.. like Blender or Gimp or whatever. But it's not like they are going to replace existing or performance critical things for the sake of programming it in python any time soon.
Python slithers into Wesnoth Posted Jan 15, 2009 14:23 UTC (Thu) by NAR (subscriber, #1313) [Link]
I think avoiding pointers in C++ is really not that big deal. I've worked on a C++ project with about 100k LOC and we had about 5 new statements in the whole code (one of them actually led to memory leak). Of course, it depends on the problem field, maybe in a game it's unavoidable.
On the other hand if the developers are not familiar with e.g. STL, but familiar with Python, by all means they should start using Python more and more.
Python slithers into Wesnoth Posted Jan 15, 2009 22:02 UTC (Thu) by boudewijn (subscriber, #14185) [Link]
Is that open source? I would like to study it. In Krita, we use shared
pointers a lot -- but that's also just papering over new statements.
Python slithers into Wesnoth Posted Jan 15, 2009 22:57 UTC (Thu) by njs (guest, #40338) [Link]
shared_ptr<> is handy -- much nicer than the traditional way of tossing delete statements about and praying -- and sometimes the only option, if you really have objects with arbitrary and unpredictable extent. (Somewhat uncommon but very handy jargon: "extent" = "the chunk of your program's runtime during which the object exists".)
But depending on the program, often most of your variables do not have arbitrary extent at all; the example I'm familiar with is "monotone", which is OSS, and basically doesn't call new or indeed use pointers at all. (This also means that you never segfault -- because segfaults are caused by dereferencing pointers, so if you never do that...) I know of two main tricks. The obvious one is to use containers (from STL or whatever), that manage their own memory. Duh. The more subtle one is about function return values: one of the main places you end up using unnecessary pointers is when a function needs to return a complex object, you don't want to copy it around everywhere, so you make the function heap-allocate it and then return a pointer. Then you end up using pointers for all your local variables, object fields, etc., because they get filled in by calling one of those functions. The better convention is to define such functions to take their return value as a reference argument -- the caller allocates it however it wants (on the stack, nested inside another object, whatever), and then passes it to the callee to be filled in.
There are probably some other ones I'm forgetting too... but anyway, good luck! C++ is *so* much nicer if you don't use pointers.
Python slithers into Wesnoth Posted Jan 15, 2009 23:42 UTC (Thu) by tialaramex (subscriber, #21167) [Link]
the caller allocates it however it wants
That's all very well if you don't care about abstractions or compatibility.
The advantage of the pointer is that when you change your rectangles from being two 16-bit co-ordinate pairs to one 32-bit co-ordinate pair and a 16-bit size pair, you know the rest of the code doesn't care about the difference between it was dealing with an abstract pointer, not a transparent structure with temptingly accessible named contents. And you know code compiled against the earlier version of the library won't now crash because the allocations are too small, because you're doing the allocations of the structure whose size changed in your library, which is also the only place accessing the structure contents.
Of course you don't /have/ to solve this with pointers. There's an equally valid tradition elsewhere of always having an array or a list, and returning only an opaque offset into the array or list. The calling code needn't care which convention you use.
"You never segfault" is one of those fun promises which assumes on the one hand, that if you'd done things some other way you'd have made mistakes (a correct program needn't take a segfault, at least not a fatal one) yet if you do things the supposedly superior way you won't make the equivalent mistakes. A fatal error is just as fatal if it's the runtime saying you accessed an uninitialised array member as if it's the OS saying you accessed an unmapped address.
-- And yes, maybe you could trap and ignore the array access error, but then your program will probably do something unpredictable and difficult to debug, just as it would if you signal(SIGSEGV, SIG_IGN);
Python slithers into Wesnoth Posted Jan 16, 2009 0:02 UTC (Fri) by stijn (subscriber, #570) [Link] A fatal error is just as fatal if it's the runtime saying you accessed an uninitialised array member as if it's the OS saying you accessed an unmapped address. Yep. In my early days of C I was told: a segfault is what you get if you are lucky. Silent data corruption is worse.
Python slithers into Wesnoth Posted Mar 15, 2009 16:14 UTC (Sun) by anomalizer (subscriber, #53112) [Link]
Anyone who has done serious C coding will agree with that.
Python slithers into Wesnoth Posted Jan 16, 2009 4:44 UTC (Fri) by njs (guest, #40338) [Link]
>That's all very well if you don't care about abstractions or compatibility.
I had to stare at this for a while to figure out what you were saying...
I think you mean, "that's all very well if you aren't claiming to return a Foo but in fact returning a FooBar (a subclass of Foo chosen at runtime)"? That's a good point. In practice I find that the only times dynamic dispatch (virtual methods) pull their weight in C++ are when using the "strategy" pattern, and then it's usually the caller passing the subclass instance down and references work fine, but that's for my code in my projects; YMMV. I'm happy to add that to the list of exceptional cases where (smart) pointers are called for.
But in response to the rest of your comment, giving the calling code the ability to allocate the object does not mean that you have violated any abstractions. If my code says "give me a rectangle", and someone changes the size of a rectangle in the header file, then my code will allocate a differently sized object from then on. Binary (as opposed to source) compatibility is a little more work, but it just requires the usual pimpl technique that you usually need anyway if you want binary compatibility in a C++ library.
And of course there are always trade-offs, but that doesn't mean this is a zero-sum game. Pointer arithmetic/memory management is one of the most error-prone aspects of classic C/C++ programming (probably only threads are worse); eliminating it will, all else being equal, reduce your bug rate -- or at least, it worked for me.
It's also true that the other way you can segfault is by walking off the end of an array (well, arguably this is a special case of pointer arithmetic, but whatever). The answer to that is even easier: along with not using pointers, don't use arrays :-). Avoiding pointers in C++ takes some tricks, but avoiding arrays (and unchecked wrappers around them) is trivial and worthwhile. It won't stop you making indexing mistakes, but it can turn unreliable crashes/random memory corruptions into reliable clean exits that print the offending file and line number.
Python slithers into Wesnoth Posted Jan 18, 2009 12:25 UTC (Sun) by rwmj (subscriber, #5474) [Link]
shared_ptr<> is just reference counting, ie. the worst, slowest, most invasive form of garbage
collection. Please don't confuse this with advanced garbage collectors.
Python slithers into Wesnoth Posted Jan 18, 2009 20:44 UTC (Sun) by zlynx (subscriber, #2285) [Link]
Reference counting is "worst" or "best" depending on your requirements. The kind of garbage collection all the "modern" JVMs like to do requires about FIVE TIMES as much RAM as reference counting.
Speaking of garbage collection, it is using reference counting itself. The difference is that a collection cycle counts up the references on the spot instead of tracking them continuously.
The Linux kernel uses reference counting quite a lot. It isn't the only thing used in the kernel, of course. RCU is another RAM hungry technique used there. But the counting seems to work rather well in a kernel, much better than garbage collection.
The Boost collection has some other reference counting options that can work better than shared_ptr, like the intrusive pointers which store the reference count inside the object, giving much better cache locality.
Python slithers into Wesnoth Posted Jan 18, 2009 21:29 UTC (Sun) by rwmj (subscriber, #5474) [Link] I'm assuming by "FIVE TIMES" you're referring to this paper. Unfortunately the paper is flawed with respect to reference counting, because it doesn't include the reference counts - it just assumes that the manually managed version "knows" somehow exactly when to free stuff. With ref counts you bloat every object by 4-8 bytes, and of course that has an effect on cache and memory, which they don't measure. You can find a more detailed response here.
Python slithers into Wesnoth Posted Jan 19, 2009 3:29 UTC (Mon) by njs (guest, #40338) [Link]
> shared_ptr<> is just reference counting, ie. the worst, slowest, most invasive form of garbage collection. Please don't confuse this with advanced garbage collectors.
Good points -- but I'm not sure who you're arguing with?
(Though of course everything old is new again -- I have heard that some cutting-edge true garbage collectors use reference counting because it allows them to optimize reference graph walking -- there cannot exist an unreachable cycle rooted at any object that has not recently had its reference count decremented to a non-zero value. I just like sharing that because while almost certainly useless to know, it's a *super* neat trick.)
Python slithers into Wesnoth Posted Jan 16, 2009 10:40 UTC (Fri) by NAR (subscriber, #1313) [Link]
No, it's not open source.
Python slithers into Wesnoth Posted Jan 15, 2009 7:33 UTC (Thu) by k8to (subscriber, #15413) [Link]
Right, just make the code bug-free.
Tone of Discussion Posted Jan 15, 2009 7:42 UTC (Thu) by cmot (subscriber, #53097) [Link]
Could somebody please write a "Debian refactoring and future direction"
article and publish it in a prominent place?
Debian refactoring and future direction Posted Jan 15, 2009 8:16 UTC (Thu) by DeletedUser32991 ((unknown), #32991) [Link]
Last weekend I actually posted an overly long and whiny list of things that I believe are at the root causes of not wanting to be involved any more for anyone who cares. You could add "let's get rid of those" to any of them and call it a plan for future direction. But I also had the decency to take my blog off Planet Debian after I resigned, so its not at a prominent place.
Debian refactoring and future direction Posted Jan 16, 2009 10:16 UTC (Fri) by syntaxis (guest, #18897) [Link]
"machines are still getting faster" Posted Jan 15, 2009 16:18 UTC (Thu) by fandom (subscriber, #4028) [Link]
Are they really? This last year we got a flood of netbooks and mobile
gadgets which are actually quite slower.
Eventually they will be as fast as a current desktop computer, but it
"machines are still getting faster" Posted Jan 15, 2009 16:32 UTC (Thu) by droundy (subscriber, #4559) [Link]
That is an excellent point, and one I wish more software developers would consider. Of course, the new netbooks are still far faster than any of *my* computers, but it'll be a relief when the end of Moore's law means my computer can stop being obsoleted after ten years. Back in the day, one of the advantages of linux was being able to run a modern system on older hardware. Fortunately, Vista has made it relatively easy to stay ahead of Microsoft, but we can't always rely on Microsoft compensating for poor choices by free software developers.
"machines are still getting faster" Posted Jan 15, 2009 18:11 UTC (Thu) by dennisdjensen (subscriber, #25165) [Link]
Nitpicking: Moore's law ended in 2003 when the curve broke.
Software gets slower much faster than hardware gets faster. In a few years assembler will get a
Kudos to White for handling this with such patience, kindness and respect.
I recently used C++ on a large project. Even though I know how to code decently in it, it
Good analysis by the way! I've subscribed to Linux Weekly News for years now, and it's still top
"machines are still getting faster" Posted Jan 15, 2009 22:05 UTC (Thu) by paulj (subscriber, #341) [Link] Nitpicking: Moore's law ended in 2003 when the curve broke. I think you may be confused about Moore's law: it's not about clock-speed, rather it's about transistor counts. I'm reasonably sure Moore's law still applies. It's being expressed in increasing number of functional units/chip rather than clock cycles..
"machines are still getting faster" Posted Jan 17, 2009 18:39 UTC (Sat) by man_ls (subscriber, #15091) [Link] So true. Another way of looking at it (instead of doubling the number of transistors every 18 months) is how big each transistor is; the size should divide by 2 every 3 years. How has it worked out in practice? Most companies have now built 45 nm fabs, from 65 nm; the 90 nm process of 6 years ago is quite outdated. Notice that it took 6 years instead of the predicted three; it seems that there was a noticeable bump at 65 nm, and the jump from 90 nm took 3 years instead of 1.5.There is still plenty of room at the bottom. According to wikipedia, we are still 6 to 13 years away from the 11 nm process; and that could lead to even smaller nanoelectronics processes, where transistors are packed even more tightly. I predict that the wrist computers of 2030 will be cleverer than your average HAL of 2001, and that hearing aids will run a flavor of Debian in their 128 cores.
"machines are still getting faster" Posted Jan 15, 2009 23:26 UTC (Thu) by zooko (subscriber, #2589) [Link]
My sons both love Wesnoth, and it runs very slowly and then locks up when they try it on their OLPC XOs, because it uses too much memory. Whether using more Python and less C++ in Wesnoth will make that problem worse or better remains to be seen.
"machines are still getting faster" Posted Jan 16, 2009 10:21 UTC (Fri) by eru (subscriber, #2753) [Link] Eventually they will be as fast as a current desktop computer, but it will take some years. ... by which time desktop computers and the expectations of software for them will have grown even more hungry! The CPU and memory specs of the EEE PC (any version) would have been quite decent for a 1999 desktop computer buyer. I got a 633 Mhz Celeron system at that time with 256K RAM, and it was plenty for the typical OS'es (both Windows and Linux) and applications of the time.
"machines are still getting faster" Posted Jan 16, 2009 22:38 UTC (Fri) by nix (subscriber, #2304) [Link]
256K RAM in 1999? 256Mb, surely ;)
"machines are still getting faster" Posted Jan 17, 2009 21:12 UTC (Sat) by eru (subscriber, #2753) [Link] Yes, of course 256Mb. Actually I used it at home until 2 years ago (with the minor upgrades of adding a DVD burner and changing the Celeron to the corresponding Pentium III, giving a slight speedup).
"machines are still getting faster" Posted Jan 22, 2009 11:51 UTC (Thu) by Janne (guest, #40891) [Link]
You should only compare machines that are actually comparable. You can't take a desktop/laptop from 2007, and compare it to netbook from 2008 and proclaim "computers are getting slower!". If you want to do such a comparison, you should compare desktops to desktops, laptops to laptops and netbooks to netbooks.
That said, I really can't see how performance of the computer could be an issue in a game like Wesnoth. Maybe for some really, EEALLY low-end computers, but should the goal of a game (any game) be that "it must be playable on every computer in existence, no matter how old or slow!"?
Python slithers into Wesnoth Posted Jan 15, 2009 16:42 UTC (Thu) by sayler (guest, #3164) [Link]
"Eric Raymond is the Wesnoth developer proposing this shift"
I feel an ELER episode coming on..
Python slithers into Wesnoth Posted Jan 15, 2009 17:50 UTC (Thu) by roblucid (subscriber, #48964) [Link]
ESR does come out of this one much better, than past coverage of Kernel Configurator and the Fedora bust up. He's obviously been passionate about Python, but he appears to have consulted, and been sensitive to the project.
Really the way David White handles is admirable. Normally this kind of story, treats you to a very nasty flame fest, like Theo de Raadt v Richard Stallman.
But Linus does make awfully good copy :)
Python slithers into Wesnoth Posted Jan 15, 2009 18:42 UTC (Thu) by drag (subscriber, #31333) [Link]
Every once and a while ESR does do something pretty good.
I like Gpsd, for example (I don't know how responsible exactly he is for it, but it's something that he's worked on at least). It's a nice approach to the problem instead of the traditional method of simply programming individual gps device support into the gps-using applications. Then you get network support to boot.
Python slithers into Wesnoth Posted Jan 15, 2009 23:13 UTC (Thu) by roblucid (subscriber, #48964) [Link]
ESR also goes out of his way to share small stuff likes small tools, that are handy but aren't going to earn kudos.
One I have is a bash script, for simulating RH's chkconfig for init script link management in an rc.d directory, complete with a man page. When I found it in email archive, it was about a month later so I wasn't able to ask him if he wouldn't mind re-writing it in python *wink*
If ESR reads this, then belated thanks for that contribution.
Python slithers into Wesnoth Posted Jan 15, 2009 18:49 UTC (Thu) by bronson (subscriber, #4806) [Link]
"You [have] done such an inept and -- at times -- arrogant-seeming job of presenting yourself..."
This is ESR saying this???
Soul mirrors Posted Jan 17, 2009 18:51 UTC (Sat) by man_ls (subscriber, #15091) [Link] It is sad that mirrors of the soul do not exist. I am sure we could all use one.
Python slithers into Wesnoth Posted Jan 15, 2009 22:57 UTC (Thu) by jengelh (subscriber, #33263) [Link]
Computers getting faster is an illusion. Games like Wesnoth provide a refuge for older boxes that, for example manage to run the Quake3 engine, get shaky at Unreal2003 and just do not make a usable game out of America's Army, to give a perspective what we're talking 'bout. And now he is gonna take even that away by making it slower - and the P programming functions with their current VMs are ten or more times slower than pure-compiled stuff and the SunJava JIT engine.
And then this: the X Window System sucks for 2D graphics - I have seen a GL context being more effective (more F/s) in low-end graphic game (-> ZDoom) than drawing to Xshm. (Most of these apps do not support output to Xvideo; and who knows how much extra cycles would be needed just to make SDL convert RGB to Xvideo acceptable formats like YUV.) Even if it were not for X, other programs running in the background (say, an audio player) would suffer. Or that kernel compile (running at nice 20).
Python slithers into Wesnoth Posted Jan 15, 2009 23:17 UTC (Thu) by roblucid (subscriber, #48964) [Link]
> Computers getting faster is an illusion
Is it Ted Tso's law that would be something like, no matter how much faster your hardware is, a kernel compile will always take me roughly 10 minutes.
Python slithers into Wesnoth Posted Jan 15, 2009 23:51 UTC (Thu) by alankila (subscriber, #47141) [Link]
The texture-from-pixmap extension should allow one to use RGBA (or more likely, ABGR) surface which is directly usable by the display hardware for displaying. It's still not as good as something like DirectX which gives memory straight from the graphics adapter for application to use. That being said, it is not such a big problem because texture fill is still insanely fast on any GPU, so it isn't a large problem to require the graphics card to blit it when the image is ready. In contrast, CPU-based data blitting is always terribly slow because of low system memory bandwidth, and can't be optimized in any meaningful way...
As I see it, the trouble with XSHM pixmaps is that they aren't actual pieces of display surface. Something still needs to blit it before it finally is on the screen surface, and for reason or other it always seems to be the main CPU, and not the GPU. I also strongly suspect the XSM pixmaps are physically stored in the system RAM, and not the graphics card RAM. One likely explanation for this is that reading from display RAM by the main CPU seems to be very slow. (You can make this mistake quite easily with DirectX surfaces -- reading from display is a lot like hitting denormal numbers on FPU: suddenly everything begins to crawl...)
So in the end, XSHM may save one copy from application's memory to X server's memory, perhaps, but there are many copies left, all which are dreadfully expensive.
Xvideo indeed is quite marvelous when it comes for displaying data. Unfortunately the U and V channels have lower bandwidth compared to Y, limiting chromatic accuracy. On the other hand, gamma correction is done by hardware, which can be awfully convenient, and you get tint and brightness/contrast correction controls for free while at it. On the whole, I do think that it may be the only reasonable way for displaying video data on X.
Computers getting faster Posted Jan 16, 2009 0:13 UTC (Fri) by giraffedata (subscriber, #1954) [Link] Yeah, I assure you my computers never get faster. I end up having to replace them constantly because applications like this keep getting slower.
Python slithers into Wesnoth Posted Jan 16, 2009 15:57 UTC (Fri) by Frej (subscriber, #4165) [Link]
At least some commercial games are done in c++ and python, like Civ4 and the new Pirates.
Probably others, as it's a third party game engine.
Python slithers into Wesnoth Posted Jan 18, 2009 10:56 UTC (Sun) by massimiliano (subscriber, #3048) [Link] Well, I know this could cause flaming responses, and I know I'm biased because I work on the project, but... ...my personal choice as a VM to write game logic would be Mono.
It is cross platform, it's JIT is typically much more efficient than interpreters, and the languages you can use to write your game can be statically or dynamically typed according to what you need.
Now, I'm not saying that Wesnoth should use Mono: if they already have Python experience, and Python is already used in the project, then it's probably a better choice (switching to a different VM would cost too much).
Have fun!
Python slithers into Wesnoth Posted Jan 18, 2009 20:32 UTC (Sun) by zlynx (subscriber, #2285) [Link]
I believe Quake2 used native code DLLs and .so files.
But in general you are right. Using an interpreter gives the game better security, portability and error recovery.
Python slithers into Wesnoth Posted Jan 18, 2009 21:01 UTC (Sun) by massimiliano (subscriber, #3048) [Link] Oh, you are right, I was forgetting Quake 2!
But for Quake 3 they got back to the VM approach... with a twist: the language was C (plain, old C), but they had a compiler from C to bytecode, and an interpreter able to execute that bytecode.
...which tells a lot about the fact that they really, really wanted a VM to solve the safety and platform compatibility issues for game mods. They decided to use a VM even if they preferred the C language for game logic!
Have fun,
Python slithers into Wesnoth Posted Jan 18, 2009 20:58 UTC (Sun) by zlynx (subscriber, #2285) [Link]
Another comment I just thought of about Mono and Python.
I've played around with IronPython on Microsoft's .NET. It is impressive, in my opinion.
It's much faster than cpython and threading really works. Or so it seemed to me.
I think that even if Wesnoth wants to stay with Python, it should be possible to use Python on top of Mono via IronPython.
Python slithers into Wesnoth Posted Jan 18, 2009 21:13 UTC (Sun) by massimiliano (subscriber, #3048) [Link] Yes, IronPython is impressive! I think that even if Wesnoth wants to stay with Python, it should be possible to use Python on top of Mono via IronPython.
Yes, it would be possible to do so.
That said, Mono would of course be usable :-)
Have fun,
|
|||||||||||||