LWN.net Logo

Wingo: the gnu extension language

Guile maintainer Andy Wingo re-examines its suitability as the official GNU extension language, showing off a number of useful features on the way. "Delimited continuations let a user extend their code with novel, expressive control structures that compose well with existing code. I'm going to give another extended example. I realize that there is a risk of losing you in the details, dear reader, but the things to keep in mind are: (1) most other programming languages don't let you do what I'm about to show you; and (2) this has seriously interesting practical applications." (The article is a few months old, but we had not encountered it previously).
(Log in to post comments)

Wingo: the gnu extension language

Posted Nov 26, 2011 17:24 UTC (Sat) by imgx64 (guest, #78590) [Link]

Well, that was disappointing.

I was expecting an article about what extension languages in general can do, why you should add Guile to your next program, and how Guile can be used to easily extend current programs (say, give an example of adding a feature to a program with few lines of code). Because frankly, ever since I heard of Guile, I haven't seen it anywhere useful.

But instead, we get a generic lisp-is-timeless-and-imperative-languages-suck article that I've read a million times before, with short blurbs of incomprehensible macro code no less!

Seriously, can someone give me a reason why Guile hasn't been a complete waste of time? And why hard-wiring a single extension language into your program is better than exposing the functionality of your program as a library or on D-Bus where it can be used by almost any language?

Wingo: the gnu extension language

Posted Nov 26, 2011 17:36 UTC (Sat) by coriordan (guest, #7544) [Link]

> Seriously, can someone give me a reason why [...] hard-wiring a single extension language into your program is better[...]?

One of Guile's goals is to include translators so that extensions can be written in Javascript, Scheme, Emacs lisp, Lua etc. and Guile will translate them as necessary and execute them. Those four languages currently work, fully or partly.

My question is whether they'd be better off using a "machine/assembly" language as the base (such as Parrot).

Wingo: the gnu extension language

Posted Nov 26, 2011 22:06 UTC (Sat) by wahern (subscriber, #37304) [Link]

One of the many beauties of Lua is its virtual machine. Lua 5.2 is fully yieldable, even across multiple C function invocations--which can store and restrieve a continuation context for resuming. IMHO, the best extension language is Lua. The second best would be a language which compiled to Lua bytecode (and you could still have your Lisp macros to boot.)

My only concern about Lua 5.2 as an extension language is handling arithmetic overflow when moving values to/from C. Everything else about the language makes integration with C almost completely effortless. But it's an issue, it's worth noting, that other extension languages also share, so it's hardly a liability in comparison.

Lua is miles ahead of everything else as an extension language. And although it's looks and acts procedural, it supports proper tail calls**, so there's no need to sacrifice elegance for practicality.

I use Lua 5.2--to be officially released very soon--because it's fixed all the headaches of Lua 5.1, and it's been a joy to work with. But Lua 5.1 is still heads and tails above all the others.

[**] At least in the Lua VM. I've spotted patches to support tail calls across C function invocations as well. It's this level of detail and capability which makes Lua shine as an extension language.

Wingo: the gnu extension language

Posted Nov 27, 2011 10:47 UTC (Sun) by elanthis (guest, #6227) [Link]

Lua is pretty popular as an extension language for the reasons you list, but _mostly_ because it's so damn dead easy to integrate into an existing project. Really, that's all that does matter, because extension languages are mostly hindered by the capabilities exposed to them from the host application, not by the language itself.

two questions about Lua

Posted Nov 27, 2011 11:26 UTC (Sun) by coriordan (guest, #7544) [Link]

I've only superficial knowledge of Lua.

When you say it integrates with C, you mean it's easy to write the core program in C and add a Lua extensions engine and have extensions written in Lua, right? (Or do you mean you an application can have a Lua extensions engine and have extensions written in optionally Lua or C or C+Lua?)

Are there already projects that compiler other languages to Lua bytecode?

If a Scheme->LuaByteCode project exists, and if using LuaBC as a base instead of Scheme would make Guile's goals easier, then maybe Guile should consider switching. (Otherwise I'd pose the same questions for switching to ParrotByteCode.)

two questions about Lua

Posted Nov 27, 2011 14:50 UTC (Sun) by oldtomas (guest, #72579) [Link]

"When you say it integrates with C, you mean it's easy to write the core program in C and add a Lua extensions engine and have extensions written in Lua, right?"

Yes. (Meaning both :)

I've interfaced Tcl, Guile and Lua to C. In terms of ease, Tcl wins hands down, with Lua a (not-so-close) second. Guile came last (to be fair: my experiments with Guile were 2001 or so. I'm aware that many things changed since).

As to performance... Tcl isn't looking that bad (they have a VM under the belt as nearly everybody else these days). Lua's appeal is its ~70K core (including a *real* garbage collector, not this refcount thingy sported by the P* languages ;-)

I think it's about time the FSF buries its axe towards Tcl. It's a vibrant and welcoming community with a sense for good engineering. Not that I'm advocating to adopt Tcl instead of Guile. Neither I'm trying to imply that all goals are common. But there's a lot to learn from other approaches, and the hackability of Tcl (and Tk) sure is one of them (remember: hackability is the practical side to FSF's freedom #2).

And hey, Tcl and Lua are free software, after all.

two questions about Lua

Posted Nov 27, 2011 15:04 UTC (Sun) by HelloWorld (guest, #56129) [Link]

(including a *real* garbage collector, not this refcount thingy sported by the P* languages ;-) I was actually pretty horrified when I recently found out about Perl's lack of a proper garbage collector. Something as simple as
sub f {
    my $x;
    $x = \$x;
}
f while (1);
will eat up arbitrary amounts of RAM. It's ludicrous.

Garbage collector

Posted Nov 27, 2011 15:38 UTC (Sun) by oldtomas (guest, #72579) [Link]

Yes, this is a weakness to be aware of. But don't get me wrong -- my comment was more tongue-in-cheek (and in awe that the Lua folks "got" this into such a tiny footprint!).

Still, the P* languages are immensely successful (and I'm a big fan of Perl, in spite of its limitations). So I wasn't diparaging either Perl nor Python nor PHP.

Garbage collector

Posted Nov 27, 2011 18:19 UTC (Sun) by pboddie (subscriber, #50784) [Link]

CPython has cycle detection to alleviate problems with reference counting, and other Python implementations use different garbage collection strategies altogether.

two questions about Lua

Posted Nov 28, 2011 10:48 UTC (Mon) by niner (subscriber, #26151) [Link]

Well, Nasal the integrated scripting language in the FlightGear flight simulator has a "real" garbage collector featuring second long hangs of the simulation during GC runs. That's obviously much better than having to use weak references here and there to avoid memory leaks.

Refcounting is a primitive mechanism and has obivous problems. But on the other hand it's simple to implement, simple to understand and very deterministic. For example in Perl I can rely on destructors getting called and most of the time even know when. In CPython, desctructors may be called or the may not, same as for example Java. So in fact, they are pretty useless.

two questions about Lua

Posted Nov 28, 2011 13:50 UTC (Mon) by nix (subscriber, #2304) [Link]

Well, Nasal the integrated scripting language in the FlightGear flight simulator has a "real" garbage collector featuring second long hangs of the simulation during GC runs.
That's not a real garbage collector. That's a crap garbage collector. It sounds like it's not incremental at all.

two questions about Lua

Posted Dec 4, 2011 2:13 UTC (Sun) by jzbiciak (✭ supporter ✭, #5246) [Link]

I think your sarcasm collector hasn't run yet.

+1 for Tcl

Posted Nov 27, 2011 15:37 UTC (Sun) by dskoll (subscriber, #1630) [Link]

+1 for Tcl. I'm mystified why it isn't more popular. Tcl was designed from the ground up as an extension language. It's a reasonable language, has an excellent and robust implementation (compare the C implementation of Tcl to that of Perl5... no contest), and a liberal license. Yet outside the electronic design automation niche, it hasn't really caught on. :(

+1 for Tcl

Posted Nov 27, 2011 16:17 UTC (Sun) by oldtomas (guest, #72579) [Link]

Don't underestimate Tcl.

For one, AOLServer is Tcl. SQLite is at least Tcl-affine (Mr. Hipp was (is?) a known Tcler (I wouldn't be surprised if SQLite had some hidden, stripped down Tcl engine inside). But at least, its test framework and its home repository plus bug tracker (Fossil) is Tcl.

Gitk is Tcl/Tk. When you start looking, you find many instances (it's like looking for mushrooms :-)

+1 for Tcl

Posted Nov 27, 2011 16:54 UTC (Sun) by imgx64 (guest, #78590) [Link]

> its home repository plus bug tracker (Fossil) is Tcl.

I was under the impression Fossil was written in C. Take a look at http://www.fossil-scm.org/index.html/dir?name=src .

+1 for Tcl

Posted Nov 27, 2011 17:31 UTC (Sun) by oldtomas (guest, #72579) [Link]

Yes and no.

It's not Tcl but jim, have a look at this:

<http://www.fossil-scm.org/index.html/artifact/5640a8acb3d...>

I guess the same engine is hidden in SQLite.

Once again: I think the Tcl folks have hit a very sweet spot on the "embedded interpreter" continuum and there's a lot to learn from them.

Now, of course you might argue that jim ain't Tcl, and you would be right. Still, the culture, that's it.

Remember: Tcl delivered starkits long before Java was doing JNLP. At a fraction of the cost. And the Tclers have been perfectioning all that. Quietly and thoroughly.

Someone said Lisp is "alien technology", and I tend to concur. Tcl is likewise alien technology. Quite a different planet (though they might be connected by a wormhole, mind you), but still amazing.

"Real" garbage collectors are way, way, WAY overrated

Posted Nov 27, 2011 22:54 UTC (Sun) by khim (subscriber, #9252) [Link]

I was actually pretty horrified when I recently found out about Perl's lack of a proper garbage collector.

Why? Garbage collectors are useless for most tasks. iOS still does not have one (iOS5 promised garbage collector and delivered ARC instead) - yet there are hudreds of thousands applications.

On the first glance garbage collector is huge win - and it may be win sometimes, but then to make sure memory does not leak out in complex cases you must keep track of who owns what anyway (you do know that memory can leak in a language with "proper" garbage colletor, right?) - and if you do that then to break cycle once in a while is not a big deal. It can be argued that it's good thing to have garbage collector for embedded langauge because people who'll use said language can not keep track of things, but even there I'm not convinced.

"Real" garbage collectors are way, way, WAY overrated

Posted Nov 28, 2011 1:56 UTC (Mon) by nix (subscriber, #2304) [Link]

to make sure memory does not leak out in complex cases you must keep track of who owns what anyway (you do know that memory can leak in a language with "proper" garbage colletor, right?) - and if you do that then to break cycle once in a while is not a big deal.
I'm sorry, but that is sheerest nonsense, every word. You could only come out with it if you'd never written a program manipulating nontrivial datastructures with intricate lifespans. As soon as you do that, GC of some kind (even trivial reference counting) is a huge, huge win.

Of course, most programs are not like that (or, at least, should not be) -- but for those which must be, garbage collection is crucial.

(btw, memory can only leak in languages with a proper non-buggy garbage collector if you redefine 'leak' to include program bugs which cause references to persist when they should not -- but in this case, the GC is doing its job, because those erroneous references are potentially traversable, so you cannot possibly discard that memory in case you choose to traverse them. Fixing this requires solving the halting problem *and* foretelling the future. I don't think so. Now other resources *can* leak: GC is not a cure-all.)

On the contrary...

Posted Nov 28, 2011 6:58 UTC (Mon) by khim (subscriber, #9252) [Link]

You could only come out with it if you'd never written a program manipulating nontrivial datastructures with intricate lifespans.

On the contrary: I've come to this conclusion after my work on few quite large projects with nontrivial datastructures with intricate lifespans.

As soon as you do that, GC of some kind (even trivial reference counting) is a huge, huge win.

Of course, most programs are not like that (or, at least, should not be) -- but for those which must be, garbage collection is crucial.

Well automation is good and refcounting has it's place - but it just removes the boilerplate. I've found scoped_ptr variation (with ubiquitous "release" method added which makes it immensely more useful... albeit significantly less powerful) enough for 99% case and the remaining 1% is usually covered with things like shared_ptr.

btw, memory can only leak in languages with a proper non-buggy garbage collector if you redefine 'leak' to include program bugs which cause references to persist when they should not

Memory leak is memory leak:

A memory leak, in computer science (or leakage, in this context), occurs when a computer program consumes memory but is unable to release it back to the operating system.

It only becomes something different when you want to justify needless complexity introduced by "proper GC"...

but in this case, the GC is doing its job, because those erroneous references are potentially traversable, so you cannot possibly discard that memory in case you choose to traverse them. Fixing this requires solving the halting problem *and* foretelling the future. I don't think so.

Right - this is basically impossible. But since it's impossible you still need to keep track of your nontrivial datastructures with intricate lifespans with enough precision to guarantee that you'll never keep pointers to needlessly growing pile of objects which are not really needed. I've found that difference between doing that and doing proper memory management without GC is minuscule while reaching this point is quite hard.

This is funny: we have millions of lines of code here. C++ programs tends to work for months without any problems (there are few black sheeps here, too, but they are rare) while Java programs are more often then not have memory leaks problems (in this case white sheeps exist but they are rare) and need elaborate babysitter schemes to keep them working. This is because C++ programmers know in their very bones that they can not ever forget about memory management and it's their responsibility to keep everything in order while Java programmers often invent elaborate designs which rely on their fake "freedom" (granted by GC) and end up a disaster because if your datastructures are convoluted enough and lifespans are fuzzy enough it's just impossible to understand what really keeps things in memory and wastes all that memory.

There is one important case...

Posted Nov 28, 2011 7:07 UTC (Mon) by khim (subscriber, #9252) [Link]

Forgot to add: there are one important case where GC is immensely useful: batch jobs.

If your goal is not to work indefinitely and serve user requests but to process finite amount of data and die then it does not really mater if you program has memory leaks or on: when it'll finish OS will reclaim all the memory anyway. What does matter is the ability to process the data using known amount of memory. In this case GC's responsibility is different: it does not need remove all memory leaks (as you've rightfully pointed out it can not really do that), but if it can collect enough memory for the program to fit in quota till it finishes - then it's enough.

On the contrary...

Posted Nov 28, 2011 9:28 UTC (Mon) by dgm (subscriber, #49227) [Link]

Amen.

It's surprising how far you can get without even smart pointers. Simple memory copying can do wonders (reference passing is really an _optimization_). Astonishing, but true.

Another technique that works surprisingly well in many situations is... (dramatical pause) static allocation! Yes, static allocation. Just allocate your structures beforehand and forever, and you don't need to track or free a single bit. What's that you say? You cannot work with arbitrary large datasets? True, but in many situations (say, when writing critical systems that have to run 24/7) this is a _feature_.

On the contrary...

Posted Nov 28, 2011 13:43 UTC (Mon) by nix (subscriber, #2304) [Link]

I don't know what you think garbage collection *is*, but the deallocate-when-it-goes-out-of-scope technique exemplified by (e.g.) C++'s auto_ptr<> is definitely a simple GC technique. So too is reference counting. Reread my comment in that light and it should make more sense.

(The rest of your comment is predicated on the false assumption that simple GC is not GC, so there's no point responding to it.)

Garbage collection need not be a complicated type-accurate generational scheme in order to be GC.

Well, it question of terminology then...

Posted Nov 28, 2011 17:06 UTC (Mon) by khim (subscriber, #9252) [Link]

(The rest of your comment is predicated on the false assumption that simple GC is not GC, so there's no point responding to it.)

There are sizable confusion here about refcounting. Sure, things like RAII, scope_ptr and refcounting can be considered "kind of garbage collection", but usually when people talk about GC they assume something more then "mere refcounting". see Wikipedia, for example where it says: other dynamic languages, such as Ruby (but not Perl 5, or PHP, which use reference counting), also tend to use GC - this clearly assumes refcounting is not GC position.

Rule of thumb: ownership constructs like scoped_ptr and unique_ptr are fine, refcounting schemes like shared_ptr and weak_ptr are warning sign and if you start pining for "real GC" then it's time to stop and think about you design: it's obviously too complicated and it's doubtful you'll ever make it work reliably.

On the contrary...

Posted Dec 2, 2011 10:59 UTC (Fri) by Los__D (guest, #15263) [Link]

"This is funny: we have millions of lines of code here. C++ programs tends to work for months without any problems (there are few black sheeps here, too, but they are rare) while Java programs are more often then not have memory leaks problems (in this case white sheeps exist but they are rare) and need elaborate babysitter schemes to keep them working."

[Citation needed]

My experiences are quite the opposite.

"Real" garbage collectors are way, way, WAY overrated

Posted Nov 28, 2011 4:34 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]

To be fair, you don't really need GC for GUI since your objects' lifetimes are mostly determined by the lifetime of UI.

But once you start writing something non-trivial you really miss a good GC.

Besides, a real precise GC for C/C++/Objective C is not possible anyway. The best you can do is to use conservative GC like Boehm-GC.

Wingo: the gnu extension language

Posted Nov 28, 2011 21:27 UTC (Mon) by marm (guest, #53705) [Link]

Do we really need any translators? There is no real need to stick to a single language or a single virtual machine. Instead, we can make a universal glue layer with the application at one side and many different languages at the other. Think SWIG, but dynamic.

Wingo: the gnu extension language

Posted Nov 28, 2011 22:10 UTC (Mon) by wahern (subscriber, #37304) [Link]

If we're just talking about languages, that's a nice idea. But with extension languages the implementation is critical. Lua has a nice one. As others have pointed out, Tcl is a great extension language, but I just can't see how Tcl's C API stacks up against Lua's. Ditto for the VM. Tcl 8.5/8.6 has a great VM, but, again, it's all about how it exposes itself to the C application. It's just not as elegant and capable. Sun HotSpot has an amazing VM, but it'd be a nightmare as an extension component. I confess I've only tinkered with Tcl, though, and have spent much more time reading the documentation than using it.

Wingo: the gnu extension language

Posted Nov 30, 2011 16:41 UTC (Wed) by dskoll (subscriber, #1630) [Link]

I haven't looked at Lua, so can't comment on its C bindings, but I think Tcl's are excellent. You say they are "not as elegant and capable". Could you provide examples of inelegance or lack of capability? I'm stumped trying to think of any.

Wingo: the gnu extension language

Posted Nov 30, 2011 17:49 UTC (Wed) by anselm (subscriber, #2796) [Link]

We're way out in opinion country here. It is currently fashionable in some circles to diss Tcl while hailing Lua as the greatest invention since sliced bread. Frequently these circles haven't bothered to actually look at Tcl in any sort of detail and/or are turned off by Tcl's somewhat idiosyncratic syntax (which however is not without its advantages) and think that the implementation must suck, too.

I don't know a lot about Lua, either, but Tcl was originally invented as an extension language and has over 20 years of solid engineering by some of the best minds in the industry behind it (John Ousterhout received the ACM Software System Award for it in 1997). Thus as far as integration with C is concerned I'd expect any actual differences in utility or elegance between the two to be slight at best. Mind you, Lua may be a wonderful language for all I know, but Tcl isn't that bad either.

Wingo: the gnu extension language

Posted Nov 30, 2011 23:07 UTC (Wed) by wahern (subscriber, #37304) [Link]

When I was choosing between Lua and Tcl about 6 years ago, Tcl's implemention _did_ suck compare to Lua's. It also didn't have coroutines, and since most of what I do involves server-side programming juggling hundreds and thousands of requests, coroutines are extremely nice.

Tcl has coroutines now, but the issue is how they're exposed. With Lua, coroutines can be used and manipulated equally well from either C or Lua code. This is particularly true with Lua 5.2, whereas 5.1 had some caveats about yielding across C stack frames, which 5.2 solved without playing games with the C stack at all. AFAICT, Tcl just doesn't offer this sort of symmetry.

Of course, Tcl has it's own event model baked in, which is probably why coroutines aren't as powerful in Tcl as in Lua. This is a plus for Tcl as a language, but not so much for embedding Tcl in random applications with sundry task models.

Wingo: the gnu extension language

Posted Nov 30, 2011 22:30 UTC (Wed) by elanthis (guest, #6227) [Link]

TCL's C bindings have some 600 functions.

The entire lua.h header is only around 450 lines, including comments, structs, whitespace, etc.

Smaller doesn't always mean better, but having looked through the TCL bindings API, I can't even begin to guess what large chunks of it do or why I should have to care about those details. TCL looks to have 6 or more different types of strings. It has multiple types that serve similar purposes. It has its own threading system and requires the C API to manage all the details, from what I can tell. It has a large number of complex abstractions where a small number of simple abstractions would do.

I have a lot of beefs with Lua. I don't think it's a particularly great language, the VM was state of the art a decade ago but it's pretty behind the times for today (even excluding JIT-based runtimes from consideration), and it's stack-based C API is not the panacea some people would have you believe. That said, it blows TCL out of the water in every possible metric I can come up with besides "size of standard library." (That one is a mixed big; in my uses, I don't want a huge standard library, but for more general purpose programming you want one.)

Wingo: the gnu extension language

Posted Dec 1, 2011 0:15 UTC (Thu) by anselm (subscriber, #2796) [Link]

having looked through the TCL bindings API, I can't even begin to guess what large chunks of it do or why I should have to care about those details.

Much of Tcl's rich C-language API is irrelevant to day-to-day use. You can probably do most of what is generally needed with a dozen or so functions. The rest deals with event handling, virtual file systems, execution of untrusted scripts, etc. and comes in very useful at times. You can mostly ignore it but when you need it, it's very good to have. The threading system is quite nice because it actually works portably.

TCL looks to have 6 or more different types of strings.

Not really. It turns out that Tcl has been able to deal with UTF-8 – something most other similar languages have figured out only fairly recently – since about 1998 or so. Basically Tcl distinguishes between »strings« (in UTF-8) and »byte arrays« (sequences of arbitrary bytes). This is what other languages do, too, but in Tcl it has been working transparently for more than a decade.

Finally, Tcl does have the distinct advantage of being very well documented, not only by means of man pages that actually say something (where other APIs basically tell you the names of functions and the names and types of their parameters, as extracted automatically from the source code), but also inside the code itself. There are also some very good books out that deal with all aspects of the language at various levels.

Lua may also be very nice. I don't know a lot about Lua, but then again I don't post wrong stuff about Lua on LWN. Maybe – just as a friendly suggestion – if you don't know a lot about Tcl then you shouldn't post wrong stuff about it to LWN?

Wingo: the gnu extension language

Posted Dec 1, 2011 0:26 UTC (Thu) by elanthis (guest, #6227) [Link]

What exactly did I post that was wrong? You offered rational for a few things, not corrections.

Wingo: the gnu extension language

Posted Dec 1, 2011 18:16 UTC (Thu) by dskoll (subscriber, #1630) [Link]

Smaller doesn't always mean better, but having looked through the TCL bindings API, I can't even begin to guess what large chunks of it do or why I should have to care about those details.

All of the Tcl C bindings have man pages. And yes, Tcl is pretty big. To get an overview, it's best to read a book. "Tcl and the Tk Toolkit" by Ousterhout is dated, but still gives a good big-picture description. There are probably more up-to-date books available.

That said, it [Lua] blows TCL out of the water in every possible metric I can come up with...

For someone who hasn't used Tcl much or studied it in any great depth, you're pretty quick to judge. (I haven't used Lua so I'm not judging it, but I don't think you have used Tcl enough to make an honest comparison.)

Hmm... two losses already...

Posted Dec 1, 2011 20:43 UTC (Thu) by khim (subscriber, #9252) [Link]

And yes, Tcl is pretty big.

Hmm... So it loses at the third most important "embedded language" criteria.

I haven't used Lua so I'm not judging it, but I don't think you have used Tcl enough to make an honest comparison.

And that phrase accepts defeat in the second most important criteria for the embedded language...

And people still say I'm mystified why it isn't more popular. Gosh...

I'm already discussed this before: embedded languages have their own peculiarities. They are not judged with the same criteria as a regular language. In particular efficiency is not all that important. Expressiveness is just a bonus point - but it's not a requirement.

What are the requirements, then?

1. It should be easy to embed in the existing program. This is the most important criteria.
Rationale: If it's hard to embed it then nobody will do that - end of story, case closed.

2. It must be easy to hack for the novice. Stupid errors should be easy to spot and fix, you should not need manuals, or, god forbid, courses to start hacking.
Rationale: embedded languages are designed to be used by users, not by developers. Users don't have much patience. The most you can expect is one weekend. If it's not enough to learn some rudiments of the language and solve some simple problem using it - you'll lose the contributor. And embedded language without lots of scripts to use is not worth much.

3. It must be small and efficient to embed. Note: I'm not talking about the ability to do many calculations for second (this is not as important). I'm talking, again, about overhead which is added to the existing program when you add embedded language to it.
Rationale: If you use embedded language in one project you'll tend to use it for other projects, too. And for some projects tens of megabytes of embedded language is not an option.

Now, this is the boilerplate. Some languages can score higher then "natural" on some items. For example python is often used as an extension language in Linux world where it's preinstalled so it's size if efficiently zero. And the same can be said about .NET in Windows world. But switch over to "another size" and you'll see that popularity of python goes down and popularity of lua shots up because size matters now. And you can forget about using Mono in Linux world altogether. JavaScript can be difficult to learn and use but most potential contributors need to tolerate it to some degree because it's the old language to program in the web - but still it's hard to embed so it's rarely used, but the projects which managed to do that enjoy high popularity among contributors.

Now, TCL. I'm not a big fan of TCL, but when I see advice like throw out your "main()", quickly define all the rest as one or more extensions, and get going again quickly with tclsh or wish or tclkit... Well, it sure does not expire confidence. And as you've pointed out it scores badly one two other criterias, too.

TCL is not quite is bad as GUILE (it's scores are FAIL, FAIL, and FAIL while for GUILE it's EPIC FAIL, EPIC FAIL, and FAIL), but still definitely is not good enough to be a contender...

Hmm... two losses already...

Posted Dec 1, 2011 22:04 UTC (Thu) by anselm (subscriber, #2796) [Link]

And yes, Tcl is pretty big.
Hmm... So it loses at the third most important "embedded language" criteria.

Tcl is big but it is not that big.

$ ls -l /usr/lib/liblua5.1.so.0.0.0 /usr/lib/libtcl8.5.so.0 /usr/lib/libpython2.7.so.1.0 /usr/lib/libperl.so.5.14.2
-rw-r--r-- 1 root root 1565272 Nov 29 00:57 /usr/lib/libperl.so.5.14.2
-rw-r--r-- 1 root root 3059672 Okt  5 13:55 /usr/lib/libpython2.7.so.1.0
-rw-r--r-- 1 root root 1175688 Jun 26 13:11 /usr/lib/libtcl8.5.so.0
-rw-r--r-- 1 root root  180896 Jul 25 10:32 /usr/lib/liblua5.1.so.0.0.0
The size difference seems to come mostly from stuff that Tcl does which Lua doesn't. This includes things that are arguably very useful, such as regular expressions (which Lua supports only sort-of, in a stripped-down flavour with different syntax). There are applications where 150k vs. 1.5M will make a difference but there are also very many applications where it doesn't, and where the convenience of having useful stuff as a built-in part of the language is more important.

In addition, there are ways of leaving stuff out of Tcl which will bring its size way down – Jim Tcl is an implementation of Tcl that concentrates on the essentials, while keeping compatibility to regular Tcl, and that is about as big as Lua (i.e., in the 100k-200k range). And it does regular expressions, too! The advantage is that pretty much all of the collected Tcl wisdom applies to Jim Tcl, too, so if you know regular Tcl it is reasonably easy to write code for Jim Tcl since most of the techniques are identical.

1. It should be easy to embed in the existing program. This is the most important criteria. Rationale: If it's hard to embed it then nobody will do that - end of story, case closed.

People have been embedding Tcl into existing programs for 20 years and they keep doing it. All the time. Ask at IBM or Cisco. This is not an issue.

2. It must be easy to hack for the novice. Stupid errors should be easy to spot and fix, you should not need manuals, or, god forbid, courses to start hacking. […] And embedded language without lots of scripts to use is not worth much.

The complete syntax of Tcl only consists of 11 fairly simple rules. It doesn't get a lot easier than that. There is a very good wiki (implemented in Tcl) with literally thousands of code examples from all kinds of applications.

Some users don't mind learning a language from scratch on their own. Others prefer being taught, and there's nothing wrong with that. I have taught commercial Tcl classes and I'm pleased to say that even non-programmers tend to get going in Tcl fairly quickly if you explain things right. The 11 rules are very helpful there – it is really not a complicated language.

3. It must be small and efficient to embed. […] And for some projects tens of megabytes of embedded language is not an option.

I think we have pretty much laid that one to rest at the beginning of this comment.

Now, TCL. I'm not a big fan of TCL, but when I see advice like throw out your "main()", quickly define all the rest as one or more extensions, and get going again quickly with tclsh or wish or tclkit... Well, it sure does not expire confidence.

There is nothing wrong with keeping your »main()« when embedding Tcl into a program. It works, and has been working for 20 years. However, experience has shown that in many cases going the other way – i.e., implementing your own functionality as a Tcl extension – makes more sense because that makes it easy to embed your functionality in other Tcl-using programs, which can come in very useful at times.

Hmm... two losses already...

Posted Dec 2, 2011 8:26 UTC (Fri) by khim (subscriber, #9252) [Link]

People have been embedding Tcl into existing programs for 20 years and they keep doing it. All the time. Ask at IBM or Cisco. This is not an issue.

It is an issue - and quite serious one. Large companies can do lots of things and they can embed anything in anything. You wrote that TCL is "surprisingly unpopular" among embedders, not me. I'm just explaining why. When you google for "embedding Lua" - you find tons of very simple instructions (including one on the official site), when you do that with python - you find instructions, too (including one on the official site), but for tcl... the first result you are seeing is why adding Tcl calls to a C/C++ application is a bad idea. This is... discouraging. Ok, you ignore that, find another page... and hit the wall: your embedded TCL does not have the ability to use most of the TCL's abilities! I mean script library, of course. And the ability to supply embedded scripts with C/C++ objects. Not via additional library, but directly. Both official LUA manual and official Python manual explain how to do that - because it's quite natural thing to do if you want to extend C/C++ program, right? For TCL it's clearly an afterthought.

Now, TCL embedding is actually easier then GUILE embedding, but still... it's far cry from popular embedded languages like LUA or Python.

The complete syntax of Tcl only consists of 11 fairly simple rules. It doesn't get a lot easier than that.

Sure it does! There are even simpler languages out there. Let's use them to extend your program.

All the jokes aside you are talking about totally different problem: how to learn the language. Most potential users of the embedded languages don't want to do that! Not at first, at least. They just want to hack something for their own enjoyment. They may decide to learn language later, but first they want to "hack something". This is where "11 fairly simple rules" are actively harmful. They mean that nonsense combinations of googled programs produce equally nonsense results instead of helpful error messages.

Some users don't mind learning a language from scratch on their own. Others prefer being taught, and there's nothing wrong with that.

You've ignored the overwhelming majority of users: the ones who don't want to learn language at all but want to do something with extensible program in the spur of the moment.

Sure, when people reach the stage where they want to finally learn tcl this simplicity helps immensely - but you are losing so many potential contributors at first stage that this does not change anything. And another psychological moment: a lot of python users don't really know python (when I use for with else it surprises a lot of people who played with python for years), yet they can happily hack away some programs. This is harder with tcl.

However, experience has shown that in many cases going the other way – i.e., implementing your own functionality as a Tcl extension – makes more sense because that makes it easy to embed your functionality in other Tcl-using programs, which can come in very useful at times.

Sure, but this argument only makes sense when you already joined the tcl community - and it's actively harmful WRT bringing new players "in the circle".

Basically both GUILE (the subject of discussed article, remember) and TCL are designed to keep users happy and satisfied, they are not designed to bring users in - and for embedded languages it's the most important quality. Because most potential developers and users have quite weak motivation to join the fray at first - if you hamper this process (as both GUILE and TCL do) then you'll not get many users no matter how cool it looks to the "enlightened" people inside.

Hmm... two losses already...

Posted Dec 2, 2011 9:57 UTC (Fri) by anselm (subscriber, #2796) [Link]

the first result you are seeing is why adding Tcl calls to a C/C++ application is a bad idea. This is... discouraging. Ok, you ignore that, find another page... and hit the wall: your embedded TCL does not have the ability to use most of the TCL's abilities!

I just did this and Google must work considerably differently for you than for me. The pages I see do give simple example programs. The pages themselves appear to have acquired some barnacles over the years but the main stuff is there.

Now, TCL embedding is actually easier then GUILE embedding, but still... it's far cry from popular embedded languages like LUA or Python.

I can't speak for Lua, but I would encourage anybody to compare the simple examples (from the docs) of embedding Tcl and Python into a program. Claiming that Python is easier to embed into a C program than Tcl means you cannot have looked very closely at doing either the one or the other or both.

Also, the fact that the Python document in question is called »Extending and embedding the Python interpreter« should tell you something – namely that packaging functionality as a general Python (or Tcl) extension is often preferable to adding Python (or Tcl) to an existing program as an extension language.

Most potential users of the embedded languages don't want to do that! Not at first, at least.

Yes, and of course nobody ever needs to »learn« Lua, right? I think by now you're really floundering.

I have been introducing people – developers and »users« – to Tcl, off and on, for the last 20 years or so. I even implemented a prototype for a product, in Tcl, for a company which is now going for an IPO mostly based on that product, still mostly in Tcl, and I mentored developers in that company, many of whom had no prior exposure to Tcl. You can take it from me that Tcl is not difficult to get used to. (As another anecdote, a sister company decided to base the main bits of their product on Lua. The product mostly sucked, turned out to be a maintenance and security nightmare, and nobody wanted to buy it. That company no longer exists. Which is very probably not Lua's fault, but it still makes one think.)

You've ignored the overwhelming majority of users: the ones who don't want to learn language at all but want to do something with extensible program in the spur of the moment.

Yes, and again with Lua that somehow magically works? Why (according to you) is Lua so much easier than Tcl that you don't have to learn to use the language in order to write code in it?

Sure, but this argument only makes sense when you already joined the tcl community - and it's actively harmful WRT bringing new players "in the circle".

No, it doesn't. As far as extending a program in Tcl is concerned, it doesn't matter whether that program is based on a pre-existing main program in C, with Tcl added, or on tclsh or wish, possibly with extensions in C added.

The only place where it might possibly make a difference is when you decide you want an extension language in your existing program. However, then the fact of the matter is that you will have to do some restructuring of your code, whatever extension language you choose. It probably depends on how your existing code is structured, but based on my experience, if you have a reasonably modular codebase with sensible inter-module APIs, it is quite easy to put Tcl shims on top of your APIs to make Tcl extensions. (If nothing else, this lets you use the Tcl unit-test infrastructure to test your C code, which is already very useful even if you never actually write a real extension in Tcl.) There are even programs like Swig which will do most of the heavy lifting for you. Going the other way round is often more inconvenient.

Incidentally, one giant success story in that respect is SQLite, which basically started out as a C library packaged as a Tcl extension. All the other language bindings came later.

Are you sure?

Posted Dec 2, 2011 14:20 UTC (Fri) by khim (subscriber, #9252) [Link]

The pages themselves appear to have acquired some barnacles over the years but the main stuff is there.

Care to share a link? I know google sometimes returns different results for different people/country/etc, but I've honestly tried to find short usable example of TCL embedding on one page - and was unable to do so.

Claiming that Python is easier to embed into a C program than Tcl means you cannot have looked very closely at doing either the one or the other or both.

Well, that's exactly my point: sure, I haven't. Why should I? To create a program which can call a script in an embedded language and then call C/C++ function back you don't need a lot of docs or theory if your language is designed for such use case. I needed about 15 minutes for both python and lua, why should tcl require more?

When I want to do that with Lua or Python - I find everything I need right in manual and can copy-paste and start using lua or python in about 5 minutes. Well, 15, if I'm slow reader. Starting from scratch, without any knowledge of python or lua! Sure, I may hit problems later, but I can start using it very, very, VERY fast. Minutes, not hours. For tcl I'll need day or two and for GUILE... don't even start.

Also, the fact that the Python document in question is called »Extending and embedding the Python interpreter« should tell you something – namely that packaging functionality as a general Python (or Tcl) extension is often preferable to adding Python (or Tcl) to an existing program as an extension language.

Whatever. The code and explanations are there, they work, if I have few hours and I want to replace my homebrew parser with lua or python - I can do that even if knew nothing about lua or python at the start of the endeavour. With tcl - it's only enough to start learning various concepts. And with GUILE - it's only enough to find that you must study a lot to understand even basic concepts.

Yes, and of course nobody ever needs to »learn« Lua, right? I think by now you're really floundering.

Of course you need to learn it! After you've embedded it in your program and started using it you'll find that some pieces which you grabbed from the net and added to you program work and some don't work. You spend some time trying to understand what goes on there and at some point you'll actually need to know how the whole thing works. But this happens later, sometimes much later. You start with just copy-pasting code from random samples and trying to make the whole thing work somehow.

I even implemented a prototype for a product, in Tcl, for a company which is now going for an IPO mostly based on that product, still mostly in Tcl, and I mentored developers in that company, many of whom had no prior exposure to Tcl.

Well, the keywords here are mostly in Tcl. Tcl is pretty decent scripting language (and GUILE is not bad, really), they just suck as embedded languages.

As another anecdote, a sister company decided to base the main bits of their product on Lua. The product mostly sucked, turned out to be a maintenance and security nightmare, and nobody wanted to buy it.

Well, sure. LUA is great embedded language, cement solution for your project. But just as is real life: buildings created entirely from clay suck - you need bricks or wood to make them decent.

The only place where it might possibly make a difference is when you decide you want an extension language in your existing program.

In other words: it only matters 9 times out of 10.

However, then the fact of the matter is that you will have to do some restructuring of your code, whatever extension language you choose.

Well, sure. For Lua and python you need to add couple of calls like lua_open or Py_Initialize and may be do some small changes here and there, but nothing beyond that. For GUILE or TCL you'll need to do a lot more. Take a look on the initial Python support in GDB, for example. Changes are pretty extensive, but they don't alter the program structure all that much.

Incidentally, one giant success story in that respect is SQLite, which basically started out as a C library packaged as a Tcl extension. All the other language bindings came later.

AGIAN - it's opposite example: when something was designed for TCL and then was cut out and turned to separate library.

I think I forgot to mention important fact which forms the basis for everything else: most programs which contain embedded languages were not designed to contain them initially, nobody panned to have them organized that way, it's just kind of happened at some point. This is just fact of life. It's pointless to discuss that and preach the virtue of other approaches. And when you finally decide to add extension language you are dealing with existing mature codebase, not with something designed with GUILE or TCL needs in mind. If you don't accept it then you are not a contender on the embedded language field.

Both GUILE and TCL are quite suitable for cases where you design you application from the ground with them in mind. Python is also great while Lua is horrible (it's too chaotic and limited to support large projects and if you build it in you application from the start then it's hard to resist the temptation to white large pieces in Lua). But this is side-show when we discuss embedded languages!

Are you sure?

Posted Dec 2, 2011 17:56 UTC (Fri) by anselm (subscriber, #2796) [Link]

Well, that's exactly my point: sure, I haven't. Why should I?

You don't have to, but then perhaps you should also refrain from pretending that your opinions are established facts.

Whatever. The code and explanations are there, they work, if I have few hours and I want to replace my homebrew parser with lua or python - I can do that even if knew nothing about lua or python at the start of the endeavour. With tcl - it's only enough to start learning various concepts.

This is basically either utter ignorance or deliberate falsehood. Please explain where the actual difference lies between embedding either Lua or Python and Tcl, such that doing it will result in such a large effect.

Of course you need to learn it! After you've embedded it in your program and started using it you'll find that some pieces which you grabbed from the net and added to you program work and some don't work.

Please explain why the same approach would not apply to Tcl.

Tcl is pretty decent scripting language (and GUILE is not bad, really), they just suck as embedded languages.

This is rather an interesting claim considering that Tcl essentially invented the idea of an embedded language in Unix, received a prestigious award for exactly that contribution to the field, and has been used for exactly that purpose by all sorts of companies for two decades now. Apparently all these people – including the ACM, IBM, Cisco, Sun Microsystems, Motorola, Oracle, most of the VLSI design software companies, the manufacturers of various CAD systems, etc., etc. – must have missed something important.

Both GUILE and TCL are quite suitable for cases where you design you application from the ground with them in mind.

If you consider the normal best design/implementation practices like modularity, documented interfaces, abstract data types, etc., »designing from the ground up with Tcl in mind« then you may be right. However aren't these the things that you'd want to do in the first place!? If your software is so messed-up that it is difficult to make it into a Tcl extension then embedding Lua probably isn't going to be all that easy, either, nor is it going to improve your software a whole lot in the long run. This would be like putting wallpaper over the cracks in the wall.

[…] while Lua is horrible (it's too chaotic and limited to support large projects and if you build it in you application from the start then it's hard to resist the temptation to white large pieces in Lua).

If you say so. But as far as I'm concerned that comment alone – from somebody who has been arguing vehemently in favour of Lua – would be sufficient to disqualify Lua from consideration for any project. Tcl is demonstrably suitable for large projects, so why should one bother with a language that is nice (if »chaotic«) for small ones but doesn't cut it for large ones? At what point does a project become so »large« that Lua is no longer a viable extension language? Especially since many projects that started small turn out to become large after a while. Why should anybody want to expose themselves to having to change extension language in mid-project?

Are you sure?

Posted Dec 3, 2011 9:35 UTC (Sat) by oldtomas (guest, #72579) [Link]

Care to share a link? I know google sometimes returns different results for different people/country/etc, but I've honestly tried to find short usable example of TCL embedding on one page - and was unable to do so.
Here ya go.. Go to the section "Tcl C API".

If you want to have more seamless fun, this is for you: you embed C code in your Tcl and Tcl manages the compile process transparently. Yesyes, "Python, too can do it", I know. Still, this is one example where Tcl's syntax (or non-syntax?) shines.

And have a look at these too. More than the technical virtues I actually appreciate the community around Tcl:

Adding Tcl/Tk to a C application
How to embed...
Yikes. I'm sounding more and more like a Tcl fanboi. I do like Tcl, but ideologically I'm more in the FSF camp. I'm just trying to advocate the "learn from each other" mantra which some people seem to forget all too often.

Hmm... two losses already...

Posted Dec 2, 2011 16:12 UTC (Fri) by dskoll (subscriber, #1630) [Link]

the first result you are seeing is why adding Tcl calls to a C/C++ application is a bad idea.

The page you linked to is one person's opinion. I think your hostility to Tcl is based on religion rather than facts (so I won't continue this thread.)

But I will make one last response: Here's a simple example of embedding Tcl.

#include <tcl.h>

int
main(int argc, char *argv[])
{
    Tcl_FindExecutable(argv[0]);
    interp = Tcl_CreateInterp();
    Tcl_CreateObjCommand(interp, "whatever", whatever_Cmd, NULL, NULL);
    /*...*/
}

Done in 3 lines.

Hmm... two losses already...

Posted Dec 5, 2011 10:15 UTC (Mon) by smurf (subscriber, #17840) [Link]

>> The complete syntax of Tcl only consists of 11 fairly simple rules.

Only? I'd guess that Python gets by with fewer.

In any case, the problem is that tcl doesn't do what everybody else is doing. Case in point: to compute 2*3, you don't just write (2*3) as in almost every other language out there, you write [expr 2*3]. Except that within an expr you can suddenly use both forms.

There are two problems here. Both make learning tcl harder than necessary.
One is that previous experience is now worthless or, worse, in the way.
The second is that this forces you to learn two distinct sets of rules.

One of the few advantages of Lisp is that there's only three simple syntax rules or so. That kind of simplicity/elegance/take-your-pick comes at a price, though; it seems that that price is too high for Guile to be a universally-useful language for embedding.

Wingo: the gnu extension language

Posted Dec 1, 2011 21:30 UTC (Thu) by anselm (subscriber, #2796) [Link]

To get an overview, it's best to read a book. "Tcl and the Tk Toolkit" by Ousterhout is dated, but still gives a good big-picture description.

Actually that book came out last year in an updated second edition which is very, very, very good. As I said earlier, as far as popular scripting languages are concerned Tcl is one of the better-documented, if not the best-documented one.

Wingo: the gnu extension language

Posted Nov 26, 2011 19:19 UTC (Sat) by tetromino (subscriber, #33846) [Link]

> And why hard-wiring a single extension language into your program is better than exposing the functionality of your program as a library or on D-Bus where it can be used by almost any language?

Because an extension language may well be used not just for extensions, but also for implementing the core features of your application. Many applications use the model where the performance-critical pieces are written in a traditional compiled language, and the pieces where the ability to implement new features quickly is critical are written in an extension language. For this model, you generally want to pick just one extension language to encourage code reuse and to ensure that all developers on the team would be able to work on any part of the application without having to pause and learn a new language.

Wingo: the gnu extension language

Posted Nov 26, 2011 22:33 UTC (Sat) by theophrastus (guest, #80847) [Link]

'brown barbaloots may well not be brown, but also purple' [wink]

Wingo: the gnu extension language

Posted Nov 27, 2011 10:25 UTC (Sun) by dgm (subscriber, #49227) [Link]

Then I would not call it "extension" language.

Wingo: the gnu extension language

Posted Nov 27, 2011 15:22 UTC (Sun) by welinder (guest, #4699) [Link]

Well, you should.

The reasons for using the extension system to implement most of the
functionality are...

1. It makes certain that the exposed API is powerful enough because
it stuff in-tree and out-tree modules on equal footing.

2. It can offer better fault protection. Think emacs-lisp vs. C.

3. Extension language API is typically more stable than the rest of
the program. Less rewriting.

...and probably more.

Wingo: the gnu extension language

Posted Nov 27, 2011 19:51 UTC (Sun) by dgm (subscriber, #49227) [Link]

How can you call extension to the core of a program? Is that people don't want to admit they are writing software in scripting languages?

Wingo: the gnu extension language

Posted Nov 27, 2011 22:42 UTC (Sun) by engla (guest, #47454) [Link]

The structure is completely like an extension language.. i.e. the host application exposes an API, and that's what you implement the rest of the application in. Compare to directly implementing an application using the normal libraries of a scripting language.

Wingo: the gnu extension language

Posted Nov 28, 2011 2:35 UTC (Mon) by dgm (subscriber, #49227) [Link]

OK, you have your own main(). Who cares? Most of the application is still implemented in an scripting language. I would NOT call it an "extension" if it's the core of the application.

Wingo: the gnu extension language

Posted Nov 28, 2011 13:45 UTC (Mon) by nix (subscriber, #2304) [Link]

What if it's the core of the application and also the extension language? Emacs is the classic example here: without any elisp it is basically incapable of doing anything, but with elisp it's Emacs. Yet elisp is clearly its extension language as well. There need be no bright line here.

Wingo: the gnu extension language

Posted Nov 27, 2011 10:26 UTC (Sun) by macc (subscriber, #510) [Link]

Guile isn't usefull.

Tcl is perfect for that job and used to be used in all the places that mattered. ( i.e. EDA, .. )

Unfortunately Mr. Stallmann had to do a personal war on Tcl/Sun/Ousterhout.

related discussions

Posted Nov 26, 2011 19:30 UTC (Sat) by wingo (guest, #26929) [Link]

related discussions

Posted Nov 27, 2011 16:36 UTC (Sun) by oldtomas (guest, #72579) [Link]

Thanks for the links. The only downside is that I wanted to get some boring tasks done this Sunday which will haunt me for the whole week now :-)

And thanks, btw. for your Herculean work on Guile. Don't get me wrong: my posts might come across as if I were advocating Tcl instead of Guile. What I'm advocating is cross-pollination.

Wingo: the gnu extension language

Posted Nov 26, 2011 21:19 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link]

Oh well...

GNU is really losing it. They are becoming more and more irrelevant to modern computing and instead of doing something useful they just continue sliding into irrelevance.

Right now we have just about a metric ton of extension languages (LUA, Python, JavaScript, even Perl). Why should I use Guile which has no community to speak off and has a more restrictive license compared to alternatives? Oh, and as an icing on the cake - it uses LISP syntax which is almost universally hated.

An argument that Guile can act as a virtual machine for other languages is stupid. It's neither going to be faster than moder JIT-compiling JS/LUA interpreters not it is going to be compatible with external libraries.

Wingo: the gnu extension language

Posted Nov 27, 2011 7:05 UTC (Sun) by rsidd (subscriber, #2582) [Link]

Lisp has been relevant for 50 years and will stay relevant. I haven't used guile, but have been intending to get serious for scheme for a while now. Probably I didn't look at guile earlier because of its marketing as an "extension language", but it seems to be more than that. I agree the GNU project doesn't seem to be keeping up with modern computing in many respects, but in this case I think they're doing a good service.

Wingo: the gnu extension language

Posted Nov 27, 2011 7:18 UTC (Sun) by Cyberax (✭ supporter ✭, #52523) [Link]

Well, Fortran and COBOL are also still relevant.

LISP is ideologically beautiful and with a lot of great stories of past glory, true. But people just don't seem to like using it. Ditto for Smalltalk.

Wingo: the gnu extension language

Posted Nov 27, 2011 7:48 UTC (Sun) by rsidd (subscriber, #2582) [Link]

Fortran and Cobol are relevant only for historical reasons: nobody chooses to learn them these days except to deal with legacy code. People (in small numbers, admittedly) still choose to learn Lisp. Fortran and Cobol offer nothing that is not available in more modern languages. Lisp offers something (macros -- not to be confused with their C namesakes) that does not exist in any other language (and as I think Paul Graham pointed out, if you supply that, you've reimplemented Lisp...)

Wingo: the gnu extension language

Posted Nov 27, 2011 9:13 UTC (Sun) by danieldk (guest, #27876) [Link]

But for a language with so many vocal proponents, it has very few success stories. Besides emacs, it is very hard to come up with examples of widely used successful applications written in Lisp.

Wingo: the gnu extension language

Posted Nov 27, 2011 9:51 UTC (Sun) by rsidd (subscriber, #2582) [Link]

Autocad is probably better known than emacs. In symbolic math there is maxima (evolved from macsyma). There are probably many more at the backend level (eg the e-commerce thing that Paul Graham sold to Yahoo).

Wingo: the gnu extension language

Posted Nov 27, 2011 14:10 UTC (Sun) by welinder (guest, #4699) [Link]

> Lisp offers something (macros [...]) that does not exist in
> any other language

FORTH has even more powerful macros. You can change syntax (and, if you
really want to, tokenizing) of the language.

(Brought to you by the department of nit-picking)

Wingo: the gnu extension language

Posted Nov 27, 2011 15:07 UTC (Sun) by oldtomas (guest, #72579) [Link]

I don't know whether you're talking about Lisp, Scheme or Guile in particular. Common Lisp has reader macros, which get a go at the input instead of the "standard" parser. That should be more or less equivalent to FORTH's immediate words, I guess (but you get to write your parser in Common Lisp, which is an ALIEN TECHNOLOGY ;^)

I don't know about Guile (last time I looked it hadn't, but that's a while ago).

Wingo: the gnu extension language

Posted Nov 27, 2011 21:16 UTC (Sun) by Cyberax (✭ supporter ✭, #52523) [Link]

Macros are certainly there in other languages.

OCaml has wonderful macro facility, for example. Nemerle in CLR world by now has the best implementation of a macrosystem. Then there's Boo (statically typed Python) with pretty good macrosystem and so on.

In short, rich macrosystem is not something unique to LISP.

Wingo: the gnu extension language

Posted Nov 28, 2011 1:46 UTC (Mon) by rsidd (subscriber, #2582) [Link]

With ocaml, I guess you are referring to camlp4. Not really part of the language, but yes, it does seem powerful, if even less readable (to me) than lisp. That's because with camlp4 and the like, you have to deal with abstract syntax trees. Ditto with boo (which I just looked up -- I wasn't familiar with it). A Lisp program is already its own abstract syntax tree -- so if you have got past the initial brackets barrier and are able to read lisp code comfortably, macros are not so much harder.

I'm not arguing that other languages are not more widely used. I mainly use python myself. I think it is good to keep lisp alive for cultural reasons, and I'm happy if the GNU project does it. I came to python from C, but I found that after I spent a bit of time in ocaml and haskell, my python coding improved sharply. Those who have tried say the same is true of lisp -- it improves your ability to think about programming.

Wingo: the gnu extension language

Posted Nov 28, 2011 17:31 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]

LISP/Scheme approach to macros is simplistic (since your code is an AST then just treat it as data), the state-of-the-art has already moved.

Nemerle has macros with quasi-quoting which is more restricted but way easier to use technique: http://en.wikipedia.org/wiki/Nemerle#Metaprogramming Boo has quasi-quoting and direct AST manipulation.

Also, Scheme/LISP show their age. There is no good pattern matching for example.

Wingo: the gnu extension language

Posted Nov 30, 2011 6:46 UTC (Wed) by oldtomas (guest, #72579) [Link]

Two questions:

- in which way is the quasi-quotation (I looked at the example in Wikipedia) different to Lisp's "quasi-quote"? As far as I can see the tokenizer has already had a hit at it. That looks to me absolutely equivalent to Lisp's backquote (which one looks more elegant is in the eye of the beholder -- I definitely prefer Lisp ;-)

- What do you mean by pattern matching? Matching a "data structure", I guess? Would "destructuring-bind" <http://asymmetrical-view.com/2008/09/18/destructuring-bin...> fit the bill?

Wingo: the gnu extension language

Posted Nov 30, 2011 16:45 UTC (Wed) by nybble41 (subscriber, #55106) [Link]

> What do you mean by pattern matching? Matching a "data structure", I guess? Would "destructuring-bind" ... fit the bill?

I was thinking more along the lines of the 'match' library[1] for Racket (formerly known as PLT Scheme). Unlike destructuring-bind, which only matches against nested lists, the 'match' library works with all the built-in data types, and can even be extended to include user-defined pattern forms.

[1] http://docs.racket-lang.org/reference/match.html

Pattern matching

Posted Dec 3, 2011 9:40 UTC (Sat) by oldtomas (guest, #72579) [Link]

Thanks! very enlightening

Wingo: the gnu extension language

Posted Nov 30, 2011 12:39 UTC (Wed) by marcH (subscriber, #57642) [Link]

> LISP/Scheme approach to macros is simplistic (since your code is an AST then just treat it as data), the state-of-the-art has already moved.

What, even more abstracted and clean ways NOT to share your code with your colleagues? Great job security news in today's recession times!

PS: for the serious version read this comment below: http://lwn.net/Articles/469640/

Wingo: the gnu extension language

Posted Nov 28, 2011 0:32 UTC (Mon) by droundy (subscriber, #4559) [Link]

Actually, Fortran continues to have a vibrant community, and is certainly used for new projects in the scientific computing field. I consider this somewhat unfortunate, since I can't stand the language, but's it's sort of embarrassing to be unable to point to a superior alternative...

Wingo: the gnu extension language

Posted Nov 28, 2011 18:36 UTC (Mon) by daglwn (subscriber, #65432) [Link]

> Fortran and Cobol are relevant only for historical reasons: nobody chooses
> to learn them these days except to deal with legacy code.

Scientists would like to disagree with you on Fortran. We just had a new standard published in 2008. Scientists use it for new codes.

> Fortran and Cobol offer nothing that is not available in more modern
> languages.

This is just wrong. Fortran offers all sorts of semantic guarantees which aren't available in C, for example and probably aren't available in many other languages either due to its treatment of arrays.

Wingo: the gnu extension language

Posted Nov 27, 2011 15:25 UTC (Sun) by smurf (subscriber, #17840) [Link]

Case in point. Your "I have been intending to get serious for a while now" suggests that the effort ot learn the thing may ultimately not be worth the gain.

Just look at gnucash. Its higher-order stuff is written in Guile, more or less. What did they do recently? They implemented Python bindings. Presumably, they're doing this for a reason, and the reason is unlikely to be "Guile is the best extension language there is".

Not unless you define "best" somehow other than they do, anyway.

Anothe famous example

Posted Nov 27, 2011 22:40 UTC (Sun) by khim (subscriber, #9252) [Link]

Note that AutoCAD also went this way: it supports AutoLISP, Visual LISP, VBA, .NET and ObjectARX.

Why? Because LISP and Guile are unimaginably horrible extension languages. Not the worst available but close to it.

It was discussed "why" many, many, MANY (last one, probably, is here): most people can not understand functional programming and can not understand LISP.

Now, you may say what you want about these poor sods, but this is just a fact or life. You don't need to like it - but you must accept it if you want to create usable extension language. Sure, most such folks are not programmers (among programmers percentage of LISP-capable folks is higher and among good programmers it's higher still), but this is the target audience for the extension language.

It does not matter how powerful you language is. It does matter how beautiful it is. If 90% of it's potential users can't use it then it's unsuitable. Period. End of story.

Now, if I choose language to write complex (but not performance-critical) core of the application... and if all controbutors are good programmers and are familiar with LISP... then may be, just may be, GUILE will be good choice. But this happens so rarely in practice...

P.S. And don't start of the famous GUILE "translators". They should be in a dictionary for the "vaporware" article right next to HURD. Duke Nukem Forever took 15 years to develop, GUILE promised "translators" 17 years ago - but still have nothing to show today. Yes, I know, there are few half-broken ones and one almost "production ready". Sadly this one is for Emacs Lisp so does not help at all.

Anothe famous example

Posted Nov 28, 2011 7:03 UTC (Mon) by oldtomas (guest, #72579) [Link]

We all know by now that you don't like Lisp. That's OK. Lisp's beauty is a bit... hidden ;-)

But you should do your homework before repeating often-quoted but untrue sound-bits like

Now, if I choose language to write complex (but not performance-critical) core of the application...

Lisp is used in high-performance applications. See here for a discussion (the title is misleading, btw. It's not Sabre, but Orbitz -- but things get clarified there).

And then there's AllegroStore. And so on.

And: do have a look at The Language Shootout: SBCL (a Common Lisp implementation) consistently ranges there among the first four -- way beyond your trusty old Perls, PHPs and Pythons (and clearly beyond Lua).

Had to be said.

Wrong again...

Posted Nov 28, 2011 7:51 UTC (Mon) by khim (subscriber, #9252) [Link]

We all know by now that you don't like Lisp. That's OK. Lisp's beauty is a bit... hidden ;-)

Actually I know and appreciate LISP. The saying whoever does not understand LISP, is doomed to reinvent it is quite apt: LISP is reinvented repeatedly and in quite unexpected places (think autoconf, for example: LISP experience helps immensely with M4 work). But I also am quite ready to accept realities of the world. GUILE maintainers (like AT) apparently can not.

Times are changing. If you develop extension language today then you should accept that it should not be LISP. If you a LISP fanatic and you want to present some kind of "imperative face" for the "lamers who don't appreciate LISP", but keep LISP in the core then creation of said "imperative face" should be at the very top of your tasklist, not some kind of afterthought in the footnote.

As I've said before: timing is critical, perfection is not. Sure, you need some measure of quality: if you system is utter crap and is basically unusable then it'll fail even if you'll beat the cheetah in bringing your system to the light, but once it passes certain threshold it becomes impossible to dislodge because of POGE. Usually such system can only be replaced when maintainers become sidetracked. Think Netscape (lost the market when grandiose "rewrite project" stalled it's development for years), then MS IE (which was developed in tandem with Windows so Vista troubles kept MS IE at version 6 for year), etc. Actually today is good time to push new extension language: cross-platform development is in favor again (because of iOS and Android) and Mono is slowly dying so yesterday favorite (.NET, I mean) is not as popular today. There are many contenders (JavaScript, Lua, Python, etc) and perhaps GUILE can be a contender, too... but they need compelling embedded language story. Not tales about macros, but solid embedding story first and good imperative face second - all the things which are promised yet totally neglected by GUILE community.

Lisp is used in high-performance applications. See here for a discussion (the title is misleading, btw. It's not Sabre, but Orbitz -- but things get clarified there).

LISP is used in high-performance applications. Not often, but it does happen. GUILE on the other hand is not. Is it supposed too? I'm not sure. Perhaps if they switched to "extension language" approach to "performance critical language" approach they should say so: then the fact that it's dialect of LISP will not matter as much but then questions about performance will arise - and I'm not sure how GUILE works here. Have anyone seriously benchmarked GUILE and compared it with other Scheme implementations out there? I was under impression that tight integration with C (which is kind of requirement for an "extentions language") precluded a lot of useful optimizations....

Had to be said.

Not really. LISP fate is different kettle of fish, we are discussing GUILE here. Do you imply that GUILE in it's all-new bytecode incarnation is now ready for performance-critical tasks? Hard to believe, really, but if so then perhaps GUILE with all it's goodies may find a different niche...

Wrong again...

Posted Nov 28, 2011 9:07 UTC (Mon) by rsidd (subscriber, #2582) [Link]

Your previous comments seemed to be complaints about Lisp generally, not Guile specifically. I don't know how Guile performs, but for those who need to interface C with scheme, one question I'd like to see answered is: why Guile, why not bigloo? What are the similarities and differences? Which is easier, which performs better, which is better supported, which is more actively developed? That would be a useful article to read. (I remember bigloo generally did well in the language shootout, but it seems to have disappeared -- with all other scheme implementations -- in the current edition.)

Yes, this was about LISP too... as an extension language...

Posted Nov 28, 2011 9:53 UTC (Mon) by khim (subscriber, #9252) [Link]

Your previous comments seemed to be complaints about Lisp generally, not Guile specifically.

Sure. I'll even cite myself: LISP and Guile are unimaginably horrible extension languages. If you consider an extension language for your project then it's good idea to pick one which will be accepted by sizable amount of potential contributors. And today it means one simple thing: LISP in general and GUILE in particular are NOT suitable.

If I want to write project where number of potential contributors is observable (some internal project, for example) and all of them as Ok with LISP (this means that this list is not just observable today, but that it does not change much over time) then GUILE becomes a contender. But in this case you may decide to drop the "performance critical code in C/C++ and logic in slow LISP" architecture altogether and go with full-blown LISP implementation which is not limited by the need to work as an extension language.

This is niche for LISP extension languages: it's tiny and does not grow all that much over time - and there are quite a few contenders in this area, GUILE is just one of them.

I don't know how Guile performs, but for those who need to interface C with scheme, one question I'd like to see answered is: why Guile, why not bigloo? What are the similarities and differences? Which is easier, which performs better, which is better supported, which is more actively developed?

Does it really matter? In the rare case where you'll be faced with the need to embed Scheme in your C program (perhaps because you already have library written in scheme which you want use) you can just measure speed directly (if your library is even compatible with more then one scheme implementation). In all other cases you should pick some different (i.e.: imperative) language.

There are quite a few good contenders nowadays: Lua, JavaScript (using V8, for example), Python, Ruby, etc. What you absolutely should not do is to even think about LISP or Scheme. As example of GNU Cash and AutoCAD shows you'll need to eventually add imperative language anyway so why not start with this and spare the pain of two extension languages? Indeed, even GNU projects are seeing the light.

GUILE offered the clever solution to the dilemma: it promised "scheme power" under the hood with translators from other, "more mainstream" languages (including translators from imperative languages like ECMAScript). Well, today, 17 years later this promise is still unfulfilled and we are still treated to the repeat of tired and old "everyone should just learn to love scheme" show. Sorry guys, it was pathetic back then, it's even more pathetic today.

P.S. Note that I'm not saying lua or python are great languages. LISP is so much better by a lot of measures. But people seems to accept lua and python much easier - and this the one measure you want very much in an extension language. Everything else is secondary.

Sorry.

Posted Dec 3, 2011 9:45 UTC (Sat) by oldtomas (guest, #72579) [Link]

My "attack" was based on wrong assumptions.

Especially, I misread this:

Now, if I choose language to write complex (but not performance-critical) core of the application... [...] just may be, GUILE will be good choice. But this happens so rarely in practice...

Turns out I was attacking a straw-man.

Wingo: the gnu extension language

Posted Nov 28, 2011 10:38 UTC (Mon) by hthoma (subscriber, #4743) [Link]

> Just look at gnucash. Its higher-order stuff is written in Guile, more or less. What did they do recently? They implemented Python bindings. Presumably, they're doing this for a reason, and the reason is unlikely to be "Guile is the best extension language there is".

GnuCash was my first and only encounter with Guile/Scheme. I wrote some reports that are part of the GnuCash code base. My reports are the result of modifying other reports and lots of try and error.

Eventually I bought "Structure and Interpretation of Computer Programs" and I admit that I learned a lot. But still don't like it. I just don't need functional programming to take some numbers from some accounts and display them in a nice way. For such a task a more C-like language which I am familiar with just suits me better.

I guess you need a degree in computer science to love LISP/Scheme. I am a mere electrical engineer so I love the beauty of a soldering iron or may be assembler language.

Wingo: the gnu extension language

Posted Nov 28, 2011 12:19 UTC (Mon) by spaetz (subscriber, #32870) [Link]

> Just look at gnucash. Its higher-order stuff is written in Guile, more or less. What did they do recently? They implemented Python bindings.

I admire and use gnucash, but it's probably not the best example to point out as a vibrant and meritocratic developer community.

But I agree that *I* am more likely to look at python extensions than to learn new, err, old language paradigms (independent of their technical merit).

Wingo: the gnu extension language

Posted Nov 30, 2011 12:33 UTC (Wed) by marcH (subscriber, #57642) [Link]

> I guess you need a degree in computer science to love LISP/Scheme. I am a mere electrical engineer so I love the beauty of a soldering iron or may be assembler language.

Nice summary.

Functional languages are good for abstraction and proofs. Including proving programs themselves are correct. If you did like maths in school then you will never like functional languages (it's the same thing).

Data/side-effect languages are much better for pushing bits. And writing bugs.

Wingo: the gnu extension language

Posted Nov 28, 2011 0:46 UTC (Mon) by geofft (subscriber, #59789) [Link]

> Lisp has been relevant for 50 years and will stay relevant.

You know even MIT's dropped teaching Scheme in favor of teaching Python, right?

Having worked with the curriculum, it's actually wonderful to be able to assume that MIT students know a language that you can teach real things in (you can teach a web security class worth something using Python, but you can't in Lisp/Scheme -- even Reddit no longer uses Lisp), you can have them work on random FOSS projects in, and you can have them communicate to others in.

Meanwhile, if you want first-class higher-order functions, Python has them. If you want macros, ... you don't want macros.

Wingo: the gnu extension language

Posted Nov 28, 2011 1:56 UTC (Mon) by nix (subscriber, #2304) [Link]

What if you do want macros? Metalua?

Wingo: the gnu extension language

Posted Nov 28, 2011 10:11 UTC (Mon) by epa (subscriber, #39769) [Link]

Ironically Python works well for purely functional higher-order code. It's the more imperative higher-order styles (such as mutable variables captured by a closure) which it lacks.

Wingo: the gnu extension language

Posted Nov 28, 2011 15:24 UTC (Mon) by deepfire (subscriber, #26138) [Link]

> It's the more imperative higher-order styles (such as mutable variables
> captured by a closure) which it lacks.

If "lacks" in this context implies impossibility, then it is false.

If it merely implies inconvenience (you cannot use statements in lambda expressions, you need to punt to a named function), then yes.

Wingo: the gnu extension language

Posted Nov 28, 2011 17:55 UTC (Mon) by epa (subscriber, #39769) [Link]

What I meant is that you cannot create and return a closure over a local variable. The statement/expression distinction and the restriction of lambda functions to expressions only is annoying, but just an inconvenience. As far as I know Python does not have a way to take a reference to a local variable and carry it with you in a returned function reference. You can, of course, create an object instead and return that.

Wingo: the gnu extension language

Posted Nov 28, 2011 18:38 UTC (Mon) by juliank (subscriber, #45896) [Link]

You can do stuff like:
  def make(start):
 
    def f():
      f.n += 1
      return start + f.n
    f.n = 0

    return f
If you want to

Wingo: the gnu extension language

Posted Nov 29, 2011 22:28 UTC (Tue) by tseaver (subscriber, #1544) [Link]

> As far as I know Python does not have a way to take a reference to a
> local variable and carry it with you in a returned function reference.

Python gained lexical scoping by default in version 2.2 (December 2001).
The feature was available as a "source option" in 2.1b2 (July 2001).

damn the torpedoes, i love scheme

Posted Nov 27, 2011 5:52 UTC (Sun) by b7j0c (subscriber, #27559) [Link]

i don't know what it means for guile to be the "official" gnu extension language and it doesn't seem that the label matters. that said, guile2 has a lot going for it and scheme in my mind is still a great language.

even though guile2 has some concurrency support lacking in chicken, it doesn't have the libraries or community. i also like where racket is going with types, which is a true novel direction for scheme that will hopefully attract interest.

anyway, keep up the good work andy, the more the merrier

Wingo: the gnu extension language

Posted Nov 28, 2011 7:39 UTC (Mon) by slashdot (guest, #22014) [Link]

Guile is the GNU extension language just as much as Hurd is the GNU kernel.

Lisp syntax is unnatural and doesn't have easy language support for OOP, so that won't change, while better VM languages support functional programming just as well.

This is exactly what I've talked about...

Posted Nov 28, 2011 8:03 UTC (Mon) by khim (subscriber, #9252) [Link]

Heh. This is exactly what I've talked about here.

Lisp syntax is unnatural and doesn't have easy language support for OOP

Have you actually used LISP? Syntax it acquired taste, yes, but as for language support for OOP is concerned... nothing can be farther from truth. LISP had OOP before other langauges were even conceived. CLOS and GOOPS are both significantly simpler and capable then "modern" alternatives.

while better VM languages support functional programming just as well

And just like OOP they do it in clunky, awkward way with lots of corner cases and other strange limitations.

As far as language itself is concerned GUILE is pretty decent, but as an extension language... this is utter failure, as I've already said.

Wingo: the gnu extension language

Posted Nov 28, 2011 15:07 UTC (Mon) by cmccabe (guest, #60281) [Link]

I wish I knew Lisp a little bit better than I do. Carnegie Mellon teaches SML/OCaml but not so much Lisp any more. The faculty has somewhat of a bias against dynamically typed languages. Well, ok, not so much a bias as an absolute loathing.

It's good for a language to be expressive, but I have to wonder how maintainable the code is going to be when everyone is rolling his own regexp library. I can just open up any random Python or Ruby script and know pretty much what is going on with those regular expressions. But decoding a custom Lisp macro just to understand your little script? That doesn't sound like fun.

I feel like a lot of the newer languages have tackled the real human problems of programming a little bit better than the old-timers think. It's important to have a huge, and highly usable standard library. It's important to have a syntax that is clean and readable. It's important to have frameworks like Ruby on Rails or NumPy. It's important to have style guides that are actually followed by most users so that things written by person A can be understood by person B. Guile doesn't seem to have any of that. Maybe it will in the future, but it doesn't now.

As far as TCL goes... I'm going to have to agree with Andy Wingo's comment that it's "past its prime." Ian Lance Taylor wrote about some of the problems here: http://www.airs.com/blog/archives/495

Wingo: the gnu extension language

Posted Nov 28, 2011 16:38 UTC (Mon) by alankila (subscriber, #47141) [Link]

Whoa. I never knew anything about TCL before, but based on what is written there, TCL is absolutely horrible. Insufficient, in a word. No wonder it isn't more popular.

Wingo: the gnu extension language

Posted Nov 28, 2011 18:08 UTC (Mon) by epa (subscriber, #39769) [Link]

If you haven't had the opportunity to try Tcl then you have missed out. It's really quite fun. Part of it is the absurd simplicity: how just three or four few simple rules like {} for quoting and [] for evaluating can build a powerful language. Tcl is the concept of the 'interpreted language' taken to its logical conclusion, where there is no distinction between program source text and character data.

The traditional way of using Tcl has been to embed it in a larger program (usually in C), whereas with Perl or Python, there is a big interpreter in /usr/bin/perl or /usr/bin/python into which you can then link C libraries at runtime. That has meant you tend to learn the main application first, and Tcl second.

FWIW, I do agree that Tcl is a bit too simple to become the standard language for general-purpose scripting, although it excels at being "the simplest thing that could possibly work" when you need to add scripting facilities to an existing program. I don't see how it follows that the answer is to use a dialect of Scheme together with a set of translator programs to turn other languages into Scheme. (Translating other languages into Tcl would be about as sensible.) Ironically the main criticism of Tcl that RMS originally gave was that "hackers like it" but ordinary users wanting to do basic automation find its syntax awkward. That goes double for any kind of Lisp.

Wingo: the gnu extension language

Posted Nov 28, 2011 18:51 UTC (Mon) by dskoll (subscriber, #1630) [Link]

FWIW, I do agree that Tcl is a bit too simple to become the standard language for general-purpose scripting

I disagree. As others have pointed out, gitk is written in Tcl/Tk. I've written quite a few standalone programs in Tcl/Tk as well as a couple of very large systems that used Tcl as an embedded language. I haven't felt that the language has constrained me in any way.

Wingo: the gnu extension language

Posted Nov 28, 2011 18:55 UTC (Mon) by dlang (✭ supporter ✭, #313) [Link]

most of the code on a tivo is written in Tcl

Wingo: the gnu extension language

Posted Dec 4, 2011 21:11 UTC (Sun) by jzbiciak (✭ supporter ✭, #5246) [Link]

I've written some TCL, and as long as you're not trying to do too much with it, it isn't too bad. But, I still find it somewhat frustrating to work with.

I guess part of my problem is that I'm using it in the context of a large design simulation, and the part I'm writing gets invoked around 15-20 minutes in to the simulation. So, fiddly bits like forgetting that new-lines have special meaning in TCL (and so the open curly of a block needs to be on the same line as the control construct), or using {} around the test of an if, rather than parentheses like so many other languages, end up biting me again and again. It's not that I don't know these rules, it's that my fingers and eyes are simply unaccustomed to them. This situation would not be improved by using Guile.

I guess part of the problem is that I don't program TCL in any other context whatsoever, and so I get reacquainted with its quirks in the most annoying and frustrating way possible: with a deadline, trying to glue two things together while dodging quirks, and with a 20-minute edit-debug cycle. I don't care one whit about how easy it was to embed. If it looked more like some other language I might program elsewhere (say Perl, Python or C), I'd take fewer loops through that edit-debug cycle.

Any TCL I write (and likely any extension code anyone writes) is not the focus or the end product of what that person is doing. Someone using TCL to write a quick extensions (in my case, a mechanism for transporting data out of a CPU running in simulation) is doing so to bridge a gap on the path to the real goal. It's duct tape and baling wire most of the time, pure and simple. Conceptual cleanliness isn't really required. "It works" is mandatory. Quick to write and debug is paramount. That's why I argue "maximum overlap with other kinds of coding you'll have to do" is a great direction to go in for a scripting engine.

At least for me, Guile is a 180 from that--heavy on the conceptual cleanliness, and completely unlike any of the other programming I'd do elsewhere. TCL fares a little better, but still has plenty of quirks that drive me batty.

Debugging embedded

Posted Dec 5, 2011 7:26 UTC (Mon) by oldtomas (guest, #72579) [Link]

I guess part of my problem is that I'm using it in the context of a large design simulation, and the part I'm writing gets invoked around 15-20 minutes in to the simulation. So, fiddly bits like forgetting that new-lines have special meaning in TCL [...] or using {} around the test of an if [...] end up biting me again and again [...]

Wow, thanks for the insight. That's a very strong reminder for people providing embedded interpreters: think of debugging. The environment is often sufficiently different from a "standard REPL" (aka interactive interpreter) that the usual tools/recourses are not readily available.

Two examples which "get it right" are Emacs and Firefox (the latter with a little help from Firebug).

Wingo: the gnu extension language

Posted Dec 5, 2011 9:54 UTC (Mon) by smurf (subscriber, #17840) [Link]

>> This situation would not be improved by using Guile.

At least Guile has fewer than the aforementioned 11 syntax rules. :-P

Wingo: the gnu extension language

Posted Dec 18, 2011 17:17 UTC (Sun) by epa (subscriber, #39769) [Link]

The problem you're banging your head against is not Tcl's quirky syntax but the lack of a separate compile stage. It is a purely interpreted language so even if your 'if' statement is totally malformed you won't find out until run time. If you were using Perl, at least some syntax errors would be caught immediately, and I imagine the same is true for Guile. I suppose one could define a 'strict Tcl' which is a subset of the full language and can be syntax-checked statically.

Wingo: the gnu extension language

Posted Nov 28, 2011 20:38 UTC (Mon) by oldtomas (guest, #72579) [Link]

Thanks for the link. I read the article. I do hold Ian Lance Taylor in high esteem, but IMHO he's being unfair to Tcl.

For one, it does have namespaces. It has its weaknesses (one of them is -- paradoxically! at the same time its greatest stregth -- the "Everything Is A String" principle.

Of course, these days, the internal representation of objects is richer and more efficient: but still every object is "visible" through its canonical string representation. Sometimes this has amusing consequences, e.g. hashes have to be sorted, to guarantee a canonical representation :-)

In my view, the greatest strength is the nearly fanatical attention to detail in the language design and development (a trait Tcl has in common with Lua). The interfaces are usually tidy and predictable -- really nice compared to the steaming mess experienced e.g. in the Java world.

Then there are goodies like Tk (I don't know an easier way to whip up a GUI except perhaps Javascript and the browser. And this one is *heavy*.

Another goody: the event loop.

Sure, Tcl has its weaknesses. But there's a lot to learn from it.

Have a look at <http://wiki.tcl.tk/1369>, just for chuckles :-)

Wingo: the gnu extension language

Posted Nov 28, 2011 22:59 UTC (Mon) by cmccabe (guest, #60281) [Link]

The ski lift thing looks fun :)

TCL sounds like a very regular language without all the weird warts that developed in some other scripting languages, like Bash or Javascript. However, I think if I were building a GUI, I would still use HTML5 + Javascript. You can get HTML looking pretty nice (see GMail or Facebook for an example.) With Tk, you kind of hit a plateau where you just can't improve the way it looks any more. I'm sorry to say that, but that's just the way it is. HTML has other advantages like network transparency, of course.

But I'm not really a GUI guy. I honestly don't know that much about building them. Becoming proficient in any of these languages has been on the TODO list for years, but there's always something higher priority to do. When I have a bash script that outgrows about a page in length, I usually reach for Python or Ruby. These languages just feel familiar somehow-- there's classes and types and a standard library with the usual data structures. I guess other people with different backgrounds might feel differently, though.

Wingo: the gnu extension language

Posted Nov 29, 2011 2:20 UTC (Tue) by dskoll (subscriber, #1630) [Link]

However, I think if I were building a GUI, I would still use HTML5 + Javascript.

Having built GUIs in both HTML/JS and Tcl/Tk, I have to say Tcl/Tk wins hands down. Unfortunately, deploying an HTML "GUI" tends to be easier because everyone already has the required client-side software. :(

Wingo: the gnu extension language

Posted Nov 29, 2011 22:42 UTC (Tue) by wahern (subscriber, #37304) [Link]

One of the cool surprises about OS X I discovered a couple of years ago was that Tcl/Tk appears to be installed by default. Unless it was the developer toolkit which installed it.

I had done next to no GUI work before (outside of the web domain), but had a little RFC viewer up-and-running in a few hours.

Wingo: the gnu extension language

Posted Dec 3, 2011 9:54 UTC (Sat) by oldtomas (guest, #72579) [Link]

One of the cool surprises about OS X

Heh. You made my day!

Lots of OSX at my workplace (I'm the curiosity there, with my Debian GNU/Linux thingie :)

I rushed to the next OSX box there (usually I tend to avoid them a bit), opened a terminal and typed in "wish" -- there it was, in its full glory!

And a decently receent version, 8.5.

Wingo: the gnu extension language

Posted Nov 30, 2011 20:01 UTC (Wed) by dashesy (subscriber, #74652) [Link]

Unless it is some odd platform (that Qt cannot build) why not build native GUI using Qt (then use ECMAScript-based QtScript for extending it)? I would also prefer QML over Tcl/Tk.

Wingo: the gnu extension language

Posted Nov 30, 2011 21:27 UTC (Wed) by dskoll (subscriber, #1630) [Link]

It's just a matter of taste, I suppose. I really like Tcl/Tk so it tends to be my first choice for non-browser-based GUIs.

Wingo: the gnu extension language

Posted Nov 30, 2011 23:49 UTC (Wed) by anselm (subscriber, #2796) [Link]

why not build native GUI using Qt (then use ECMAScript-based QtScript for extending it)?

Because that would be like going grocery shopping with an M1A2 Abrams tank? It'll get you there in the end, but the squashed SUVs and the gas bill are a bit of a nuisance.

Wingo: the gnu extension language

Posted Dec 1, 2011 0:32 UTC (Thu) by HelloWorld (guest, #56129) [Link]

I have never heard of anybody's SUV being squashed by a Qt-based application, so I fail to see your point.

SUV

Posted Dec 3, 2011 9:58 UTC (Sat) by oldtomas (guest, #72579) [Link]

No, but I've seen people pining for more hardware because of all this bloat craze. And this increases our CO2 footprint as well, doesn't it?

SUV

Posted Dec 5, 2011 18:25 UTC (Mon) by dashesy (subscriber, #74652) [Link]

A language that at least compiles to machine code should have a smaller impact on CO2 footprint than one in which everything is string. In addition, (unless hard disk is concerned) older hardware should be able to stick to the basic functionalists of a more powerful library leaving much of the code untouched on the disk. But at the end nothing will stop people from demanding newer hardware when they see it on the other side of the fence.

Copyright © 2011, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds