Using Common Lisp in Emacs

Posted Nov 15, 2023 7:45 UTC (Wed) by rsidd (subscriber, #2582)
In reply to: Using Common Lisp in Emacs by neggles
Parent article: Using Common Lisp in Emacs

And it's not even clear what's not to like. What's wrong with keyword arguments? It generally makes code easier to read.

Using Common Lisp in Emacs

Posted Nov 15, 2023 7:59 UTC (Wed) by jem (subscriber, #24231) [Link] (17 responses)

This is not just about keyword arguments. From the mailing list:

>Btw, the above is a very simple use of cl-loop. We have quite a few of much more complex ones. For example:
>(cl-loop
>  with comp-ctxt = (make-comp-cstr-ctxt)
>  with h = (make-hash-table :test #'eq)
>  for (f type-spec) in comp-known-type-specifiers
>  for cstr = (comp-type-spec-to-cstr type-spec)
>  do (puthash f cstr h)
>  finally return h)

Boy that is hard to understand.

Using Common Lisp in Emacs

Posted Nov 15, 2023 10:33 UTC (Wed) by spacefrogg (subscriber, #119608) [Link] (16 responses)

I believe you know already that the loop code is equivalent to something like:

(let* ((comp-ctxt (make-comp-cstr-ctxt))
       (h (make-hash-table :test #'eq)))
  (dolist (x comp-known-type-specifiers h)
    (let* ((f (car x)
           (type-spec (cdr x))
           (cstr (comp-type-spec-to-cstr type-spec)))
      (puthash f cstr h))))

And now tell my how this, anywhere on earth, is easier to understand than the cl-loop macro. If you know anything about the cl-loop macro, you realise that "with ... =" is a let binding before the loop and "for ... =" is a let binding inside the loop. "for ... in" is obviously the loop itself. And even if you don't know cl-loop specifically. You can roughly infer the idea by just using common understanding of the English language and common use of trigger words like "with" and "for". Even elisp itself uses the with-* metaphor to signify temporary bindings. So I call this whole previous comment a straw man. You may dislike the loop macro anyway you want. I do, too. But calling it unreadable is not serving any constructive purpose.

Using Common Lisp in Emacs

Posted Nov 15, 2023 10:58 UTC (Wed) by Phantom_Hoover (subscriber, #167627) [Link] (6 responses)

The code snippet you posted is in Lisp, which most Lisp programmers can be presumed to know. The snippet using loop is written in CL’s loop-macro sublanguage, which Elisp programmers are understandably entirely unfamiliar.

The fact that some Lisp advocates seem to seriously think ‘you use macros to create a DSL to solve your problems’ is an incredible selling point frankly baffles me. At best you’re reducing a variety of medium-difficulty problems into the hard problem of designing a decent computer language, and at worst… well if you asked me to maintain a legacy codebase full of ad-hoc half-finished DSLs I’d run a mile in the other direction.

Using Common Lisp in Emacs

Posted Nov 16, 2023 3:02 UTC (Thu) by NYKevin (subscriber, #129325) [Link] (5 responses)

> The fact that some Lisp advocates seem to seriously think ‘you use macros to create a DSL to solve your problems’ is an incredible selling point frankly baffles me.

As a complete outsider to this discussion... every time I read someone's argument for Why Lisp Is Great™, DSLs are always at the top of the list. Have I been misinformed?

I don't use Lisp, so I can't judge for myself.

Using Common Lisp in Emacs

Posted Nov 16, 2023 6:51 UTC (Thu) by jem (subscriber, #24231) [Link]

In this specific case, the "domain" is writing loops in Lisp programs. That is, somebody thought normal Lisp syntax is not expressive enough, or gets too unwieldy for writing loops, so a totally new sub language had to be invented. Not only that, the sub language is very un-lispy, including, among other things, an infix "=" operator instead of the standard "let" form, for no obvious reason.

This reminded me of the attempt to introduce infix notation in Scheme, because prefix notation "does not feel natural". https://srfi.schemers.org/srfi-105/srfi-105.html

Using Common Lisp in Emacs

Posted Nov 16, 2023 13:02 UTC (Thu) by Phantom_Hoover (subscriber, #167627) [Link] (3 responses)

I wouldn’t say you’re ‘misinformed’, this is very much my own hot take. But I have had to maintain and extend software written in a DSL by someone who was clearly bored and had since left. Fortunately it was a simple script that could be unpicked in an hour or two, and easily rewritten in the base language; if it had been substantially more complex, and used more of the DSL’s unique functionality, understanding what it was actually doing and fixing or extending it would have been extremely difficult — I’d probably still have extracted all the logic and rewritten it in the base language that I and my colleagues are familiar with.

One of the most difficult and important challenges in software design is making architectures that are flexible, extensible and maintainable by many people over many years, and language design is a fairly pure example of that. It’s insane to me that anyone thinks it’s a good choice of tool for problems like formatting a string or writing loops. Remember that CL’s format and loop DSLs are mature standards, despite all their complexity — imagine trying to work with them in the more common scenario where they’re legacy code, half finished and written by someone you can’t contact.

I honestly think the reason Lisp fans talk so glowingly about this approach is that CL is only used by hobbyists or small companies with low churn. (I do still like it a lot though, it was the second language I learned and I’d happily work with it over most mainstream languages.)

Using Common Lisp in Emacs

Posted Nov 16, 2023 17:11 UTC (Thu) by Wol (subscriber, #4433) [Link] (2 responses)

> I honestly think the reason Lisp fans talk so glowingly about this approach is that CL is only used by hobbyists or small companies with low churn. (I do still like it a lot though, it was the second language I learned and I’d happily work with it over most mainstream languages.)

The other thing is, like me with Pick and Relational, Lisp is actually fundamentally different to other languages in many ways. Like Forth is fundamentally different. You know the quote about "BASIC considered harmful"?

The languages you know shape the way you think. For example, comparing those three languages, BASIC encourages/d long monolithic procedural code. There's no reason for it to be spaghetti, but it easily slips into it. Forth it's pretty much impossible to write monolithic code, it is structured functional code. Lisp, as its name implies, is very much data-driven structures. Those are three completely different ways of thinking, and for example comaparing C code written by a C expert with C code written by a Lisp expert who is good at C, even with them both tackling the same project their code will have a completely different feel. And quite possibly they would have difficulty understanding WHY the other one had done things a certain way, even if both ways worked very well. Lisp programmers think their approach is superior, and actually I believe the evidence bears that out. Even if writing C in the Lisp style actually results in a more efficient program!

Like me, my data tables have a completely different feel, in all likelihood, to a Relational guy. Because my Pick heritage means tables "just want" to be fourth normal form, because I do an EAR. My mindeset, the way I see data and code, is completely different to the way a Relational guy sees it. And of course, I think my way is best :-)

(The same analysis applies to natural languages. I'm multi-lingual - European languages only - and there are plenty of examples where different nations see things differently because only one language has the words to express something. Or the overtones of a direct translation give completely the wrong meaning - compare the English "borgeois" with the German "gut burgerlich", for example!)

Cheers,
Wol

Using Common Lisp in Emacs

Posted Nov 16, 2023 18:32 UTC (Thu) by mpr22 (subscriber, #60784) [Link] (1 responses)

You know the quote about "BASIC considered harmful"?

I do.

I also know people who studied computer science in the 1990s having had their first exposure to computer programming be 8-bit microcomputer BASICs; quite a few of those BASICs were arguably worse than the Dartmouth BASIC of 1975 that Dr Dijkstra was familiar with.

Their minds have never struck me as "mutilated".

Using Common Lisp in Emacs

Posted Nov 16, 2023 23:02 UTC (Thu) by Wol (subscriber, #4433) [Link]

Well, my programming language of choice is DataBASIC.

But as I say, my point is not that one language is better or worse than another, but that it shapes the way you see the world. The first languages you are taught often have a major impact on how easily you learn others. My first language was FORTRAN, and I'm sure (if you know FORTRAN), you would see the heritage even in the code I write today, 40 years on. As a simple example, when writing C I almost always use arrays, not pointers ... they may be functionally identical (near enough), but that's what seems natural to me. (And when given a C programming test by a recruitment consultant, my colleague and I got some of the highest scores the consultant had seen ...)

Cheers,
Wol

Using Common Lisp in Emacs

Posted Nov 15, 2023 11:40 UTC (Wed) by mti (subscriber, #5390) [Link] (7 responses)

Actually for me the first version with cl-loop was completely unreadable while the version without cl-loop was mostly understandable from what I remember from the Scheme course I read 30 years ago.

When not using a language every day it is much easier to read code that uses a few simple constructs.

When using a language every day it may be different.

Using Common Lisp in Emacs

Posted Nov 15, 2023 12:37 UTC (Wed) by spacefrogg (subscriber, #119608) [Link] (6 responses)

With all due respect, I don't believe a single word of "completely unreadable". You had a well enough understanding of the loop to identify it as such. In the elisp code, you literally have to first spot the dolist deep in the middle of the code block, to even identify that it is a loop, we are talking about. Otherwise it is just a stack of bindings with no context.

The loop, with all its weaknesses, I grant you that, captures the meaning of the code much better, because everything that belongs to the loop is encapsulated by it. To do such abstractions is an essential property of high-level programming languages.

I find arbitrarily selected "I reject any further abstractions from this point on" to be no contribution to any debate about computer programming. There are valid situations to reject needless abstractions, but capturing loops is certainly not one of them.

All programs invent abstractions. When those abstractions are not represented in the programming language, they are called protocols or API.

Using Common Lisp in Emacs

Posted Nov 15, 2023 14:05 UTC (Wed) by mti (subscriber, #5390) [Link] (5 responses)

OK, maybe not completely unreadable. I assumed that cl-loop was some kind of loop based on the name. But my experience from other programming languages would also lead me to assume that the two 'for' also introduced loops. So I assumed it was three nested loops. And 'finally' reminds me of finally in Java and other languages. But I don't think that is the meaning here.

So, almost, but not completely unreadable ;-)

There is a balance in how much abstraction you use at the syntax level. Short common idioms are good for the experienced developer but can be hard to understand for the occasional developer.

Or expressed in another way. If I was developing Common Lisp software I would probably use cl-loop myself (*). When occasionally looking at elisp code I would prefer that the code does not use cl-loop.

(*) I assume it is called just 'loop' in CL, 'cl-' just being a prefix used in elisp?

Using Common Lisp in Emacs

Posted Nov 15, 2023 17:41 UTC (Wed) by louai (guest, #58033) [Link] (4 responses)

Yes in Common Lisp it's just LOOP. You might write something like this:


(defun frob-type-specifiers (specifiers)
  (loop with comp-ctxt = (make-comp-cstr-ctxt)
        with h = (make-hash-table :test 'eq)
        for (f type-spec) in specifiers
        for cstr = (comp-type-spec-to-cstr type-spec)
        do (setf (gethash f h) cstr)
        finally return h))

Roughly equivalent to this C++ code with made-up types:


std::unordered_map<f_t, cstr_t>
frob_type_specifiers(const std::vector<std::pair<f_t, spec_t>> &specifiers) {
  auto comp_ctx = make_comp_cstr_ctxt();
  std::unordered_map<f_t, cstr_t> h;

  for (auto &it : specifiers) {
    auto f = it.second.first;
    auto type_spec = it.second.second;
    auto cstr = comp_type_spec_to_cstr(type_spec);
    h[f] = cstr;
  }

  return h;
}

To a trained eye the loop construct is much easier to grok since it encapsulates all the relevant variables in one place. The with clauses create variables that have block scope. The for clauses create variables that have loop iteration scope. The do clause does something on every iteration. The finally clause can be used to do something when the loop terminates.

Using Common Lisp in Emacs

Posted Nov 15, 2023 17:44 UTC (Wed) by louai (guest, #58033) [Link]

Of course those C++ assignments should be


    auto f = it.first;
    auto type_spec = it.second;

Using Common Lisp in Emacs

Posted Nov 16, 2023 18:52 UTC (Thu) by sfink (guest, #6405) [Link] (2 responses)

I'm with @mti, I have a dim distant familiarity with Lisp, and the `cl-loop` version is much harder to understand.

First of all, what's the point of `with` bindings when they're outside of the loop anyway? Why should they be considered "part of the loop"? What's wrong with using `let*` to create an environment with the correct scope, given that that's what the programmer is expressing? I don't get it, and that made it harder for me to understand the `cl-loop` thing since I thought it couldn't possibly be doing the thing that seemed like a bad idea to do, so I tried to come up with some other meaning.

Second, the `for` keyword is absolutely baffling if you aren't already familiar. I would also assume it was doing some sort of triply-nested loop. If you want bindings in the loop body, wouldn't the most straightforward way to do that be to... put the bindings in the loop body?

`cl-loop`/`loop` just feels to me like it's trying too hard to be declarative or something, which creates a level-of-abstraction separation between the loop and its surroundings that feels jarring. Declarative is great. Imperative is ok. Mixing the two is a mess, and the benefits really have to pay for themselves.

I feel like I'm just not getting it. Which is unsurprising, since I've written almost no code in either CL or Elisp for multiple decades, and I'm unfamiliar with the conventions and space. So my opinion shouldn't carry much weight. But I'm offering it up as a datapoint about an external perspective.

Using Common Lisp in Emacs

Posted Nov 16, 2023 20:01 UTC (Thu) by louai (guest, #58033) [Link] (1 responses)

Here is a detailed look at the LOOP macro. Admittedly it is divisive - some people love it, some hate it. But it's also very powerful. I'll pick an example from the link above with a couple comments:


;; List of 100 random numbers under 10000
(defparameter *random* (loop repeat 100 collect (random 10000)))

;; Iterate over *random* counting even and odd numbers, calculating
;; a total sum, and grabbing smallest and largest elements
(loop for i in *random*
   counting (evenp i) into evens
   counting (oddp i) into odds
   summing i into total
   maximizing i into max
   minimizing i into min
   finally (return (list min max total evens odds)))

Using Common Lisp in Emacs

Posted Nov 16, 2023 20:07 UTC (Thu) by louai (guest, #58033) [Link]

Apologies for the self-reply, I accidentally hit publish when I meant preview. Here's an example from a personal project, to convert LEB128 octets to integers:


(declaim (inline unsigned->signed))
(defun unsigned->signed (n size)
  (logior n (- (mask-field (byte 1 (1- size)) n))))

(defun leb128->integer (chunks &optional signed)
  (loop for bits from 0 by 7
	for chunk in chunks
	for septet = (mask-field (byte 7 0) chunk)
	for result = septet then (dpb septet (byte 7 bits) result)
	finally (return (if signed
			    (unsigned->signed result bits)
			    result))))

It's a powerful tool that makes for very succinct loops. Like any powerful tool it does require some practice to use properly. I feel that is very much in the Emacs tradition though.

Using Common Lisp in Emacs

Posted Dec 21, 2024 15:22 UTC (Sat) by bpearlmutter (subscriber, #14693) [Link]

Minor bug: you forgot to return h. You need an h after the (dolist ...).

Using Common Lisp in Emacs

Posted Nov 15, 2023 17:57 UTC (Wed) by iabervon (subscriber, #722) [Link]

I think that the syntax of keyword arguments in Common Lisp is very hard to follow if you learned a Lisp dialect without keyword arguments. It's a big departure from how members of an S-expression can relate to each other in R4RS Scheme, for example, where tokens in the middle of an S-expression simply don't gather up other tokens into implicit logical units.