Why go on with LaTeX front-ends? [LWN.net]

Why go on with LaTeX front-ends?

Posted Jun 6, 2002 21:48 UTC (Thu) by DeletedUser1650 ((unknown), #1650)
Parent article: LyX 1.2.0 released

First of all, I want to thank the LyX developpers for their continued good work in making LaTeX a more generally useful tool.

But then, I want to ask a question which I think is important: why is there so much work and investment in developping user-interface layers over LaTeX, and so little in good projects free of the TeX legacy? Here I do not speak only of LyX, but also of other projects with a similar goal, like whizzyTeX, or the various LaTeX editing environments.

LaTeX is no longer the Right Way

Do not misunderstand me, I agree that TeX was an considerable breakthrough in computer typography, back when D. Knuth designed it, and that LaTeX does a decent job at making TeX a little more usable. But it seems to me that those LaTeX front-ends are a bit schizophrenic... On one hand they recognize that the era of command line interfaces and batch processing is over, and that the reasonable way to work today is with graphical interfaces and real-time processing; but on the other hand, they stay faithful to the common sense belief that "with a visual design system, authors usually produce aesthetically pleasing, but poorly designed documents" (quoting Leslie Lamport).

Actually that is true with "traditional" word processors derived from MacWrite, and including M$-Word, which were initially designed as glorified typewriters. However, WYSIWYG document editors do not necessarily incite the user to produce documents with no or little logical structure. There are other ways to do visual word processing than the M$-Word way.

LyX developers argue that "What You See Is What You Mean" (WYSIWYM) is a good feature. That belief allows them to conciliate their desire for a graphical editor and their belief that WYSIWYG can only lead to unstructured documents. However I argue that WYSIWYM is only a work-around the extreme difficulty to use TeX in a truly WYSIWYG and interactive way.

Actually, the good feature is to be able to choose the exact level of WYSIWYGness you need at a given point. When producing content, WYSIWYM is fine, since at this point visual layout is a distraction, moreover it allows to display structure information which would otherwise be hidden (like index entries) or less handily editable (like footnotes and floating objects). But sometimes you need to show more structure (in way similar to what is allowed by SGML editors). And sometimes you want a very WYSIWYG display, for example, when fine-tuning line and page breaking, or when designing tables or writing formulas.

Additionnaly, anyone who tried to do some elaborate document presentation with LaTeX understand what a hack is the whole TeX/LaTeX system. A useful hack, indeed, but a hack nevertheless. Actually TeX is just batch typesetter with a very elaborate macro processor, LaTeX tries to make it structured. But LaTeX stays very difficult to use when you venture off the road marked by the twenty-or-so basic macro you can make you mother use. Worse, LaTeX documents are notoriously hard to convert to any other format, loosing one of the most interesting features of structured documents: media independance.

To make a parallel with programming languagues, C++ is arguably a pain to program in, and a lot of other programming languagues (like lisp, Perl, Python, Eiffel, Erlang) are vastly superior for the vast majority of software. However, C++ is still widely used because it is useful though ugly. Still, a lot of effort is going in more modern languages. But why do everybody who want high quality typesetting and structured documents stick to TeX?

The LyX developpers do their best to work around the fact that TeX is a 30 years old software which does not know what "user feedback" means, and actually has no support for structured documents behind the LaTeX facade. But they end up with many compromises. They end up implementing a lot of ad-hoc code to provide support for every individual feature of TeX without improving TeX itself.

The next generation

I hope that if you are still reading me, you are assuming that there exists no alternative to TeX which is as good at typesetting, with cleaner typesetting language, complete support for WYSIWYG interactive edition, and as inherently structured document model.

Actually there is one, and it is called GNU TeXmacs. Its only dependences on Knuth work are METAFONT and BibTeX, it does not use TeX at any level. It already implements the features needed for at least 90% of the documents (though floating object support is yet a bit problematic), including a very good mathematic typesetter. It is very mature software, being 4 years old and having already blown its v1.0 candles. And it introduces a number of very innovative features which make it fast and easy to use.

Some of the fine features that I can point out to the tech audience of LWN are the very strong LISP-ish-ness of the document model. Actually, all documents are trees (does not that sounds like XML?) and they are interpreted as a program by the typesetter (does not that sound like Lisp). The page breaking uses a global algorithm which produces better results than TeX, the line stacker properly "shoves" successive lines with big descents (height of ink below the baseline) and big ascents (idem, above the line), and yes, it reimplements the famous line-breaking algorithm of TeX. It can also use its mathematical typesetting abilities to provide a user interface for computer algebra systems, actually putting the CAS session inside your document, and finally, it is extensible with GUILE.

You can find more information on the TeXmacs web site and on the TeXmacs Wiki.

Yet this projet is very usable, has great promises, and has a very high quality code, the core developper team just doubled to reach two persons, and they are having incredibly hard time to find financial and institutional support for the project. So yes, the existing LaTeX converters are not perfect, and no we do not provide (yet) abilities matching every TeX feature, but there is only so much a single person working half-time can do...

So, please, can you tell me why so many people are investing in TeX front-ends, and why so little come to help us making something much, much better?

(Log in to post comments)

Why go on with LaTeX front-ends?

Posted Jun 7, 2002 3:40 UTC (Fri) by DeletedUser1677 ((unknown), #1677) [Link]

The reason to go on with LaTeX front ends is the same reason to go on with using ASCII text files; in fact it is the same reason to use Fortran instead of ADA, and C instead of Haskell. The propellor-head "new and improved abstraction" ways of doing things almost universally offer nothing to the end user (that is to say, me, the scientist who actually uses both LyX and TeXmacs to generate LaTeX output, LaTeX being the lingua franca of peer reviewed journals in the hard sciences). We don't care if the way TeXmacs works internally lights the jets of some computer propellorhead if it doesn't generate crisp, utile LaTeX output. LaTeX output is pretty much the only reason to use such tools. LyX is actually pretty good at doing that. TeXmacs presently isn't, and lacks even the rudiments of the amount of documentation necessary to make it useful or even usable by the average schnook. It has some features which are much better than those in LyX (for example, its interaction with Maxima), but you presently can't use the thing easily enough to take advantage of these features; and some of TeXmacs (the bibliography features for instance) simply doesn't work at all. The fact that it isn't LaTeX internally is not presently an advantage of TeXmacs, possibly excepting for the people who write code for it. People who use LyX almost exclusively come from the technical and scientific community; and we use it for the LaTeX output. We don't care about the "next great thing" in how the program represents data internally -the only reason TeXmacs is useful at all is its LaTeX compatabilities.

Why go on with LaTeX front-ends? Because of LaTeX!

Posted Jun 7, 2002 8:16 UTC (Fri) by DeletedUser1713 ((unknown), #1713) [Link]

There are a few reasons why LaTeX remains prevalent: it is a human-readable established document interchange format, there are loads of available existing solutions for numerous tasks, there is a host of available expertise available.

Both of LyX and TeXmacs don't address this properly IMO: both don't use LaTeX as their external format, both have to export and particularly to import into their own system, and the reexport does not properly reflect the input again. This makes both of them unsuitable for interchange of documents with others: you can not receive one version of a document and send the author a version with a few corrections in it.

The basic design difference between LyX and TeXmacs is the choice of WYSIWYG: TeXmacs' input window is the same as its print output, LyX' isn't.

The disadvantage of this choice for LyX is of course lack of immediate visual feedback. The advantage is that line wrapping, colors, markup of text elements and structures and the choice of screen fonts can be chosen for maximum editing convenience instead of having to reflect the printed output. It is somewhat ironic that the font defaults for LyX do not exploit this possibility: they default to fonts not designed for screen display. My Emacs source buffers are more legible. Also LyX overdoes a few things unnecessarily: like with TeXmacs, inserting text into existing justified paragraphs leads to the distractive concertina effect where a line shrinks on entry until the display engine decides to rewrap the paragraph, and this happens for about every word.

A disadvantage of both systems is massive reinvention of the wheel: apart from implementing their WYSIWYG paradigms, the systems also have to provide editing features. Over the time, literally thousands of editors were created for Free systems. Very few have prevailed. Those that did have received several man decades of development. Just like starting from scratch with regard to the typesetting, starting from scratch with regard to the editing is something which only makes limited sense.

Another disadvantage of both systems is that they have to reimplement every feature available in LaTeX just for the sake of editing. They can in some manner export LaTeX they don't understand (ERT in LyX, conditional LaTeX in TeXmacs), but editing it becomes inconvenient. That is, both systems are not really LaTeX frontends, but merely export to it. With TeXmacs, this is a design decision, with LyX I would more tend to call it a historical accident: LyX at the beginning was intended to be a frontend to LaTeX.

Personally, I am main developer of a system called preview-latex which is used in conjunction with Emacs editing modes. It merely focuses on providing WYSIWYG display of selected and identifiable compositions (math text, figures, section headers, included graphics per default). Both the selection and identification is done by LaTeX, so the system does not get confused easily by intricate TeX programming and is easily extensible. The system does not interfere with normal LaTeX source editing and does not touch the text, merely changes the way in which it is displayed.

It is for that reason entirely suitable for copy editing and document interchange. I have repeatedly been asserted that "merely" the visual feedback offered by this system has been sufficient to cause Windows users to switch to Emacs (XEmacs in this case because GNU Emacs still lacks a released version supporting inline images under Windows) and even to convert vi users. Since it does not obstruct either access to the power of LaTeX nor of Emacs, it saves itself some reinvention of the wheel and makes it likable to power users.

Since the acceptance of this experiment suggests that far less intrusive amounts of WYSIWYG with a focus merely on the display can also be successful, in the long term it would be beneficial if the display engine related handling of TeX typesetting (font handling, line wrapping, page breaking) were abstracted into libraries in such a way as to make them ubiquitous. Ultimately, there is no sense in systems like LyX, TeXmacs, Abiword, OpenWrite and the like all to implement their own versions of math editing, math display, font handling, user interface and so on from scratch. I hope that this may be one aspect that future development from the Omega project might lead to.