Weekly edition Kernel Security Distributions Contact Us Search Archives Calendar Subscribe Write for LWN LWN.net FAQ Sponsors

# More behind the scenes changes...

## More behind the scenes changes...

Posted Jun 18, 2013 17:12 UTC (Tue) by Cyberax (✭ supporter ✭, #52523)
In reply to: More behind the scenes changes... by nix
Parent article: Meeks: LibreOffice's under-the-hood progress in 4.1.0 (beta)

As someone who used to help to publish a journal with papers in TeX (about 10 years ago, AFAIR), I still want to commit a genocide of everyone who uses TeX.

There's no efficient version control of packages, packages constantly break (sometimes each other), they are written in a language that makes M4 look like a paragon of clarity, etc.

It's a pity that nobody is really interested in a TeX replacement.

More behind the scenes changes...

Posted Jun 18, 2013 19:10 UTC (Tue) by jimparis (subscriber, #38647) [Link]

> It's a pity that nobody is really interested in a TeX replacement.

Do you not count, or are you simply unaware, of projects like http://luatex.org/ and http://www.contextgarden.net/‎? Even if they don't meet your goals yet, there are clearly people working towards improving and replacing TeX in an evolutionary way.

More behind the scenes changes...

Posted Jun 19, 2013 10:35 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link]

LuaTeX is pretty dormant at the moment. And ConTeXt is just another macro package, like LaTeX only with a different emphasis.

So no, there is no real movement to make a TeX replacement.

More behind the scenes changes...

Posted Jun 19, 2013 11:11 UTC (Wed) by anselm (subscriber, #2796) [Link]

As the briefest of glances at the LuaTeX web site would have told you, LuaTeX is doing fine and is being actively developed further. If anything the code has settled down to a point where the first books are coming out that expose LuaTeX to a wider audience, which is a good thing. There are reasonably busy mailing lists for developers and users. The project may not be as large and high-profile as, say, the Linux kernel, but to call it »pretty dormant« is basically denying reality.

There is »no real movement to make a TeX replacement« because apparently no such thing is required desperately enough. Those people who care enough about typography don't seem to mind TeX's limitations or else try to address them by evolution rather than replacement (viz. LuaTeX), while people working on similar tools apparently aren't interested enough in good typography to come up with something that rivals TeX's output quality (those projects that do take an interest generally use TeX as a backend for typesetting).

In particular, the office-suite people don't appear to worry about issues like line breaking and hyphenation enough to make use of the theoretical work pioneered in TeX 30 years ago or so. As long as that doesn't happen, TeX is going to stick around (and probably a lot longer).

More behind the scenes changes...

Posted Jun 19, 2013 12:35 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link]

The problem is that TeX is pretty useless for complicated formatting. It also sucks for casual documents because of huge entry barriers. LuaTeX does not look much better, actually (yes, I did check it). The end result is that TeX is mostly used for scientific papers with lots of formulae. Such papers usually follow a pre-set pattern and don't need anything more complicated than a simple table or a chart.

And the lack of a good replacement shows. We now have Markdown, reStructuredText and other similar attempts to do something that TeX should have made easy but haven't.

And about hyphenation, I remember spending DAYS trying to find out how to make TeX to correctly hyphenate German text blocks (with all those nice long compound words) inside a mostly-Russian paper (and Russian has its own hyphenation rules, of course). Microsoft Word handled it out of the box.

More behind the scenes changes...

Posted Jun 19, 2013 13:30 UTC (Wed) by anselm (subscriber, #2796) [Link]

The problem is that TeX is pretty useless for complicated formatting.

Speak for yourself. I've personally used TeX to format all sorts of unusual and complicated things from CD inlays to magazines and complete books. »Useless« is a strong word to describe something that is very powerful indeed and often able to do things that seemingly no other piece of software can. I've talked to people who use TeX to make interactive PDF files out of gel electrophoresis scans because Adobe's famous tools couldn't handle the document sizes concerned.

We now have Markdown, reStructuredText and other similar attempts to do something that TeX should have made easy but haven't.

And guess what these tools do when publication-quality output is desired? Right. They generate input for TeX. Markdown etc. are good for what they do – making it possible to format simple documents for the web and printed output – but they really suck at complicated formatting.

And about hyphenation, I remember spending DAYS trying to find out how to make TeX to correctly hyphenate German text blocks

It seems to be a pattern with you to conclude that if you can't do something, nobody else can do it either. Consider that years ago TeX made it possible to hyphenate Greek text inside (right-to-left) Hebrew text inside English text correctly. Now that we have things like Unicode and OpenType fonts, things are only getting easier.

More behind the scenes changes...

Posted Jun 19, 2013 13:51 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link]

"Useless" is a strong word, but it's also true. Even Knuth's works are not that complicated compared to some chart-rich PDFs that I've seen.

In particular, tables are a total mess. Paginated tables with correct handling of subheaders and automatic column widths are non-existent. I've seen people resorting to manual size tweaking all too often. You certainly can do whatever you want with absolute sizes, but that kinda negates the purpose of a markup language.

> And guess what these tools do when publication-quality output is desired? Right. They generate input for TeX.
No they don't. People simply make HTML and use something like InDesign.

> It seems to be a pattern with you to conclude that if you can't do something, nobody else can do it either.
I consider that a I'm at least as advanced as an average user. And if I can't get something to work within a reasonable timeframe then most users probably won't be able to do it as well.

And that's a not a mark of a good tool.

>Consider that years ago TeX made it possible to hyphenate Greek text inside (right-to-left) Hebrew text inside English text correctly. Now that we have things like Unicode and OpenType fonts, things are only getting easier.
Yeah. Except that TeX still has only recently gained full UTF support (with tons of packages still not rendering text correctly sometimes). Back then it was the bibliography entries with references that gave me tons of problems.

More behind the scenes changes...

Posted Jun 19, 2013 14:21 UTC (Wed) by anselm (subscriber, #2796) [Link]

Even Knuth's works are not that complicated compared to some chart-rich PDFs that I've seen.

These charts presumably aren't drawn using Microsoft Word or Adobe InDesign but using some other more specialised program. This means that these charts could just as well be included in a TeX document. As a matter of fact, there are absolutely amazing things you can do with graphics in pure TeX these days, without having to resort to external programs. Check out this site for some examples.

Paginated tables with correct handling of subheaders and automatic column widths are non-existent.

Funny but I just finished a LaTeX document the other day which is basically one long 20-plus-page variable-column-width table. It's probably just as well that I didn't listen to you first.

People simply make HTML and use something like InDesign.

That's not what the Sphinx manual says to do :^) Also, I would be very surprised if, say, ReST+HTML+InDesign led to nicer output than ReST+TeX, at least without extensive manual tweaking. (Incidentally, InDesign internally uses various algorithms from TeX.)

Back then it was the bibliography entries with references that gave me tons of problems.

I can see where that might be a problem, but unless I'm gravely mistaken this has nothing to do with TeX, which has been able to do multi-language hyphenation for 25 years or so.

It is always interesting to see how people are prepared to diss TeX based on some terrible experience they had years ago. It turns out that many things that used to be fairly difficult (like designing chapter openings) have now become quite straightforward. TeX may have its idiosyncrasies (although not more of them than other popular and well-known software systems) but as far as output quality is concerned it is still second to none, and it is used productively in many more places than one would at first suspect – which is a lot more than can be said for most other 30-year-old application programs. It is safe to say that TeX will be with us for a very long time.

More behind the scenes changes...

Posted Jun 19, 2013 14:45 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link]

> These charts presumably aren't drawn using Microsoft Word or Adobe InDesign but using some other more specialised program.

>This means that these charts could just as well be included in a TeX document. As a matter of fact, there are absolutely amazing things you can do with graphics in pure TeX these days, without having to resort to external programs. Check out this site for some examples.
Yes, and you might note the curious absence of complicated tables in these examples. Or stuff like angled text (surprisingly often necessary and very complicated in TeX).

Ability to generate diagrams directly in TeX is nice, until you try to understand how it actually works.

Damn, even mentioning angled and rotated text is enough to trigger a PTSD in me. I remember countless hours trying to make tables work with \rotatebox correctly with automatic resizing.

> That's not what the Sphinx manual says to do :^) Also, I would be very surprised if, say, ReST+HTML+InDesign led to nicer output than ReST+TeX, at least without extensive manual tweaking. (Incidentally, InDesign internally uses various algorithms from TeX.)
There's nothing wrong with TeX core algorithms. It's just everything else that sucks.

> Funny but I just finished a LaTeX document the other day which is basically one long 20-plus-page variable-column-width table. It's probably just as well that I didn't listen to you first.
What package have you used for it? I'm totally interested in replacing JasperReports with a TeX-based report generator. So far I haven't found anything working out there.

> I can see where that might be a problem, but unless I'm gravely mistaken this has nothing to do with TeX, which has been able to do multi-language hyphenation for 25 years or so.
Yes, the problem with TeX is the lack of a coherent core. It's a hodge-podge of various macro packages, that don't really cooperate well. And the gulf between a casual user and macro package writer is a huge one.

More behind the scenes changes...

Posted Jun 19, 2013 15:13 UTC (Wed) by anselm (subscriber, #2796) [Link]

you might note the curious absence of complicated tables in these examples.

Which should not come as a huge surprise given that PGF and TikZ are tools for graphics, not tables. As a matter of fact angled text is not a problem for TikZ at all – see, for example, this picture. TikZ can even typeset text along arbitrary curved paths.

What package have you used for it?

ltxtable (nothing special there). There are some limitations with this, notably the fact that page breaks can only occur between cells, not within a cell – which wasn't an issue in my application. If that is a problem then, depending on the source material, there are various other approaches that one could take, including using a sequence of appropriately constructed paragraphs instead of a table. I would expect other (non-TeX) packages to have similar issues, simply because doing that sort of thing isn't exactly great typography to begin with.

And the gulf between a casual user and macro package writer is a huge one.

Which is one of the issues that LuaTeX is trying to address. I'm personally not a great fan of Lua, but in TeX it does make a few things easier that used to be very difficult.

More behind the scenes changes...

Posted Jun 20, 2013 0:20 UTC (Thu) by mathstuf (subscriber, #69389) [Link]

making it possible to format simple documents for the web and printed output – but they really suck at complicated formatting.
For reStructuredText in particular, getting nested markup is an exercise in frustration. Try making a bolded link with the title "TikZ" in an rst file.

More behind the scenes changes...

Posted Jun 20, 2013 7:24 UTC (Thu) by nim-nim (subscriber, #34454) [Link]

> Now that we have things like Unicode and OpenType fonts, things are only
> getting easier.

TEX completely missed the OpenType revolution, it's still playing catch up just to use them at the basic level (using advanced OpenType features that LibreOffice can at least partially use, despite the years of SUN mismanagement, is not on the horizon).

Take any TEX distribution, remove all pre-OpenType fonts, and things are going to break right and left. Even the parts which are supposed to be opentype-ready fall back on legacy fonts now and then.

Even the STIX project, which took years to produce comprehensive technical/math fonts, aiming squarely for TEX's sweet spot, had to take another year to release a special 1.1.0-latex font distribution (I kid you not, that's the official version name) to workaround TEX limitations. TEX was completely, utterly, distressingly unable to exploit the previous opentype STIX fonts distribution that all other software suites could happily use. Taking one year to borgify the fonts to the n-th level and convert them to the obsolete type1 format was deemed easier than fixing TEX.

When TEX distributions drop all special TEXified fonts and make use of the system OpenType fonts without any preprocessing (including wide-encoding smart fonts), exploiting a generic OpenType engine such as harfbuzz (and not a pile of brittle one-shot macros), then they'll be able to claim OpenType support.

More behind the scenes changes...

Posted Jun 20, 2013 8:14 UTC (Thu) by anselm (subscriber, #2796) [Link]

TEX completely missed the OpenType revolution, it's still playing catch up just to use them at the basic level.

On the other hand, it took the rest of the industry 20 years, several false starts and finally OpenType to catch up with what TeX could do with fonts in the 1980s. The font side of TeX (Metafont) was way ahead of anything else at the time and still is in various respects; it's too bad that the rest of the world went with the Adobe stuff (which incidentally postdates TeX and Metafont).

When TEX distributions drop all special TEXified fonts and make use of the system OpenType fonts without any preprocessing (including wide-encoding smart fonts), exploiting a generic OpenType engine such as harfbuzz (and not a pile of brittle one-shot macros), then they'll be able to claim OpenType support.

One of the ideas behind projects like LuaTeX is to make that sort of thing easier. In the meantime TeX isn't going away; as long as the other (free, in particular) projects suck as much at professional-quality typography as they do TeX has nothing to fear. What's the point of having good fonts if you can't typeset worth a damn with them?

More behind the scenes changes...

Posted Jun 20, 2013 10:01 UTC (Thu) by nim-nim (subscriber, #34454) [Link]

> One of the ideas behind projects like LuaTeX is to make that sort of
> thing easier.

I wouldn't be that optimistic
http://newsgroups.derkeiler.com/Archive/Comp/comp.text.te...

TEX is still searching its OpenType way.

More behind the scenes changes...

Posted Jun 20, 2013 10:25 UTC (Thu) by anselm (subscriber, #2796) [Link]

If you're really interested in OpenType, the TeX implementation to use is probably XeTeX. The recently-published TeXLive 2013 contains the current version of XeTeX, which uses HarfBuzz (among other things).

As far as the message you quoted is concerned, I don't read this as »this will never get done«. We'll probably get there eventually.

In the meantime TeX is still a very viable contender in its field, if only because (a) it is very well-established and (b) those programs that produce good output (even the pricey ones like Adobe InDesign) suck in various other respects when it comes to, e.g., book-length documents that TeX handles as a matter of course, while those programs that are less weird and idiosyncratic can't touch TeX's output quality or are otherwise limited in what they can do.

More behind the scenes changes...

Posted Jun 20, 2013 17:14 UTC (Thu) by nim-nim (subscriber, #34454) [Link]

I never stated it would never be done. The first message of mine you replied to stated it was not done yet “TEX completely missed the OpenType revolution, it's still playing catch up”.

I guess you now agree “We'll probably get there eventually.”

More behind the scenes changes...

Posted Jun 20, 2013 17:23 UTC (Thu) by nim-nim (subscriber, #34454) [Link]

BTW the advancement Xetex-side can be attributed in a large part to the wonderful Khaled Hosny, who is also active at LibreOffice and fonts-side.

More behind the scenes changes...

Posted Jun 20, 2013 17:49 UTC (Thu) by anselm (subscriber, #2796) [Link]

Yep, that is one man who should not have to pay for any of his drinks whenever he is around TeX or LibreOffice people.

More behind the scenes changes...

Posted Jun 20, 2013 17:39 UTC (Thu) by anselm (subscriber, #2796) [Link]

XeTeX (which I haven't personally used a lot) is in various respects farther along than LuaTeX as far as OpenType support is concerned and in fact seems to be rather good at OpenType these days. It is of course missing all the Lua stuff that is in LuaTeX.

On the other hand, LuaTeX – which the message you quoted referred to – has some catching-up to do in the OpenType department, although it can do a few things even now that XeTeX can't (for example certain »microtypography« tricks); I'm told it is also better at, e.g., Arabic typesetting, which is one of the places where the Lua support really comes in handy.

It's complicated ;^)