LWN: Comments on "LuaTeX comes of age"

LuaTeX comes of age

Hybbel — Thu, 04 Mar 2021 13:40:30 +0000

> using TeX and it failed miserably - all the table packages could really deal only with simplest layouts

Tables in LaTeX is still a mess. It can be seen by the number of packages that do table typesetting. Each solves a particular aspect of the madness, but there is not a single one that can be used for every occasion. tabu I think was the most advanced, but now it is not supported. Whenever the are a lot of tables, it's always a voodoo dance.

LuaTeX comes of age

nix — Fri, 08 Sep 2017 21:37:53 +0000

In a similar but inverted vein, there was PassiveTeX, because *obviously* TeX macros are the right tool to use for XSL processing.

LuaTeX comes of age

michal_h21 — Mon, 04 Sep 2017 10:56:40 +0000

Yes, tex4ht uses real TeX interpreter, it only inserts html codes in some macros such as sections or lists to add some meaning to the output. It also supports conversion of generated DVI to images, which is useful for various diagrams.

I've also created experimental project which uses Lua callbacks instead of the DVI processing, but the downside is that it doesn't support tex4ht features which requires some DVI tricks, such as the image processing or some tricks with math.

It can be found here: https://github.com/michal-h21/lua4ht

LuaTeX comes of age

marm — Sun, 03 Sep 2017 10:06:25 +0000

Please send me an e-mail to mj@ucw.cz and I will reply when it's ready.

LuaTeX comes of age

callegar — Sat, 02 Sep 2017 16:54:27 +0000

If I remember correctly also tex4ht used the "real" tex interpreter. In that case there was an auxiliary program to "disassemble" the dvi stream. Being able to do the same /inside/ luatex with no external programs and working at the TeX list level seems a very nice improvement! Looking forward to try it!

LuaTeX comes of age

leephillips — Sat, 02 Sep 2017 13:28:12 +0000

I'd be really interested in staying informed about progress on this project. Any way of doing that?

LuaTeX comes of age

marm — Sat, 02 Sep 2017 12:58:05 +0000

Ad 5) I have a soon-to-be published experimental code which bends LuaTeX to produce HTML/EPUB instead of PDF. It still has a number of ugly corner cases (mostly due to limitations and bugs of EPUB readers), but overall it works quite well.

Unlike usual TeX-to-anything convertors, it does not try to parse the source file. Instead, it lets the TeX engine interpret macros and build paragraphs and then it uses Lua code to disassemble the resulting lists and translate them to HTML.

LuaTeX comes of age

callegar — Sat, 02 Sep 2017 06:48:21 +0000

Thanks for the quick and helpful answer.

With respect to point 2) you are certainly right to say that you should select fonts that have good glyph coverage. Unfortunately, in many cases the choice is severely conditioned by other people choices and/or by the need to rely on free fonts that match sufficiently well some professional font. In these cases having partial glyph coverage is unfortunately quite frequently. I got the feeling that the ucharclasses package might help, but when I tested it did not work right. Time to test again, probably.

LuaTeX comes of age

sfeam — Fri, 01 Sep 2017 16:57:55 +0000

I also have been using LuaTeX (via LyX + texlive) for a while now. I can't answer any of your questions, but at least with regard to point 2 (missing glyphs in fonts) I have had the opposite experience. In fact the reason I started using LuaTeX in the first place was that it solved a problem I had been having with xetex/pdftex that some glyphs in embedded pdf figures would disappear after processing even though the pdf figure displayed correctly when viewed on its own. For whatever reason, LuaTeX has been more successful in retrieving, converting, or whatever it does when encountering an uncommon glyph.

LuaTeX comes of age

leephillips — Fri, 01 Sep 2017 16:29:50 +0000

Many interesting questions here. Right now I have a minute to try to reply to a few:

1) I've seen claims that the slowness you've discovered is mostly in the font loading routines. I think this has a good chance of being fixed some day not too far off.

2) If you're going to use Unicode you need to make sure that you've selected fonts that have good glyph coverage. But I think you can do what you want ("Use this font as the main font, but take the Greek characters from that font") with some fontspec commands (can be used with XeTeX as well as LuaTeX, but you may need to specify font names differently in the two cases), like:

\newfontfamily\greekfont[Script=Greek]{fontname}

4) This has been improving; I think we will eventually have full microtype support in LuaTeX.

5) In https://lwn.net/Articles/640302/, I show automated network graph layouts, which are done behind the scenes with Lua scripting. I think some of your other ideas are being worked on. These are hard problems!

LuaTeX comes of age

callegar — Fri, 01 Sep 2017 10:34:44 +0000

I am using it extensively, since TeXLive 2016, which I have in Ubuntu 17.04, and I am finding some minor problems not knowing whether they originate from my own ignorance or not, so I take the occasion to ask:

1) Slowliness. This seems to be (at least partially, but probably in a major way) dependent on the fonts being used. Conversely xetex is incredibly fast with the same fonts. Is there any fundamental reason why LuaTex cannot be speed competitive with xetex? Is the slowliness of LuaTex under consideration now that the goal of reaching function-completeness is reached? I have a feeling that being so computationally intensive, LuaTex won't find much space on cloud based LaTeX implementations such as Overleaf, Sharelatex, etc. (because the same hardware handles way less users with lualatex than with pdflatex). This could severely hinder the LuaTex acceptance.

2) Missing glyphs in fonts. Many widely used fonts miss some glyph sets. Typically, some popular fonts miss Greek letters, which are an important part of scientific typesetting. When using things like Word or Libreoffice, you do not even notice this fact, because the missing glyphs are seamlessly taken from other fonts. Is there a similar mechanism in LuaTex? Can I say "Use this font as the main font, but take the Greek characters from that font"?

3) Conversion of legacy classes to luatex. Is there some "porting" guide? I'd be delighted to see the IEEEtran class ported to lualatex.

4) Full microtype support. Is there some fundamental reason why some microtype features cannot be supported in LuaTex? I know that this is true for xetex, and I wonder if it is only a matter of time for microtype to catch up or if luatex misses some features that pdftex had.

5) "killer" luatex packages. So far, the set of packages using luatex is still very small and, most important, too thin to offer many compelling reasons for using luatex over xetex if you just seek unicode support. I wonder if luatex packages are on development for "fixing" things that so far proved too hard to address seamlessly in tex. I am thinking of things like:

- typesetting on a grid (which improves appearance when printing on thin paper);

- balanced colunms on the last page;

- correct handling and ordering of wide floats in two column mode;

- better support for multipage tables with automatically computed column widths (which incidentally would greatly benefit things like sphynx);

- shaded/boxed text that supports page/column breaks without glitches;

- margin comments that know how to push both the comment and the related text on the next page when the comment would come too low on the page (preferring a shorter page to a page with a broken margin paragraph)

- frame based formatting with frames that flow one into the other;

- figures anchored to the page with text flowing around them (which is in fact a special case of the previous point).

For all these things (pdf)latex has some packages, but all of them have broken corner cases. The ugly of working with current latex is precisely that too often you run in some corner case causing you to spend a huge amount of time to work it around.

I am also curious at how the luatex authors consider projects like "sile" where the premise is that "Most of the development of TeX since Knuth’s era has involved removing his early decisions and replacing them with technologies which have become the industry standard [... so that ...] the parts of TeX that people actually use are 1) the box-and-glue model, 2) the hyphenation algorithm, and 3) the line-breaking algorithm".

LuaTeX comes of age

daurnimator — Sat, 26 Aug 2017 13:47:40 +0000

> The version that I'm using returns results if I prepend "=", but not "`".

Yep, sorry that's what I meant to write but got caught up in formatting.

LuaTeX comes of age

leephillips — Sat, 26 Aug 2017 13:39:05 +0000

Thanks for these useful comments!

Glad to know that 5.3 returns results in the REPL. My experiments were with 5.2.4.

The version that I'm using returns results if I prepend "=", but not "`".

"Concatenating in a loop in lua is bad for performance."

True, just as in Python. I do it your way if the string will be of significant size.
Also, this way I could include the example without explaining tables (crucial for
using Lua non-trivially, but not needed for these simple examples).

LuaTeX comes of age

daurnimator — Sat, 26 Aug 2017 07:57:28 +0000

Its JIT compiler and advanced garbage collection make for good performance in both time and space.

LuaJIT is a separate codebase from 'normal' (PUC-Rio) Lua.

unlike Python and Lisp, typing a something won't return its result unfortunately; you need to use the print statement.

This is only true in old lua versions 5.1 and 5.2. In 5.3 (released 2015) the output will be printed by default.

Additionally, in lua 5.1 and 5.2 you can prefix a line in the REPL with ` to have the result printed

shape = shape.." "..indent.."cm "..length.."cm"

Concatenating in a loop in lua is bad for performance. Consider building a table and then using table.concat instead.

GLYPH = node.id("glyph")

Make sure to use local!

Incorrect information in LuaTeX comes of age

tferro — Fri, 25 Aug 2017 16:14:24 +0000

What! "C code"? It should have been written in WEB! :-)

Damn kids are on my lawn again.....

Small extension language

canatella — Thu, 24 Aug 2017 15:08:25 +0000

If you enjoy ruby, nowadays there is mruby: https://github.com/mruby/mruby which clocs at 16610.

Incorrect information in LuaTeX comes of age

leephillips — Thu, 24 Aug 2017 13:12:28 +0000

Another advantage for embedding that I didn't mention is that multiple Lua intrepreters can be run in the same process; each interpreter can run independently in a separate thread. Compare the "global interpreter lock" that exists in (most standard implementations of) Python, for example. (Comparing Python because it's also a popular language for embedding scripting when size constraints are not severe.)

Incorrect information in LuaTeX comes of age

eru — Thu, 24 Aug 2017 12:24:19 +0000

That is still a quite small footprint compared to most alternatives. A few years ago, I wanted to allow user-written hooks to one (inhouse) program I am maintaining, and researched a bit the available extension languages. Of the languages I looked at, which were sufficiently capable, Lua was the only one whose implementation did not totally dwarf the original application.

Incorrect information in LuaTeX comes of age

leephillips — Wed, 23 Aug 2017 12:48:59 +0000

Replying to my own comment:

I downloaded the Lua source and used cloc (http://cloc.sourceforge.net/) to compile these statistics:

The source directory contains 14585 lines of C code and (in addition) 2481 lines of comments; there are also 2010 lines of code in the C/C++ header files and 1132 lines of comments in those files.

Thanks for your comment, which inspired me to get the accurate numbers.

Incorrect information in LuaTeX comes of age

leephillips — Wed, 23 Aug 2017 12:02:49 +0000

I've seen figures ranging from less than the one I quoted to more than the one you found. I guess it depends on whether you count the core language or include the standard library (and headers, comments, etc.).

Incorrect information in LuaTeX comes of age

pravinkc — Wed, 23 Aug 2017 10:15:35 +0000

Lua source is 24K lines of C
https://www.lua.org/about.html

LuaTeX comes of age

Cyberax — Tue, 22 Aug 2017 23:30:31 +0000

This is seriously cool. Are there any examples of table-based data formatting with callbacks?

About 10 years ago I tried to do huge (50-500 pages) reports using TeX and it failed miserably - all the table packages could really deal only with simplest layouts. E.g. dynamic column sizing for each page was not implemented consistently, ditto for row interleaving.