Replacing AWK with Python in GCC? [LWN.net]

Replacing AWK with Python in GCC?

Posted Jul 25, 2018 19:58 UTC (Wed) by madscientist (subscriber, #16861)
In reply to: Replacing AWK with Python in GCC? by nirbheek
Parent article: Replacing AWK with Python in GCC?

There's a reason something is the status quo: it works. If you're proposing to replace something that works, it needs to be replaced in a way that doesn't cause other people pain.

Saying "guess what, your previously working environment is now working Even Better(tm), but oh by the way now you need to install a whole bunch of extra stuff onto your system before you can do the same things you were doing yesterday" is not, to everyone, actually an improvement.

I build GCC regularly. Many of the systems I do this on do not have Python 3 installed. I am NOT excited about having to go find/download/build Python 3 on all my build systems.

The GCC build system already relies (at least tangentially) on Perl. So why introduce Yet Another Interpreter Dependency? If you don't like awk (and who does?) then Perl is more than powerful and expressive enough for this job: rewrite the awk scripts in Perl and avoid adding new prerequisites for the build system.

Change has consequence and just because someone did some work doesn't mean the results have no cost and should be accepted without thought. It's not bikeshedding if the change has a major repercussion like increasing the dependency footprint of the software.

to post comments

Replacing AWK with Python in GCC?

Posted Jul 25, 2018 20:03 UTC (Wed) by rahvin (guest, #16953) [Link] (4 responses)

Are you volunteering to rewrite the AWK scripts in perl?

Replacing AWK with Python in GCC?

Posted Jul 25, 2018 20:07 UTC (Wed) by epa (subscriber, #39769) [Link] (1 responses)

There is an awk to Perl translator. It used to ship as part of the Perl core and is still available.

Replacing AWK with Python in GCC?

Posted Jul 25, 2018 20:49 UTC (Wed) by smurf (subscriber, #17840) [Link]

The a2p translator is likely to create an even greater mess of unreadable code from the current awk scripts.
They are also complex enough that the translated version is unlikely to work at all, given that a2p is essentially unmaintained AFAIK.

WRT old machines without Python >= 2.7: there's always cross compilation.

Replacing AWK with Python in GCC?

Posted Jul 25, 2018 20:37 UTC (Wed) by madscientist (subscriber, #16861) [Link] (1 responses)

What I'm saying is that the complaint in the post I replied to is inappropriate for (at least) this situation. It's perfectly reasonable, and not whining or bikeshedding, to consider and discuss carefully and fully a decision like adding a huge new build dependency.

However, to answer your question, if the choice ends up being either someone comes up with a Perl implementation in which case we'd prefer to use that, or if no one does that then we're going to start requiring Python 3 as a GCC build dependency, then sure, I'd be willing to put in that effort. It does not appear that these are the options on offer, however.

Replacing AWK with Python in GCC?

Posted Jul 25, 2018 21:22 UTC (Wed) by rahvin (guest, #16953) [Link]

Maybe you should make your offer to convert these scripts to perl officially on the list so they can be considered. Mind you, they'd probably also want a commitment from you to maintain them as well.

Commenting on it here is all well and good, but the discussion is moving towards python, at least partly, because no one has come forward and said they'd move the scripts to perl. If you don't make the offer it's not there for consideration.

Replacing AWK with Python in GCC?

Posted Jul 26, 2018 4:42 UTC (Thu) by marcH (subscriber, #57642) [Link]

> rewrite the awk scripts in Perl

Replacing certainly unreadable code by likely unreadable code doesn't sound like a major upgrade IMHO, not worth the disruption.

> and avoid adding new prerequisites for the build system.

Granted.

Replacing AWK with Python in GCC?

Posted Jul 26, 2018 20:14 UTC (Thu) by MrWim (subscriber, #47432) [Link] (2 responses)

Saying "guess what, your previously working environment is now working Even Better(tm), but oh by the way now you need to install a whole bunch of extra stuff onto your system before you can do the same things you were doing yesterday" is not, to everyone, actually an improvement.

One trick they could apply is to run these scripts during make dist and include the output in the source tarball. This would mean that there are no additional dependencies required for people who just want to build GCC, only for those who want to develop GCC. This would make bring-up on new targets just as easy as now.

Of course this assumes that the process is deterministic and that the generated C files are themselves portable - but this should not be too much of an inconvenience.

You can even go further and include the generated C files in the git repo. It sounds icky I know, but the downsides of the duplication can be mitigated as long as:

The generation is deterministic - it depends only on files in the same git repo and isn't dependent on the architecture of the machine it's running on or the environment in which it is run.
You have a CI system to reject git commits with an inconsistency between the .opt and .c file
Concurrent modifications to the .opt files are fairly rare - you don't want to be dealing with merge conflicts in generated code - even if they can be resolved just by rerunning the tool on the merge commit.

I've used this approach successfully in some of my projects where I want a build dependency, but I don't want to burden other developers (including casual, drive-by ones) with the requirement to install that dependency.

Replacing AWK with Python in GCC?

Posted Jul 26, 2018 20:54 UTC (Thu) by jwilk (subscriber, #63328) [Link] (1 responses)

They already have configure (generated from configure.ac) committed to the repo, so the eww factor shouldn't be a problem.

(BTW, it's Subversion, not git.)

Replacing AWK with Python in GCC?

Posted Jul 27, 2018 21:19 UTC (Fri) by paulj (subscriber, #341) [Link]

The general approach is to have those built "source" files in the 'dist' tarball (generated by 'make dist'), so that that is buildable without dependencies on the tools needed for those built "source" files.