LWN: Comments on "Extracting the abstract syntax tree from GCC"

I've abandonded Emacs for serious development

yoe — Mon, 02 Feb 2015 18:40:47 +0000

It depends on what you call 'full-featured', but qt-creator is pretty good.

Extracting the abstract syntax tree from GCC

dlang — Sun, 25 Jan 2015 22:32:07 +0000

Are those Free Software companies (as Stallman defines it) or Open Source Software companies?

I'm pretty sure that Stallman wouldn't count them as Free Software companies.

Extracting the abstract syntax tree from GCC

cortana — Sun, 25 Jan 2015 15:59:48 +0000

Plus Redhat, Fluendo...

Extracting the abstract syntax tree from GCC

spaetz — Sun, 25 Jan 2015 15:58:20 +0000

> Few Open Source companies as compared to none whatsoever in the Free Software camp says what?

PLEASE! Can we leave the snarky infighting polemics out?

There are not 'none' free software companiess by the way. Think of g10 code (gnupg), Igalia (calling themselves bnoth open source and Free software consultancy etc).

There are companies out there if you look for them, just no 800 pound gorillas.

Extracting the abstract syntax tree from GCC

vonbrand — Sun, 25 Jan 2015 14:22:24 +0000

Few Open Source companies as compared to none whatsoever in the Free Software camp says what?

Extracting the abstract syntax tree from GCC

dakas — Sun, 25 Jan 2015 13:39:05 +0000

Uh, Eric Raymond explicitly created the "Open Source" label exactly to appeal to industry players who considered bothering with principles suspicious. So yes, Open Source is for people without principles. That's not Stallman's pitch, but an explicit design goal of the Open Source agenda. To replace the appeal to principles, a bunch of technical and marketing criteria are propounded.

The dearth of actually successful Open Source companies (where Open Source is part of a permanent rather than an exit strategy) makes it pretty clear that this "realistic" advertising strategy is not actually founded in much realism.

Laws and society still largely favor proprietary solutions. RedHat is one company that is doing a lot for free software based on a proprietary service model. Google may be bleeding money from its Android enterprise, but comparatively little compared to the size of the project. But then the Android universe is not much interested in enabling free software. You may be able to get the source for much of the stuff, but tinkering with it requires jailbreaking and lots of work.

So even where Open Source/Free Software is successful in deployment, the financing has to come from less than free models.

Which means that principles are required after all to keep the freedom a running part of the whole: it does not pull its weight in the market all by itself. Never mind on whether you call it "Open Source".

Extracting the abstract syntax tree from GCC

blujay — Fri, 23 Jan 2015 12:40:08 +0000

And like politics, everyone is affected by it, even the onlookers. Therefore everyone has an interest in it, even if they have no interest in it. It's a bikeshed we all live in, even if we don't realize it.

Why not get the AST from LLVM ?

rodgerd — Thu, 22 Jan 2015 20:58:28 +0000

If I've understood the saga correctly, she was going to use LLVM until Stallman persuaded her to use the full GNU toolchain, and *then* he sat on his hands before coming out against the idea. I think that's what rankles so much for people - he's wasted a lot of this developer's time.

Extracting the abstract syntax tree from GCC

mlopezibanez — Thu, 22 Jan 2015 14:57:26 +0000

I agree wholeheartedly with your whole comment except for:

"But these days, anyone who wants an AST can use LLVM; they can also write a GCC plugin to dump the GCC AST but it will be basically equivalent."

The problem is that they are not equivalent at all, and this simple fact makes the whole "debate" pointless. GCC AST is simply still not good enough to do the kind of source code analysis that editors and refactoring tools require. Not even accessing GCC internals will give you that. RMS blessing is not going to change that.

Also, my experience after 10 years working on GCC's C/C++ FEs tells me that a GCC plugin is definitely not the right approach to achieve this. If the issues with GCC's AST were fixed today, then the next step would be to create a library within GCC's source repository that wrapped GCC's internals in a nice interface. This is precisely what the GDB devs have done for the new compile command:

* https://sourceware.org/gdb/current/onlinedocs/gdb/Compili...
* https://github.com/gcc-mirror/gcc/tree/master/libcc1

Extracting the abstract syntax tree from GCC

mlopezibanez — Thu, 22 Jan 2015 13:56:32 +0000

This widespread misconception seems to come from an idealized view of how a compiler works. GCC at least does not work this way: the FE often calls the back-end to get some values and the middle-end calls into the FE to get others. Moreover, there is not "an" intermediate representation, but a collection of data structures tightly coupled. It is possible to serialize some of this data into an on-disk representation (in fact, LTO is doing precisely this), but many issues remain to be solved and the representation would still be tightly coupled with GCC internals.

Any company seriously thinking about using such an approach to extend GCC is doomed to fail (and they have failed in the past) and, in the case of proprietary extensions, they will get no help from the community in solving their issues.

Reality check

Wol — Thu, 22 Jan 2015 13:45:04 +0000

> or vice-versa if you don't like C++ being the race track

Race tracks don't go anywhere, or at least, only in circles :-)

Cheers,
Wol

Extracting the abstract syntax tree from GCC

mlopezibanez — Thu, 22 Jan 2015 13:44:41 +0000

What is missing from the discussion and the article (and what any *active* GCC developer will tell you) is that the AST of GCC C/C++ FEs has fundamental limitations for source code analysis, specially for refactoring, thus, nobody should expect that dumping GCC AST will somehow magically allow Emacs hackers to not need to "hack on GCC for every feature we want to add to Emacs". Even motivated devs such as AdaCore and Mozilla have tried to use GCC AST and given up because it is not complete enough. It is not a question of dumping the AST or not: There is a huge amount of work to do before GCC's AST can be useful for any source code analysis.

In my humble opinion, given Clang vs. GCC status as FEs, even if GCC dumped its AST today, nobody in their sane mind will spend any effort whatsoever to build a GCC AST reader just to enable some proprietary source code analysis, middle-end optimization or target backend. (There are good reasons why each of these potential uses would not become easier by dumping GCC's AST).

As one of the GCC developers working during the last ten years on reducing the gap between GCC and Clang, I would suggest that Emacs developers who are happy with Clang should just use it to improve Emacs (I'm also an Emacs user!) and those that are unhappy with Clang should just join the GCC team and help us such that in some years, GCC could be an alternative. Anything else is just noise.

Why not get the AST from LLVM ?

moltonel — Thu, 22 Jan 2015 10:07:03 +0000

If there's such a political barrier about gcc ast, why doesn't the author of the emacs completion work switch to using llvm, and get on with life ?

I can understand the desire to remain "GNU-pure" by using gcc with emacs, but at some point a pragmatism check is needed. Switching backend sounds like a much less dramatic outcome than abandoning the project.

But maybe the contributor has been burned once too many, and doesn't have the energy to refactor and/or justify himself again. Such a waste :(

I've abandonded Emacs for serious development

mgedmin — Wed, 21 Jan 2015 14:29:41 +0000

> Unfortunately, there's no decent, full-featured IDE using another framework (Qt for example).

GNOME Builder aims to become such, and is currently having a pretty successful fundraiser: https://www.indiegogo.com/projects/builder-an-ide-of-our-...

Extracting the abstract syntax tree from GCC

edelsohn — Wed, 21 Jan 2015 03:05:36 +0000

Richard unfortunately does not recognize that the more he tries to prevent uncomfortable changes to GNU Software, the more pain he causes himself and the GNU Project.

Richard's approach hinges on the now false premise that software such as GCC have such a high barrier to recreate that developers will come to the FSF plantation. As Joe said, there is no secret sauce to protect, but leaders of both proprietary products and FLOSS projects from an earlier era cling to that as the foundation of their "businesses".

Extracting the abstract syntax tree from GCC

pinskia — Tue, 20 Jan 2015 23:31:58 +0000

I have not seen one discussion about this on the GCC mailing list. So until someone actually start programming this up, there is no way really to say if it is going to have the issues that RMS is talking about.

Extracting the abstract syntax tree from GCC

ms_43 — Tue, 20 Jan 2015 18:32:04 +0000

The obvious way to do it is modify GCC to dump some intermediate representation (there are several at different levels, including the AST the article is about) to a file, and then read that file with a second process that contains the proprietary secret sauce. No linkage necessary.

Extracting the abstract syntax tree from GCC

paulj — Tue, 20 Jan 2015 11:40:27 +0000

Highly unlikely!

That FSF didn't make a violation stink about this back in the day suggests it must have been GPL compatible someway. I'd assume by way of the exception in the GPL that allows linking to system libraries. ??

Though, reading the thread it seems to me the libs GCC was modified to work with were delivered by the Sun Studio compiler packages - which weren't ever part of base Solaris, certainly not then anyway.

Extracting the abstract syntax tree from GCC

etienne — Tue, 20 Jan 2015 11:13:03 +0000

> My point being the alternative in the GNU/Free monoculture etienne is pining for is non-existent.

I was not talking about Nvidia but Ati.
My point was you look at their very powerful hardware, you say I want to play with that.
You look at how to program the hardware, you do not see a locked-down proprietary language but something reasonably described (OpenCl), you begin to look at the different hardware configurations you can buy.
You look at the compiler, you see LLVM, you say great! an open source compiler: I shall not have too many problems, and even if I have I will be able to fix them. You buy the hardware you have selected.

Then you realize, when you finally achieve to configure the hardware to work as a basic video card on your distribution (did take a long time, had to disable package management and other dirty things), that the version of LLVM used is completely out of date, that most of the toolchain is completely proprietary and no part can be re-generated, then you hit your first compiler bug (usually it locks down the PCIe bus so forces a hardware reboot with unclean filesystems because the disks are on PCIe Sata interface).

Then you understand that they declared their compiler BugFree (TM), and that they will not consider a bug if you cannot reproduce it on the latest video game (which do not seem to use OpenCl but the "intermediate" representation directly).

> Acting *offended* that LLVM exists and allows FOSS to work for its users is a repulsive attitude.

*offended* is a big word, that is not the first time I spent money on hardware I cannot really use, and I have seen a lot of broken compilers where the supplier denies officially that a bug may exists.

I am also *not saying* LLVM is buggy, I am saying the version they used has been so modified it now has a bug that bite me hard - I can't do anything about it.

I think LLVM gets bad advertisement with that story, final user should really understand the difference of:
- compiler based on "Open Source" software
- compiler itself "Open Source"
- compiler itself "Open Source" and can be fixed (regenerated) locally
- compiler GPL

Extracting the abstract syntax tree from GCC

mfukar — Tue, 20 Jan 2015 07:56:24 +0000

Imagine all the ruckus it would raise if that were true, huh?

Extracting the abstract syntax tree from GCC

k8to — Tue, 20 Jan 2015 04:48:17 +0000

I don't think it was claimed that proprietary software is equivalent to this situation.

However, it's *also* not as simple as "just fork if you want" when that would be a pretty expensive action for a project with the history and culture of Emacs.

It's not like we don't have history to go by here.

Reality check

k8to — Tue, 20 Jan 2015 04:46:40 +0000

Personally, I think avoiding C++ shows good taste.

However, not really knowing much about it at all isn't as encouraging.

Extracting the abstract syntax tree from GCC

rahulsundaram — Tue, 20 Jan 2015 02:32:30 +0000

Are you honestly suggesting that FSF allowed a vendor to get a proprietary license for any software from them?

Extracting the abstract syntax tree from GCC

dgm — Tue, 20 Jan 2015 02:25:42 +0000

The FSF is the proprietor of all copyrights to GCC. Maybe they licensed the GCC front end?

Extracting the abstract syntax tree from GCC

ms_43 — Mon, 19 Jan 2015 20:15:18 +0000

This is quite tragic, because it's not like corporations were in any way prevented from hooking up GCC's front end to a proprietary optimizer in the past.

For example, Sun had a "GCC for SPARC Systems" product:

https://gcc.gnu.org/ml/gcc/2006-03/msg00140.html

This blog claims that (at the time) the proprietary PathScale compiler was based on a GCC front-end:

http://tirania.org/blog/archive/2004/Apr-18.html
https://web.archive.org/web/20080103174738/http://pathsca...

Extracting the abstract syntax tree from GCC

flussence — Mon, 19 Jan 2015 19:05:27 +0000

My point being the alternative in the GNU/Free monoculture etienne is pining for is non-existent.

Acting *offended* that LLVM exists and allows FOSS to work for its users is a repulsive attitude.

Extracting the abstract syntax tree from GCC

tialaramex — Mon, 19 Jan 2015 16:30:54 +0000

Everybody's diagnostics have been getting better. Over my career we've gone from "Oh! This new system tells you where the problem is, rather than just saying Syntax Error" to GCC telling me that (a) My fscanf() format string seems not to match the provided parameters (b) what the format seems to ask for (c) what the parameter I supplied is (d) how to turn off the warning if I don't care.

The tools I have today are many times better than the tools I had twenty, ten, even five years ago.

Extracting the abstract syntax tree from GCC

danieldk — Mon, 19 Jan 2015 16:01:27 +0000

> It is clear that Stallman has not used any of the "competing" IDEs (e.g. Xcode, IntelliJ IDEA), which is not a huge surprise,

Note, in contrast to what Metzger says, IntelliJ itself is open source under the Apache 2 License:

http://www.jetbrains.org/display/IJOS/Ultimate+Edition+vs...

They have proprietary extensions, which are bundled in the Ultimate edition.

Extracting the abstract syntax tree from GCC

Frej — Mon, 19 Jan 2015 09:07:54 +0000

Clang error messages tend to be more helpful as well.

Extracting the abstract syntax tree from GCC

seyman — Sat, 17 Jan 2015 12:53:33 +0000

> he is limiting freedom of devs, users

He is doing no such thing. If others feel that Emacs badly needs a specific feature, they are free to fork it or fund someone else to do so.

Saying that FLOSS users are as helpless before the editor of their software as users of proprietary software is a confession that the speaker understands very little about Free Software.

Extracting the abstract syntax tree from GCC

smitty_one_each — Sat, 17 Jan 2015 02:30:22 +0000

They have one? I'd always understood legal to take refreshment from the neck. . .

Reality check

rahvin — Sat, 17 Jan 2015 02:01:35 +0000

Something that's been bothering me about this is I thought RMS lost his "overlord" role with GCC with the former split and that the only software he still only had any real control over (as in project dictator/director/decision maker) was emacs.

It would seem my memory of history is either distorted or out of date because he must have a role or no one would be consulting him about what GCC can do. (as google is failing me here) Is RMS still the main decision maker for GCC these days or hold a key decision making role? Because I thought the whole egcc split thing sidelined him into an advisory role when they re-merged.

Honestly if he's so out of date that he's never even used C++ he really shouldn't be in any management role for software which is used in programming, either GCC or emacs for that matter. To provide a car analogy, I wouldn't find myself qualified to design race tracks if the only thing I'd ever driven was a go-cart (or vice-versa if you don't like C++ being the race track). The differences between programming 20 years ago and today are vast and if you haven't stayed up to date you don't even really know what the issues are let alone the proper feature set.

Don't get me wrong, I like RMS. Free software needs an uncompromising evangelist to keep the movement steered towards the goals. But that doesn't mean he needs to be making feature decisions on a software package if his knowledge and actual programming experience is dated.

Extracting the abstract syntax tree from GCC

rdc — Sat, 17 Jan 2015 00:00:30 +0000

I am glad to see people pointing out the hypocrisy of Stallman's position, in protecting freedom he is limiting freedom of devs, users and usefulness of the tools he has championed for decades, and appears unable to see the problem

Extracting the abstract syntax tree from GCC

mathstuf — Fri, 16 Jan 2015 20:39:09 +0000

Nvidia isn't standards conforming (they allow all sorts of things the spec says shouldn't be possible). They also care very little about OpenCL.

Reality check

k8to — Fri, 16 Jan 2015 18:14:48 +0000

I'd rather say that RMS is disrespecting everyone else, by insisting that things he hasn't put any effort into understanding (for decades) must go through his personal understanding and say-so on grounds that are not really technical at all nor practical.

There is certainly a relationship between the RMS extreme stubbornness and this behavior where he believes that his sole arbitration of technology he does not seem to be interested in understanding is reasonable. The personality traits can be beneficial sometimes, but they're certainly not right now.

Extracting the abstract syntax tree from GCC

k8to — Fri, 16 Jan 2015 18:04:40 +0000

How even-handed.

Extracting the abstract syntax tree from GCC

marcH — Fri, 16 Jan 2015 16:03:03 +0000

For your next tea/coffee break at work, I recommend the machine which is located in the middle of your legal department.

Extracting the abstract syntax tree from GCC

jake — Fri, 16 Jan 2015 15:37:06 +0000

> Is there any policy when sending a free link to other sites (Hacker News etc.) is ok?

There is some information in the FAQ: https://lwn.net/op/FAQ.lwn#slinks

It says:

Where is it appropriate to post a subscriber link?

Almost anywhere. Private mail, messages to project mailing lists, and blog entries are all appropriate. As long as people do not use subscriber links as a way to defeat our attempts to gain subscribers, we are happy to see them shared.

I don't think we have seen really any abuse of the subscriber links over the years. We should probably be a bit more explicit in the FAQ about things like G+, Hacker News, etc. -- links have been posted to those places without any noticeable problems, at least so far ...

jake

Extracting the abstract syntax tree from GCC

meyert — Fri, 16 Jan 2015 12:10:13 +0000

I actually would like to see Code-Pointer Integrity - http://nebelwelt.net/publications/1431c3/ - as an GCC plugin! But they decided to implement it as an LLVM extension.

Extracting the abstract syntax tree from GCC

smitty_one_each — Fri, 16 Jan 2015 12:05:20 +0000

>Stallman would like to see some kind of investigation to determine what pieces of information are needed for which purposes (e.g. auto-completion, refactoring, and so on), but that seems short-sighted to some.

In other words, somebody be a prophet and predict all future, currently unanticipated, uses.
With all the massive respect due RMS, this seems a trifle myopic.