I'm confused...

Posted Oct 1, 2018 5:40 UTC (Mon) by pr1268 (guest, #24648)
Parent article: Nuitka 0.60 released

From the article:

Nuitka is a Python compiler. [...] You feed it your Python app, it does a lot of clever things, and spits out an executable or extension module.

And later (on the same page):

It translates the Python into a C program [...]

So which is it? A native code compiler or a C-language translator?

And assuming the latter, then why not go straight from Python to assembly?

A compiler that translated code straight from any of the contemporarily-popular languages to arch-specific native assembly would really be nice. Just my $0.02.

I'm confused...

Posted Oct 1, 2018 7:11 UTC (Mon) by jtaylor (subscriber, #91739) [Link] (2 responses)

It is likely translated to C so it can conveniently use the Python C-API and make use of the optimizations C compilers can already do.

I'm confused...

Posted Oct 1, 2018 14:39 UTC (Mon) by jamesh (guest, #1159) [Link] (1 responses)

And by generating code that uses the libpython C API, it means that it can trivially call into CPython extension modules.

I guess the main opportunities for optimisation would come from identifying occasions when it is safe _not_ to use the libpython API. For example, if you know the two values you are adding are integers then generate code that adds two ints rather than calling PyNumber_Add() on two PyObjects.

I'm confused...

Posted Oct 1, 2018 15:10 UTC (Mon) by kayhayen (guest, #127589) [Link]

That's exactly what it does. Initially it basically only meant to replace bytecode with Python C/API call, then avoid that as much as possible, and lately, with 0.6.0 first for bool, for 0.6.1 for void, and hopefully soon for int, to stay within C types as much as possible.

I call it graceful degradation. Code that can be fast should be. And if you go all in on special Python features it will become less optimized, but still faster.

Obviously a very hard project that I am in there, but from now on I am hoping to improve performance with every release for the 0.6.x series, whereas 0.5.x had 33 teleases preparing that in the background mostly.

Yours,
Kay (Author of Nuitka)

I'm confused...

Posted Oct 1, 2018 10:19 UTC (Mon) by wazoox (subscriber, #69624) [Link] (4 responses)

That's detailed in the developer manual: http://nuitka.net/doc/developer-manual.html
It generates C11 code with a fall-back on C++03 for the cases where a good C11 compiler isn't available (Windows).

I'm confused...

Posted Oct 1, 2018 15:13 UTC (Mon) by kayhayen (guest, #127589) [Link] (3 responses)

Actually there is MinGW64 which appears to generate substantially faster code than MSVC (64 bits Python2 pystone is what I looked at only), and it does C11. And there is also Clang, which upcoming versions will support as well via "clang-cl.exe", where I am hoping to benefit from the fact that Clang does C11. People are claiming that Clang is better than MSVC too. In my benchmark it fell into the middle of MSVC and MinGW64.

Yours,
Kay

I'm confused...

Posted Oct 1, 2018 23:17 UTC (Mon) by atai (subscriber, #10977) [Link] (2 responses)

MinGW64 actually means gcc

I'm confused...

Posted Oct 1, 2018 23:42 UTC (Mon) by k8to (guest, #15413) [Link]

Sure, though I think the runtime for the produced executables is sometimes a significant factor for this type of situation, so it's useful to provide that.

I'm confused...

Posted Oct 2, 2018 13:21 UTC (Tue) by kayhayen (guest, #127589) [Link]

Yes, it uses gcc code at the core, but e.g. MinGW is apparently less Windows compatible, doesn't offer all the tools, and doesn't work for Nuitka. The C runtime actually also matters a bit and I think that is where gcc ports differ. I for my part wished there was less variations for gcc on Windows.