LWN: Comments on "The accelerating adoption of Julia"

The accelerating adoption of Julia

callegar — Thu, 05 Nov 2020 09:06:44 +0000

The article revived my curiosity and I made some *very initial* steps in taking a look at julia and its ecosystem.

Impressions were rather mixed.

- on the pros side:

some aspects of the language seem great and the way in which some packages make available in a consistent and flexible way the best of what is available in terms of numeric and scientific software looks similarly great.

- on the cons one, mentioning two or three

the time taken by every interactive `import` or `using` statement is huge and makes python feel like the fastest thing in the world. True, it will be compensated by faster runs, but this "startup" time is really unpleasant.

the approach by which the software ecosystem interacts with the rest of your system looks a bit like a takeover. You install juno and atom is changed for anything else you might want to use it at so you cannot have both an IDE and your editor layout arranged as you want for anything else, but you must pick one. You install tools relying on python (e.g., pyplot) and making them cope with the way in which python is setup on your system (e.g. with virtualenvs) will give you the impression that is your system that should actually be adapted to julia... Recently, you also get the impression that to have the golden "standard" environment (you need that if you want to interact with people and be able to give/receive instructions on what to do) you cannot do without Visual Studio...

no standard packaging of the latest release on linux (no traditional distro packages, no modern ones such as snaps)...

The accelerating adoption of Julia

spigot — Mon, 26 Oct 2020 05:13:18 +0000

Java has had jshell since JDK 9. I suspect it may be awkward to use when compared with REPLs designed hand-in-hand with their languages, but it exists...

The accelerating adoption of Julia

ssmith32 — Sun, 25 Oct 2020 06:31:27 +0000

You're stretching any sensible definition of "siloed language" wayyy too far:

COBOL: being "business oriented" is a fairly general goal

ADA: being designed with security in mind is not the same thing as being designed to address the security problem domain: the goal was a language that worked in many problem domains, all of which had secure functioning as a desired attribute.

C: Seriously? A language targeting "problems that can be solved with a assembled program" is about a general purpose as it gets..

C++, Java were explicitly general purpose.

Even LISP had a pretty general problem domain in mind:

"The system was designed to facilitate experiments with a proposed system called the Advice Taker, whereby a machine could be instructed to handle declarative as well as imperative sentences and could exhibit ``common sense'' in carrying out its instructions."

I think even the creators of Prolog *hoped* it could somehow solve problems in a wide variety of domains.

I think the reverse is probably more true: the authors of a language *think* they found a way to encode solutions that if the world just thought exactly like them, we'd see that it really is a general purpose language. Stuff like Erlang, ostensibly for a specific use, but the creators obviously suffer under a delusion that it would be really useful everywhere.

The accelerating adoption of Julia

Cyberax — Sat, 24 Oct 2020 21:48:52 +0000

> What specifically are you lacking? And why is this so important? Julia is a dynamic language with a very good REPL environment. If you do REPL style development you normally don't really rely on a sophisticated IDE, unless you try to program Julia as you would Java or C#.
REPL is NOT a replacement for debugger. It can sometimes be abused to do that (by putting manual breakpoints that drop into a REPL shell), but it's just not.

Debugger use is a skill that many people lack, it can reduce the time to solve problems drastically. This is doubly true for complicated environments where setting up stuff from scratch for REPL is not always practical, especially if you have a complicated reproduction case.

> With Revise, you can also easily add logging functionality such as @debug straight to code you are running, to see what is going on. If you use Git it is quick to remove code changes later, that you used for debugging. This style of debugging often doesn't work well in a C++ or Java environment because you need to recompile and reload the code. But you don't in Julia.
Java has supported hot code swap since forever, including fully dynamic changes with https://www.jrebel.com/products/jrebel

The accelerating adoption of Julia

dvdeug — Sat, 24 Oct 2020 15:30:53 +0000

My company offers an online training course to its employees in the IBM mainframe debugger for COBOL, PL/1 and assembly. Not my department, but I think it says something about how much more impact PL/1 had than generally given credit for. It never was the juggernaut IBM wanted, but it got a good 15, 20 year run and a bunch of legacy code is around even today.

The accelerating adoption of Julia

marcH — Sat, 24 Oct 2020 04:12:14 +0000

> Although honestly I don't understand why people rely a lot on debuggers when the have a powerful REPL at their disposal.

Thanks: you just made me realize that the lack of an REPL for languages like C or Java is exactly why I tend to use a debugger to starting "touring" and making sense of a brand new codebase. A debugger is not as flexible as an REPL but it's as close as it gets when there is no REPL. A debugger is really not just to find bugs, it's a live explorer.

Now to be fair I often use a Python debugger to tour a brand new Python codebase too. Either because it's not possible to import and use an REPL on a large Python codebase (because too much context and complex dependencies?), or because I don't know how to do it...

The accelerating adoption of Julia

niner — Fri, 23 Oct 2020 11:04:30 +0000

Raku

(now that was easy)

The accelerating adoption of Julia

Cyberax — Thu, 22 Oct 2020 19:41:24 +0000

> How is this hard?
If you have a method with 4-5 parameters and several candidates, the error message can span a page.

The accelerating adoption of Julia

atomiczep — Thu, 22 Oct 2020 15:21:50 +0000

> The other problem with Julia is that error messages (a dreaded "method not found") can compete with C++ template errors in being incomprehensible.

How is this hard? You check the type of the arguments you are trying to pass, and you can lookup what types the given function is able to accept. You should be quickly able to figure out which argument is of the wrong type or whether you need to add another method.

This error message is usually quite clear. It may be that you are not accustomed to reading them. Perhaps you could give an example and explain why you find this example hard to read.

> And so far I'm just not seeing it from Julia, the IDE integrations are not at all polished

What specifically are you lacking? And why is this so important? Julia is a dynamic language with a very good REPL environment. If you do REPL style development you normally don't really rely on a sophisticated IDE, unless you try to program Julia as you would Java or C#.

> debugger support is lacking (remote debugging is a PITA for one thing)
Could you not debug remotely from within the Julia REPL? I find that debugging in the REPL works quite fine. Although honestly I don't understand why people rely a lot on debuggers when the have a powerful REPL at their disposal. You write your code in a functional style with easily testable functions which you can try out in the REPL. If you hit a problem it is usually a question of looking at where the problem happened and try to feed similar data into functions that caused problem.

With Revise, you can also easily add logging functionality such as @debug straight to code you are running, to see what is going on. If you use Git it is quick to remove code changes later, that you used for debugging. This style of debugging often doesn't work well in a C++ or Java environment because you need to recompile and reload the code. But you don't in Julia.

The accelerating adoption of Julia

richiejp — Thu, 22 Oct 2020 13:55:53 +0000

FWIW I created a testing/data analyses project in Julia which we use to automatically tag failing kernel tests:
https://gitlab.com/Palethorpe/jdp
https://archive.fosdem.org/2020/schedule/event/testing_op...

It's a bit complicated for what we actually use it for now, but it works (although I should probably re-implement it in language people understand with a lot of my original assumptions removed). Julia is a great language, but rough round the edges. I spent quite a lot of time fixing bugs in and optimising libraries and decided just to reinvent the wheel on some occasions instead of using an existing library because of the stuff going on there, but I'm inclined to do that anyway.

Would happily use it for the things we usually do in Perl or Python. I can't convince anyone else to learn it though.

The accelerating adoption of Julia

NicolasLs — Thu, 22 Oct 2020 13:23:28 +0000

I have an even stronger opinion that [X] is useless if it doesn't have a good infrastructure around it, where X = Internet, electricity, trains or cars. It's a mystery to me why any of them ever got adopted.

The accelerating adoption of Julia

Cyberax — Thu, 22 Oct 2020 04:42:16 +0000

It's better. Stack traces are more reliable, if still unreadable (full file paths are either missing or wrong). Ahead-of-time compilation (PackageCompliler) is there, but it's not quite completely robust.

I wouldn't start a serious project in Julia, but it's great for notebook-style development.

The accelerating adoption of Julia

leephillips — Thu, 22 Oct 2020 02:32:32 +0000

That’s a six-year-old article about an alpha version of the language.

The accelerating adoption of Julia

devkev — Thu, 22 Oct 2020 01:32:45 +0000

My only exposure to Julia is this post https://danluu.com/julialang/ from the end of 2014 (or 2015, with the updates), so I'm wondering if anyone knows how things might have changed since then?

Julia notebook

walters — Wed, 21 Oct 2020 23:53:55 +0000

Sounds like https://en.wikipedia.org/wiki/Hysteresis ?

The accelerating adoption of Julia

leephillips — Wed, 21 Oct 2020 23:19:39 +0000

In graduate school (1980s, physics) one astronomer, our resident computer expert, was pushing PL/1 hard. He believed it was the perfect language and would rule the future. I was recommending APL to everyone. Of course, eventually, I wound up using Fortran for all my simulations, as did everyone else.

The accelerating adoption of Julia

jkowing — Wed, 21 Oct 2020 22:15:59 +0000

> Oh yes, possibly PL/1 which was meant to be all things

Picking classes to take with no guidance, I remember (long ago) taking Programming Language 1 offered in the computer science department at my college. I was thinking - hey - this sounds like THE programming language to know - after all, it is number 1. I was feeling pretty good about myself. Then, after completing the course, I was told that will be the last time they offer it and it was removed from the curriculum. And that was the last of my PL/1 programming.

Oh well. At least my C (which doesn't sound very important at all - and certainly must not as good as fancy and powerful as B and A) still keeps going strong!

The accelerating adoption of Julia

Cyberax — Wed, 21 Oct 2020 17:03:55 +0000

OK, it would be useless for me. We have a fair amount of computation code here and I checked Julia for a possible replacement for Python and C++. It didn't work that well.

Whily notebook-style development is nice, the traditional workflow with unit tests, continuous integration, code coverage analysis, etc. was severely underwhelming.

The accelerating adoption of Julia

Wol — Wed, 21 Oct 2020 16:14:38 +0000

> I don't care for the idea of a siloed language

Name me just ONE language that was *conceived* as a general-purpose language. Okay, many have morphed into general purpose languages, but ...

COmmon Business Oriented Language
FORmula TRANslation
ADA was designed as a secure language
C was designed as glorified assembler for system stuff
Beginners All-purpose Symbolic Instruction C??? okay it says all-purpose but it also says beginners ...
Pascal was designed as a teaching language - Modula-2 was a more all-purpose evolution
APL and J were mathematical notations intended to be evaluated by a computer

That's SEVEN (or eight) siloed languages off the top of my head ... oh yes, add another - one of my favourites - DataBASIC for programming databases.

As I said, can you name ONE that was DESIGNED as a general purpose language? Oh yes, possibly PL/1 which was meant to be all things to all men, and mostly was nothing to anybody.

Cheers,
Wol

The accelerating adoption of Julia

auc — Wed, 21 Oct 2020 16:07:55 +0000

I tend to see Julia as Python with the GIL issue solved (and some fancy features on top). It kind of might eat Python's lunch in the scientific community.

The accelerating adoption of Julia

leephillips — Wed, 21 Oct 2020 15:10:53 +0000

What is a siloed language? Julia is used in areas aside from scientific computing; not surprising, as it is a general purpose language. Although you shouldn’t write a driver in it.

Julia notebook

leephillips — Wed, 21 Oct 2020 15:07:36 +0000

I will talk more about this in the sequel article, but, briefy, although they have a superficial resemblence, the two notebooks work profoundly differently.

The accelerating adoption of Julia

droundy — Wed, 21 Oct 2020 14:51:52 +0000

I do scientific computing, and the fact that Julia is primarily for scientific computing is actually the biggest this that has turned me off from it. I don't care for the idea of a siloed language. If it's not a good language, I don't want to teach it to my students, and if it is a good language, why doesn't anyone want to use it for non scientific programming? It's not like scientists are the only ones who want their code to run fast, or are the only ones who want programming to be easy and fast.

Julia notebook

droundy — Wed, 21 Oct 2020 14:44:57 +0000

That is really encouraging. The thing that I've always hated about Mathematica-like notebook systems is precisely the fact that their output depends on an invisible history of user interactions.

The accelerating adoption of Julia

Paf — Wed, 21 Oct 2020 14:02:58 +0000

Well, it’s primarily for scientific computing.

If you don’t do that (and probably most lwn readers do not), then it’s not really for you. That’s fine. I’m certainly not going to write a driver in it.

Julia notebook

Paf — Wed, 21 Oct 2020 14:00:59 +0000

Given the apparent connections to the existing Jupyter project, can you say why this wasn’t done as an extension to Jupyter?

The accelerating adoption of Julia

BirAdam — Wed, 21 Oct 2020 12:50:48 +0000

I forgot to note, this particular article was not of the ad type that I have seen elsewhere, I am just at this point somewhat repulsed by the language in general due to ad-like news articles elsewhere.

The accelerating adoption of Julia

BirAdam — Wed, 21 Oct 2020 12:49:21 +0000

I have been seeing "news" about Julia out of nowhere recently, and a lot of it seems like cleverly disguised ads. I personally do not see any reason to use Julia, and I generally don't care much about hypelangs.

The accelerating adoption of Julia

rsidd — Wed, 21 Oct 2020 08:05:29 +0000

Indeed. And the infrastructure is growing rapidly -- right now almost everything I want seems to be there in terms of libraries, often much better than python. For the IDE, I have been using jupyter; but will check out pluto.

My workflow is, roughly:

prototype quickly in jupyter
save the "done" functions to a library file; delete from the jupyter notebook and import the library
at the end of the day, if I want a commandline program, just write it as a wrapper around the library file.

It looks like pluto will make this sort of iterative development even easier. On my next-to-try list.

And the performance, for such an easy-to-write language, is incredible. And when it isn't, the Julia people take notice. See this for a very interesting example: Julia folks got their biosequences package to run as fast as Seq, a highly optimised special purpose language for bioinformatics (helped by the transparency and accessibility of the Seq folks, to their great credit).

Julia notebook

rsidd — Wed, 21 Oct 2020 07:54:02 +0000

I love Julia; I use jupyter; julia with jupyter is buggy (randomly disappearing cell content; I'm not able to reproduce predictably). Python with the same jupyter install is fine. So I wonder how actively the julia frontend is being used by julia folks. I didn't know about pluto until now. Will definitely give it a try.

Julia notebook

tnoo — Wed, 21 Oct 2020 06:25:31 +0000

This sounds very promising. Personally, I don't like the Jupyter notebooks, mainly for the reasons outlined in this thread. The sad thing is, that Pluto will work for one language only, and thus split the development efforts and communities again.

Now, I'm looking very much forward to your next article.

Thanks

The accelerating adoption of Julia

gerdesj — Wed, 21 Oct 2020 00:29:34 +0000

"I have a rather strong opinion that a language is useless if it doesn't have a ..."

Take a long hard look at that statement and consider why it might be bollocks. Julia is a programming language. It was conceived, designed and coded by some people who want to do <something>. If you don't like it then please feel free to not contribute.

Julia notebook

leephillips — Tue, 20 Oct 2020 21:52:06 +0000

Well, now I don’t have to write it. You just concisely described the main point!

Julia notebook

leephillips — Tue, 20 Oct 2020 21:50:02 +0000

I wrote about Jupyterlab here at https://lwn.net/Articles/748937/.

As you will learn in the followup article, there are some longstanding shortcomings in Jupyter notebooks, nice as they are, that are fixed in Pluto. Some of the differences go deep enough that I don’t think Jupyter would have been likely to evolve in this direction.

Julia notebook

garrison — Tue, 20 Oct 2020 21:39:29 +0000

Pluto is an important innovation beyond Jupyter notebooks: as the project describes, it is "reactive - when changing a function or variable, Pluto automatically updates all affected cells."

Pluto offers an environment where changed code takes effect instantly and where deleted code leaves no trace. Unlike Jupyter or Matlab, there is no mutable workspace, but rather, an important guarantee:
At any instant, the program state is completely described by the code you see.
No hidden state, no hidden bugs.

Pluto works the way I always wished Jupyter would work, and as a bonus, its notebooks are plain text files, easy to store and work with in git (including git-diff). I expect the forthcoming LWN article will explore the differences with Jupyter in depth.

Julia notebook

joib — Tue, 20 Oct 2020 21:35:27 +0000

I haven't heard of Pluto before, but from a brief look at the github page, it does seem to have slightly different goals than jupyter.

Jupyterlab is just the next generation web UI for jupyter. All the other jupyter infrastructure remains the same.

Julia notebook

tnoo — Tue, 20 Oct 2020 21:09:59 +0000

> new computational notebook for Julia, called Pluto, that is akin to Jupyter notebooks.

This is ironic, since the "Ju" in "Jupyter" stands for "Julia" (actually Julia, Python, R). Is there a NIH syndrome at work?

And now there's even Jupyterlab. Why are they splitting efforts?

The accelerating adoption of Julia

Cyberax — Tue, 20 Oct 2020 18:42:23 +0000

I have a rather strong opinion that a language is useless if it doesn't have a good infrastructure around it. And so far I'm just not seeing it from Julia, the IDE integrations are not at all polished and debugger support is lacking (remote debugging is a PITA for one thing).

The other problem with Julia is that error messages (a dreaded "method not found") can compete with C++ template errors in being incomprehensible.