Carrez: The real problem with Java in Linux distros

Posted Sep 29, 2010 22:07 UTC (Wed) by HelloWorld (guest, #56129)
In reply to: Carrez: The real problem with Java in Linux distros by marcH
Parent article: Carrez: The real problem with Java in Linux distros

Writing fast software requires some understanding of smart datastructures, algorithms, complexity theory etc., and poor developers generally don't have that kind of understanding. Since to my knowledge Java can't magically turn an O(n^2) algorithm into a O(n) one, Java programs written by poor developers will perform poorly.

(Why do I even have to explain this?)

Carrez: The real problem with Java in Linux distros

Posted Sep 30, 2010 6:24 UTC (Thu) by marcH (subscriber, #57642) [Link] (13 responses)

> (Why do I even have to explain this?)

Because writing "enterprise" software, of which you apparently know little (lucky you), is almost not about algorithms but mainly about large amounts of boring boilerplate code for persistence, serialization and user interfaces. Have a look at what J2EE is (now JEE).

In the real world, grunts do not get to spend time reading Knuth and carefully select algorithms.

Carrez: The real problem with Java in Linux distros

Posted Sep 30, 2010 18:51 UTC (Thu) by HelloWorld (guest, #56129) [Link] (8 responses)

Algorithms are being written all the time. It's about trivial stuff like removing even numbers from a list. Don't tell me this doesn't happen in the real world, _especially_ when dumb programmers (who don't know about higher-order functions like filter) and dumb languages (like Java, which also doesn't know about higher-order functions) are involved.

A dumb programmer would do this:

void removeEvenNumbers(ArrayList<Integer> l) {
    for(int i = 0; i < l.size(); ++i)
        if (l.get(i) % 2 == 0)
            l.remove(i);
}

A smart one would do this:

void removeEvenNumbers(ArrayList<Integer> l) {
    int j = 0;
    for(int i = 0; i < l.size(); ++i) 
        if (l.get(i) % 2 != 0)
            l.set(j++, l.get(i))
    l.removeRange(j, l.size()
}

Now please don't tell me that this kind of stuff isn't done in the real world.

Carrez: The real problem with Java in Linux distros

Posted Oct 1, 2010 7:00 UTC (Fri) by jschrod (subscriber, #1646) [Link] (4 responses)

IMO, both are not good. Iterator.remove() should be used.
Or, if there is such a utility function in Apache commons, it should be used.

Carrez: The real problem with Java in Linux distros

Posted Oct 1, 2010 14:11 UTC (Fri) by HelloWorld (guest, #56129) [Link] (3 responses)

IMO, both are not good. Iterator.remove() should be used.

Please, if you make such statements, provide arguments for them. Otherwise, it's just a waste of time for both of us. Also, as far as I can see, there is no analogon to the removeRange method that takes iterators instead of indices.

Or, if there is such a utility function in Apache commons, it should be used.

I don't think there is. It's basically impossible to abstract this kind of things in Java enough to make it useful. A method that removes even numbers from a list isn't really that useful, since one day you may want to remove odd numbers or primes or negatives or whatever. So what you need to do is pass a small chunk of code that determines whether an element will be removed. The best Java will give you is an anonymous inner class, which makes it so verbose that it's pointless.

filter(new Predicate<Integer>() { public boolean doIt(Integer i) { return i % 2 == 0; } }, someArrayList);

Carrez: The real problem with Java in Linux distros

Posted Oct 3, 2010 10:20 UTC (Sun) by marcH (subscriber, #57642) [Link]

> which makes it so verbose that it's pointless.

You really have no idea how much money boilerplate code can make.

Carrez: The real problem with Java in Linux distros

Posted Oct 4, 2010 0:18 UTC (Mon) by sbishop (guest, #33061) [Link] (1 responses)

You asked for arguments, so here's one. You're "stupid" example is inefficient, yes. But it's also buggy. Consider what would happen if there were two even numbers in a row.

And that's a good reason to use commonly used libraries for these kinds of things, of course. I'm sure you know that, but you don't appear to be a Java developer, so I wouldn't expect you to know what those libraries are. The parent post mentioned Apache Commons. That would work, but their Collections framework doesn't do generics. These days Guava would be used.

Carrez: The real problem with Java in Linux distros

Posted Oct 4, 2010 1:13 UTC (Mon) by HelloWorld (guest, #56129) [Link]

You asked for arguments, so here's one. You're "stupid" example is inefficient, yes. But it's also buggy. Consider what would happen if there were two even numbers in a row.

That's actually a good point, that wouldn't have happened with an iterator.

And that's a good reason to use commonly used libraries for these kinds of things, of course. I'm sure you know that, but you don't appear to be a Java developer, so I wouldn't expect you to know what those libraries are. The parent post mentioned Apache Commons. That would work, but their Collections framework doesn't do generics. These days Guava would be used.

This kind of stuff makes sense in theory, but as I pointed out elsewhere in this thread, the Java language makes this kind of stuff way too hard and verbose. Programmers will just continue writing loops as long as it stays that way (It works perfectly well in more expressive languages though). Perhaps Java 8 will remedy this situation.

Carrez: The real problem with Java in Linux distros

Posted Oct 1, 2010 10:29 UTC (Fri) by marcH (subscriber, #57642) [Link] (2 responses)

Without really looking at your example in detail, I like the fact that you first show off with some functional-speak, and then demonstrate an example with for loops.

Carrez: The real problem with Java in Linux distros

Posted Oct 1, 2010 11:19 UTC (Fri) by HelloWorld (guest, #56129) [Link]

Given that this is a discussion about poor developers writing poor code in Java, an example in Java seemed fit.

Anyway, if you want an example in a functional language, here is how you'd do it in Haskell:

removeEvenNumbers = filter odd

Carrez: The real problem with Java in Linux distros

Posted Oct 5, 2010 22:18 UTC (Tue) by nix (subscriber, #2304) [Link]

An example with a missing bracket so it doesn't compile ;P

Carrez: The real problem with Java in Linux distros

Posted Sep 30, 2010 19:15 UTC (Thu) by HelloWorld (guest, #56129) [Link] (3 responses)

Oh by the way, you still haven't told why you think that Java makes stupid things harder. I think you're underestimating the dumb developers' imagination when it comes to doing dumb things.

Carrez: The real problem with Java in Linux distros

Posted Oct 1, 2010 10:22 UTC (Fri) by marcH (subscriber, #57642) [Link] (2 responses)

> Oh by the way, you still haven't told why you think that Java makes stupid things harder.

Yes I left this as an exercise (hint: cheat and use the Internet).

Carrez: The real problem with Java in Linux distros

Posted Oct 1, 2010 11:38 UTC (Fri) by HelloWorld (guest, #56129) [Link] (1 responses)

This discussion is going nowhere. Making claims and not providing any arguments has a name. It's called trolling.

Carrez: The real problem with Java in Linux distros

Posted Oct 2, 2010 15:12 UTC (Sat) by marcH (subscriber, #57642) [Link]

Sorry if you felt "trolled" but I actually never meant to discuss this *at all*. I just meant to briefly share a number of converging experiences, and such a forum is the appropriate place for that. Formally demonstrating the conclusions of these experiences with hard facts is not possible in such a forum, anything serious would require an entire book. We all understood that your experience differs, thanks.

Carrez: The real problem with Java in Linux distros

Posted Sep 30, 2010 10:38 UTC (Thu) by jschrod (subscriber, #1646) [Link] (6 responses)

In the Real World(tm), a.k.a. business programs, programmers don't have the chance to select between an O(n^2) and an O(n) algorithm[*], they use libraries and frameworks that supply such algorithms.

Obviously you don't develop in a business context. There, the interesting parts of software development are in the requirement, architectural, and testing parts of a project, not in any algorithm designs which are 99.999% of the time *VERY* boring, being `get data, change some attribute, move it somewhere else, finished'.

[*] Being pedantic, there are few algorithms anyhow, where one *could* make such a choice -- most of the time, it's O(n^2) vs. O(n \log n). Equally well, it often doesn't matter at all, because the factors that O-notation abstracts away, and also $n$ that needs to be handled are too small anyhow. There's a reason why DEK does full analysis in his books, and not just O notation.

Carrez: The real problem with Java in Linux distros

Posted Sep 30, 2010 18:54 UTC (Thu) by HelloWorld (guest, #56129) [Link] (5 responses)

See above for a trivial O(n^2) algorithm that is being written all the time.

Also, if the problem size is small anyway, why worry about performance at all and not just write the stuff in some other, more productive language?

Carrez: The real problem with Java in Linux distros

Posted Oct 1, 2010 7:14 UTC (Fri) by jschrod (subscriber, #1646) [Link] (4 responses)

> See above for a trivial O(n^2) algorithm that is being written all the time.

See my answer there.

> Also, if the problem size is small anyway, why worry about performance at
> all and not just write the stuff in some other, more productive language?

First of all, a large problem size does by no means equate with complex algorithms. E.g., in a current project of us, phone events of 40,000,000 customers must be rated and billed. Processing each event is simple, no algorithmic complexity at all. The challenge is an architectural design that supports or even furthers horizontal scalibity, i.e., the ability to distribute the processing over many servers.

Second, in business applications performance issues are more often caused by bad interface decisions, I/O problems, and bad database designs than by algorithms. Business applications are not about computing, they are about moving data from A to B.

As I wrote, complexity and challenges for business applications are found in the requirement and design phases, not in the programming phases. Actually, also in the testing phase if one goes beyond module unit tests and tests if a system really delivers the functionality and serviceability the customer *needs* (that's most of the time not the same he's ordered).

Third, I have yet to meet a non-toy project -- let's say 50+ people working on it -- where one is allowed to select its implementation language on purely technical grounds. I hope I don't have to spell out all the reasons for that. In the real world, implementation language(s) selection is as much a management and political decision as it is a technical decision, for better and worse.

Carrez: The real problem with Java in Linux distros

Posted Oct 1, 2010 14:55 UTC (Fri) by HelloWorld (guest, #56129) [Link] (3 responses)

And how does Java help with all that? Does Java magically create good database designs? I don't think so. And it probably won't magically make your program scalable and all that. So what it boils down to is that to make fast software, you need smart developers.

You could say that smart people could figure out the design and let grunt programmers implement it. But I don't think that works well in practice. The devil is in the details, and for a bad programmer, it's easy to write a horrible program from a design he didn't really understand.

That's true, but you don't have a choice.

Posted Oct 2, 2010 22:50 UTC (Sat) by khim (subscriber, #9252) [Link] (2 responses)

You could say that smart people could figure out the design and let grunt programmers implement it. But I don't think that works well in practice. The devil is in the details, and for a bad programmer, it's easy to write a horrible program from a design he didn't really understand.

Yes - and it means most "business programs" are horrible. But with Java they work. With other, less restricting languages, they don't. The problem here is the fact that it's usually very hard to fire bad developer from "big business" so they must be used somehow - and Java is the best language for that. C# is not because it's nicer language and actually includes many higher-level features!

That's true, but you don't have a choice.

Posted Oct 3, 2010 1:23 UTC (Sun) by HelloWorld (guest, #56129) [Link] (1 responses)

Yes - and it means most "business programs" are horrible. But with Java they work. With other, less restricting languages, they don't.

You'll have to give reasons for this if you want anybody to believe this who doesn't already.

The problem here is the fact that it's usually very hard to fire bad developer from "big business" so they must be used somehow - and Java is the best language for that.

Those developers shouldn't have been hired in the first place. After all, this is what the probation time is good for: keeping lunkheads out of your team. Instead, you're even trying to accommodate those lunkheads. This is like letting a would-be carpenter use a hand saw instead of a power saw since he might cut his fingers off instead of telling him that he's simply not the right person for the job.

If someone don't want to believe something then he'll not believe something

Posted Oct 3, 2010 9:07 UTC (Sun) by khim (subscriber, #9252) [Link]

You'll have to give reasons for this if you want anybody to believe this who doesn't already.

Sorry, but no. You can not disprove honest belief so easily: that's why we hava Judaism, Christianity, Islam, Buddhism, etc. I've already said why Java is "better": instead of small number of constructs which can be combined in many ways you have lots and lots of stuff which can be combined in limited way. This is easier to use if you try to randomly connect things and see what will finally work.

Those developers shouldn't have been hired in the first place. After all, this is what the probation time is good for: keeping lunkheads out of your team.

Welcome to real world. Yes, probation time helps, but not absolutely: if you reject candidates one after another then HR department will eventually decide that it's better to fire you (citing unaccommodating as reason) rather then tolerate problems you cause (they lose bonuses if you reject lunkheads, you know).

So in the end only absolutely hopeless imbeciles are fired. Everyone else stays and you must accommodate them somehow. Especially if they have good recommendations and finished prestigious universities.

This is like letting a would-be carpenter use a hand saw instead of a power saw since he might cut his fingers off instead of telling him that he's simply not the right person for the job.

Yes and no. It's more like giving him power saw but only asking him to produce identical straight planks - and this is exactly what happens in real life. If you take a look on what is in stores then you'll see that work of genuine carpenters-designers are rare and expensive. Most furniture are mass-produced using the same design for thousands of items. Why? It's easy: it's cheaper this way. People who can design exist, often they can even use power saw, but they are rare and expensive. So furniture production is optimized for lunkheads: they know how to operate power saw or even more complex equipment, but they don't know how to design things and how to cope with timber blemishes (unless they've gotten explicit instructions). So other people (designers) are doing all planning work and low-lever people are just following the rules.

Java world is designed for just such a use. And the main ingredients are not even language capabilities but IDE capabilities - but they are enabled by language capabilities. I've conducted hundred of interviews and "Java developers" are worst bunch: often they can not even write simple programs like you've shown above without help of IDE! And they will happily write O(N^2) solution you've presented - if it'll work then Ok, if it'll not work - some higher level person will speedup the thing by factor ten and will get separate bonuses for that work.

Business world is not about good code. It's about strict adherence to specifications - and Java helps immensely here.