Unladen Swallow 2009Q2 released [LWN.net]

Unladen Swallow 2009Q2 released

Posted Jul 15, 2009 18:08 UTC (Wed) by iabervon (subscriber, #722)
In reply to: Unladen Swallow 2009Q2 released by drag
Parent article: Unladen Swallow 2009Q2 released

Even if Python were as fast as C, Python is much harder to debug than most other languages, because just about any Python code could actually be correct, if some distant part of the program inserted variables in its scope or added attributes to its objects.

There are some applications Python is good for, and some of them care about performance, but there are others where it's much more useful to get compiler errors for large classes of common mistakes.

to post comments

Unladen Swallow 2009Q2 released

Posted Jul 15, 2009 21:51 UTC (Wed) by dw (subscriber, #12017) [Link] (16 responses)

Relying on type checks from the C compiler hardly gets you much, it's a 1% improvement on ensuring the code is grammatically correct. I'm not sure which classes of common mistakes you're referring to, but preventing addition of integers and strings, and so on, is not my idea of a big win.

As for arbitrarily munging with module namespaces, and so on, well, you can also write inline asm from C or write jumps between function bodies. ;)

Unit/system tests that exercise code catch a hell of a lot more classes of bugs than C's idea of type checking ever could, which you should probably be doing for C projects too. The question is which language leaves you with more time to write such tests..

Unladen Swallow 2009Q2 released

Posted Jul 15, 2009 22:00 UTC (Wed) by clugstj (subscriber, #4020) [Link] (4 responses)

I'd have to say that it is MUCH MORE than 1% improvement. You can compile just about anything in Python. With Python you have to write a lot more unit tests to verify that the code is to the level of what a clean compile would tell you in a more strongly typed language.

Of course that flexibility can be useful, but it isn't free.

Unladen Swallow 2009Q2 released

Posted Jul 16, 2009 0:17 UTC (Thu) by drag (guest, #31333) [Link] (2 responses)

> I'd have to say that it is MUCH MORE than 1% improvement. You can compile just about anything in Python. With Python you have to write a lot more unit tests to verify that the code is to the level of what a clean compile would tell you in a more strongly typed language.

Python is a strongly typed language....

$ python
Python 2.5.4 (r254:67916, Feb 17 2009, 20:16:45)
[GCC 4.3.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> cow = "5"
>>> cow = 1 + cow
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +: 'int' and 'str'

-------------------

You can 'compile' it also:

$ cat > hello_world.py
print("hello world")

$ python
Python 2.5.4 (r254:67916, Feb 17 2009, 20:16:45)
[GCC 4.3.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import py_compile
>>> py_compile.compile("hello_world.py")
>>> exit()

$ rm hello_world.py ; chmod +x hello_world.pyc

$ ./hello_world.pyc
hello world

Well.. it's byte-compiled, not native machine code. Same difference, pretty much. What don't you get from executing a python program that you don't get from compiling C++?

There are plenty of tools for analyzing python code for correctness and whatnot....

Unladen Swallow 2009Q2 released

Posted Jul 16, 2009 0:22 UTC (Thu) by drag (guest, #31333) [Link] (1 responses)

Sorry, I know I am being a bit foolish.

I think you meant 'statically' typed, not 'strongly' typed.

Unladen Swallow 2009Q2 released

Posted Jul 16, 2009 4:12 UTC (Thu) by k8to (guest, #15413) [Link]

Python is definitely *not* statically typed.

>>> myvar = 4
>>> myvar = "hi"
>>>

Notice no error.

Unladen Swallow 2009Q2 released

Posted Jul 16, 2009 4:14 UTC (Thu) by k8to (guest, #15413) [Link]

With all languages, you have to write a lot more unit tests. It's just the statically typed crowd who operate under some fantasy that they don't.

That python forces you into this is a feature.

Unladen Swallow 2009Q2 released

Posted Jul 15, 2009 22:11 UTC (Wed) by iabervon (subscriber, #722) [Link]

The useful thing isn't the type checking but rather the fact that variables have to be declared in a lexically enclosing scope, which means that trivial typos are compile-time errors rather than runtime errors (when that line is actually reached). Also important are things like calling functions with the wrong number of arguments (swapping % and , in function calls). And it's hard to write unit tests for cases which shouldn't be reachable, but which are important for reporting when unexpected things happen.

Unladen Swallow 2009Q2 released

Posted Jul 16, 2009 10:54 UTC (Thu) by NAR (subscriber, #1313) [Link] (9 responses)

The type checks are really useful when the code is being refactored. The type is modified at one place, then the compiler shows all the other places where the code should be updated. In this case the unit tests might be not that useful, because e.g. it takes too long time to run the unit tests or the unit tests don't provide 100% coverage (happens in real world, especially with error cases).

Unladen Swallow 2009Q2 released

Posted Jul 16, 2009 10:56 UTC (Thu) by dw (subscriber, #12017) [Link] (8 responses)

This is not a feature of static languages. See for example Rope, http://rope.sourceforge.net/

Hmm... May be I misunderstood something

Posted Jul 17, 2009 4:41 UTC (Fri) by khim (subscriber, #9252) [Link] (7 responses)

How can the rope help in real usecase: change of the interface of in big codebase? I can explain how it's done with C:
1. Old version of feature (class, function, etc) is marked as __attribute__ ((deprecated)) (with comment about possible replacements).
2. After month or two feature is removed.

Note: the codebase is big and that means you can not get access to all users of the feature (they are in diffirent repositories, etc). How can this be done with rope?

Sure, the rope looks nice, but does it work for big projects? I was under impression that it can only do "IDE-style refactoring": where all project can be loaded in one tool. This is where Java IDEs are great, but this NOT the real advantage of static typing. The real advantage is simple fact that interface is checked at compile time and, the most iportant, changes to the interface are checked at compile time!

If you start adding unittests required to check the interface for all classes you are using you are losing the "number of lines" advantage Python offers over C/C++ really fast... Sure C/C++ are awful languages too, but it's related to stupid low-level C pointers, not to the static typing...

It's not rocket science..

Posted Jul 17, 2009 4:52 UTC (Fri) by dw (subscriber, #12017) [Link] (2 responses)

--- module.py.bak	2009-07-17 05:47:46.000000000 +0100
+++ module.py	2009-07-17 05:48:25.000000000 +0100
@@ -1,5 +1,11 @@
 
+import warnings
+
 def some_function():
+    warnings.warn('deprecated, use amazing_function() instead!')
+    return some_function_deprecated()
+
+def some_function_deprecated():
     do_old_fashioned_stuff()
     if True and 0x62:
         return

By some accounts, relying on GNU C features to 'increase' maintainability is on a road straight to hell. ;)

It's not rocket science..

Posted Jul 17, 2009 8:22 UTC (Fri) by amonnet (guest, #54852) [Link]

Sure,

this works well for commonly used codepath that will be covered by testings. For occasionnal codepath, i'm pretty sure no one will catch the warning until production.

Worse then useless

Posted Jul 17, 2009 15:32 UTC (Fri) by khim (subscriber, #9252) [Link]

Not only it fails to catch the errors at build time it gives end-user warnings about some program internals - the last thing user needs.

Sure, the Python is great. Sure, with right amount of testing you can make small or even medium programs work. Yet all this is breaking apart for a big programs.

This is not a disaster (one-size-fit-all is not good approach, really), hust something to keep in mind.

Hmm... May be I misunderstood something

Posted Jul 18, 2009 10:47 UTC (Sat) by bockman (guest, #3650) [Link] (3 responses)

Sure, compiler checks are useful. But are not enough, and this is a point that sometime is forgotten in comparing static and dynamic typing. An API can mantain its formal interface but change its meaning (e.g. the parameter 'length' was inches and now is centimeters ) and no compiler will help you with that.

And I don't think that in a dynamic language one has to do explicit unit test for wrong parameter types : misusage of a function is bound to trigger some exception or make anyway fail the test. The resulting error message might not be the most informative, but usually lead you to the original problem. What you have to do is ensure full code coverage, which is a good practice also in statically-typed languages.

Not that I would dislike having more compiler-time checks in python: there are tools that help with that, and maybe they will help more in the future making clever usage of python3 annotations.

Ciao
------
FB

Hmm... May be I misunderstood something

Posted Jul 18, 2009 19:34 UTC (Sat) by farnz (subscriber, #17727) [Link] (1 responses)

Actually, given an expressive type system, the formal interface would change for that change in meaning. For example, using a type system comparable to that of Haskell (but C-ish syntax), I could define a function Storeys buildingHeightInStoreys( Metres<Int> length );. When I change that function to work in inches, it becomes Storeys buildingHeightInStoreys( Inches<Int> length );.

Throw in type inference, and you get some of the benefits of dynamic languages like Python. Again, using C-ish syntax, I could declare a function USDollars estimatedRent( auto height );, and write suitable behaviour for each variation on height (perhaps by unifying on height in storeys, and estimating from there).

Having said all that, it's a tradeoff in terms of work - the question is not whether static types are enough to stop all bugs on their own, but whether the reduction in tests that they permit is outweighed by the cost of static types. I believe that in C, this isn't true. In Haskell, however, people claim that it is true; this is one that remains to be tested more formally.

Hmm... May be I misunderstood something

Posted Jul 18, 2009 19:45 UTC (Sat) by dw (subscriber, #12017) [Link]

In this particular case I think 'the pen is mightier than the sword'.. just have a paragraph in your style guide requiring the scale and unit be included in the variable name, or at least documented at the variable's declaration site.

I personally work on one project where units of time are stored in things like "lengthMs", "startDelayMs", etc. Seems to work well for me.

Compiler is not substitute for brain, sorry...

Posted Jul 19, 2009 6:32 UTC (Sun) by khim (subscriber, #9252) [Link]

Sure, compiler checks are useful. But are not enough, and this is a point that sometime is forgotten in comparing static and dynamic typing.

Sorry, but this is bull-shit.

An API can mantain its formal interface but change its meaning (e.g. the parameter 'length' was inches and now is centimeters ) and no compiler will help you with that.

Fire the person who commited such a change (or revoke his commit rights) - problem solved. It's resposibility of programmer to guarantee that formal interface changes if the semantic changes: new method should get some suffix prefix so it's impossible to mix old and new version, it's not a rocket science. Bad programmer can write awful unmaintable program in any language.

And I don't think that in a dynamic language one has to do explicit unit test for wrong parameter types : misusage of a function is bound to trigger some exception or make anyway fail the test.

Not at all. Take your own example: suppose you have an interface where you enter paraments of sattelite and it returns some stats. Suppose you have two implementations (different approximation methods or something). Unittest for interface will test that everything works for trivial case, unitests for implementations will check error margins and other such things. The exception will be triggered, alright: when orbiter will either try to orbit Mars below surface or "NoSuchMethod" exception will be thrown and the system will be shut down. By that time it's kinda too late...

What you have to do is ensure full code coverage, which is a good practice also in statically-typed languages.

Full code coverage is nice and all, but unfortunatelly it's not an option for a big system: you can only test one component a time, because the whole system is too big.