PyCon: Evangelizing Python [LWN.net]

PyCon: Evangelizing Python

Posted Mar 30, 2013 14:26 UTC (Sat) by intgr (subscriber, #39733)
In reply to: PyCon: Evangelizing Python by dpquigl
Parent article: PyCon: Evangelizing Python

> I needed to validate the types of data coming in from a json string and to do this I checked if the values I got back matched certain data types which apparently is a big no no in the world of python. Instead I'm supposed to use them as if they were those data types and wait for the world to explode when they aren't and pick up the pieces.

I believe there's a misunderstanding there. You shouldn't do type-checking of data passed in internal APIs; if your function accepts a string parameter, you don't check that it's string every time -- just assume that the caller got it right.

However, if you're decoding JSON received from untrusted sources, then that kind of type checking makes sense and is often required to prevent some kinds of security bugs.

Case in point with Python 2:

def check_age(data):
    d = json.loads(data)
    if d['age'] < 18:
        print "You must be 18 or older to continue."
    else:
        print "OK!"

>>> check_age('{"age": 13}')
You must be 18 or older to continue.

>>> check_age('{"age": "13"}')
OK!

This hole is fixed in Python 3, but in general there are still surprises you can run into if you allow arbitrary types from untrusted input.

(Log in to post comments)

PyCon: Evangelizing Python

Posted Mar 30, 2013 20:53 UTC (Sat) by tnoo (subscriber, #20427) [Link]

> This hole is fixed in Python 3,

Maybe I don't get it, but how should an undefined value d['age'] be compared to a number? The natural way is to compare int(d['age']) < 18. which won't give random behaviour. and which would be done in any sane language. Nice that it raises Type error in Python3.

PyCon: Evangelizing Python

Posted Mar 31, 2013 9:23 UTC (Sun) by marcH (subscriber, #57642) [Link]

> You shouldn't do type-checking of data passed in internal APIs; if your function accepts a string parameter, you don't check that it's string every time -- just assume that the caller got it right.

"internal" can mean internal to either: your host, your process, your jar, your file, your company, your department, your team,... where does the trust stop? The answer probably depends on the task at hand. There is no simple answer.

Searching for "Defensive Programming" should return a lot of simple - and extreme - answers. Make your own opinion somewhere in the middle...

PyCon: Evangelizing Python

Posted Mar 31, 2013 9:48 UTC (Sun) by dark (subscriber, #8483) [Link]

It's not really about trust. It's about embracing duck typing.

If you're parsing data from strings then by all means check that it's well-formed and that it's the kind of data you expect -- it's all under your control at that point.

If you're accepting a value from elsewhere, then it may make sense to do sanity checks on it, but those should be along the lines of "does it support the methods I expect to call on it", not "does it have the specific type I expect". The best example of this is file objects -- you really shouldn't enforce that it's a "file", you should let the caller pass in whatever file-like object is most convenient.

Most python code is simple enough that you can do those 'checks' just by calling the methods when you need them and getting an exception if they fail. But if you're about to insert such a received value into a complex system and you want to catch errors before they're deep in the stack then it may make sense to inspect the value a bit. Or just convert it -- calling int(value) or str(value) will throw exceptions right away, and the latter will let the value handle its own conversion.