bytes vs. characters

Posted Apr 15, 2015 18:22 UTC (Wed) by njs (subscriber, #40338)
In reply to: bytes vs. characters by david.a.wheeler
Parent article: Report from the Python Language Summit

> In particular, data sources routinely lie about their encoding, and Python 3 interferes with handling the real world instead of helping with it.

I'm curious if you could elaborate on what interference you're thinking of? I don't have a dog in the fight or anything, but my experience with py3 has been pretty pleasant so far, and I don't see off the top of my head how py3 could do worse than py2 in this case. It seems like at worse you would end writing the same code in both cases to treat the data as bytes, try different encodings or whatever you want, with the main difference that in py3 at least you don't have to deal with random functions deciding to help out by spontaneously encoding/decoding with some random codec? Or depending on what you're doing, surrogate-escape could be pretty useful too, and that's a py3 feature.