bytes vs. characters [LWN.net]

bytes vs. characters

Posted Apr 23, 2015 22:07 UTC (Thu) by dlang (guest, #313)
In reply to: bytes vs. characters by marcH
Parent article: Report from the Python Language Summit

they are only invalid if you decide ahead of time that they are supposed to be UTF8 strings.

the spec allows them to be a string of bytes (excluding null and /), no encoding is required.

to post comments

bytes vs. characters

Posted Apr 23, 2015 22:30 UTC (Thu) by marcH (subscriber, #57642) [Link]

> they are only invalid if you decide ahead of time that they are supposed to be UTF8 strings.

They are invalid if you decide that they are supposed to be in some encoding and some filename uses any *other* encoding. Then garbage gets displayed: for real.

https://en.wikipedia.org/wiki/Mojibake (search page for "garbage")

It's less rare with removable media or ID3
https://en.wikipedia.org/wiki/ID3 (search page for "mojibake")

> the spec allows them to be a string of bytes (excluding null and /), no encoding is required.

As far as filenames are concerned, you meant: *the lack of* a spec.

http://www.dwheeler.com/essays/fixing-unix-linux-filename... (search page for... "garbage")

> no encoding is required.

Which command do you typically use instead of "ls"? hexdump?