Encoding in Kate [LWN.net]

Encoding in Kate

Posted Jun 13, 2006 9:12 UTC (Tue) by halla (subscriber, #14185)
In reply to: Encoding in Kate by man_ls
Parent article: Gedit Review (Softpedia.com)

Autodetecting a text encoding is definitely not simple at all -- in fact,
most people who've tried it agree that it's impossible do do correctly
and reliably.

to post comments

Encoding in Kate

Posted Jun 13, 2006 9:16 UTC (Tue) by piman (guest, #8957) [Link] (2 responses)

Autodetecting UTF-8 is pretty easy, since very few valid sequences are valid UTF-8. So UTF-8, fallback to locale encoding, is very effective. If a UTF-8 or UTF-16 BOM is present, it's also almost always right (though ugly).

When downloading web pages, detecting the encoding is very easy, the server gives it (if the server is wrong, it's not your editor's fault, and it will be right more often than not).

It's only when forced to choose between old legacy encodings (Latin 1 versus Latin 2, for example) that it becomes difficult-to-impossible to guess the encoding.

The final good solution is to assume everything is UTF-8 unless otherwise specified and force the last legacy users to migrate to something sane. This is often best on a multiuser system where LC_CTYPE won't apply outside the current user's /home resources.

Encoding in Kate

Posted Jun 13, 2006 14:29 UTC (Tue) by tjc (guest, #137) [Link]

The final good solution is to assume everything is UTF-8 unless otherwise specified and force the last legacy users to migrate to something sane.

Sounds a bit Orwellian to me. Even if you mostly succeed, a few people will still meet in abandoned building at the dead of night and read ASCII text...

Encoding in Kate

Posted Jun 13, 2006 14:44 UTC (Tue) by man_ls (guest, #15091) [Link]

People will use whatever works best for them. For me, it's latin-1 or ISO 8859-15. It is not a legacy encoding; it is completely sound if all you want is to write European characters. I'm fairly sure I will never mix any Chinese text in my web pages, and with ISO 8859-15 they seem to work best across editors and browsers.

This web page is written in ISO 8859-1; or so it says in the header:

CONTENT="text/html; charset=iso-8859-1"

so the character might not work. In fact it does work in Firefox, maybe because UTF-8 is really the charset used?

Encoding in Kate

Posted Jun 13, 2006 9:25 UTC (Tue) by man_ls (guest, #15091) [Link] (1 responses)

Note that I'm not suggesting to autodetect the encoding blindly, but based on the XML header:

<?xml version="1.0" encoding="ISO-8859-15"?>

that appears as the first line of an XHTML file. Maybe the choice of word "autodetect" was unfortunate. The encoding in an XML document is declared and should be respected, right?

Encoding in Kate

Posted Jun 14, 2006 11:28 UTC (Wed) by niner (guest, #26151) [Link]

But kate part is no XML editor. It may have some plugins making work with XML files easier, but it's still a normal text editor that knows nothing about XML.