[PATCH]format=flowed [LWN.net]

[PATCH]format=flowed

Posted Jun 7, 2007 10:31 UTC (Thu) by xoddam (subscriber, #2322)
In reply to: [PATCH]format=flowed by foom
Parent article: Re: [PATCH -mm] 1/2: MMCONFIG: validate against ACPI motherboard resources

> No, _not_ using format=flowed wreaks havoc upon structured text.

The purpose of format=flowed is to let some paragraphs be
re-wrapped for narrow displays. The intention was that
anything other than a wrappable paragraph would be a
fixed text line and be treated like plain text.

But the inventors in their wisdom decided to fix other
things that also weren't broken (like wrapping of paragraphs
quoted with ">"), and invented "space stuffing":

"On reception, if the first character of a line is a space,
it is logically deleted."

"On generation, any unquoted lines which start with ">", and any
lines which start with a space or "From " MUST be space-stuffed.
Other lines MAY be space-stuffed as desired."

( from http://www.ietf.org/rfc/rfc3676.txt )

If this 'flowed text' were a separate mime-type, and text
was always converted to and from it consistently (which
requires a given display width and an unambiguous way to
distinguish wrappable paragraphs from "fixed lines"), there
would be no problem. But it's also supposed to be plain
text, and display (and parse!) the same as plain text.
It doesn't, and it can't.

> assume that the receiver also uses a client that
> wasn't written in the dark ages, and thus knows
> about the format=flowed parameter.

Knowing about it and implementing it *correctly* aren't quite the same.

There are so many bugs persisting in eg. Thunderbird (follow
links from https://bugzilla.mozilla.org/show_bug.cgi?id=168420)
that when I use the same program as composer and reader,
indentation is messed up *every time*.

As long as anyone on a list is using an MUA which either (a) does
format=flowed *wrong*, like Thunderbird (and many other graphical
mail clients), or (b) doesn't do format=flowed at all, it makes
sense to deprecate it.

> lots of people on OSS mailing lists use clients that don't deal
> with attachments very well either (I guess?), so those are also
> disfavored...

I think the preference for in-line patches is so that they can be
read in-line, and so that the patch and accompanying message can
be saved together as a plain text file which works with 'patch'.
Attachments have an annoying habit of getting silently encoded as
quoted-printable or base64 -- which is almost as annoying as
format=flowed.

The habit of piping raw email messages to patch is compelling.
If format=flowed were implemented sufficiently consistently that
leading spaces were preserved across all MUAs which supported
it, there would be a compelling reason for 'patch' itself to
recognise the format=flowed line and parse the text accordingly.

As long as popular MUAs do it wrong, there is no point trying
to support it at all.

(Log in to post comments)

[PATCH]format=flowed

Posted Jun 7, 2007 14:36 UTC (Thu) by DonDiego (subscriber, #24141) [Link]

Not if the patch is flagged as text/plain, text/x-diff or text/x-patch. Your mail client can then quote it inline for you in replies.

If people send patches marked as binaries, mutt allows you to change the mime-type on the fly. I'm soooo in love with this feature ...

[PATCH]format=flowed

Posted Jun 15, 2007 21:50 UTC (Fri) by slamb (guest, #1070) [Link]

I don't get this either, but sometimes you have to let the wookie win. git users are very insistent on a particular format (see git's own Documentation/SubmittingPatches), and they merge enough patches that they don't want to mess around with figuring out your format. Whether their format is right or wrong, the consistency makes sense.

However, you may be able to just use git send-email when sending patches to these people, rather than alter your MUA's settings. format=flowed does serve a purpose, even if some people don't like the details of how it is done.

[PATCH]format=flowed

Posted Jun 7, 2007 17:59 UTC (Thu) by foom (subscriber, #14868) [Link]

> If this 'flowed text' were a separate mime-type, and text was always converted to and from it
> consistently (which requires a given display width and an unambiguous way to distinguish
> wrappable paragraphs from "fixed lines"), there would be no problem. But it's also supposed
> to be plain text, and display (and parse!) the same as plain text. It doesn't, and it can't.

It is an explicit attribute on the text/plain mime type. It doesn't require a fixed display width. It is
unambiguously convertible back into the unwrapped form. It is not supposed to parse the same
as plain text. (where are you coming up with this?)

You aren't suppsed to randomly apply format=flowed rules to all incoming text. Just that which
explicitly states it was encoded using that scheme...

Prior to format=flowed, to send plaintext mail which round-trips paragraphs properly, the only
option was quoted-printable encoding. That worked and works great, but doesn't degrade very
well for people using crappy clients. With format=flowed, the emails roundtrip in useful clients,
and don't look completely terrible on lame clients. It is a total win.

> Knowing about it and implementing it *correctly* aren't quite the same.
>
> There are so many bugs persisting in eg. Thunderbird (follow
> links from https://bugzilla.mozilla.org/show_bug.cgi?id=168420)
> that when I use the same program as composer and reader,
> indentation is messed up *every time*.

Okay, I've never used Thunderbird. Maybe it's absolutely totally worthless at email, I can believe
that. But it having a defective implementation doesn't make the thing it implements bad. Apple
Mail composes plaintext mail which round-trips perfectly. It's really not that mysterious a
process, the rules are plainly spelled out in the RFC.

> Attachments have an annoying habit of getting silently encoded as
> quoted-printable or base64

Well, yeah...Any mail client worth its salt will decode q-p and base64 encodings before you ever
see them. What are you using as a client, "/usr/bin/mail"?

[PATCH]format=flowed

Posted Jun 7, 2007 19:27 UTC (Thu) by foom (subscriber, #14868) [Link]

It's pretty ironic that my comments in this discussion are nearly unreadable in my RSS reader,
because LWN uses the HTML textarea wrap="physical" cols="75", and thus my browser is
hard-wrapping the comment text to 75 columns.

Sigh.

[PATCH]format=flowed

Posted Jun 8, 2007 1:13 UTC (Fri) by xoddam (subscriber, #2322) [Link]

> It is an explicit attribute on the text/plain mime type.
> It doesn't require a fixed display width.

(I just meant you need a display width to render it)

> It is unambiguously convertible back into the unwrapped form.

Round-trips are broken in several MUAs.

> It is not supposed to parse the same as plain text.

Well alright, that wasn't a requirement. But it is supposed
to *display* the same as plain text in user agents which
do not support it. It doesn't. Moreover, it doesn't parse
the same.

> (where are you coming up with this?)

RFC 2646 and RFC 3676, which make a false assumption:

"Since lines which require space-stuffing rarely occur, and
the aesthetic consequences of unreversed space-stuffing
are minimal, this is not expected to be a significant problem."

Programs parse email. Email messages can contain structured
text where leading white-space is significant, to human and
mechanical readers. Space stuffing stuffs it up. This *is*
a significant problem.

> You aren't suppsed to randomly apply format=flowed rules
> to all incoming text. Just that which explicitly states
> it was encoded using that scheme...

The problem (with Mozilla Mail aka Thuderbird) is that
*all* *outgoing* plain text mail is format=flowed,
whether or not you want your text to be parseable at
the other end. A further problem is that the users
mostly have no idea they're sending email which isn't
(quite) text/plain. It took me years to discover what
the problem was, and how to fix it. Literally.

FYI (or rather for the information of people who don't
like this behaviour), you *can* turn off format=flowed
for all outgoing mail:

Tools -> Options -> Advanced -> Config Editor

then set mailnews.send_plaintext_flowed false.

Unfortunately you can't turn it off easily per-message
while composing.

I have no problem with mime attachments for the purpose
of sending files. But patches are supposed to be readable
by humans and by the patch program. They are often sent
for discussion to a mailing list.

I have no problem with format=flowed for text which
is just paragraphs and non-indented lines. I have
not turned off the preference for *displaying* flowed
text. But as soon as whitespace (leading or trailing)
becomes significant, format=flowed ceases to be
appropriate. The obvious cases are patches, inline code
or pseudocode, ASCII art and "manually" indented
paragraphs such as outline lists.

> What are you using as a client, "/usr/bin/mail"?

No, Thunderbird. Outlook. Evolution. KMail. Various
Web mail services. Piping emails to patch.

Thunderbird is admittedly the worst of them, and maybe
the only one which consistently breaks round-trips. But
none of them give any indication that what they're sending
is not "fixed" text/plain; you have to dig deep into
preferences, FAQs and bug databases to learn the most basic
things about what is a very basic computing task.

[PATCH]format=flowed

Posted Mar 25, 2010 23:52 UTC (Thu) by jschrod (subscriber, #1646) [Link]

> [about:config] then set mailnews.send_plaintext_flowed [to] false.

Thanks, thanks, thanks!

There are some client environments where I have to use Thunderbird. Now I can make them behave a bit better.

[PATCH]format=flowed

Posted Mar 26, 2010 14:05 UTC (Fri) by foom (subscriber, #14868) [Link]

I've been using format=flowed in my mail-reader for years (Apple Mail) -- and have not had the problems you have. In my experience, the assumption in the RFC about space-stuffing not causing a big problem is spot on. Maybe it's because when I have something like a patch, I attach it to the mail instead of pasting it into the body.

I don't really understand why anyone likes to do it the other way. Having it as an attachment makes it easier to extract (save the file, vs find the beginning and end of the patch and copy/paste), and many mail-readers will display plaintext attachments inline, too...

Unfortunately, as of OSX 10.6.2, Apple Mail stopped sending format=flowed, and now always sends MIME quoted-printable encoding (without any added line-breaks encoded), which of course is a complete disaster, since mail-list archives are generally too stupid to do anything but decode the quoted-printable...and it ends up all on a single line, with soft-wrapping disallowed. :(

So you get things like: http://lists.cs.uiuc.edu/pipermail/llvmdev/2010-March/030...