The trouble with text-only email [LWN.net]

By Jonathan Corbet
October 12, 2017

Mozilla's manifesto commits the organization to a number of principles, including support for individual privacy and an individual's right to control how they experience the Internet. As a result, when Mozilla recently stated its intent to remove the "text only" option from its mailing lists — for the purpose of tracking whether recipients are reading its emails — the reaction was, to put it lightly, not entirely positive. The text-only option has been saved, but the motivation behind this change is indicative of the challenges facing independent senders of email.

The announcement that the text-only option would be removed was made (on the mozilla-governance list) by Michele Warther in late September. It quickly became clear that the community viewed this idea with a bit of skepticism, leading Warther to explain the reasoning behind the change:

Unfortunately text emails don’t have the same feedback loops available as HTML. A lot of people, of course, consider this to be a feature. But an unintended result is the negative effect that zero-interaction signals have on our reputation scoring: we have seen an increase in greylisting/blacklisting as a direct result of our text only emails, and that typically means we can’t send any email to *anyone* until it gets resolved.

The spam-ridden nature of the Internet forces anybody receiving email to be somewhat selective about what is allowed through. Only the most obscure and privately held email accounts can be exposed to an unfiltered mail stream without driving the owner insane. Much of the filtering applied is content-based, but much of it, especially at certain large web-based email providers, also takes into account the "reputation" of the sending site.

Some aspects of email reputation are straightforward. An IP address that is observed to be the source of volumes of spam will quickly find its way onto various online blacklists. When an IPv6 address is involved, the resulting block can cover a significant part of the address space, causing considerable collateral damage; this is why LWN's server is configured to send email via IPv4 whenever possible. IP addresses known to be used for residential Internet access are often penalized — if they are allowed to originate mail at all. Anybody who maintains their own email system can attest that reputation scoring also seems to have a significant random factor.

One metric that some sites evidently use is email sent to accounts that are known to be inactive, which is seen as a sign of a spammy originator. This, seemingly, is where Mozilla has run into trouble. One way to avoid this problem is to track which recipients are actually reading their email; any recipient who doesn't look at any messages for a period of time can then be unsubscribed. Then, in theory, email providers can see that the emails from a given source are actually of interest and refrain from putting up obstacles in their path.

The problem, of course, is that this tracking requires the "feedback loops" mentioned in Warther's message. These loops tend to take the form of tracking images that are fetched from a server belonging to the sender. The privacy implications of this kind of tracking are obvious: not everybody wants email senders to know when their mail was read and where the reader was at the time. Requiring this sort of disclosure would seem to run afoul of Mozilla Manifesto #4: "Individuals’ security and privacy on the Internet are fundamental and must not be treated as optional." But the alternative, Warther said, is an ongoing series of delivery problems for Mozilla's email in general.

There are other problems with tracking images and related mechanisms, starting with the fact that people who are paying attention tend to disable the loading of such images. Your editor recently received a complaint from a financial company that its emails were not being read; those emails were indeed read, they just weren't allowed to phone home and report that fact. Chances are good that this kind of blocking will increase in the future; not everybody wants to be a part of an unrequested "feedback loop".

In this particular case, it would seem that an acceptable compromise has been found, and the text-only option will remain. But, once a year, those subscribers will get a message asking them to click on a link to confirm their continued interest in remaining on the list. That should allow Mozilla to prune its inactive readers — a useful activity even without the reputation issues — without the need for involuntary tracking.

The fact that even principled organizations like Mozilla feel the need to employ tracking says something discouraging about the state of email, though, not that there was really any need for more evidence that the email system is broken. As the reputational checks become harder to pass, more users will be forced to use one of a small number of huge webmail providers (which have no trouble with "feedback loops") just to get their work done. Every kernel merge window features one or more developers having trouble getting their pull requests through to Linus Torvalds's Gmail account, for example. It's not clear what the solution to the email problem is, but the need is obvious.

to post comments

The trouble with text-only email

Posted Oct 12, 2017 15:53 UTC (Thu) by pabs (subscriber, #43278) [Link] (15 responses)

Which email clients actually still allow this sort of tracking of HTML email?

The trouble with text-only email

Posted Oct 12, 2017 16:03 UTC (Thu) by dbaker (guest, #89236) [Link] (13 responses)

All of them. The way this generally works is that there's a unique, invisible, element in the email, like a 1x1 pixel image. When that image is requested, the server marks that the email has been read. Since most email clients blindly load all HTML elements this works. Even for the more careful clients that don't load elements from remote servers by default and ask the user to click a "load full message" (k-9 mail on android does this), most HTML mail is unreadable without those remote assets because the layout is completely broken.

This is even harder for mail client's that have to rely on a third-party rendering engine for HTML mail, since they don't necessarily have the hooks into the renderer to tell it not to load remote content.

The trouble with text-only email

Posted Oct 12, 2017 16:20 UTC (Thu) by rjek (subscriber, #94501) [Link] (1 responses)

Mozilla's own mail client turns this off by default, on privacy grounds.

The trouble with text-only email

Posted Oct 15, 2017 20:23 UTC (Sun) by rahvin (guest, #16953) [Link]

Almost all email clients do an I believe that was the originators point, this even includes Microsoft's outlook which won't download the external elements without you saying you want to. The original posters point is important, you think you're going to track people with a hidden HTML element but the reality is almost all email clients block viewing of external elements for security and privacy reasons.

Although privacy is mentioned lets not forget it's possible to put javascript into an HTML email and most email readers just pipe the HTML code straight into one of the browsers for display. Now you have a script from god knows where running by default in whatever browser code the email reader uses (which is explorer for Outlook on windows) if you are using HTML email. Because of this I consider HTML emails to be a security threat.

Not blindly loading elements works just fine

Posted Oct 12, 2017 17:18 UTC (Thu) by david.a.wheeler (subscriber, #72896) [Link] (4 responses)

> All of them. The way this generally works is that there's a unique, invisible, element in the email, like a 1x1 pixel image. When that image is requested, the server marks that the email has been read. Since most email clients blindly load all HTML elements this works. Even for the more careful clients that don't load elements from remote servers by default and ask the user to click a "load full message" (k-9 mail on android does this), most HTML mail is unreadable without those remote assets because the layout is completely broken.

I don't load elements from remote servers by default, and practically never have problems with desirable email. In my experience, email that's broken without remote loading is practically always spam.

> This is even harder for mail client's that have to rely on a third-party rendering engine for HTML mail, since they don't necessarily have the hooks into the renderer to tell it not to load remote content.

At least some third parties DO support this functionality.

Not blindly loading elements works just fine

Posted Oct 14, 2017 16:39 UTC (Sat) by rbrito (guest, #66188) [Link] (3 responses)

I wrote a python script that is meant to be run by procmail and that discards the HTML part of an email if it is only composed of plain text and HTML.

The code is here: https://github.com/rbrito/scripts/blob/master/drop-altern...

Not blindly loading elements works just fine

Posted Oct 15, 2017 14:54 UTC (Sun) by mathstuf (subscriber, #69389) [Link] (2 responses)

I've seen multipart emails where the plain text is just "sorry, but your client doesn't support HTML messages". Why even bother at that point?

Not blindly loading elements works just fine

Posted Oct 15, 2017 21:11 UTC (Sun) by klempner (subscriber, #69940) [Link] (1 responses)

Worse, that's not even standards compliant. In multipart/alternative the "alternatives" are supposed to be alternatives. It should be fine (if suboptimal) to see the text only version.

For a while I used a mail client configuration which fell back to rendering html emails through links, but would prefer a plain text alternative which would usually be better than the output of this hack.

Not blindly loading elements works just fine

Posted Oct 16, 2017 14:43 UTC (Mon) by mstone_ (subscriber, #66309) [Link]

I gave up and default to displaying a text rendering of the html, because the majority of multiparts I was getting had the stupid "haha, you can't read the message" text alternative. Nothing gets more traction than a dumb idea.

Text-only email discouraged in US government

Posted Oct 12, 2017 20:44 UTC (Thu) by david.a.wheeler (subscriber, #72896) [Link]

There are discouragements to using HTML email in the US government, because of their security risks. There really aren't many upsides to using them either.

The Department of Homeland Security (DHS)'s "Enhanced Analysis of GRIZZLY STEPPE Activity" (February 10, 2017) reminds organizations to disable HTML in email. Here's the text:

Organizations should ensure that they have disabled HTML from being used in emails, as well as disabling links. Everything should be forced to plain text. This will reduce the likelihood of potentially dangerous scripts or links being sent in the body of the email, and also will reduce the likelihood of a user just clicking something without thinking about it. With plain text, the user would have to go through the process of either typing in the link or copying and pasting. This additional step will allow the user an extra opportunity for thought and analysis before clicking on the link.

The US DoD bars the use of HTML email in heightened alerts.

The trouble with text-only email

Posted Oct 12, 2017 21:39 UTC (Thu) by davidstrauss (subscriber, #85867) [Link]

The email system (client or server) will try to load the image, but this sometimes occurs even without the user opening the message: https://blog.mailchimp.com/how-gmails-image-caching-affec...

The trouble with text-only email

Posted Oct 13, 2017 8:39 UTC (Fri) by NAR (subscriber, #1313) [Link] (3 responses)

The gmail web interface (by default) does not load remote images, only if I explicitly ask for images.

The trouble with text-only email

Posted Oct 17, 2017 2:13 UTC (Tue) by Mithrandir (guest, #3031) [Link] (2 responses)

IIRC, Google has now moved to a system where they download a copy of the image on their server, caching it, thereby removing any ability of IP-based tracking: the sender only sees a single hit, from a Google server, no matter how many messages they send to gmail accounts.

The trouble with text-only email

Posted Oct 17, 2017 5:53 UTC (Tue) by johill (subscriber, #25196) [Link] (1 responses)

That can't possibly work, it's trivial to embed the tracking cookie in the URL of the image.

The trouble with text-only email

Posted Oct 17, 2017 11:11 UTC (Tue) by mathstuf (subscriber, #69389) [Link]

It depends on when the image is cached. Upon receipt? No different than a lack of a bounce message. On viewing the parent email? That's problematic. Showing external images? Well, the user requested it then.

The trouble with text-only email

Posted Oct 12, 2017 18:40 UTC (Thu) by jhoblitt (guest, #77733) [Link]

I'd argue that HTML mime segments should have been limited to referring to other mime segments from the get-go. Virtually everyone agrees that XSS is bad -- so why is email loading remote entities a good idea?

The trouble with text-only email

Posted Oct 12, 2017 17:21 UTC (Thu) by alvieboy (guest, #51617) [Link] (8 responses)

Still, this was recently published (unsure about when the original report was, though, or how serious it is): https://www.scientificamerican.com/article/the-only-safe-e-mail-is-text-only-e-mail

Alvie

The trouble with text-only email

Posted Oct 13, 2017 5:22 UTC (Fri) by eru (subscriber, #2753) [Link] (7 responses)

It is a kind of obvious claim, since text-only cannot have active content (unless the client really, really blows it - two decades ago, GNU Emacs used to have a bug where loading a plain text file could trigger the execution of arbitrary lisp code from the file). The bad thing is that some clients work overtime to make text mail unusable (Hi, Outlook), with "helpful" wrapping and replacement of some character sequences. Then there is the issue of characters outside 7-bit ASCII range. Text mail has multiple alternative representations (for historical reasons), a client dealing with all of them gets complex, and therefore has added possibilities for bugs.

The trouble with text-only email

Posted Oct 13, 2017 5:58 UTC (Fri) by dd9jn (✭ supporter ✭, #4459) [Link] (6 responses)

Do you mean QP and Base64 encodings by "alternative representations"? If so, I cannot see the difference to HTML parts, which may use the very same encodings (on top of HTML entity references). Same goes for the charset conversion.

The trouble with text-only email

Posted Oct 13, 2017 6:49 UTC (Fri) by eru (subscriber, #2753) [Link] (5 responses)

That, and the fact that many different character sets can be used, which may not be understood or processed correctly by the client. I agree these issues are not unique to text mail. It would be nice, not that UTF-8 is getting adopted, and transports no longer gag on bytes with MSB set (unlike in the serial modem days), to have text mails always sent as UTF-8, as-is. But cannot happen, too much historical baggage.

The trouble with text-only email

Posted Oct 13, 2017 12:23 UTC (Fri) by hkario (subscriber, #94864) [Link] (4 responses)

> But cannot happen,

it definitely won't happen with such attitude

> too much historical baggage.

SSLv2 and SSLv3 got deprecated without too much crying and gnashing of teeth, I think we can make software recognise and default to encoding quarter of a century old...

The trouble with text-only email

Posted Oct 13, 2017 18:44 UTC (Fri) by tialaramex (subscriber, #21167) [Link] (3 responses)

SSLv2 and SSLv3 got deprecated _from the public Web_. That's a completely different kettle of fish. You'll still see it enabled on email servers, not as commonly as once upon a time but it's hardly rare.

It is much, much harder to deprecate anything from MXs. That's why we have all this "opportunistic encryption" stuff for SMTP. GMail is actually an excellent example. On the _web_ side Gmail simply refuses access to archaic clients that can't reasonable modern security. But when delivering mail from a GMail user over SMTP, if Google finds you can't speak any secure protocols it shrugs and delivers insecurely.

The trouble with text-only email

Posted Oct 14, 2017 16:10 UTC (Sat) by joib (subscriber, #8541) [Link] (1 responses)

> On the _web_ side Gmail simply refuses access to archaic clients that can't reasonable modern security. But when delivering mail from a GMail user over SMTP, if Google finds you can't speak any secure protocols it shrugs and delivers insecurely.

I don't think they accept anything over smtp. At least I had to go to the gmail settings page and explicitly disable some security setting in order to be able to use git send-email via my gmail account.

The trouble with text-only email

Posted Oct 14, 2017 21:39 UTC (Sat) by tialaramex (subscriber, #21167) [Link]

When I wrote about "delivering mail" I actually meant the step where Google calls an MX and tries to _deliver_ the email to its recipient. The clue is probably in that word "delivering". You're talking about a completely different step, so that's why your experience doesn't match up at all.

Forcing secure SMTP with MTA-STS

Posted Oct 15, 2017 16:46 UTC (Sun) by dkg (subscriber, #55359) [Link]

The administrators of the recipient domain can actually tell google (and other providers) not to fall back to cleartext delivery using a protocol called MTA-STS (or at least to report such a problem so that it can be diagnosed). If your e-mail provider doesn't do that, you should encourage them to do so.

You can test this for the domain you're interested in with the following commands:

dig +short -t txt _mta-sts.example.net
wget -q -O- https://mta-sts.example.net/.well-known/mta-sts.txt

Here's how that status looks for gmail itself (which is in mode: report):

0 dkg@host:~$ dig +short -t txt _mta-sts.gmail.com
"v=STSv1; id=20160707T010757;"
0 dkg@host:~$ wget -q -O- https://mta-sts.gmail.com/.well-known/mta-sts.txt
version: STSv1
mode: report
policy_id: 1503837332
mx: gmail-smtp-in.l.google.com
mx: .gmail-smtp-in.l.google.com
max_age: 86400
0 dkg@host:~$

The trouble with text-only email

Posted Oct 12, 2017 18:26 UTC (Thu) by nybble41 (subscriber, #55106) [Link] (4 responses)

> ... the text-only option will remain. But, once a year, those subscribers will get a message asking them to click on a link to confirm their continued interest in remaining on the list.

So HTML readers _won't_ get the link? No doubt there are subscribers who receive e-mails in HTML format but configure their clients not to load external resources; will they be automatically unsubscribed without prior notice?

The trouble with text-only email

Posted Oct 12, 2017 19:55 UTC (Thu) by tome (subscriber, #3171) [Link] (3 responses)

I get these confirmation emails as text with the link as URL in the text. Reading with mutt, I paste the link into a browser and stay subscribed.

The trouble with text-only email

Posted Oct 12, 2017 20:14 UTC (Thu) by nybble41 (subscriber, #55106) [Link] (2 responses)

> I get these confirmation emails as text with the link as URL in the text.

But do you normally get your mailing-list emails in HTML format, or are you a text-only subscriber? If they're sending these notices to everyone they're unable to track then all is well, but (AIUI) the article implied that only text-only subscribers would get the links.

The trouble with text-only email

Posted Oct 12, 2017 20:49 UTC (Thu) by tome (subscriber, #3171) [Link] (1 responses)

I see your point. Yes, I'm a text-only subscriber. It's not clear whether everyone gets these confirmation requests, though the article does say "...the text-only option will remain. But, once a year, *those* subscribers will get a message..." (emphasis mine). Given the problems with suppressed loads of confirmation images, it seems they should send the requests to everyone, and send them all as text.

The trouble with text-only email

Posted Oct 13, 2017 14:32 UTC (Fri) by madscientist (subscriber, #16861) [Link]

A reasonable way to implement this would be to keep a "last known contact" time with each subscription and if that time is >1 year, a ping message is sent asking for confirmation. This contact time would be updated by all sorts of actions, including response to the ping message, reading an HTML mail with images loaded, maybe even sending email (if the return address is accurate), etc.

In that situation people reading HTML mail with images loaded would never get pinged, but it can't reasonably be called preferential treatment.

The trouble with text-only email

Posted Oct 12, 2017 18:59 UTC (Thu) by jkingweb (subscriber, #113039) [Link]

Given Mozilla's reputation generally, I'm surprised they're not whitelisted in the first place. I am not and never have been a webmail user, and it's news to me that these techniques are used to build reputations. It seems, actually, to be counterproductive. Wouldn't an effective phishing scam actually increase its reputation by fooling people, then?

The trouble with text-only email

Posted Oct 12, 2017 19:42 UTC (Thu) by ernstp (guest, #13694) [Link] (12 responses)

Don't they get bounces when they send to inactive email accounts?

The trouble with text-only email

Posted Oct 12, 2017 20:54 UTC (Thu) by pbonzini (subscriber, #60935) [Link] (9 responses)

These are "dormant" accounts, which are almost never read. For example I get my bills on an email account shared with my wife so that both of us can have access on case of emergency. I hardly ever open it since I also see the amounts on the bank's statements, and never use it to send email from.

While my surname is pretty uncommon, this account's username is just our first names strung together, so it is not unlikely that someone else has a very similar email address. In fact, it is surprising how much crap I get that is destined to others who, presumably, used the wrong domain. O_o

This address has only ever been given by me to the electric, gas and phone companies; yet, through the years, it has also been included in several mailing lists, including a parents list at a public school 400 km from here and a marketing list at Diners Club, whose messages are clearly destined to current owners of a Diners credit card.

Given this story, and the abundance of NameNumber@gmail.com addresses, I am not surprised that 1) companies want to know if you read their messages; 2) enough people get subscribed to mailing lists by mistake, and report it as spam, to throw a wrench in the filters' classification algorithms.

The trouble with text-only email

Posted Oct 12, 2017 23:13 UTC (Thu) by josh (subscriber, #17465) [Link] (8 responses)

> enough people get subscribed to mailing lists by mistake, and report it as spam, to throw a wrench in the filters' classification algorithms.

You don't get subscribed to mailing lists "by mistake", and reporting it as spam is completely correct. If the kind of account you have gets subscribed to a mailing list, that mailing list is a spam source and should be blacklisted.

The trouble with text-only email

Posted Oct 13, 2017 8:14 UTC (Fri) by mpr22 (subscriber, #60784) [Link] (7 responses)

Working in a contact centre, I get to field angry telephone calls from people whose public transport discount smartcards are weeks late because they didn't receive the email that we sent to the wrong address they entered and which tells them "sorry, your application has problem XYZ, please log on and fix it". Given that the site asks them to enter their email address twice to guard against typos before allowing them to get anywhere near the ordering interface, I can confidently assure you that humans can't be relied on to type their email addresses correctly even when asked to type them twice for confirmation.

So while you're right to say "report as spam", I regret to say you're wrong about "you don't get subscribed by mistake".

The trouble with text-only email

Posted Oct 13, 2017 8:35 UTC (Fri) by gevaerts (subscriber, #21521) [Link] (4 responses)

You might get one email by mistake. You do *not* get subscribed by mistake. Proper subscription uses a confirmation system.

The trouble with text-only email

Posted Oct 13, 2017 22:20 UTC (Fri) by pbonzini (subscriber, #60935) [Link] (1 responses)

I cannot really blame a middle school for not using Mailman or even Google Groups...

The trouble with text-only email

Posted Oct 15, 2017 18:26 UTC (Sun) by matthias (subscriber, #94967) [Link]

Even if they use mailman, in closed groups, it is quite common that the mailing list administrator manages the subscriptions. The administrator gets a list of hand written addresses and subscribes all of them to the list. If have done this myself for some time for our student homes. There always was a significant amount of typos.

The trouble with text-only email

Posted Oct 19, 2017 9:51 UTC (Thu) by rmv (subscriber, #48328) [Link] (1 responses)

You *shouldn't* get subscribed by mistake. But you do.

I'm on the mailing list of two US schools, a Texas Obgyn clinic, two US based banks, an Indian pharmaceutical sales organisation, a German home furnishings company and a Washington based VW dealership, a food delivery service in Mumbai and a matrimonial service in Kerala. Amongst others.

Being based in northern Europe, I'm pretty sure I didn't at any point sign up for any of those on a whim and then forget about it. These are all from organisations who've received an application from a genuine client and have taken absolutely no steps to corroborate the email address before using it.

If there's an unsubscribe link, I use it. If there's not, or I receive another email after using the unsubscribe link, it gets reported as spam.

The banks were the most persistent as they have enough reputation that reporting them as spam has little effect and the only way to unsubscribe was for the account holder to change their mail preferences and as I wasn't the account holder they couldn't do anything.

The trouble with text-only email

Posted Oct 20, 2017 3:54 UTC (Fri) by mrshiny (guest, #4266) [Link]

I feel your pain. People around the world with the same name as me somehow enter my email address instead of their own. I've been subscribed to dating sites, job sites, payroll sites, real estate, ecommerce sites, you name it, from every English-speaking country. Unsubscribing doesn't work well - thankfully gmail's spam button works pretty well.

The trouble with text-only email

Posted Oct 22, 2017 10:41 UTC (Sun) by gezza (subscriber, #40700) [Link] (1 responses)

Entering the same data twice on a web page is just asking for copy-paste.
If it was wrong the first time, it will be wrong the second.

The trouble with text-only email

Posted Oct 26, 2017 15:14 UTC (Thu) by itvirta (guest, #49997) [Link]

Do you think a regular user knows how to copy-paste? ;)

I seem to remember that Paypal used scripts to prevent copy-pasting, at least when changing
the password (which is annoying if you're using password storage software), I can't remember if
it was for the email address too.

The trouble with text-only email

Posted Oct 26, 2017 8:45 UTC (Thu) by Wol (subscriber, #4433) [Link] (1 responses)

> Don't they get bounces when they send to inactive email accounts?

NOPE. That is not standard.

My email domain is owned by my brother, and he has configured his provider to silently dump unrecognised users.

When I managed a domain ages ago, I configured that to dump all unrecognised users into a single account, used pretty much to just "scan and dump" (when spam volumes were low enough :-)

Cheers,
Wol

The trouble with text-only email

Posted Oct 26, 2017 8:57 UTC (Thu) by yaneti (subscriber, #641) [Link]

Well, what you and your brother did/do is just a cop out.

Its understandable for situations where mail is accepted on machines that don't have the resources to scan for backscatter spam.
Otherwise its just rude and inconsiderate. Imagine if you normal postal service acted like that..

The trouble with text-only email

Posted Oct 12, 2017 19:51 UTC (Thu) by marduk (guest, #3831) [Link] (3 responses)

So one thing that no-ip.com does is for "free" users every month they send you a link to renew your hostname. And you have some number of days to to that or your hostname expires.

Might Mozilla do something similar with mailing lists? Every so often (month, year?) they could mail each subscriber a unique link and give them a number of days to renew their subscription. If the user doesn't go to the link to confirm their subscription within the time frame then they are automatically unsubscribed from the list. I don't think most people would have a problem with that.

The trouble with text-only email

Posted Oct 12, 2017 20:36 UTC (Thu) by karkhaz (subscriber, #99844) [Link]

This is exactly what will happen, as noted in the second-last paragraph of the article

The trouble with text-only email

Posted Oct 13, 2017 0:32 UTC (Fri) by linuxrocks123 (subscriber, #34648) [Link] (1 responses)

God I hated that (the automatic domain delisting thing).

Hence I switched to YDNS. I suggest others do also.

The trouble with text-only email

Posted Oct 13, 2017 11:47 UTC (Fri) by imphil (subscriber, #62487) [Link]

Thanks for the hint on YDNS! I've been looking for a solid alternative for no-ip.com for a long time now, and YDNS looks about right.

The trouble with text-only email

Posted Oct 12, 2017 21:07 UTC (Thu) by frostsnow (subscriber, #114957) [Link] (1 responses)

I was under the naieve assumption that the resources associated with an HTML e-mail were all MIME attachments. Now that I've learned otherwise, and that resource loading is used for *tracking* purposes, I'm rather infuriated and have lost even more faith in the Web/HTML community.

The trouble with text-only email

Posted Nov 5, 2017 21:54 UTC (Sun) by pboddie (guest, #50784) [Link]

In messages I get, I would say that it is almost unusual for images to be other attachments in the same message as the HTML payload. To take one recent example, KMail shows just a single HTML part with no other parts for images, and not even the courtesy of a plain text apology, presumably because the sales and marketing people responsible imagine other people reading it in Outlook just as they do.

What the sender wants in such cases is for the mail client to pull down all those resources dynamically, thus getting the message under any payload size restrictions and also giving them the ability of seeing when you read the mail, plus what you clicked on since the images will also be hyperlinked. This lets them hypothesize/fantasize about all sorts of "insights" they might get your habits without actually having to ask you anything, although you can be sure that these organisations also like to pester you to fill out stupid questionnaires.

As for losing faith in the Web/HTML community, it wouldn't surprise me if this apparent tracking requirement weren't also part of some similar "insight"-mining process where someone at a higher management level in Mozilla is agonizing about "what our users want" while continuing to ignore all feedback that doesn't reinforce their existing notions on the topic.

"the email system is broken"

Posted Oct 12, 2017 21:45 UTC (Thu) by debacle (subscriber, #7114) [Link] (7 responses)

Sure. But isn't HTML email part of the problem, and not the solution?

Anyway, who cares? Those who prefer text (like us mutt users) can just ignore the HTML attachment.

"the email system is broken"

Posted Oct 13, 2017 1:38 UTC (Fri) by Tara_Li (guest, #26706) [Link] (3 responses)

Unfortunately, I'm seeing e-mails that I do need that are coming as HTML, with no text segment.

"the email system is broken"

Posted Oct 13, 2017 8:55 UTC (Fri) by karkhaz (subscriber, #99844) [Link]

Yes, but this is an edge case. I'm also a mutt user, and I deal with this by having a maildir just for HTML-only emails. When I receive a HTML email that I need (airplane ticket, bank statement etc) I don't even bother reading it in mutt, but just save it into the maildir. I can then browse the cur/ directory of the maildir with an actual web browser, using a browser profile that doesn't load remote images unless I ask it to. This is also great because I don't trust anybody who isn't Google or Mozilla to keep their Blink/WebKit/whatever plugins up-to-date and safe from vulnerabilities, so the only program on my system that I view HTML with is an actual browser.

This is also convenient because once you know which senders are guaranteed to send HTML-only mail, you can write a procmail rule or the like to put them straight into the maildir. (I use a hosted email with Kolab Now, so I set the rule up server-side instead). If you're paying per-gigabyte for email storage and are worried about the size of HTML email, you can even have a cronjob that moves emails out of the maildir (deleting them from the server) and into local storage somewhere, perhaps adding it to a git repo or whatever.

Double bonus: if you get a lot of HTML email from one sender, and the emails are in a fairly fixed format, then you can parse the information you need out of that email and not bother reading the email at all. For example, whenever I get a delivery to my postbox, I get an email telling me so, with tracking pixels etc. But the email also contains a QR code as an attachment that I need to open my postbox for that delivery. So again, cronjob to find new mails from the post company, save all the QR codes, upload them to a URL on my web site, and I then browse to my website when I visit the post room---no need to even open the email. That's only worthwhile if you enough email from one sender to make it worth writing a custom parsing script for them, but (a) it's nerdy and (b) by doing this for the most prolific senders, I significantly reduce the time that I need to spend dealing with content-free email.

"the email system is broken"

Posted Oct 14, 2017 21:47 UTC (Sat) by debacle (subscriber, #7114) [Link]

$ sudo apt install w3m

and then:

~/.mutt/muttrc:
auto_view text/html                                      # view html automatically
alternative_order text/plain text/enriched text/html     # save html for last
~/.mutt/mailcap:
text/html; w3m -I %{charset} -T text/html; copiousoutput;

(from: http://jasonwryan.com/blog/2012/05/12/mutt/)

"the email system is broken"

Posted Oct 16, 2017 21:36 UTC (Mon) by juliank (guest, #45896) [Link]

And I see emails with an empty text segment. That's even worse; given that I set up automatic mail rendering with w3m -dump in mutt...

"the email system is broken"

Posted Oct 15, 2017 12:07 UTC (Sun) by gdt (subscriber, #6284) [Link] (2 responses)

USASCII e-mail isn't the solution either, it simply can't represent the first-learned language of most Internet users.

"the email system is broken"

Posted Oct 15, 2017 12:49 UTC (Sun) by NAR (subscriber, #1313) [Link]


Content-Type: text/plain; charset="UTF-8"

Content-Transfer-Encoding: quoted-printable

And there's no need for HTML to represent non-ASCII characters.

"the email system is broken"

Posted Oct 26, 2017 8:49 UTC (Thu) by Wol (subscriber, #4433) [Link]

It can't even represent English properly ... :-)

Cheers,
Wol

The trouble with text-only email

Posted Oct 12, 2017 22:44 UTC (Thu) by dskoll (subscriber, #1630) [Link] (19 responses)

we have seen an increase in greylisting/blacklisting as a direct result of our text only emails

I don't believe that. It sounds like complete nonsense to me. And I work in the anti-spam industry.

Greylisting is independent of the message content, so it cannot possibly penalize text/plain any more than text/html. And I challenge Mozilla to prove that plain-text emails are a causal factor in getting them blacklisted.

Total claptrap IMO.

The trouble with text-only email

Posted Oct 12, 2017 23:28 UTC (Thu) by tialaramex (subscriber, #21167) [Link] (10 responses)

In the heads of the same sort of people who aren't sure what the difference is between "Windows" "Microsoft" and "Office" so use them all interchangeably, the word "Spam" means "Any email I didn't expect to receive".

Sending email that people don't like results in getting labelled "spam". A fraction (but not an insignificant one) of email users mark any email they didn't expect "spam". Even the fact that they've _specifically asked to receive this email_ is something they pay no heed to in adding this label, if on Monday they think to themselves "Oh, I love puppies, I want to read all about puppies" yes, yes please, sign me up to the Puppy Lover mailing list, that doesn't prevent them on Tuesday receiving an email "Dear Puppy Lover, thanks for signing up to read about puppies" and thinking "Ugh, not this" and pressing the Spam button.

If you're big, the receiving side will put up with this anyway, they'll have to accept that yes, 1% (or whatever) of all Puppy Lover emails are labelled spam and (possibly while rolling their eyes) tell customers they're working hard to eliminate spam and thanks for clicking that "Spam" button religiously.

[ This happens for other categories too. The usual thing to do with reports of "phishing" is to thank the customer for being on their guard and re-assure them you take phishing seriously. It's not important whether the email they forwarded is actually a phishing scam, or an advert your agency sent to them, or just a routine email about say, confirmation of delivery of an order they just literally made, just tell them thanks for being such a clever customer who definitely isn't an idiot that doesn't know what "phishing" is. ]

But if you're a bit smaller, what happens is you get blocked as "too spammy" where that turns out to mean "Users are incompetent but we can't be bothered to do anything about that, so instead we're throwing your email in the garbage, too bad".

I know about this because although my employer owns a huge commercial "email marketing" company which falls into that "big" category I run a service for them which mostly doesn't use that company to send email, it sends only actionable alerts to customers who've usually either paid for the privilege of getting these alerts or signed up as part of some sort of premium package (e.g. they have a Platinum payment card and this service was one of the "freebies"), and nevertheless a non-trivial fraction of the alerts are marked as "Spam" by the recipient. Using the commercial "marketing" company (ie people who actually send spam for a living) gets the mail through.

Personally I would suggest we ask the same sites that offer them this "Spam" button to collect and publish lists of these people's email addresses so we can avoid "harassing" them altogether by refusing to let them sign up for anything ever again. It's not really important to me whether they learn anything from the experience, we're all better off regardless.

The trouble with text-only email

Posted Oct 13, 2017 11:44 UTC (Fri) by dskoll (subscriber, #1630) [Link]

Yes, that's all common knowledge, but has nothing to do with greylisting nor with text/plain mail being especially penalized.

The trouble with text-only email

Posted Oct 13, 2017 15:27 UTC (Fri) by nybble41 (subscriber, #55106) [Link] (8 responses)

> ... or signed up as part of some sort of premium package (e.g. they have a Platinum payment card and this service was one of the "freebies") ...

To be perfectly frank, I would probably consider that spam as well, unless there was a simple and obvious way to opt out of this "freebie" up front. If you're signing up specifically to receive the e-mails that's one thing, but if it's just bundled with some other service (so you might want that service but not the e-mails) then that isn't substantially different from any other source of unsolicited e-mail. If I recognized the source and there was an "unsubscribe" link I would probably use it, but in the absence of such a link—or if it didn't take effect promptly, by the end of the next business day at the latest—I would have no qualms about reporting the e-mails as spam. If the source isn't recognized, of course, then one must assume that any links could be an attempt at phishing, so that isn't an option.

If a company doesn't want me to consider their message spam: 1) Don't send anything without asking first. I don't care whether it's opt-in or opt-out as long as the option is clearly presented. (One-time "welcome" e-mails can be an exception, as can critical security messages on the order of "your account has been compromised".) 2) Provide an option to disable e-mails separate from other services, if any. If you send different categories of e-mails, e.g. relevant account notifications and general product updates, provide separate settings for each type. 3) Provide a functional "unsubscribe" link. "Please allow up to two weeks for the change to take effect" is not good enough. Unsubscribing should be an automated process. Click the link, maybe click another button to confirm that's what you want, and receive no further e-mails—allowing for ones already in transit, of course.

The trouble with text-only email

Posted Oct 13, 2017 19:04 UTC (Fri) by rgmoore (✭ supporter ✭, #75) [Link] (4 responses)

I would agree with essentially everything here, except that I would not consider "opt-out" OK. Users should have to specifically request everything except essential communication like receipts, shipping notices, bills, and security alerts. It can be as simple as including a check box, but if that's the way you do it the box should default to not checked. Users should have to take active steps before they start receiving email. The only time a user should have to take any step to stop receiving email is when they've previously taken an active step to receive it and they've changed their mind.

The trouble with text-only email

Posted Oct 13, 2017 19:43 UTC (Fri) by karkhaz (subscriber, #99844) [Link] (3 responses)

Note that in the EU, opt-out is prohibited by law. This isn't the case in the US, where it's legal to start spamming your users without asking, but in both the EU and USA it is against the law to not provide an "unsubscribe" link in every email.

This doesn't stop plenty of companies (presumably based in the US) from not complying with EU regulations though, I assume somebody would have to take them to court in the EU to enforce that they offer their user to opt-in to email.

The trouble with text-only email

Posted Oct 14, 2017 10:44 UTC (Sat) by Jonno (guest, #49613) [Link] (2 responses)

> Note that in the EU, opt-out is prohibited by law.

Well, only sort of. While a customer *technically* needs to opt-in before you can spam them, it is perfectly legal to make opt-in the default choice.

(Both a checked-by-default "I want to subscribe" checkbox and an unchecked-by-default "I don't want to subscribe" checkbox is fine, but only providing an after-the-fact "click here to unsubscribe" link is not).

The trouble with text-only email

Posted Oct 18, 2017 11:01 UTC (Wed) by hkario (subscriber, #94864) [Link] (1 responses)

point is, there needs to be a checkbox, and having subscribed to half a dozen web stores last month, it really is easy to remember to keep them unchecked

definitely much easier than reading T&C of any website

The trouble with text-only email

Posted Oct 19, 2017 9:29 UTC (Thu) by rmv (subscriber, #48328) [Link]

Ah, so you haven't come across the random mix of opt-out and opt-in checkboxes? So if you either tick or untick all of them blindly, you're guaranteed to have signed up to something.

The trouble with text-only email

Posted Oct 13, 2017 19:10 UTC (Fri) by tialaramex (subscriber, #21167) [Link]

The customer will have signed up as in, they had to fill out all the stuff needed to make it work. Typically they've filled out payment card details, home address, email addresses, phone numbers, and maybe also stuff like passport numbers and whatever type of identity card people have in the locale the product they purchased is for (e.g. SSNs in the US)

I'm sure a tremendous number of people who have such a benefit available to them never sign up, just as when insurers hand over tens of thousands of codes to victims of a data breach that entitle the recipient to a free account almost none of those are ever used. Since the alerts are based on finding their data, if they haven't told us any data they don't get any alerts. But those who do are still depressingly likely to label the actual alerts spam.

In some cases the company white labelling the service asks for non-alert emails, those all go out through that "marketing" service I talked about, and doubtless any number of sensible customers label _those_ as spam, which has no effect because they're a big successful marketing company.

The trouble with text-only email

Posted Oct 19, 2017 7:44 UTC (Thu) by nim-nim (subscriber, #34454) [Link] (1 responses)

Even one-time welcome mails are not ok, since many marketing departments consider it fine to reset their system as soon as enough people have unsubscribed, and resend "welcome" messages (usually one-line welcome, three screens of marketing materials). Or use one-message mailings, you can unsubscribe all your want, you've already received the only message associated with the mailing and the next one will use another list (and the unsubscribing marks an active e-mail that can be sold for profit for another spamming round).

If this scum was not systematically using deceptive practices they could complain users are tagging legitimate messages as spam. As it is the tagging reflects what the customer actually wants, not what he's been tricked into.

The trouble with text-only email

Posted Oct 19, 2017 15:50 UTC (Thu) by nybble41 (subscriber, #55106) [Link]

> Even one-time welcome mails are not ok, since many marketing departments consider it fine to reset their system as soon as enough people have unsubscribed, and resend "welcome" messages (usually one-line welcome, three screens of marketing materials).

Such messages would not be "one-time welcome mails", as they are neither one-time nor plausibly intended to welcome a new user. The fact that they reset their system has no bearing whatsoever on whether the messages are spam.

> Or use one-message mailings, you can unsubscribe all your want, you've already received the only message associated with the mailing and the next one will use another list

The problem here is that the recipient never signed up for those one-message lists in the first place. "Don't send anything without asking first." If you're sending mail to people who didn't sign up, whether it's one message or a hundred, you're spamming.

The trouble with text-only email

Posted Oct 13, 2017 1:07 UTC (Fri) by wahern (subscriber, #37304) [Link] (1 responses)

> Greylisting is independent of the message content

Most implementations temporarily reject receipt before DATA, but there's nothing inherent in this approach. (See https://tools.ietf.org/html/rfc6647#section-2.5) You might want to process the message so you can apply whitelisting rules, including DKIM exemptions. Too many people disable greylisting because of the delay. If you can hasten delivery of desirable e-mail, fewer companies will disable greylisting.

Maybe it's not common because of weird bugs in Exchange that break redelivery if you reject on EOM. But IME the reason it's not more common is because the server-side frameworks weren't performant enough to handle many late rejections. In particular, it's difficult to design (and keep other engineers from breaking!) a filtering engine that supports inline, streaming processing with the equivalent of "redirect to /dev/null and reject with 451" short-circuiting. Most engines seem to quickly degenerate to slurping in the entire message body into a string in someone's favorite scripting language, which will never scale if you're not rejecting most messages upfront.

The trouble with text-only email

Posted Oct 13, 2017 11:43 UTC (Fri) by dskoll (subscriber, #1630) [Link]

Yes, our implementation actually does greylist post-DATA and has various mitigations to reduce greylist delay. But it still doesn't distinguish between text/plain and text/html

The trouble with text-only email

Posted Oct 13, 2017 4:38 UTC (Fri) by fratti (subscriber, #105722) [Link] (3 responses)

In addition to this, it comes at a peculiar time when Mozilla is starting to put spyware extensions into random Firefox downloads, and Google Analytics tracking into about: pages.

I feel like someone at the top completely misunderstands Mozilla's mission.

The trouble with text-only email

Posted Oct 16, 2017 8:33 UTC (Mon) by jezuch (subscriber, #52988) [Link]

It's my impression as well. It's as if they hired someone to do this job and they started applying "industry best practices" without reflection.

The trouble with text-only email

Posted Oct 22, 2017 10:57 UTC (Sun) by gezza (subscriber, #40700) [Link] (1 responses)

Do you have a link for those two comments?
about: triggering google analytics is fairly concerning.
Thanks.

The trouble with text-only email

Posted Oct 27, 2017 4:38 UTC (Fri) by Mook (subscriber, #71173) [Link]

I'm not the person you're replying to, and I don't have a page for this (just tried things out right now), so I'll have to go with instructions:

Downloaded new Firefox, from https://download.mozilla.org/?product=firefox-latest&os=linux64&lang=en-US
Started Firefox with a new profile, firefox -no-remote -profile $(mktemp -d) -offline about:blank
At this point you should be seeing a window with a blank tab (selected), and a background tab with network errors (because you're offline)
Tools → Web Developer → Web Console, switch to the Network tab
File → (uncheck) Work Offline
Tools → Add-ons (which loads about:addons)

That last step loads a page off discovery.addons.mozilla.org, which appears to include Google Analytics.

The trouble with text-only email

Posted Oct 13, 2017 5:14 UTC (Fri) by rsidd (guest, #2582) [Link]

My understanding of what they are saying is different. If they routinely send mail to defunct addresses (that don't bounce), they are classified as possible spammers. (Do organizations retain defunct addresses with the hope of identifying spammers this way? Weird.) With plain-text mail there is no way for the sender to know whether their mails are being read or not. HTML mail is an attempt to fix this problem and weed out defunct addresses.

I don't read it as saying plain text mail is flagged as spam!

The trouble with text-only email

Posted Oct 13, 2017 20:59 UTC (Fri) by Sesse (subscriber, #53779) [Link]

> Greylisting is independent of the message content

Nothing is saying you can't greylist by sending 4xx on the DATA command (after seeing the content and deeming it dubious).

Dormant accounts

Posted Oct 12, 2017 22:48 UTC (Thu) by dskoll (subscriber, #1630) [Link]

Second point: Yes, some people will use honeypot addresses to trap spammers. But I find it incredibly difficult to believe that anyone would use an email account as a honeypot that was formerly an active, legitimate account that actually signed up for a mailing list.

The practice of sending a "Click here if you still want to subscribe" link periodically is all that's necessary to prune the list of people who don't read it.

The trouble with text-only email

Posted Oct 13, 2017 0:01 UTC (Fri) by rgmoore (✭ supporter ✭, #75) [Link]

The fact that even principled organizations like Mozilla feel the need to employ tracking says something discouraging about the state of email, though, not that there was really any need for more evidence that the email system is broken.

This says a lot more about contemporary internet mentality than anything. They want to present this as a way of making life easier for people, but it's as much about their own convenience as their users. People have worked so hard to destroy privacy that it's easier for a company to achieve their goal by ignoring their users' privacy than by protecting it, and even organizations that are nominally interested in privacy will often ignore it when privacy is even a minor inconvenience to their goals.

The trouble with text-only email

Posted Oct 13, 2017 2:31 UTC (Fri) by fredex (subscriber, #11727) [Link] (1 responses)

I've always been of the persuasion that "When God invented email, He intended it to be plain text."
;-)

The trouble with text-only email

Posted Oct 13, 2017 7:27 UTC (Fri) by FLHerne (guest, #105373) [Link]

This is nonsense. Shiva Ayyadurai invented EMAIL! (and will sue you for suggesting he didn't).

The trouble with text-only email

Posted Oct 13, 2017 4:03 UTC (Fri) by marcH (subscriber, #57642) [Link] (16 responses)

> It's not clear what the solution to the email problem is, but the need is obvious.

At least for the very specific case of *one-way newsletters* the solution is obvious: any "pull" system (RSS, read-only NNTP,...) is better. It's very simple really: "don't call us, we'll call you" and *poof*, all problems like spam, filtering, subscription, etc. are gone instantly.
Of course these are not mutually exclusive: you can perfectly keep email on life support in parallel with another, actually working solution.

Some would argue that the same "pull" superiority applies to any non-private communication, but... let's not go there https://lwn.net/Articles/702177/ :-)

The trouble with text-only email

Posted Oct 13, 2017 5:01 UTC (Fri) by neilbrown (subscriber, #359) [Link] (15 responses)

> the solution is obvious: any "pull" system

Yes, if only we had a 'pull' system as good as email..

I experimented with using rss2email to read lwn comments in my mail reader a while back.
One problem was that all threading was lost. I thought I should get grumpy at the grumpy editor, but on looking into the details, I discovered that rss had no way to export "in-reply-to" or "references". So I had to keep my gumpyness to myself.

Maybe we could use anonymous-POP as a pull system which supports everything the email supports.
i.e. I connect to the pop server at lwn.net, login as guest/guest, use UIDL to list the uids of the 100 most recent comments, and collect the ones that I haven't seen....

The trouble with text-only email

Posted Oct 13, 2017 6:01 UTC (Fri) by johill (subscriber, #25196) [Link] (13 responses)

Did you just reinvent NNTP?

The trouble with text-only email

Posted Oct 14, 2017 7:00 UTC (Sat) by marcH (subscriber, #57642) [Link] (12 responses)

I really don't understand why NNTP seems to get so little love. Even with that little it still rocks anyway. When gmane.org almost died I thought I would die with it.
Imagine how much more NNTP would rock if it received just 1/10th of the massive (https://lwn.net/Articles/732570/) efforts trying (and failing) to fix email's unfixable design flaws.
It shouldn't even be that much effort considering NNTP is literally email's cousin, sharing so many genes and a lot of compatibility.

(on NNTP)

Posted Oct 17, 2017 12:15 UTC (Tue) by Herve5 (guest, #115399) [Link] (5 responses)

I thought NNTP disappeared when Google bought Déjà News?

I had been an intense user of NNTP, with client softwares that were able to show the discussion trees, change news servers on the fly etc.
And for me all of this disappeared, almost over one single year, once Google imposed logging in specificaly for each dedicated newsgroup, etc.

I remember seeing my favorite discussion groups, be it philosophy or specific sharewares (do this word still exist, 'shareware'?) -my favorite groups *voluntarily* moving to Google groups...

Please tell me I am wrong, but I hardly remember even the name of my softwares -ah yes, MacSOUP on macintosh at the time... but that was when I was young...

But if of all places there is one that could resurrect it, it's LWN!

(on NNTP)

Posted Oct 17, 2017 14:45 UTC (Tue) by sfeam (subscriber, #2841) [Link] (1 responses)

eternal-september.org

(on NNTP)

Posted Oct 18, 2017 12:30 UTC (Wed) by Herve5 (guest, #115399) [Link]

Thank you very much sfeam!
(but my worry also was about all those guys that voluntarily migrated to Google...
Still I find worth re-checking what is left today on NNTP!

(on NNTP)

Posted Oct 18, 2017 11:44 UTC (Wed) by anselm (subscriber, #2796) [Link] (2 responses)

I thought NNTP disappeared when Google bought Déjà News?

NNTP disappeared when people figured out that it is easier to put up a web forum or blog than to get a new USENET newsgroup approved.

(on NNTP)

Posted Oct 18, 2017 12:33 UTC (Wed) by Herve5 (guest, #115399) [Link] (1 responses)

Well, easier, with a different GUI and separate credentials for each -until Google ate them all...

(on NNTP)

Posted Oct 18, 2017 14:57 UTC (Wed) by anselm (subscriber, #2796) [Link]

Easier from the POV of the forum/blog operators, not necessarily that of arbitrary users.

network news transfer protocol

Posted Oct 26, 2017 6:15 UTC (Thu) by Garak (guest, #99377) [Link] (5 responses)

I really don't understand why NNTP seems to get so little love.

I think I understand it. Several big factors. One big factor is the difficulty of operating an NNTP server. Sure, it's not that hard to pony up for a linode or aws vhost. But just imagine if that hurdle was removed, and instead every ISP woke up tomorrow and removed the annoying server prohibition clauses from their terms of service. If that happened, I think we'd see a rapid innovation in nntp server appliances that many home users would dabble in operating. I.e. Razpi+usbdisk+enablingarouterportforward. Now shortly after gaining more traction, we'd start to encounter and resolve NNTP's problems. Whitelists and simple reputation systems would do the trick without much work (from the perspective of a community of hundreds of reasonably adept home server operators working in traditional FOSS ways). Or cryptocurrency like solutions (I recall before bitcoin, several basic ideas of cryptocurrency were kicked around as a spam-paid-postage solution, probably in a dusty LWN/Schneier article).

The next big factor is how the kinds of idealized communications systems like this are opposed by large forces. 'Streaming' non-live content would never have been a thing if there wasn't effectively a war on S/FTP by big copyright moneyrakers. Napster should never have been a thing in a world where s/ftp and nntp exist in foss ways for everybody. But that didn't happen because the home-server persecution campaign was successful. Likewise the NSA/FBI don't like secure communication systems, which is what s/ftp+nntp would have turned into. Instead we got facebook and googleplus which NSA's PRISMs cut like a hot knife through butter. Seriously, Facebook only exists because of home server persecution IMHO. And it's a weird thing, to get away with it, they had to obfuscate it. And it seems to have worked. The original promise of the internet as a balanced truly two-way street, where everyone could easily be a web server, as well as a web browser, was more than the establishment could allow to disrupt.

In other words- I Love the Network News Transfer Protocol. I fear that government sponsored propaganda campaigns effectively trolled usenet to death with racism provocation tactics (obviously I venture this far unreservedly after the recent issues with that and Russia/US/2016election). In my analysis, the way that FOSS/NNTP could have survived that onslaught was if everyone could run an nntp server at home on Network Neutrality non-discrimination terms. For a long time, the linode/aws workaround was simply too high a bar, preventing the needed critical mass of operator/developers. Even now, ethical people read the linode/aws ToS and see the bigger picture and history, and decide it's not worth the headache. The solution I see is to actually tackle the real problem of defining how Free Speech exists on the internet. By my math, it requires the ability to operate a server, and thus be able to have any kind of conversation you want with anybody, without considering whether it may cause you to be illegally violating a contractual terms of service. And at the end of the day, the unethical people don't care about that, and have all the darknet communication they want already. It's akin to the old- If you outlaw guns(home servers) then only outlaws with have guns(home servers and the communcation security advantages they enable).

$0.02...

network news transfer protocol

Posted Oct 26, 2017 7:02 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link] (4 responses)

> But just imagine if that hurdle was removed, and instead every ISP woke up tomorrow and removed the annoying server prohibition clauses from their terms of service. If that happened, I think we'd see a rapid innovation in nntp server appliances that many home users would dabble in operating.
Nothing will happen. There are no "no-server" clauses in ISP contracts in Russia or Ukraine. Yet the density of mail and NNTP servers is not greater than in the US.

network news transfer protocol

Posted Oct 26, 2017 22:03 UTC (Thu) by Garak (guest, #99377) [Link] (3 responses)

That's your opinion/analysis. My own includes perhaps more focus on the how and why earlier internet innovation came more from the U.S. than other countries. While I certainly am cogniscent of the myriad of ways the U.S. governmental system has been atrocious in recent years and not so recent decades, I do think that when it comes to the nexus Free Speech and capitalism accelerated consumer goods innovation and availability, there is some valuable difference that explains the U.S.-centric development of tech.

In Russia, there could be a full page advertisement every week in newspapers with a personal message signed by Putin telling the public how free they are to engage in Free Speech even harshly critical of his regime. But that doesn't mean I think it would be a wise idea to believe that and act on that belief. I.e. read the wikipedia page on journalist murders in Russia over the past couple decades, and various government's beliefs about those cases, and worse, U.S. policies that seemed to both see a problem there, but not discuss it much.

I really think the path of nntp server appliance innovation would be wholly different in the U.S. OTOH, one of the saving graces of my analysis is that I believe this is an opportunity for nations other than the U.S. (but probably not Russia) to outshine the U.S. in future internet communications innovation. And while as a U.S. citizen I feel bad about that, I recognize all the blood on our hands and see it as a just desert that our own corruptness may cause history to look back on this failing of ours, with a brighter view of the countries that managed not to squander the opportunity.

Just my opinion/analysis/$0.02...

network news transfer protocol

Posted Oct 26, 2017 22:08 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link] (2 responses)

No it won't. You simply have an unhealthy obsession with SMTP (and NNTP).

NNTP died not because ISPs are cracking down on users the moment they start a listening socket on port 119. It died because the whole NNTP infrastructure requires a lot of attention and maintenance. And nobody cares about it, since simple web forums are MORE functional for an average user.

And there actually is a fair number of self-hosted blogs/forums.

network news transfer protocol

Posted Oct 26, 2017 23:49 UTC (Thu) by rodgerd (guest, #58896) [Link]

Even more important, in my opinion, is the ability of forums to exercise some sort of control over their own content without requiring (easily bypassed) per-post moderation.

network news transfer protocol

Posted Oct 29, 2017 3:40 UTC (Sun) by Garak (guest, #99377) [Link]

NNTP died not because ISPs are cracking down on users the moment they start a listening socket on port 119. It died because the whole NNTP infrastructure requires a lot of attention and maintenance. And nobody cares about it, since simple web forums are MORE functional for an average user.
And there actually is a fair number of self-hosted blogs/forums.

I didn't say an ISP crackdown was the cause of NNTP's death. I again highlight the aspect of my analysis that you seem indifferent to. Things didn't come to where they are all at one, it was a remarkably fascinating evolution of computer network communication tool development over the last several decades. My theory is that what NNTP required to be successful was *development*, because indeed, as it was/is, the attention and maintenance requirements were/are too high for it to succeed in competition with a multitude of smaller scale web forum platforms. My theory is that if there was a FreeSpeech/NetworkNeutrality protection that allowed anyone to compete and test further development without paying a premium to aws/linode or their own ISP for 'business class' service that allows the operation of an NNTP server, then we would see the *development* increase to a point where NNTP could outcompete the myriad of alternatives that have outcompeted it in the last decade or two.

At the end of the day, I see significant value in what usenet achieved for end-user experience in some important ways. An analogy would be the difference between the historical ubiquitous cable television channel line-ups, versus a different front-end app for netflix/hulu/disney/cbs/nbc/etc. We needn't explore all the nuance in this thread, but there was something pretty amazing about the breadth of channels all in one place, with users having full control over the development of the front-end. I think if users could have full control over the back-end as well, development would reach a critical mass point where NNTP could easily once again outshine the currently more popular alternatives.

And I have no particular attachment to NNTP(and I'll admit, you were right to throw SMTP alongside it) as it was or is. My attachment is to what it basically accomplished. I think there are important facets of what SMTP and NNTP accomplished, that as they fade away, would be a shame to lose. I think ordinary users with FOSS techniques could easily resolve the serious deficiencies that existed and still exist. But a big problem is that there are big money interests, Google with GMail and googlegroups, and Reddit amongst others that depend on the high bar to ordinary people being able to operate group/email back-end servers. I think that Google's Network Neutrality stance is entirely corrupted by their monetary interest in how things evolved. And I think that is having very negative effects on how things continue to evolve.

The trouble with text-only email

Posted Oct 13, 2017 16:17 UTC (Fri) by perennialmind (guest, #45817) [Link]

A "pull" system can still be email. There was a time when the mailing lists for the Cyrus IMAP server were accessible over anonymous IMAP. <quick search> Huh. It looks like they still are. That's not the use case I expect mail a network mail store to optimize for, but is it really any stranger than publishing all the mail over HTTP? The more I think about it, the more natural it seems. As a nice side-effect, it would dodge some of the problems sender authentication strategies like SPF have with forwarding.

The trouble with text-only email

Posted Oct 13, 2017 8:59 UTC (Fri) by mjthayer (guest, #39183) [Link] (4 responses)

The idea of making people actively renew mailing list subscriptions once a year rather than making them actively cancel them sounds quite nice anyway.

The trouble with text-only email

Posted Oct 13, 2017 11:47 UTC (Fri) by dskoll (subscriber, #1630) [Link]

I agree. I run a few mailing lists and I might look at implementing this.

The trouble with text-only email

Posted Oct 13, 2017 14:20 UTC (Fri) by cesarb (subscriber, #6266) [Link] (2 responses)

The only time I've ever seen that happen was last week, when the Linux Foundation newsletter sent me a "Confirm Your Subscription" link (with the implied threat that, if I didn't follow the link, I'd be unsubscribed from the newsletter).

I'm not sure I like that idea much. It means that, if you miss a single email (or read it too late), your mailing list subscription can be lost.

The trouble with text-only email

Posted Oct 14, 2017 6:48 UTC (Sat) by marcH (subscriber, #57642) [Link] (1 responses)

Easy: ask for confirmation 3 or 4 times in row before auto-unsubscribing.

The trouble with text-only email

Posted Oct 15, 2017 19:18 UTC (Sun) by dd9jn (✭ supporter ✭, #4459) [Link]

... and modify the subject of all messages delivered to that address with something like "[confirmation required]" so that it will even be noticed when not reading the mails.

The trouble with text-only email

Posted Oct 13, 2017 20:08 UTC (Fri) by flussence (guest, #85566) [Link] (3 responses)

>One metric that some sites evidently use is email to sent accounts that are known to be inactive, which is seen as a sign of a spammy originator. This, seemingly, is where Mozilla has run into trouble. One way to avoid this problem is to track which recipients are actually reading their email; any recipient who doesn't look at any messages for a period of time can then be unsubscribed.
The polite and correct way is to ask the user on the other end to refresh their opt-in status every few months.

The last correspondence I allowed Mozilla to send me was a horribly bungled unsolicited marketing newsletter from the Spread Firefox website, sent from a third party server I'd never heard of, with an invalid return address and a gibberish text/plain part which screamed “phish attempt”. As I recall there was quite a bit of outrage about that and they were every bit as deflective and dismissive back then as they are now. They need to get off their high horse.

The trouble with text-only email

Posted Oct 14, 2017 0:23 UTC (Sat) by roc (subscriber, #30627) [Link] (2 responses)

Did you read the article through to the part where it says Mozilla listened to users and is implementing exactly the compromise you suggested?

The trouble with text-only email

Posted Oct 14, 2017 19:23 UTC (Sat) by flussence (guest, #85566) [Link] (1 responses)

Your words say one thing, but this unsolicited spam I received for Pocket a few minutes ago without a one-click unsubscribe says something completely different.

The trouble with text-only email

Posted Oct 14, 2017 19:28 UTC (Sat) by flussence (guest, #85566) [Link]

My mistake, after fighting through login screens I've discovered it has *no* unsubscribe. Disgusting.

The trouble with text-only email

Posted Oct 15, 2017 21:05 UTC (Sun) by mlinksva (subscriber, #38268) [Link] (3 responses)

> When people open an email, click on links, don’t mark us as spam, filter to the trash etc [...] Unfortunately text emails don’t have the same feedback loops available as HTML.

I don't understand. Are there some feedback loops not mentioned that text emails do not obtain but HTML emails do?

The trouble with text-only email

Posted Oct 16, 2017 19:25 UTC (Mon) by eythian (subscriber, #86862) [Link] (2 responses)

Yes, embedded images to let the sender know the message has been opened. Not totally reliable, but good enough apparently.

The trouble with text-only email

Posted Oct 19, 2017 2:35 UTC (Thu) by mlinksva (subscriber, #38268) [Link] (1 responses)

> these actions all contribute to a positive “reputation score” on those mail services

Mail services, not the sender.

I still don't understand the argument.

The trouble with text-only email

Posted Oct 21, 2017 16:50 UTC (Sat) by runekock (subscriber, #50229) [Link]

Read the LWN text, not the Mozilla quote, then it should make sense:

By tracking, the sender gets better at only sending to people who actually open the mail. And mail services can use that as a factor in their spam filter.

The trouble with text-only email

Posted Oct 19, 2017 14:11 UTC (Thu) by mirabilos (subscriber, #84359) [Link]

My mailer just considers HTML mail to be spam.

The trouble with text-only email

Posted Oct 19, 2017 14:23 UTC (Thu) by kpfleming (subscriber, #23250) [Link] (4 responses)

I'm really curious how an IPv6 address being considered a 'spam source' could block large parts of the address space. This is only possible if the system making this determination is recording the address with a very small mask (something far smaller than the /64 it should be using). Any examples of this actually happening?

IPv6

Posted Oct 19, 2017 14:45 UTC (Thu) by corbet (editor, #1) [Link] (1 responses)

Spamhaus routinely blacklists big chunks of the space, so that we were often getting nailed by a block on somebody else's address. So we finally just gave up on IPv6 delivery, which is really kind of sad.

IPv6

Posted Oct 19, 2017 14:53 UTC (Thu) by kpfleming (subscriber, #23250) [Link]

Sad indeed. Definitely not #winning.

The trouble with text-only email

Posted Oct 19, 2017 17:24 UTC (Thu) by pizza (subscriber, #46) [Link] (1 responses)

IPv6 /64s are handed out like candy. So if one address gets blocked, all you have to do is increment by one, and resume spamming.

So unfortunately it makes sense to just block a large address subset.

(I saw something similar with IPv4 and /24s; even to this day my IDS logs show that more than one IP in a subnet is typically used at once..)

The trouble with text-only email

Posted Oct 19, 2017 18:34 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]

Uhm, /64 in IPv6 is pretty much a single LAN. So you blacklist the whole /64 prefix.

An ISP should give you a /48 network at least, not a single address from their /64.

The trouble with text-only email

Posted Oct 28, 2017 19:29 UTC (Sat) by whyagaintang (guest, #97642) [Link]

I am surprised that nobody cited this recent study from https://freedom-to-tinker.com/2017/09/28/i-never-signed-u...

Quote from the paper:
Mail clients that block cookies by default, like Apple Mail, offer some level of protection. In these clients it’s more difficult for a tracker to track users across mailing lists, since the mail client doesn’t provide a persistent identifier. The same is true for webmail clients which proxy images, like Gmail and Yandex. Content proxying has the added benefit of preventing a tracker from being able to link the browser’s cookies to any identifiers received when an email is opened