LWN: Comments on "The perils of federated protocols"

The perils of federated protocols

jond — Wed, 06 Feb 2019 09:18:12 +0000

Is that figure the spam you receive after filtering, or before?

Phone numbers as identifiers

zdzichu — Fri, 12 Oct 2018 05:52:15 +0000

After writing a text, hold the "Send" icon. Menu for selecting SIM to use should appear.
One of the worst, non-discoverable interfaces. :(

Phone numbers as identifiers

tonyblackwell — Fri, 12 Oct 2018 00:28:50 +0000

... and I can't tell Signal to use a specific one of my dual-sim numbers. It seems welded to my USA SIM, despite living in Australia, defaults to +1 so any routine SMS disappear into USA blue null rather than being delivered in my country (fixable if I remember to +61 prefix all numbers, but this is tedious). Surely the fix for this would be trivial on Signal's part...

Chicken or egg

CBiX — Tue, 07 Feb 2017 01:58:12 +0000

Obviously it's an interdependent relationship: internet technology is only evolving at such an enormous speed because of its decentralized infrastructure and the fact that everyone can equally contribute content, ideas and new systems.

The perils of federated protocols

PaulWay — Mon, 30 May 2016 01:18:40 +0000

Does anyone else read "freedom to innovate" as "inability to forward plan"?

I agree with other analysis here: the examples Moxie uses for "federated" protocols failing are not only inaccurate, they're just wrong. And most of his examples are examples of people making simple assumptions that later turned out to be wrong. IPv4 has failed because it assumed that there would never be more than 2^31 (or so) hosts on the internet. HTTP 1.0 failed because it assumed that each session would make one request and terminate and that session setup (TCP and SSL) would be cheap. Etc. etc. That was partially addressed in HTTP 1.1, but there are lots of things that HTTP 2.0 does that simply can't fit into HTTP 1.1 and require a new negotiation system. Fundamentally all of these are problems with backward and forward compatibility.

So to me part of Signal's problem with using federation is caused simply by them not thinking in the long term about what they would need and making sure that things would be forward and backward compatible. You don't have to worry about compatibility if you just force your client and your server to always use matching protocols, but it means that you and no-one else can be compatible with you. And that's exactly what Moxie argues for.

Have fun,

Paul

The perils of federated protocols

Fowl — Mon, 30 May 2016 01:17:36 +0000

WhatsApp spam exists. (I've received it)

The perils of federated protocols

HelloWorld — Sun, 29 May 2016 23:47:19 +0000

So how come I *never* get any spam on WhatsApp and dozens to hundreds of spam emails every day?

Long Live The Prosperous Federation, So Say We All!!!

pizza — Thu, 26 May 2016 11:13:50 +0000

> Google and other large 'cloud' players understand well that their 'cloud' profits are threatened by decentralized and open alternatives (in an environment where network neutrality ensures the ability to operate a server at home without getting written permission from a 'gatekeeper' (aka ISP)).

I don't see this at all. There is real value in a third party providing services for folks who can't be bothered to do it themselves -- and I say this as someone who chooses to run his own.

The home-server-persecution bit is largely ISP driven because it breaks their asymmetric download-heavyish models they've based their pricing on. That, and the sad fact that most home systems' "servers" are really just spam bots and sources of various forms of malware.

The perils of federated protocols

madhatter — Thu, 26 May 2016 08:24:30 +0000

We could certainly have a discussion about the pros and cons of localised-lightweight vs centralised-heavyweight source control, and it might even be interesting, but here is probably not the right place to do it. If you think that my choice of source-control applications has any bearing on my underlying argument, please feel free to argue your case. Do, however, bear in mind Pirsig's dictum that "the world's biggest fool can say the sun is shining, but that doesn't make it dark out".

The perils of federated protocols

chojrak11 — Thu, 26 May 2016 08:18:02 +0000

> my memory is that it took me less than a decade to type and commit the above change.
Said last person in the world still using RCS routinely...

The perils of federated protocols

madhatter — Thu, 26 May 2016 06:35:24 +0000

I agree that code that's baked into firmware (especially on platforms that are infrequently upgraded by the manufacturer) is much harder to upgrade than code that's deployed in software, but that's equally true on client (eg, think weird android-esque tablets) as on server platforms. That's a pretty good argument for using general-purpose computing platforms for as much as possible, but I'm not sure it argues specifically against servers.

The perils of federated protocols

Cyberax — Thu, 26 May 2016 06:02:16 +0000

> I'd have the check my daybook to be sure, but my memory is that it took me less than a decade to type and commit the above change.
Duh. You probably don't have an expensive middlebox in front of your server doing load balancing.

The perils of federated protocols

madhatter — Thu, 26 May 2016 05:58:40 +0000

> Clients _can_ remove support for _some_ bad fallbacks, after years of gradual deployment. Servers are usually stuck pretty
> much for a decade (e.g.: SSLv3 deprecation).

# rcsdiff -r1.45 /etc/httpd/conf/httpd.conf
1113a1114
> SSLProtocol All -SSLv2 -SSLv3

I'd have the check my daybook to be sure, but my memory is that it took me less than a decade to type and commit the above change.

Long Live The Prosperous Federation, So Say We All!!!

Garak — Thu, 26 May 2016 03:33:36 +0000

For some reason the Florence song "Ship To Wreck" comes to mind. Or the old phrase "embrace, extend, extinguish". I of course (if you know my writings) believe it is a bizarre socio-stupidity thing. There are plenty of those all around us. Specifically, with the FCC's 10-201 original net neutrality, they promulgated the fantasy that the internet was a place of true free speech empowerment, where basic communication from each node was possible, regardless of the local network's preference for types of content or application or device. Google and other large 'cloud' players understand well that their 'cloud' profits are threatened by decentralized and open alternatives (in an environment where network neutrality ensures the ability to operate a server at home without getting written permission from a 'gatekeeper' (aka ISP)). I think this has resulted in some bizarre home-server-persecution propaganda campaign that ties in with Hillary Clinton and Edward Snowden and official government coverups. I know, I'll pretend to put on a tinfoil hat and go away now. Enjoy your taboos, Good Luck.

The perils of federated protocols

nybble41 — Wed, 25 May 2016 01:24:05 +0000

>> things like Xeon Phi, it's used on your mobile phone
> I don't think my phone has a 300W processor

Indeed, a mobile phone with a 300W Xeon Phi processor would last about five seconds before either draining the battery or setting itself on fire, whichever comes first. Possibly both.

There should have been a closing parenthesis after "Phi". That was meant to be read as "GPGPU is used on your mobile phone".

The perils of federated protocols

flussence — Tue, 24 May 2016 22:43:42 +0000

> Define “usable”, please.
In the way VDPAU is today. Able to do more than run the demo/test code shipped with Mesa. Reducing power consumption by offloading work to an appropriate device instead of increasing it by being dead weight to compile.

> things like Xeon Phi, it's used on your mobile phone
I don't think my phone has a 300W processor (it seems to cope with image/photo editing fine regardless). Did you mean to say Someone Else's Computers? Those kind of services are best enjoyed as schadenfreude.

The perils of federated protocols

mirabilos — Tue, 24 May 2016 15:44:12 +0000

Well, maybe Marlinspike should consider just not moving so fast. There’s something to be said for stability. (Cue Debian.)

On the other hand, I do strongly agree that the extensibility of XMPP basically killed off Jabber. I’m back to just IRC except for a (non-federated) work-internal server, and occasionally (say twice a month) firing up a Jabber client to get a message through (though I mostly eMail those people instead).

The perils of federated protocols

micka — Tue, 24 May 2016 07:35:45 +0000

I wonder how you define the federatesd/centralized aspect when writing about graphical API that are purely local. Unless you're thinking about something like rendering distributed on multiple computers? I don't think metal/vulkan does that.

The perils of federated protocols

pizza — Mon, 23 May 2016 15:52:31 +0000

> Define “usable”, please. All the HPC is built today around GPGPU and similar architectures (things like Xeon Phi, it's used on your mobile phone (to process photos in real-time and do other compute-intensive things) and so on.

I think it's fair to say that GPGPU is only now becoming "usable" without relying on (highly) proprietary software stacks.

The perils of federated protocols

khim — Mon, 23 May 2016 14:16:32 +0000

Ten years later, still no sign of it ever becoming usable.

Define “usable”, please. All the HPC is built today around GPGPU and similar architectures (things like Xeon Phi, it's used on your mobile phone (to process photos in real-time and do other compute-intensive things) and so on.

The fact that Linux distros (and desktop in general) are no longer in the center of this development is unfortunate (especially since lots of development is based on Linux), but it does not mean that all that development have just stopped and disappeared.

One day Vulkan might work, but I'm not holding my breath for it.

Vulkan works today (although not that many apps use it), Metal is used by real apps, too (look here - there are links to appstore where they could be found). Sure, you couldn't use it on your device if your insist on 100% OS but most users out there don't care and use it just fine.

And while WebGL is usable today don't forget that we are talking about technology which is almost two decades old (DirectX was released in 1996 and OpenGL is even older then that).

Sure, you could counter that other unfederated APIs (things like Glade, e.g.) have died—but there are emulators and people still play these games.

My point was that most platforms had 3D by the end of XX century while Web needed another decade before it got it and, ironically enough, exactly when web finally, finally, arrived on that decade-old platform the rest of the world already moved and shifted to significantly different 3D API! When would web have things like Metal, Renderscript or Metal? My guess is: most likely answer is “never”—and if Android will, indeed, arrive on desktop even WebGL and WebRTC could become unavailable eventually (since people would use native apps instead of webapps for things like videocalls or maps)—although that is not guaranteed, freeze a-la SMTP is more likely.

The perils of federated protocols

tincho — Sun, 22 May 2016 23:24:45 +0000

Re-posting from a comment I left on mjg59's blog.

Two years ago, after a day of talks at FOSDEM, some friends and I had a conversation about similar topics. It was triggered by the then new kid on the block: Telegram. I wrote a blog post about that shortly after.

In the next few months I kept thinking about the problem of having a user-friendly, federated, secure system for RTC. Even though it went unnoticed and I did not do any real work on it, I wrote a series of posts discussing ways in which this could be done, here.

Maybe this interests somebody who has the time and resources to help make it a reality.

The perils of federated protocols

mathstuf — Sun, 22 May 2016 19:34:56 +0000

I've done some OpenCL on R600 hardware. It works, but is not complete by any means. This was probably 2.5-3 years ago now.

The perils of federated protocols

flussence — Sun, 22 May 2016 17:35:40 +0000

I remember all the hand-wavy hype about GPGPU and OpenCL when AMD opened up all the R600 hardware docs. Ten years later, still no sign of it ever becoming usable. One day Vulkan might work, but I'm not holding my breath for it.

The perils of federated protocols

jospoortvliet — Sun, 22 May 2016 16:54:24 +0000

I guess the point parent wanted to make is that while webGL, WEBRTC etc are already very old they still don't work everywhere while Vulkan, not more than a year old, is announced to be supported almost everywhere. I agree with you that doesn't mean it IS. Time will tell if it will go as expected...

The perils of federated protocols

eternaleye — Fri, 20 May 2016 18:34:23 +0000

You misunderstood GP's comment on lock-in, then proceeded to describe exactly lock-in.

Lock-in isn't "can't switch devices" unless the provider locking you in is the device provider.

It's "can't switch away from the provider's product" - which results in the 20 additional messengers, the lack of interoperation, etc.

The perils of federated protocols

yroyon — Fri, 20 May 2016 18:32:32 +0000

Wechat is insanely popular amongst Chinese users, including in the US. LINE is popular with mobile gamers, to coordinate guilds and such. (Oh, and of course players based in China can't use LINE, it's blocked by the Great Firewall. Wechat is not.)

Slack at work, Couple with the spouse, etc, etc. So many. Each has a niche.

The perils of federated protocols

Cyberax — Fri, 20 May 2016 18:03:38 +0000

> And federated protocols CAN do that. Developers can remove support for old, insecure fallbacks.
No, they can not. Clients _can_ remove support for _some_ bad fallbacks, after years of gradual deployment. Servers are usually stuck pretty much for a decade (e.g.: SSLv3 deprecation).

So if in your book a decade to make a change is quick, then I don't want to know what is "slow".

> If I want to make Signal the input or output to a complex pipeline, say filtering piles of messages into interesting places, using some to update displays on my desktop and queueing up some for later examination, extracting information from others, populating an Org mode document with times I plan to meet people, can I? No?
You can, nobody stop you personally. However, you should be prepared for your scripts to break at any moment if Signal makes an incompatible change.

That might be OK for a homegrown project that nobody cares about, but if you try that with actual real-life users who depend on your tools...

Phone numbers as identifiers

mathstuf — Fri, 20 May 2016 16:49:41 +0000

Not just that, but not everyone has a phone number in the first place (what with data-only plans and Google's MiFi thing).

The perils of federated protocols

aemerson — Fri, 20 May 2016 16:36:57 +0000

I don't agree with Mr. Marlinspike's claims. Even the supposedly factual ones that federated protocols can't evolve and are stuck in the nineties are surprisingly false. People are evolving IRC (people other than Slack, I might add) it's a better experience with a modern client on a modern server now than it was inthe nineties. It's even better than it was in the twenty-naughties. There have been changes to DNS, changes to HTML, changes to HTTP...

Bringing IPv6 up is just a red herring. Non-Federated Internet Protocol would run the network in some circle of Hell.

Open, multiple-implementor protocols get developed quickly enough, it's the adoption of new work that can be slow. As usual, Network Effects Ruin Everything. Some things, like end-to-end encryption, really can't and shouldn't be negotiated. A flag day really IS the best way to do that.

And federated protocols CAN do that. Developers can remove support for old, insecure fallbacks. (Like SSL libraries have done.) providers can declare flag days and say they'll only federate with the new version of the open, interoperable protocol that has much better security properties so everyone knows when the deadline is coming.

(The claim that federated protocols 'rally around' large providers actually vitiates the rest of Mr. Marlinspike's argument. If there are a small number of large providers they can coordinate. Or a single provider can take the initiative and refuse to support older versions, making the others support them. All the disadavantages of ederation disappear. I'm not necessarily convinced 'large providers' are unavoidable, but if Mr. Marlinspike is, he should throw the rest of his anti-federation views in the garbage.)

Centralized protocols of course have their own problems. You may have noticed, but there isn't much in the way of running Signal on something other than a smart phone (There is an application built into Google Chrome but by all accounts it isn't very good.), or if you have no phone at all. This kind of sucks.

If someone wrote a GTK+ client for Signal or a client to run in a terminal, would Mr. Marlinspike say he doesn't want THEIR project connecting to HIS servers? And he's already ruled out federation? That sucks too, doesn't it?

If I want to make Signal the input or output to a complex pipeline, say filtering piles of messages into interesting places, using some to update displays on my desktop and queueing up some for later examination, extracting information from others, populating an Org mode document with times I plan to meet people, can I? No? Not unless I convince Mr. Marlinspike and the other Signal developers to implement it? (Sure, I could implement it myself, but they do not want Other Projects aon Their Servers and they won't talk to any other servers.) That kind of sucks, too.

Stuart Cheshire turned DNS into an awesome zero-configuration/service discovery (yes DNS-SD can and often does run over regualr DNS not just mDNS) protocol without having to convince the Centralized DNS AUthority to let him. Avahi reimplemented Cheshire's protocol and talks to the original and other implementations. It works wonderfully. They didn't have to talk anyone into it or get told that nobody would interoperate with them. (And people store all kinds of other things in the DNS, too, more variety every year.)

Google sends and receives email from anyone, and they do all kinds of things with email that RFC 822 never mentions (making your flight itinerary automatically pop up on your phone?) and they didn't have to convince the Central Email Authority to implement it for them.

The Suckless people shatter IRC into a whole pile of FIFOs that you can run through the Unix meat grinder however you like. They didn't have to ask the IRC authority.

And where are the awesome centralized protocols of the 2000s? Surely with advantages like these and a popular Internet they must have evolved quickly, leaving everything else in the dust, making ICQ and MySpace the dominant platforms in the market today....?

Thank you, no. If someone starts a project to fork Signal and create a federated version that isn't so wedded to the phone (heck, if they make an open version that let's me use something other than a phone number as an ID, I'll write the desktop client), I will happily donate to a kickstarter or send paypal or anything else to hurry it along.

I'll leave centralized network paradigms to do what they have done throughout history: suck and die.

The perils of federated protocols

paulj — Fri, 20 May 2016 08:36:44 +0000

IPv6 was slow to be rolled out not because of federation, but because of second-system-syndrome effects that led IPv6 designers to ignore backward compatibility.

SMTP hasn't developed much because it's very mature, and basically does what's needed. The maturity of SMTP hasn't stopped development at higher layers above SMTP. Also, if you want to blame SMTP for identity and abuse issues, no one has solved those any better in any other protocol that couldn't also be applied to SMTP. SMTP is actually wildly successful, because it is "federated", distributed and decentralised.

The perils of federated protocols

niner — Fri, 20 May 2016 08:28:17 +0000

Odd, I cann use WebGL, WebRTC and HTTP/2 just fine while I can use neither Metal nor Vulkan right now. Your examples seem to express the opposite of what you intended.

The perils of federated protocols

krakensden — Fri, 20 May 2016 02:12:00 +0000

I don't even know anyone using anything other than SMS and Facebook. Whatsapp, Wechat, Groupme, all seem fairly dead in my circle. Hangouts killed google chat. AIM & MSN have been dead for years.

The perils of federated protocols

flussence — Thu, 19 May 2016 21:53:50 +0000

> "so why are Signal's growth, ratings, and engagement substantially higher?"

Maybe because the silent majority of XMPP servers and clients don't phone home to one central authority? The rest of his statements are of a similarly dismissable begging-the-question variety — Signal is just another Yahoo Messenger in the grand scheme of things.

The perils of federated protocols

Seegras — Thu, 19 May 2016 18:18:07 +0000

> The only thing it brings, is user/customer lockin.

Yeah, either we've got a completely fragmented market, where every 2 weeks a new state-of-the-art messenger pops up. Like now. WhatsApp, Signal, I don't even know the IM of this week.

Or one of these manages to take off. And then we've got a lock-in. After that, it will get stale and hinder any newer development.

The perils of federated protocols

khim — Thu, 19 May 2016 18:11:30 +0000

They didn't start out large. The question you should be asking is how/why they became large, given their early disadvantages.

And that answer is ... federation.

That's right answer to the wrong question. Of course federation makes it possible to build large system and, indeed, when large system couldn't be monolithic they become federated. Even today there are many systems which are federated: not just ISPs, but cellular networks, railroads, airlines and many other systems are federated today! Heck, you could even find popular federated systems developed in XXI century (here is one, e.g.).

But they all share one important quality: they have some kind of “ceiling”. Some reason which limits growth of unfederated alternative. It may be technical reason (AOL/Compuserve growth hit the limit when it reached US borders: it was impossible to provide cheap enough access to people in Asia or Europe because intercontinental phone calls were incredibly expensive), it may be non-technical reason (RIAA and MPAA made sure that there would be no huge torrent sites with millions of user thus we've naturally gotten DHT), but if there are no “ceiling” then there are no reason for the federation. It's more cumbersome and thus less attractive solution, it's only chosen by users out of necessity, not out of desire.

The perils of federated protocols

khim — Thu, 19 May 2016 17:11:47 +0000

did we learn nothing?

Sure. The lesson is obvious: no matter how dominant is your platform if you stay dormant for years sooner or later someone will bypass you.

The web which we enjoy today is result of Microsoft's attempt to rebuild it today: architecture astronauts have won and instead of quickly adding features to MS IE which would make a breakout attempt impossible Microsoft decided to rebuild everything from scratch

The end result was something years later, with reduced functionality and insane resource consumption.

This gave chance to the Firefox/Safari/Chrome—but also gave developers of these monsters a false sense of security: they decided that since Microsoft was stupid all other contenders for the “try before you buy” app deployment platform will be just as stupid. The height of folly is, of course, stillborn Firefox OS but I think that the ball was lost when Mozilla decided that it could afford to dictate the rules to app developers: “it's my way or the highway”… most developers have chosen the highway… well some have picked some other highway, but almost everyone left anyway…

Some still believe that they will return, but I seriously doubt it: Apple and Google are not like Microsoft (at least not yet), they iterate fast and already made web development mostly irrelevant. I fully expect to see regression of web platform in the next few years—it'll be interesting to see how this process will look like.

Phone numbers as identifiers

Creideiki — Thu, 19 May 2016 17:08:16 +0000

One of the most user-friendly choices that Signal made was to use the phone numbers already stored in the contacts list as the identifiers for sending messages—exactly like regular SMS text messages.

I respectfully disagree. This stubborn insistence that one person = one phone number = one phone brings me a lot of pain.

Most of the people I want to talk to via Signal have two phones and two numbers - one personal, one for work. I go one step further and use a dual-SIM phone. Trying to control, or even ascertain, which identity is used at each endpoint of a conversation is an exercise in hair-pulling frustration.

The perils of federated protocols

smurf — Thu, 19 May 2016 16:36:43 +0000

> The only thing it brings, is user/customer lockin.

You can move WhatsApp from one phone to another. No problem. Just root the thing and backup+restore the data with TitaniumBackup.

My belly-ache with the whole fragmented-messager fiasco is twofold. I can't know beforehand which messager a peer might be using: I need to feed my phone numbers to all of them. Plus, *something* must eat all the RAM and CPU on my phone … why not install 20 additional messengers, learn the idiosyncracies of their UI, deal with crashes, deal with broken sync …

The second problem is most of these messengers don't interoperate and don't have any sort of API. I want my computer to text me? Right: install another specialized messenger. I want to create a chat group between >2 people? Right: tell all of them to install a common chat tool.

The perils of federated protocols

ortalo — Thu, 19 May 2016 15:44:16 +0000

Durability may be a natural advantage of federated protocols.
Conversely, this may be viewed as a natural disadvantage with respect to the sophisticated centralized system that offer all the bells and whistles at the risk of fast collapse.

Maybe we just need both designs in the right places (one where we need durability and the other where we need sophistication). Looks somehow like wheels to me in some sense... ;-)

The perils of federated protocols

pizza — Thu, 19 May 2016 13:41:47 +0000

> Because the list of complains about XMPP looks basically the same as 10 years ago (which sadly describes very well the state of XMPP interoperability)

The funny thing about XMPP interoperability is that, it was the big players (most notably Google) that were the worst offenders -- For example, Google Talk was subtly incompatible with the Jingle spec and reference implementation that Google itself authored.