Weekly Edition
Return to the Development page
| |||||||||||||
Tent v0.1 released
The initial release of the Tent distributed social networking protocol
implementation is out. From the introductory post:
"Tent is decentralized, not federated or centralized. Any Tent server
can connect to any other Tent server. All features are available to any
server as first-class citizens. Anyone can host their own Tent server. Tent
servers can also be run as Tor hidden services to create a social darknet
for at-risk organizers and activists. Anyone can write applications that
connect to Tent in order to display or create user content."
(Log in to post comments)
Tent v0.1 released Posted Sep 21, 2012 21:33 UTC (Fri) by job (guest, #670) [Link]
Why this obsession with overloading HTTP?
Bouncing JSON over HTTP for a global pubsub-system does not strike me as a particularly useful idea. Something more akin to netnews (where updates are batched) or jabber (where subscriptions are followed in real time) would be much more sane.
And don't give me some crap about firewalls, because email and IRC and games work perfectly even among the technically uninformed majority.
Tent v0.1 released Posted Sep 21, 2012 21:47 UTC (Fri) by lindi (subscriber, #53135) [Link]
I don't know where you live but email does not work perfectly here if you try to use something else than gmail. For example sending mail from one of the biggests ISPs of Finland to Yahoo has had problems for several years: http://www.iltasanomat.fi/kotimaa/art-1288371200415.html
IRC does not work perfectly either. A friend of mine could not figure out how to use IRC. His ISP did not have an IRC server and the generic IRC server that he found would only allow two connections from the same IP. The whole building was behind the same IP.
Tent v0.1 released Posted Sep 21, 2012 22:32 UTC (Fri) by job (guest, #670) [Link]
While your story here may not be representative for other reasons, it is clearly irrelevant what protocol you used. You would have had exactly the same e-mail problems (I take it they are reputational in nature) if your mail was pushed over JSON over HTTP. They would just be a lot harder to debug.
And the connection limit your friend encountered on IRC would be just as relevant if it was a web chat over HTTP. In fact, they would be worse. If it should be taken as an argument for anything, it is clearly that one should not try to cram even functionality over one specific command protocol.
Tent v0.1 released Posted Sep 22, 2012 17:48 UTC (Sat) by geofft (subscriber, #59789) [Link]
You misunderstand the point being made. The argument is that most home users push their mail over JSON over HTTP (or something) to an intermediary like Gmail that does trust them (because it authenticates them), and the intermediary sends the email over SMTP.
There is no proposal to make SMTP use HTTP as a transport. The point being made is that, despite most people having email work fine, most people do not directly use SMTP on their home computer, and instead use HTTP to access access some other machine that _does_ use SMTP. So it is not contradictory that port 25 on the home user's side of the connection does not work for most people, even though most people do use email.
Tent v0.1 released Posted Sep 21, 2012 22:37 UTC (Fri) by Cyberax (✭ supporter ✭, #52523) [Link]
HTTP is a nice protocol. It doesn't require reinventing the wheel and it has lots of nice infrastructure built around it (caching proxies, load balancers, etc.). So deal with it.
JSON is just a nice and easy serialization format.
Tent v0.1 released Posted Sep 22, 2012 10:07 UTC (Sat) by job (guest, #670) [Link]
But yes it does. That's the whole problem with it. HTTP is not a pubsub protocol.
Designing one that works over HTTP requires a whole lot of wheel reinvention, because of its shortlivedness, statelessness etc. It's just not a suitable transport.
The "nice" infrastructure around it is part of the problem. Caching proxies cause no end of problems when pushing state around.
Tent v0.1 released Posted Sep 22, 2012 13:47 UTC (Sat) by AlexHudson (guest, #41828) [Link]
HTTP/1.0 in a narrow sense, sure. Pipelining wasn't added long after and server push was in there very early on, now we have better systems like websockets but the principle is the same.
Caching proxies don't cause any problem so long as the system is used correctly, and HATEOAS-style services work well and benefit from the caching.
Even incompatible protocols like SIP borrow heavily from it. It's a fine technical choice.
Tent v0.1 released Posted Sep 22, 2012 18:49 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link]
Which is great! Internet in itself is not a pubsub media, it's a stateless transfer media with unreliable connectivity.
I've seen tons of protocols that just use TCP and happily build very statefull systems on top of it. And of course, they usually can't survive broken connections.
Tent v0.1 released Posted Sep 21, 2012 22:46 UTC (Fri) by geofft (subscriber, #59789) [Link]
> And don't give me some crap about firewalls, because email and IRC and games work perfectly even among the technically uninformed majority.
No, they don't. Most ISPs filter block 25 in both directions, and many block IRC since it's used for botnet coordination. Have you tried it (on an ISP that is commonly used by the technically uninformed majority)?
Tunnelling over HTTP or better yet HTTPS is becoming quite common because it's the only thing guaranteed to work. "... for various reasons, different application layer protocols are increasingly being carried over TLS using a small set of TCP port numbers, most often port 443."
It's also generally the case that few people play large-bandwidth games on public networks at e.g. cafes, whereas a social networking protocol is useful if it's accessible from those sorts of restricted environments. In particular, a cryptographically-interesting social networking protocol is also quite useful from repressive countries with incompetent national ISPs.
Tent v0.1 released Posted Sep 22, 2012 10:15 UTC (Sat) by job (guest, #670) [Link]
Considering that 100% of the population use a computer, and many have more than one Internet connection, I would say that empirical data speaks against that. Mail just works for almost everyone. While it is true that you have to enter more settings in your mail software than should be strictly necessary, because of things like SMTP filtering, that is a solved problem.
As long as your protocol survives being NAT'd you're probably fine on today's networks. Because I hope you are not considering the use case that people start putting up Tent servers inside their office networks.
In fact, overloading port 80 will cause more problems than it will solve for home users. They will not find it at all straightforward to run both a web server and a Tent server from the home connection, for example.
Tent v0.1 released Posted Sep 22, 2012 13:03 UTC (Sat) by tao (subscriber, #17563) [Link]
"Mail just works for almost everyone." doesn't mean that there isn't heavy filtering in place though.
A lot of people use web based e-mail (gmail, hotmail, etc.). I'd wager most others either use pop or imap from their providers; very few -- if any -- "normal" users have their own web servers. Very few "normal" users use IRC.
Besides, even if your home operator would allow all types of traffic, you cannot rely on that everywhere. Ever tried ssh:ing from the free wifi most cafés offer these days? Or the pay for use wifi you can find at most airports?
Tent v0.1 released Posted Sep 23, 2012 5:38 UTC (Sun) by butlerm (subscriber, #13312) [Link]
Many if not most ISPs block inbound HTTP to those without static IP addresses. It won't make a difference to the vast majority of users who do not run their own servers, but to those that do, it is hard to see how choosing inbound HTTP over practically any other (UDP or TCP based) server protocol is likely to make any difference on the filtering front.
If you want to operate a "server" without the cooperation of your local ISP or firewall administrator, about your only option is to establish a long lived outbound connection to some sort of proxy server, or worse. I take it that is not what is going on here - requiring a proxy server would defeat the purpose, and requiring outbound connections to every server that might want to communicate with you doesn't scale.
Tent v0.1 released Posted Sep 22, 2012 16:30 UTC (Sat) by smoogen (subscriber, #97) [Link]
. Mail just works for almost everyone. While it is true that you have to enter more settings in your mail software than should be strictly necessary, because of things like SMTP filtering, that is a solved problem.
----
You do realize that a) large majority of those email users use email over HTTP in some form or another.. (either via google, hotmail, yahoomail) or the fact that various mail clients send their mail over HTTP in ugly ways to get around such firewalls?
And the games are run via various proxy daemons in firewalls that basically use HTTP.
Your arguments are 13+ years too late :/. You can join me on the front porch and yell at the kids to get off the lawn.. but that is about it. The problem is that HTTP turned out to be a nicely documented protocol that had lots of stuff written around it. It sure beats the people who would use the LPD printing or the SMTP email protocol for everything before hand. Before that the majority of programmers used UUCP as their basis. I say this from dealing with various network programs written in the 1980s and looking at why they weren't working in the 2000's due to firewalls and such. Most of them didn't have a custom protocol but were using one of those three as their basis.. mainly because they were there, they had been written and the programmer didn't have to deal with too much bikeshedding over what new command was needed to fulfil some esoteric task.
HTTP a nicely documented protocol? Posted Sep 23, 2012 7:13 UTC (Sun) by oldtomas (guest, #72579) [Link]
Glad you mentioned SMTP:
"It sure beats the people who would use the LPD printing or the SMTP email protocol for everything before hand"
You do realize that HTTP basically *is* SMTP: command line (just "GET" vs. "MAIL FROM", capitals included), CRLF separated headers, blank (CRLF line) then body? They just forgot the last line with a single dot (and are a bit more forgiving about what goes in the body, an attitude which turned around and has bitten us in the ass ever since (go read about "Content-Length" in the relevant RFCs if you don't know what I mean).
Of course, a multitude of contraptions have been bolted-on afterwards, some which look like wings, other like corkscrews. Flow control. A streaming (100 CONTINUE et al) protocol tacked upon a packet protocol (the original HTTP/1.0, akin to SMTP) with a rudimentary flow control, running over a much more sophisticated streaming protocol (TCP). No. If HTTP looks nice to you it's because you didn't look too much into it.
Actually it's chewing gum and bailing wire.
On a more general (and more on-topic) note: don't read in that a negative attitude towards Tent itself: I think it's a nifty idea, worth to take off -- I serioulsy hope it does (and having looked at their Git repo, the one thing I most wish would be a plain language description of the protocol they envision, rather than a fuzzball of Ruby and PostgreSQL. But I just might be blind -- or that may be in the making).
Tent v0.1 released Posted Sep 22, 2012 17:42 UTC (Sat) by geofft (subscriber, #59789) [Link]
Have you completely forgotten about webmail? Most users access their email client over HTTP. Port 25 works from one mail server to another (of course), but generally not between home users and their mail servers.
Port 80 inbound to home users also tends not to work.
Tent v0.1 released Posted Sep 23, 2012 20:39 UTC (Sun) by drag (subscriber, #31333) [Link]
The only people that I know that use email clients on their computers are Linux geeks and employees that are required to use Outlook. Even the majority of the hardcore folks that I know once managed their own email servers for personal uses have long since abandoned them for Gmail.
Email is a huge pile of shit on many levels and while everybody still uses it nobody wants to deal with it.
Maybe if it tied itself into DNSSEC and made it so people actually could trust the originator of the email to be who they claim to be it may move back into the 'things that don't totally suck' category of software, but I am pretty sure it is just too far gone at this point.
Even professionally managed webmail is getting more and more useless. Which is one of the reasons we see the rise of 'social networking'.
Tent v0.1 released Posted Sep 29, 2012 10:29 UTC (Sat) by Jandar (subscriber, #85683) [Link]
> No, they don't. Most ISPs filter block 25 in both directions, and many block IRC since it's used for botnet coordination. Have you tried it (on an ISP that is commonly used by the technically uninformed majority)?
I'm using the biggest ISP in my country, obviously the commonly used one by the technically uninformed majority. Of course it doesn't filter any ports, doing so would create very bad PR and loss of many customers. Are there really major ISPs filtering? How can such customer-screwing ISPs remain in business outside of a niche-market?
Tent v0.1 released Posted Sep 29, 2012 10:38 UTC (Sat) by nix (subscriber, #2304) [Link]
Filtering is, alas, almost ubiquitous in the UK: you have to go to specialist ISPs to find ones that let you get unfiltered access, and even *they* egress-filter port 25 because if they didn't worm-infested Windows boxes would lead to their entire netblock being blacklisted.
My sister's using an ISP that explicitly egress-filters and bans everything other than HTTP and SSL in its almost-impossible-to-find terms of service (email? use webmail), and transparent-proxies the HTTP and HTTPS over some sort of carrier-grade NAT horror that drops all connections after half a minute or so on the assumption that a web browser will immediately re-initate it and no other application matters. She's happy with it, but when I go there there's no way I can ride an SSH connection over a TCP link that unreliable, not even with proxytunnel.
(I'll admit I'm wondering about some sort of interstitial thing that would make an intermittent TCP connection appear like a constant one, re-establishing where necessary, making this sort of thing work again. Unfortunately the client end would almost certainly have to work on Windows because most systems I encounter when travelling run Windows, and that is beyond my sphere of expertise.)
Tent v0.1 released Posted Sep 22, 2012 7:37 UTC (Sat) by steveriley (subscriber, #83540) [Link]
I used to be a protocol purist too. But despite my rants, one particular layer 7 has become the new layer 4. So now -- meh. Doesn't matter much anymore.
Tent v0.1 released Posted Sep 22, 2012 7:42 UTC (Sat) by paulj (subscriber, #341) [Link] Why this obsession with overloading HTTP? Because a high proportion of ISPs are obsessed with ensuring that HTTP becomes the only protocol that works reliably across the internet, by blocking any other protocol (often because of abuse). Yes, eventually that abuse will just migrate to protocols layered on HTTP, but welcome to the tragedy of the expediency of the short-term hack.
Tent v0.1 released Posted Sep 22, 2012 21:27 UTC (Sat) by lab (subscriber, #51153) [Link]
> Because a high proportion of ISPs are obsessed with ensuring that HTTP becomes the only protocol that works reliably across the internet, by blocking any other protocol (often because of abuse). Yes, eventually that abuse will just migrate to protocols layered on HTTP, but welcome to the tragedy of the expediency of the short-term hack.
In the absence of a dedicated +1 button, I hereby award my +1 clueful commentary to paulj, for hitting the sad nail on the sad head. It's not just ISP's by the way; it's every clueless sysadmin in the world, which is a lot.
Additional +1 to the commentary below, which rephrases the problem as "multiplexing the Internet through port 80", which is another way of describing the same sad problem.
I believe the relevant debate/topic, which accurately and pointedly frames the problem, is the one called "net neutrality". It's a very important one. Maybe put in another way: The fight against tackling real problems at the wrong level. Seing as that is pretty much a human race condition, it's a tough one.
Tent v0.1 released Posted Sep 23, 2012 10:01 UTC (Sun) by paulj (subscriber, #341) [Link]
Thanks ;)
Re port 80, I think that one day “Why is there a "Must be 80" field in the Internet Protocol?” will be a question geeks ask each other to test their knowledge of arcane trivia. Few people will know why.
Tent v0.1 released Posted Sep 23, 2012 17:41 UTC (Sun) by khim (subscriber, #9252) [Link] Geeks will know about that, of course: internet may be going in the "80-port only" direction, but intranets usually have no such limitations.
Tent v0.1 released Posted Sep 24, 2012 14:03 UTC (Mon) by nim-nim (subscriber, #34454) [Link]
Meh, people are not multiplexing over port 80 (way too many caching and filtering infra which is not forbidding of protocol errors) but over port 443.
With TLS, you only have to worry endpoints speak the same syntax as you.
(and *that* is what is fuelling MITM tls attacks, since having everything crypted is intolerable to way too many organisations, that would have tolerated limited bank and passwords crypto but now whole-internet crypto)
Tent v0.1 released Posted Sep 30, 2012 1:13 UTC (Sun) by filteredperception (guest, #5692) [Link]
> I believe the relevant debate/topic, which accurately and pointedly frames the problem, is the one called "net neutrality"
Welcome to my world (and my FCC form 2000F complaint about the new Google Fiber ISP in my 'hood)
http://cloudsession.com/dawg/downloads/misc/kag-draft-2k1...
"
Google's current Terms Of Service[1] for their fixed broadband internet service being deployed
"You agree not to misuse the Services. This includes but is not limited to using the Services for
where 'here' is a hyperlink[2] to a page including this text-
"Unless you have a written agreement with Google Fiber permitting you do so, you should not
In my professional opinion as a graduate in Computer Engineering from the University of
[1] http://fiber.google.com/legal/terms.html
--- end FCC complaint ref #12-C00422224 ---
Overloading HTTP Posted Sep 22, 2012 10:25 UTC (Sat) by oldtomas (guest, #72579) [Link]
Yeah. It's what I used to call "multiplexing the whole Internet through port 80". Sad.
But yes, it's mainly the "crap about firewalls". At home, my ISP lets the things through I care about (didn't try more exotic things like UDP multicast, I don't expect it to work), but at $WORKPLACE, the firewalls are set up to just allow ports 80 (proxied) and 443 (CONNECT). I can understand that they consider a HTTP proxied port 80 somewhat more secure, because the proxy can try to trigger on known malware patterns (what about unknown malware?), but 443: you tell the proxy CONNECT blah.foo.org and there you go, an encrypted channel right through the proxy. Just more expensive.
Sad, yes.
And please, don't mention Jabber as a particularly good example. The sheer contradiction between streams (which are potentially unending) and XML documents (with an explicit beginning and end) has me split between weeping and laughing. There's lots of verbiage to that in the RFCs, but it doesn't make that better.
But hey, maybe some day network interface hardware understands HTTP headers and is unable to handle simple UDP.
Overloading HTTP Posted Sep 22, 2012 22:46 UTC (Sat) by paravoid (subscriber, #32869) [Link]
Is incoming HTTP allowed from your corporate firewall?
Would your company policy allow you to host your own 24/7 Tent server in the company's premises?
The same applies to people who mentioned their (evil) ISP. Non-HTTP traffic still works fine on hosted servers or residential connections with static IPs, thankfully.
Overloading HTTP Posted Sep 23, 2012 7:18 UTC (Sun) by oldtomas (guest, #72579) [Link]
"Is incoming HTTP allowed from your corporate firewall?"
No, of course not. I know what you mean -- but the point made was subtly different: my corporate firewall just does allow *outgoing* 80 and 443. Several (many?) ISPs seem to do that too. Thus, services "out there", having a "real" Internet connection make less and less sense if they sit on (say) port 22.
Overloading HTTP Posted Sep 23, 2012 16:51 UTC (Sun) by man_ls (subscriber, #15091) [Link] Egress filtering used to be my pet peeve: why limit outbound connections to certain ports? At some point clueless (or perhaps fearful) sysadmins started doing it to protect who knows what from whatever -- perhaps internal hackers from taking over FBI websites. Right now a sysadmin at any large company who left open e.g. outbound port 22 would be considered crazy by their peers, unless some Vice-Pope signs it off.That particular fight was lost without having started, and now even home connections appear to have trouble connecting to certain ports outside the sanctioned range; not to speak about 3g connections. So we have better fight for having good port 80 support (e.g. for websockets), something where regular users are likely to help us -- if only by complaining loudly to their ISPs when weird layers of proxies and firewalls break connections.
Overloading HTTP Posted Sep 23, 2012 20:13 UTC (Sun) by butlerm (subscriber, #13312) [Link]
Most ISPs I am aware of only block a small handful of commonly abused ports for outbound connections. A much bigger problem is blocking inbound connections. A real ISP (like CenturyLink) will sell you a static IP address that is completely unblocked, for a nominal fee.
I don't see how anyone can expect to operate a Tent server without such cooperation, so the protocol used for server-server communications is almost irrelevant. It is the client-server protocol where special consideration needs to be taken, and that will naturally be a web interface in most cases.
The idea that HTTP provides some sort of filter advantage for server-to-server communication, however, seems to be entirely a red herring.
Overloading HTTP Posted Sep 23, 2012 20:54 UTC (Sun) by paravoid (subscriber, #32869) [Link]
Exactly my point. Thank you for making it clear.
Overloading HTTP Posted Sep 24, 2012 19:45 UTC (Mon) by drag (subscriber, #31333) [Link]
> I don't see how anyone can expect to operate a Tent server without such cooperation
You have to have a connection brokering service for locating servers and setting up connections.
The idea is that your content server goes out and connects to a broker server. The user's clients do this also. So if their client wants to set up a connection with your server then it sends a message to the broker. The broker then communicates back to your server, which then pushes a hole through your firewall using a mechanism like uPNP or starting a fake connection to the client to open up a hole in the NAT connection tables for the client to connect through.
All in all this is a relatively routine thing used by a huge number of popular 'p2p' protocols.
I am sure that the 'Tent' people took this into account. Personally I think that a modified Jabber server would be good for this sort of thing.
Tent v0.1 released Posted Sep 22, 2012 18:21 UTC (Sat) by tx (subscriber, #81224) [Link]
This seems interesting and I wish the project luck, but I can't help but feel that the push based approach is going to end up biting them.
Tent v0.1 released Posted Sep 23, 2012 8:19 UTC (Sun) by intgr (subscriber, #39733) [Link]
So how does this differ from the bazillion other decentralized social networks invented in the last few years?
|
|||||||||||||