An asynchronous Internet in GNOME
| LWN.net needs you! Without subscribers, LWN would simply not exist. Please consider signing up for a subscription and helping to keep LWN publishing |
At GUADEC 2016 in Karlsruhe, Germany, Jonathan Blandford challenged the GNOME project to rethink how its desktop software uses network access. The GNOME desktop assumes Internet connectivity is always available, which has the side effect of making the software stack considerably less useful and, indeed, usable to people who live in those places regarded as the developing world.
Blandford currently works for Endless Mobile, which makes computers for the developing world, he said, using GNOME as its desktop environment. He is also one of GNOME's longest-serving project members, having been a GNOME developer at Red Hat long before the 1.0 release.
The Internet is ubiquitous in people's lives and is fundamental to most software, Blandford said, so most developers lapse into thinking about it as omnipresent. But, in fact, Internet access is neither "always on" nor is it cheap in many places around the globe. "Less than half of humanity has access to it today," he said. Thus, software that takes Internet connectivity for granted fails to function for users in the developing world. "We're seeing a split between 'Internet-haves' and 'Internet have-nots.'"
But, he added, the fact that the software industry as a whole lets this market down means that there is an opportunity for GNOME to do better than simply follow the status quo. "And I'm as guilty of this as anyone," he admitted. Nine years ago, he noted, when he worked on GNOME as part of Red Hat's desktop team, that team fully embraced the concept of the "online desktop" that would seamlessly integrate with remote services—which has worked out well for the software project, but has left a lot of people behind.
The barriers that keep ubiquitous high-speed Internet connectivity out of the hands of most of the world's population are fairly well known. Infrastructure costs associated with wired networks mean that the majority of the world uses cellular connections for network access. In addition, data charges are frequently too high for many people, and the bulk of the Internet remains rather English-centric. Finally, while bandwidth is increasing rapidly in the US and Europe, speeds are creeping up far less rapidly everywhere else around the globe.
With the majority of GNOME developers hailing from Europe and North America, Blandford said, the project needs to adopt a new mindset in order to serve users in the developing world. He then showed some raw numbers regarding speeds and pricing in different areas of the world. One of the most striking was that the average desktop uses 60GB of data every month in the developed world—a number far too high to be sustainable elsewhere.
Blandford then pointed out that although a lot of time is spent discussing "instant data" like email and social media messages—data that users consider highly valuable—such data makes up only a tiny fraction of the overall Internet bandwidth. The majority is "cacheable data," he said, like web pages and video content that is produced once and requested many times.
Currently, cacheable data is "extremely expensive," he said; although media companies use commercial content-delivery networks (CDNs) for the developed world, the same data is virtually inaccessible in the developing world. So one important way GNOME could improve the computing experience for people in the developing world is to build an "asynchronous Internet" within the GNOME desktop, utilizing alternative means to distribute and store cacheable data.
People have already created various methods to distribute data in bulk outside the Internet, he noted. El Paquete Semanal, for example, is a hard drive image circulated weekly in Cuba through underground means, currently including about 2TB of data. The Android app Zapya lets users download a web site to their phone while they are online at some public location, saving it to read (and share) once at home. In both cases, the latency of these "networks" is exceptionally high, but the overall throughput of a hard drive delivered weekly via motorbike is pretty good, he noted.
There are also projects like Toosheh, which uses television satellites to "backpack" one-way data signals onto unused bandwidth, enabling users in Iran who have satellite dishes to access web content that is officially censored by the government. Endless has discussed the idea of cell network providers letting users download data at reduced rates overnight when there is substantially less demand for voice service. And, of course, peer-to-peer technologies like BitTorrent are widely used in the developing world to distribute data.
But raw support for asynchronous network access is only one piece of the puzzle: applications would need to support data access through different means, such as offline caches. This is an area where Endless has already been working, Blandford said. Endless systems include a variety of GTK+-based applications that use local data caches to provide access to informational content (from news to topical, Wikipedia-like references). They retrieve updates from Endless servers in off-peak hours, if there is Internet access.
GNOME could adapt similar techniques for many of its applications, he said. The existing GNOME Recipes application, for example, retrieves recipes live over the web; the equivalent Endless app uses a local database that is updated incrementally when it is convenient. Still, he said, his goal was not to dictate that GNOME copy Endless's designs, but rather to inspire the GNOME community to think about how it could improve offline usability in its applications.
There are several areas where GNOME applications can improve, he said. The first is to stop assuming connectivity is available and to build in ways to deal gracefully with connectivity problems and disconnects. "And I'm looking at you, Evolution," he said. Dealing with degraded connectivity includes being careful not to download data more than once, he added. The second is to be conscious of the user's bandwidth, including being explicit about when the application is using the network. "It's a precious resource," he said, "It's not 'free.'" Android works along those lines already with the "download for offline use" options in Google Maps and Translate, he said, but so far GNOME does not use that design pattern. Related to that improvement area is "aggressively caching data," he said, as in the Endless examples.
Other areas for improvement include expanding language support and developing documentation that reaches beyond GNOME's existing stable of users and developers. This is an area in which the project already does a good job, he said, but it needs to go further. Although GNOME takes a lot of flak for simplifying the desktop too much, Blandford said, Endless found in its real-world user tests that much of the documentation and the wording used on the desktop was written to serve a small niche of college-educated people with extensive experience with technology concepts. Ultimately, the point is that developing a desktop system for the entire world requires recognizing that "our needs" are not "everyone's needs."
Blandford ended by proposing that asynchronous Internet support be added to the list of concepts that GNOME cares about, like accessibility and internationalization—he suggested "a18t" as one possible abbreviation, and "Int··n·t" (for "spotty Internet) as another.
Whatever the abbreviation is, he concluded, this is a concept that GNOME should think about while it develops its software. Most of the people who are not yet online want to get online, Blandford said, and when it comes to the desktop software they use, they have not picked one yet. So GNOME's opportunity with asynchronous networking support is a chance to serve an underserved population, but also to grow the GNOME community with new people who are excited about getting access to computing.
[The author would like to thank the GNOME Foundation for travel assistance to attend GUADEC 2016.]
| Index entries for this article | |
|---|---|
| Conference | GUADEC/2016 |
(Log in to post comments)
An asynchronous Internet in GNOME
Posted Sep 8, 2016 7:37 UTC (Thu) by eru (subscriber, #2753) [Link]
An asynchronous Internet in GNOME
Posted Sep 8, 2016 8:19 UTC (Thu) by Seegras (guest, #20463) [Link]
An asynchronous Internet in GNOME
Posted Sep 8, 2016 8:21 UTC (Thu) by NRArnot (subscriber, #3033) [Link]
There's also the issue of what happens if your phone line dies on a Saturday, with a Monday deadline looming. There's little chance that the phone company will fix it over a weekend. Is it really a good idea for one's software to depend on internet connectivity?
An asynchronous Internet in GNOME
Posted Sep 9, 2016 9:22 UTC (Fri) by roblucid (guest, #48964) [Link]
This greed for bandwidth can only grow and leave a larger & larger slice of the market poorly served by mainstream always online model.
An asynchronous Internet in GNOME
Posted Sep 9, 2016 15:36 UTC (Fri) by zlynx (guest, #2285) [Link]
More bandwidth covers up problems but it really is not the answer you want. What you want is traffic shaping and bandwidth management, and buffer-bloat control. I used to use a heavily hacked WonderShaper script and RED with ECN on a Linux router box.
Try it and you'll be surprised at how useful 1 Mbps is. It's really plenty of bandwidth for most Internet uses. Even applications like Netflix worked fine at 1.5 Mbps. Simultaneously with VOIP, even.
An asynchronous Internet in GNOME
Posted Sep 15, 2016 20:20 UTC (Thu) by Wol (subscriber, #4433) [Link]
Even if you don't live in a rural community ...
Broadband over copper is "up to 20 Mb". I get 17Mb. And yet I'm plagued by unusable internet, which I think is other people saturating the network - whether its people at home on my lan, or people elsewhere saturating the exchange, I don't know.
Cheers,
Wol
An asynchronous Internet in GNOME
Posted Sep 16, 2016 7:26 UTC (Fri) by paulj (subscriber, #341) [Link]
Yay for the consequences of "Won't someone think of the children?!!" pressure policies by the UK government which have effectively forced most ISPs to implement filters - without any legislation requiring it.
An asynchronous Internet in GNOME
Posted Sep 8, 2016 13:10 UTC (Thu) by nedrichards (subscriber, #23295) [Link]
An asynchronous Internet in GNOME
Posted Sep 8, 2016 16:25 UTC (Thu) by josh (subscriber, #17465) [Link]
I ran into a hotspot once that wanted email confirmation before allowing use. How, exactly, did they think you'd check that email and handle the confirmation?
An asynchronous Internet in GNOME
Posted Sep 8, 2016 17:48 UTC (Thu) by farnz (subscriber, #17727) [Link]
In the case I encountered it, the issue was that mobile networks in the area only had GPRS (no EDGE, UMTS, no LTE - just good old 2.5G data). The thought was that you'd use your 2.5G data at ~10 kbit/s to confirm, then they'd have some e-mails for marketing purposes in return for running ADSL2+ to their premises and sticking a cheap AP on the end.
The business model behind it has gone now that there's LTE in the area from at least one network, but the registration system remains...
An asynchronous Internet in GNOME
Posted Sep 28, 2016 15:09 UTC (Wed) by mgedmin (subscriber, #34497) [Link]
An asynchronous Internet in GNOME
Posted Sep 28, 2016 15:19 UTC (Wed) by farnz (subscriber, #17727) [Link]
Definitions vary - I'm using the ones from Wikipedia, where 2G is circuit switched data only (as per the launch version of GSM), GPRS is 2.5G (using 2.5G for the first version of the network that did PSD), EDGE is 2.75G.
An asynchronous Internet in GNOME
Posted Sep 15, 2016 20:09 UTC (Thu) by Wol (subscriber, #4433) [Link]
Most hotspots I use seem to think that the only thing people use the internet for is browsing. I spent ages trying to debug why Thunderbird wasn't working, only to realise I needed to click through the licence agreement - which meant I had to fire up Firefox.
Not a problem, just a right royal pain if you don't realise what's going on!
Cheers,
Wol
An asynchronous Internet in GNOME
Posted Sep 28, 2016 15:12 UTC (Wed) by mgedmin (subscriber, #34497) [Link]
This is why captive portal detection was invented. My Android pops up a notification when it detects a captive portal, and when I tap on it, it opens the portal's web page in a new app window (so it doesn't clutter my Chrome tabs, yay), and even closes it automatically when the captive portal doesn't require you to log in. (Don't ask me why there are captive portals that collect no information and require no confirmation for terms and conditions, just redirect your first web visit to some web page. They exist.)
I believe Network Manager can do this too, but I don't know if it's enabled by default on most distros. At least I haven't seen it work :/
An asynchronous Internet in GNOME
Posted Sep 8, 2016 19:15 UTC (Thu) by flussence (subscriber, #85566) [Link]
Sometimes I tether my phone to my laptop while travelling in spite of this, but only because I've gone to great pains to ensure the software on it isn't the sort that consumes all available resources it finds.
There's a much more widespread problem here and this is just a symptom of it: far too many developers these days are spoiled rotten and blinded by their own confirmation bias. The way they build software is like having cars with no suspension, because all they know is perfectly smooth suburban roads. Everyone else is in for a rough ride if they have to pay for their internet traffic, or thought your flavour-of-the-month plain text instant messaging app would be usable with only a gigabyte of RAM, or plug one too many monitors in…
An asynchronous Internet in GNOME
Posted Sep 8, 2016 22:02 UTC (Thu) by flussence (subscriber, #85566) [Link]
Er, that number ought to read £0.50. The ISP's only behind the times, not medieval :-)
An asynchronous Internet in GNOME
Posted Sep 11, 2016 2:22 UTC (Sun) by ianmcc (subscriber, #88379) [Link]
An asynchronous Internet in GNOME
Posted Sep 9, 2016 2:43 UTC (Fri) by trentbuck (subscriber, #66356) [Link]
I just read all the code in https://github.com/lellolandi/Recipes
AFAICT it doesn't talk to the internet AT ALL.
It's just a GUI wrapper around a single-table sqlite database,
to simplify data entry & retrieval.
Am I reading the code wrong?
Is the link in the article to the wrong project?
An asynchronous Internet in GNOME
Posted Sep 9, 2016 21:04 UTC (Fri) by smoogen (subscriber, #97) [Link]
An asynchronous Internet in GNOME
Posted Sep 13, 2016 0:02 UTC (Tue) by davidstrauss (subscriber, #85867) [Link]
But the article linked to the one mentioned in the parent comment.
An asynchronous Internet in GNOME
Posted Sep 9, 2016 23:58 UTC (Fri) by gnu (subscriber, #65) [Link]
As someone from the third world where bandwidth is still a problem in most areas, I could totally relate to the article and look forward to a project like GNOME, browser plugins and other applications implementing these ideas. I have been looking at BitTorrent protocol-like functionality in certain applications (video streaming, software updates.. etc..).
An asynchronous Internet in GNOME
Posted Sep 13, 2016 0:04 UTC (Tue) by davidstrauss (subscriber, #85867) [Link]
Absolutely. I have no idea why every desktop/server isn't usable (via an on/off option) as a mirror for package updates. The packages are signed; we don't need to trust the path of the bytes.
An asynchronous Internet in GNOME
Posted Sep 13, 2016 0:27 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link]
But I can't remember its name right now and Google searches come up empty.
An asynchronous Internet in GNOME
Posted Sep 13, 2016 20:09 UTC (Tue) by flussence (subscriber, #85566) [Link]
The reference ipfs daemon has mDNS built in, but I'm not aware of any way to use it as a regular package proxy.
An asynchronous Internet in GNOME
Posted Sep 13, 2016 22:51 UTC (Tue) by nybble41 (subscriber, #55106) [Link]
It should be easy enough to set up IPFS as a package source (just use a local HTTP gateway), but some trusted individual would need to publish a mirror of the repository at a well-known IPNS address so others could find it, and keep the mirror up to date. IPFS wasn't really designed to proxy arbitrary HTTP sites. The sites like ipfs.io which can be accessed both over HTTP and over IPFS work the other way around: the IPFS version is the master and HTTP is provided by an IPFS gateway running on the server.
Personally I use squid-deb-proxy, which integrates well with apt and (after installing squid-deb-proxy-client) uses mDNS to connect clients to the server when available. If the server isn't available it automatically falls back to direct downloads, which is nice for mobile devices. The server required some configuration to let it know about extra repo domains for third-party packages and my (somewhat idiosyncratic) local network address ranges. I also adjusted the settings to coalesce simultaneous requests rather than fetch the same file twice. (The default prioritizes small-file latency over bandwidth.)
An asynchronous Internet in GNOME
Posted Sep 28, 2016 15:16 UTC (Wed) by mgedmin (subscriber, #34497) [Link]
It is full of pain[*][+], so I'm not surprised nobody installs it by default.
[*] like shipping a default config file that causes the nigthly cron script to send errors to root because the default list of exceptions happens to be empty
[+] like rejecting all 3rd-party repositories (including PPAs) by default with an unhelpful 403 Forbidden error that apt shows as originating from the real upstream mirror and don't let me start counting the hours I lost while debugging this
An asynchronous Internet in GNOME
Posted Sep 13, 2016 1:30 UTC (Tue) by raven667 (guest, #5198) [Link]
https://privacy.microsoft.com/en-US/windows-10-windows-up...
An asynchronous Internet in GNOME
Posted Sep 13, 2016 5:48 UTC (Tue) by davidstrauss (subscriber, #85867) [Link]
An asynchronous Internet in GNOME
Posted Sep 15, 2016 20:14 UTC (Thu) by Wol (subscriber, #4433) [Link]
I can't make sense of what you say, BUT! If an option to use MY bandwidth for OTHER people is an opt-OUT, what happens if I pay for said bandwidth by the MB. THAT is the problem, not the fact that the feature is there.
Indeed, my ISP offers me the option to use other peoples' routers while I'm roaming. But that is very much an opt-in - if I configure my router to let others use it, then I can use others' routers in return. (And iirc, it's on a separate network managed by BT, not on my home 192.168.1.x subnet.)
Cheers,
Wol
