NLUUG: The Open Web [LWN.net]

November 4, 2009

This article was contributed by Koen Vervloesem

At the end of October, NLUUG held its Fall Conference with the theme The Open Web. Steven Pemberton, researcher at the Centre for Mathematics and Computer Science in Amsterdam and involved with the web since the beginning, set the stage with his keynote about the different dimensions of openness.

On a conference that focuses on an open web, it's natural to sit back first and think about what we mean by "open". Pemberton gave some obvious examples of technologies that are bad for openness on the web, such as proprietary media formats, Flash, and so-called "Web 2.0" sites, which lock your own data in silos. On the other side, there are technologies such as XML, CSS, and RDFa that promote openness.

Device-independence

The discussion is not so much about technology as it is about how people are (ab)using it. Pemberton showed an example of the web site of the Dutch national airline KLM, which blocks browsers they don't "support". Other web sites block users if their screen size is "too small", or they redirect the user to a mobile version of the web site, even if the user wants to browse the full site on his smartphone with a high-resolution screen. So these web developers decide how, or if, a certain class of users sees the web site, just because they are too lazy to implement their web site in a device-independent way.

According to Pemberton, we have all the technology to design device-independent web sites at our disposal. For example, with XForms the designer can describe what a form control is meant to do instead of implementing it, e.g. as a radio button. The form description can then be mapped to specific controls dependent on the device, such as different controls for a desktop browser or a cell phone, or even a voice menu on a phone call.

Web 2.0 walled gardens

Another challenge for the open web are the myriad of Web 2.0 sites that have entered our lives during the last five years or so. Since the monetary value of a Web 2.0 site comes from advertising sales targeting its user-generated content, these web sites tend to lock-in their users. This places an immediate burden on the user: does he choose LinkedIn, Xing, or Plaxo for his professional social network? Or does he have to use them all because some of his contact use LinkedIn and other ones Xing?

According to Metcalfe's law, the value (or utility) of a network is proportional to the square of the number of nodes. This means that when you split a network (social network, instant messaging network, e-mail distribution system, etc.) in half, each part only has one quarter the value of the combined network and the total value is halved. Looking at the case of LinkedIn and Xing, this makes sense: fragmentation is not good for the users.

Moreover, without standards for migrating data between services, users that value their freedom of choice face a nightmare. What if the social network goes out of business? What if the web site crashes and has no backup? Without data portability, you lose your data in both situations. The web site's owners can impose terms of use that cause other significant problems for users: Pemberton told the story of someone's Facebook account that was deleted because he used screen scraping to download all the email addresses of his friends.

A web site for every home

The solution to these walled gardens sounds simple: "Publish your data yourself and let others aggregate it." According to Pemberton, we already have all the technology at our disposal to achieve data portability. Mark your site up with RDFa, an extensible way to embed rich metadata within web documents, and a W3C recommendation since October 2008. "RDFa is CSS for meaning", Pemberton summarized. "This is also why you should have your own web site. APIs and XML feeds are just poor substitutes for having your data right on your own web site." Drupal, a leading open source content management system, already has RDFa support.

As an example of what is possible with RDFa, take Flickr. It hosts a lot of photos for its users, and it is a convenient web site. If everyone starts publishing their photos on their own web site, a lot of this convenience gets lost. However, if these web sites use RDFa, someone can write an aggregator that can be a "Flickr killer" yet remains open.

Pemberton concluded his talk by admitting that there is still a lot to be done before the open web is user-friendly enough that users can take control from locked-in social networks. Creating your own web site is still not an easy task for John Doe, let alone creating a Facebook-like web site on your own server. Moreover, there are still some technical challenges. For example, how do we control who can see which information from our web site? A possible solution is a distributed social network using OpenID for authentication, which is a work in progress.

Another case of lock-in in social networks are the countless "social applications", widgets on a social network that give the user some information or are just for fun. A MySpace application will not work on LinkedIn. However, Google is working on a solution. Chris Chabot, Developer Advocate at the search giant, talked about OpenSocial, a set of open APIs to create applications for social networks. Applications implementing the OpenSocial APIs are interoperable with any social network that supports them. Among the supported social networks are MySpace, Plaxo, and LinkedIn.

Today's technology for the future open web

Apart from this "holistic" topic of openness, the conference also had a fair number of strong technical talks. For example, Henri Bergius talked about location-aware applications with GeoClue. Now that computing is becoming more and more mobile, location is becoming an important parameter for applications. The GeoClue project is a D-Bus service that applications can use to become geo-aware. GeoClue supports a lot of flavors of location: GPS, GSM, Wi-Fi, IP addresses, and so on.

On the KDE front, Sebastian Kügler talked about freeing the web from the browser and gave a demo of Project Silk, while Frank Karlitschek talked about the Social Desktop integration of web communities into desktop applications. Both KDE developers talked about their respective projects to LWN in mid-October.

A nice real world example of the advantages of open APIs came from Karl Vollmer, the developer of Ampache, a web-based audio and video streaming solution. In 2002, the previous developer of the project added an XML-RPC API, but never documented it. Moreover, it was an "ad hoc" format with custom date encoding. The result: for over four years there were no other implementations of the API.

Vollmer replaced the old XML-RPC API in November 2007 by a documented and simple-to-use REST (representational state transfer) API. After two years, nine successful implementations of the API have appeared: Amarok 2 has it, there is an Android implementation, a Python GTK interface (Quickplay), a plug-in for SqueezeCenter, a Rhythmbox plug-in, a WebOS plug-in, support in the UPnP media center Coherence, and even an iPhone application (Amphone). The example of Ampache is a good reminder that we don't get an open web by using undocumented and ad hoc formats.

From Flash to HTML 5

Gnash developer Bastiaan Jacques talked about the role of Flash in the open web. But why do free software proponents have to care about Flash? "Because it has nearly 100 percent market penetration," Jacques says. Indeed, over the last few years, Adobe Flash has become so ubiquitous that it is difficult to imagine the web without it.

However, with the proprietary Flash technology in its current state, the web will never be a truly open web. The Gnash developers reverse engineered parts of the Flash technology to create a free software Flash player, and this works relatively well. Gnash has even some better security and privacy features than Adobe Flash. For example, it blocks Flash cookies by default.

In the meantime, the Gnash project is facing some challenges. The Open Media Now! foundation was started in 2008 to fund Gnash development, but, because of the economic crisis, the four full-time developers were cut back to zero. Another challenge is that proprietary codecs cannot be distributed with Gnash, which may affect the end-user's experience.

To conclude his talk, Jacques stressed that Flash is not a part of the open web, but that we are stuck with it. Moreover, "Gnash is a relevant project because existing content must remain accessible and people are notoriously slow to transition to new (read: open) technologies." For new content, Jacques recommends HTML 5.

This brings us neatly to the last speaker, Paul Rouget, who is Technology Evangelist at Mozilla. He gave a bunch of demos of new technologies in Firefox 3.5 and HTML 5. His take-home message:

Today the web is full of sexy and fun stuff. With HTML (5), CSS, SVG and JavaScript we have a very powerful platform to implement nearly everything you find in Adobe Flash and Microsoft Silverlight. And contrary to the proprietary competitors, these are not black boxes and they work on nearly all modern browsers.

Conclusion

What the NLUUG conference showed clearly is that we already have almost all building blocks for the open web. XML, CSS, SVG, RDFa, XForms, OpenID, OpenSocial, and HTML 5 are all existing or emerging standards in the open web toolbox, although it still is a technical challenge to build a user-friendly open web upon them. An equally interesting evolution can be seen in KDE projects like Project Silk or the Social Desktop, that are doing their best to bring the web to the desktop. However, ultimately the openness of the web depends on the people that are creating the content. Pemberton made it clear that this not only means the web masters, but also the users: the ubiquity of user-generated content on the current web brings power to the users, who can choose to remain locked in a comfortable walled garden or to be free and boldly go where no one has gone before.

(Log in to post comments)

NLUUG: The Open Web

Posted Nov 5, 2009 5:22 UTC (Thu) by drag (subscriber, #31333) [Link]

Total kick-ass stuff.

Normally I am not a big fan of the Web 2.0 rhetoric because of misplaced commercial interests that continuously focuses on 'cloud computing' as a way to scale proprietary web applications to a large enough scale that you actually start having meaningful numbers.

But the reality is that the best way to scale everything is to effectively have one node per user.

-------------------------

The major limitation in all of this is not so much usability as much as the technical aspects of the implementation.

For example I have a friend that went through the time and expense of purchasing internet access through a cable provider that only provides private IP addresses. He can't run any services from his house. Not even SSH access or anything.

Another example is that I've been playing around with SIP. SIP is great, open, and has good performance. Security is a bit so-so, but other then that I like it and there are lots of devices that support it.

But you know what happens if you have two people on the same private network with NAT firewall and softphones? _it_won't_work_. One will not be able to connect. Sure one will work fine, and if you went through the trouble of setting up a SIP gateway or proxy or whatever then you could get that working fine, but that is a huge PITA.

Meanwhile Skype seems to have dozens of ways to work around a firewall. You can have multiple people on the same network doing the same protocol behind a NAT firewall and it'll 'just work'.

That is why things like Ekiga and Empathy suck* compared to Skype. Skype can just plow through any network, while SIP requires a smart person to operate it and set up the tunnels or know how to find a proxy service or whatever.

(*other then they are buggy and are not only difficult to configure that they rarely can be configured in a way that works well.. while things like linphone have a network configuration system that is massively easier to work (even though the application itself has oddness and UI problems typical with older Linux applications). For example I have a SIP account that is dedicated for VoIP to POTS, but it does not support P2P. So I want to use a Ekiga SIP account for P2P and a different SIP account for POTS. Ekiga can only be configured to do one or the other.. if I set up a SIP proxy it will send all traffic through it.. which is incompatible with Ekiga's service, but required for VoIP to POTS. I can do that with Linphonec just fine, but Ekiga it is very difficult to use. Empathy I can't get to work with anything in Ubuntu 9.10. (which is compounded with Ubuntu not being able to make a smart enough PulseAudio setup (but they are doing much better with 9.10)))

So like Skype you need a way to plow through networks. Not just for VoIP or one or two protocols... but with everything.

So I figure the best way is with IPv6. If you use typical tunnelling stuff it is difficult to setup in a way that is universally accessable... that is they are generally designed to create private secure networks and not really designed to allow open penetration into private secure networks.

But if you use a IPv6 tunnel over IPv4 then that is designed to be universally accessable through IPv6 networks. Not necessarily for VoIP... I know the latency will kill performance. But the advantage of this approach is that individual services and web APIs don't need to concern themselves with details about circumventing NAT firewalls or blocked ports or whatever... they can worry just about providing a nice optimized protocol without the normal restrictions that you typically have to deal with. This should make things massively simpler from a lower level network protocol point of view.

-------------------

So then the challenge is to set it up in such a way that a user who has no f-ing clue what SIP is or IPv6 is or anything else can just install a simple application in a familiar manner and have it 'just work'.

So here is a possible solution:

User installs a Vmware player or Qemu VM as if it was a application on their Windows/OSX/Ubuntu/etc box.

The VM has two network interfaces. A "private" network and a "public" network.

The public network does IPv6 over IPv4 tunnel and can be accessed by anybody on a IPv6 network. (Of course you have some heuristics that test to see if a real IPv6 is accessable and if so use that if at all possible instead of the tunnel)

The 'private' network interface is only accessible to the PC the VM is running on. And then all the configuration and communication happens from the user's browser to the private interface on the VM. That is the main user interface; a web application.

So from the point of view from the user here is how it goes:

1. Sign up for the account (which would include a OpenID and access to the required network services)

2. Download the application and installs it. The installer sets up a bookmark in the browser and/or a icon on the desktop that starts up the VM and launches the web browser with the correct address to access the web- based user interface for the VM environment.

3. The user is then prompted for the OpenID details and keys or whatever, which is then loaded into the VM, and then the VM enters 'go mode' and begins providing all the services.

At no time is the user ever made aware of anything that happens or how it happens. If they are curious they can read the documentation and get deeper access to the VM, but that is purely optional.

The actual web-based application stuff in the VM should be done in a similar way to Sugar. A standalone Python WSGI application (or similar high level language with standalone web application capabilities) that can access the open APIs or at least provide hooks into services or whatever. Whatever Web 2.0-ish thing you want to accomplish. For most people that should be enough. A stripped down Debian-style environment can run a low- usage website with a 128-256MB of RAM allocated.

And it must be developed to be 'secure by default'. It should be exceptionally difficult to make the VM insecure.

If users desire better performance or more capabilities they can upgrade to the 'home server' type of thing were they purchase a standalone hardware that is more capable... like using more WSGI python processes sitting behind a Apache-based reverse proxy and that sort of thing. That way if users desire to have LOTS of communication and LOTS of friends they can set up something that will scale well.

And you can have additional services that you can't really get in a resource-strapped VM.. like firewall/router and NAS services.

Since dedicated NASes these days are doing more and more and more in order to make their product stand out then there should be a big market for NAS OEMs to exploit to attract customers. That way you would not have to make a big production about setting up a server. You have the open source applications bundled with NAS firwmare upgrades or something like that.

And then you can use this also to promote wider adoption of IPv6 and reduce dependence on IPv4 firewalls. Tell users that if their ISPs supported something called 'IPv6' then they could get much better performance and higher levels of security.

But on private networks it should be a IT admin's worse nightmare. It should implement every trick under the book to get that IPv6 tunnel out in the world. Tunneling over TCP on port 443, for example, can easily circumvent many of the most restrictive firewall restrictions. Hell, even running in degraded mode by tunneling over DNS should be attempted! By making it exceptionally difficult to stop, you make it very easy to get working.

NLUUG: The Open Web

Posted Nov 5, 2009 17:36 UTC (Thu) by foom (subscriber, #14868) [Link]

And once again, the open source crowd misses the point. Flash is popular far more because of adobes flash authoring tools than because of flash technology.

...Really? Cause from where I'm sitting, >90% of the interesting uses of flash is simply because it's the easiest cross-platform way to display video.

Then there's flash ads, of course.

And then there's the one or two sites out there which are *actually* authored in flash. But that certainly doesn't seem like a particularly popular option...

NLUUG: The Open Web

Posted Nov 6, 2009 16:40 UTC (Fri) by jospoortvliet (subscriber, #33164) [Link]

Oooh, that's not true. I know plenty of sites, often by artists
(musicians, painters, etc) done in flash. HTML is far to limiting for
them. And plenty companies use it all around as well.

Unfortunate, but true. Yes, I'd rather have the web technologies like
(X)html and SVG properly support this stuff AND have the proper tools to
build pages with it - but currently, the flash tools are just better.

NLUUG: The Open Web

Posted Nov 6, 2009 19:30 UTC (Fri) by martinfick (subscriber, #4455) [Link]

Plenty of companies do use FLASH, and these companies usually do not have usable web sites, the have "flashy" web displays (how appropriately named flash is). The only major content provider that uses flash is youtube. The rest is marketing hype.

Product manufacturer X hires a web site designer (who is used to using flash tools and thinks they are therefore easier to use than anything else), to design a fancy looking website that satisfies the VP of the small product line company. These companies don't even know that their website is unusable, because their website is not used. But at least the VP thinks they do not have a 90s era web presence.

Their website is not part of their business model, others sell their products to consumers. If the website were part of their business model, it would not likely use flash (except for the adds, since they don't really care if that is a hurdle to their users, they get paid anyway). Even on windows, users need to install flash to get it to work and that is one barrier too high for most businesses to tolerate. Web sites which need to be used, are cross platform and mostly open.

NLUUG: The Open Web

Posted Nov 6, 2009 20:27 UTC (Fri) by ufa (subscriber, #56005) [Link]

Everytime someone buid a site in flash, a kitten dies :P

NLUUG: The Open Web

Posted Nov 6, 2009 9:04 UTC (Fri) by njs (guest, #40338) [Link]

> Marketing an "open web" will not succeed. Creating a fantastic content authoring tool that targets HTML/JavaScript/css/svg is the only way to take a significant chunk of the market from adobe.

Perhaps they are marketing to content authoring tool developers.