Solid: a new way to handle data on the web

By Jake Edge
October 31, 2018

The development of the web was a huge "sea change" in the history of the internet. The web is what brought the masses to this huge worldwide network—for good or ill. It is unlikely that Tim Berners-Lee foresaw all of that when he came up with HTTP and HTML as part of his work at CERN, but he has been in a prime spot to watch the web unfold since 1989. His latest project, Solid, is meant to allow users to claim authority over the personal data that they provide to various internet giants.

Berners-Lee announced Solid in a post on Medium in late September. In it, he noted that despite "all the good we've achieved, the web has evolved into an engine of inequity and division; swayed by powerful forces who use it for their own agendas". Part of what he is decrying is enabled by the position of power held by companies that essentially use the data they gather in ways that run directly counter to the interests of those they gather it from. "Solid is how we evolve the web in order to restore balance — by giving every one of us complete control over data, personal or not, in a revolutionary way."

Users' data will be stored in a Solid "pod" (sometimes "personal online data store" or POD) that can reside anywhere on the internet. Since Solid deliberately sets out to build on the existing web, it should not be a surprise that URLs, along with Uniform Resource Identifiers (URIs), are used to identify pods and specific objects within them. Pods also provide one place for businesses, including Inrupt, which was co-founded by Berners-Lee, to provide services for Solid. As he noted in his post, people are willing to pay companies like Dropbox for storage; hosting Solid pods would be a similar opportunity for Inrupt and others.

The vision is that users will be able to grant applications and other users read or read-write access to selected data in their pod. That pod can be hosted "in the cloud" or locally; users can install the Solid Server to host pods on their own system or get a pod from a hosting provider. Applications will access data that is provided to them by following "typed" links; this is called "Linked Data" in the Solid documentation. The example given is that a comment made by one person on another's photo could be represented as:

    <https://mypod.solid/comments/36756>
	<http://www.w3.org/ns/oa#hasTarget>
	    <https://yourpod.solid/photos/beach>.

The link type (on line 2) uses the Web Annotation Ontology to specify what kind of link is being made. The Linked Data is described using the Resource Description Framework (RDF) Turtle notation, which consists of three items: a subject, a predicate (or link type), and an object. Each element of this triple is a URI and a Turtle statement is terminated with a ".".

Solid is meant to provide ways to control access to a user's data, which implies a need for identities and authentication. The Solid specification GitHub repository provides links to all of the relevant pieces that make up Solid. Identities in Solid are provided using WebID URIs, while authentication is done using WebID-TLS. Alternative authentication methods will be supported as well; WebID-OIDC support is currently under development and other options are being explored. Web Access Control (WAC) will be used for access-control lists (ACLs) on the data in Solid pods.

One thing that is notably missing from any of the Solid marketing and documentation is any mention of encryption. If users are to turn over all of their content to a pod provider, they will likely want to know that the data is protected from both attackers and from the provider itself. That is a difficult problem to solve, however, since various different entities (applications and users) will have access to different parts of the pod. That implies that the data is either not encrypted, is decrypted by the server, or that each entity will get a key of some sort to decrypt the data it gets. There is a GitHub issue asking about encryption but, other than that, a seemingly important feature is not even discussed.

The Solid server is Node.js-based. Its installation instructions start with the always worrisome "curl | sudo" pattern. It can be run either directly from the command line or in a container using Docker. It implements many of the features envisioned for Solid and is presumably the server being used by the two existing pod providers.

There is something of a chicken-and-egg problem for Solid, though. In order for it to be adopted widely, it is going to need lots of applications that use the Solid model. Getting people to write those applications (or to add Solid support to existing applications) may be difficult without a fairly sizable user base. The ability to break that logjam will be a major factor in determining Solid's level of success.

The Solid web site provides a "Make a Solid app on your lunch break" tutorial. It uses jQuery to create a web page that handles authentication and shows information from the logged-in person's WebID Profile, including their name and the names of their friends. The friends' names are loaded from the WebID Profile on each of the friends' pods; clicking on a friend's name will load the friend's profile into the page, which is meant to show how Linked Data makes it easy to find and display data from multiple pods. The tutorial uses jQuery for simplicity, but the documentation describes creating Solid applications using the more full-featured AngularJS framework; support for other frameworks is planned.

The "semantic web" has been a longtime dream of the World Wide Web Consortium (W3C) and Berners-Lee in particular, though it has always seemed to suffer from a low adoption rate. Solid is trying to take the semantic web one step further by placing the handling of the actual content directly under the control of users. Whether that level of control is compelling enough to get over the hurdles that both the semantic web and Solid impose remains to be seen—there is certainly reason to be skeptical.

The vision that is promoted by Solid and its backers is attractive, but most consumers have shown a marked disinterest in what happens to their personal data, especially if there is any kind of cost—not just monetary, but time or inconvenience as well. Storing information like photos, contacts, videos, and so on, in ways that allows others to interact with them in various ways, sounds great—in theory. But the cost is that users will need to be cognizant of the kinds of permissions they grant and dodgy applications (and "friends") will undoubtedly try to tempt them into going astray, which leaves them in the same situation they are already in.

The incentives are too high, at least for now, for companies and others to not find ways to route around this kind of access control. Solid seems like a ... well ... solid idea, though it may be a bit overhyped (and an encryption story is needed); it is a little hard to see it gaining much traction, however. It would be nice to be wrong about that.

Solid: a new way to handle data on the web

Posted Oct 31, 2018 18:14 UTC (Wed) by cwitty (guest, #4600) [Link] (23 responses)

If I understand correctly, part of the Solid vision is that people would be responsible for hosting their own comments on (for example) LWN articles.

That seems extremely unlikely to work well long-term; I would expect that between people forgetting to pay their hosting fees (or running out of money), or losing access to their domain name, or whatever, the half-life of article comments would be something like 2 to 4 years; going back to a 10-year-old article would have most of the comments missing (and the rest wouldn't make sense, because they would be responding to missing comments).

I much prefer the current situation, where I confidently expect that the comment threads on LWN articles will be hosted as long as the article itself is (which I hope is a very long time!)

Solid: a new way to handle data on the web

Posted Oct 31, 2018 18:54 UTC (Wed) by alogghe (subscriber, #6661) [Link] (13 responses)

Well think of the long term value of other types of comments?

Like the reviews on Yelp or Amazon or Netflix (now all deleted by Netflix) that people have done.

People frequently put a lot of their personal or professional knowledge into these reviews. Thoughtful, well reasoned and even kept up to date by some people.

All tremendous value to the web but all handled by companies with frequently highly toxic agendas.

Sometimes they bury negative or positive reviews under commercial spam, sometimes they neglect or just delete everyone's work for no real reason other then a business model change.

If I say X about a product or movie, it's what I say. Not what Amazon or Netflix says about it. I'd like to host it and provide that value to the web. Companies could aggregate those reviews from real people.

Also "hosting fees".

Why is it we think these hosting fees are expensive whatsoever? This is truly trivial amount of bytes for large amounts of what people care about.

Even if someone likes to post videos all the time... the storage costs and replication costs are cheap. If there were standard ways to cache and replicate it via say IPFS (its essentially a way to protocolize the CDN networks we use today) why would this bill be expensive? It would seem like there are many paths to it being cheap or free even without the data leeching that occurs today.

The complexity of managed services comes about largely because of a lack of standardization on many of the inputs and outputs of these services not from the information itself being bulky or complex.

Solid: a new way to handle data on the web

Posted Oct 31, 2018 22:04 UTC (Wed) by Beolach (guest, #77384) [Link] (2 responses)

Also "hosting fees".
Why is it we think these hosting fees are expensive whatsoever? This is truly trivial amount of bytes for large amounts of what people care about.

I would expect that if this (or in order for this to) catches on a basic amount of Solid pod hosting would be included in essentially all ISP plans - including cell phone data plans. And it should be easy to migrate the pods from one provider to another.

I'm a big fan of decentralization, in large part because it encourages competition, and competition is the best solution to most problems on the internet IMO - especially fees/prices/costs.

Solid: a new way to handle data on the web

Posted Nov 4, 2018 9:33 UTC (Sun) by jospoortvliet (guest, #33164) [Link]

I like what storj.io and some similar projects are building ;-)

Solid: a new way to handle data on the web

Posted Nov 10, 2018 2:21 UTC (Sat) by flussence (guest, #85566) [Link]

I think you give present-day ISPs a bit too much credit. In too many parts of the world it's hard to find one that'll even give you the bare minimum of an unadulterated IPv4-only connection. The idea of giving end users any agency is completely alien to them.

Solid: a new way to handle data on the web

Posted Oct 31, 2018 22:41 UTC (Wed) by derobert (subscriber, #89569) [Link] (9 responses)

I apologize in advance that I've only read the article — not followed all the linked information — but it really seems like you wind up with either (a) the service being commented on (LWN, Netflix, whatever) can delete content when they no longer find it valuable or (b) they can't, thus all comments are spam. Or you wind up with some distributed reputation system where it'd be really hard not to make filter bubbles even worse (or, alternatively, you trust the service to tell you who you should listen to, getting right back to (a)).

Solid: a new way to handle data on the web

Posted Nov 1, 2018 3:54 UTC (Thu) by k8to (guest, #15413) [Link] (8 responses)

The commented-on items could potentially "delink" as in change their service so that the comments are not shown.

If they are truly valuable, someone could theoretically re-create the order of the comments, although in practice this seems rather implausible to me.

Solid: a new way to handle data on the web

Posted Nov 1, 2018 7:34 UTC (Thu) by derobert (subscriber, #89569) [Link] (7 responses)

This seems a rather fundamental problem to sort-of handwave away. The open protocols, open interaction model isn't just dying of corporate greed, it has been greatly weakened by spam and other unwanted content. Usenet was largely killed by spam. Email has been kept alive — barely — by herculean effort fighting spam, without which email is unusable; even with it, people's behavior changed to make it much harder to contact them (e.g., by not posting their email address). Open instant messaging is largely dead, and spam is at least one of the reasons. There are of course other reasons — such systems are often more complex to both design and use, deploying new features is harder, different implementations interacting invariably give rise to weirdness, etc.

Besides just spam, there are other forms of content a given user doesn't want, from trolling to being a jerk. The open systems haven't really had much of an answer to that, at least after they outgrew sysadmins contacting each other to resolve issues. Even if you filter an email address, it's trivial, free, and fully-automatable for a troll/jerk/etc. to create a new one and evade your filter. Non-open systems have a better answer to that: they can make it more difficult to create an account, and they can disable your account. And the larger that service becomes, the more of a deterrent that is; consider how much it'd suck to have your Google account disabled (presuming you're not one of the folks who avoids Google; I believe the same is true of Facebook, but I avoid them.)

So, getting back on topic, it sounds like its trivial to create a pod, which is also an identity. Spammers will surely create them and plenty of spam comments. If comments are to remain useful, site administrators will need a good way to stop that. The trend on forums has been to make sign-up harder, else you get spam bots signing up & posting (which quickly overwhelms the moderators).

Back in the mid-to-late 90s through the early 2000s, the Web had a lot of sins — terrible graphic design, <blink>, <marquee>, <bgsound>, the list goes on. But what it also had was tons of pages with readily available contact information, authors that welcomed random strangers to contact them about their pages, and — hard as it is to believe now — comment sections where anyone could comment, point to their own pages, etc., without any authentication/account/etc whatsoever, just by typing a pressing a button. And it worked! Some of that is gone for good, the Internet will never again be a small town. But it'd be really awesome if some of that can make a comeback.

(I spent some more time reading through https://solid.inrupt.com/ and looking at their docs and forum... and... well... I'm not sure what they're doing that's new. Maybe I'm just missing the point.)

(Obviously, these things aren't fully dead. I know there are still active Usenet groups. I run a few XMPP servers. But they are tiny compared to their height, and to the closed systems that replaced them.)

Solid: a new way to handle data on the web

Posted Nov 5, 2018 11:13 UTC (Mon) by mjthayer (guest, #39183) [Link] (6 responses)

Regarding spam, the infamous old idea of the e-mail postage stamp could help. You pay a few cents to send a message if you are not on a white list, which the recipient could choose to refund (or perhaps not: it might help finance the service). You would most likely be white-listed after the first message of course.

I recently discovered that a company contact page with no e-mail address listed, just a painful maze of contact forms, did have a simple fax number. I happily paid the twenty (Euro) cents which my e-mail provider charges to send a fax, and got a speedy answer.

Solid: a new way to handle data on the web

Posted Nov 5, 2018 16:44 UTC (Mon) by derobert (subscriber, #89569) [Link] (5 responses)

I can't help but to observe you are replacing one problem with a potentially much worse problem: now all we need to solve is international microtransactions!

(Also, the percentage of my postal mail that is junk argues it might not work anyway, though at least the volume there is manageable. The cost does at least hold down the volume.)

Solid: a new way to handle data on the web

Posted Nov 8, 2018 9:53 UTC (Thu) by jezuch (subscriber, #52988) [Link] (4 responses)

With regards to microtransactions, the idea came up about 20 years ago but we chose to go with surveillance economy instead. Don't know if this means that the problem is hard or it's because of the IQ-lowering power of "free!".

As for the spam, looks like the mass-mailers have special deals with the postal service. What is the probability that spammers will have similar special deals in the world we're discussing? :)

Solid: a new way to handle data on the web

Posted Nov 8, 2018 10:58 UTC (Thu) by anselm (subscriber, #2796) [Link] (2 responses)

Don't know if this means that the problem is hard or it's because of the IQ-lowering power of "free!".

It's easier to collect large sums of money from a few customers than tiny sums of money from very many customers.

Facebook, for example, would be a much nicer place if every user paid, say, $1/month for the privilege of using the service. At 2 billion users, that would give Facebook revenue of $24 billion/year, which is more than the GDP of various small-to-midsize countries, and without the need to engage in sleazy data-sharing shenanigans. Think of the great things Facebook could do for its users with that sort of money, and there should even be the odd million left over to pay for the upkeep of Zuckerberg's mansion.

The reason that doesn't happen is that we (the users of Facebook) would leave the platform in outrage if Facebook were to announce that starting from 2019 they'd charge the monthly equivalent of one-fifth of the price of a cup of fancy coffee in order to operate a non-privacy-invading, user-centered service. We're so used to “free” that the idea of having to pay even a nominal amount of actual money turns us off. Facebook understands this very well, which is why they prefer to have advertisers as paying customers rather than users.

Solid: a new way to handle data on the web

Posted Nov 8, 2018 16:02 UTC (Thu) by nybble41 (subscriber, #55106) [Link] (1 responses)

> The reason that doesn't happen is that we (the users of Facebook) would leave the platform in outrage if Facebook were to announce that starting from 2019 they'd charge the monthly equivalent of one-fifth of the price of a cup of fancy coffee in order to operate a non-privacy-invading, user-centered service.

It's more like: (a) most or all of a $1 payment would be wasted on transaction costs (the microtransaction problem); (b) some potential users are not in a position to make online payments of any size (the "unbanked" problem); and (c) no one would actually believe that they weren't monetizing data about their users *in addition to* the $1/mo. fee. We've been down this road before: users pay a lot more than $1/mo. for cable or satellite TV yet still are bombarded with ads.

Solid: a new way to handle data on the web

Posted Nov 8, 2018 18:44 UTC (Thu) by farnz (subscriber, #17727) [Link]

Plus see Facebook's annual revenue and user counts as published with the SEC. If we take 2012 as the year in which Facebook reached 1 billion users (somewhere between Q2 and Q3 by that chart), and 2017 as when it reached 2 billion users (between Q1 2017 and Q2 2017 from that dataset). Then compare revenue - in 2012, Facebook took in around $5 billion per year - so $5 per user per year - while by 2017, it was taking in $40 billion - so $20 per user per year.

If Facebook wanted $2/month per user in 2012 for tracking-free users, would people feel that was reasonable, given that ads were only making around $0.50 per month per user? If not, would they continue to increase the amount they paid month-on-month to the $2 per user per month that Facebook takes in today?

Further, these are big picture numbers - the value distribution will not be even, so there will be users who are currently worth much more than $2 per month to Facebook, and users in countries where $2/month would make Facebook unaffordable. Would Facebook be as financially successful if (say) US residents were asked to pay $10/month, while Thais were asked to pay $0.10 per month?

Solid: a new way to handle data on the web

Posted Nov 8, 2018 17:26 UTC (Thu) by derobert (subscriber, #89569) [Link]

It's hard to do even just inside one country. Many people tried and failed. It took at least a decade to build the advertising model the Internet now largely uses; that wasn't easy either. Remember we're not on the first or even second attempt — and we started from an existing, working advertising model (print media).

It was easier than microtransactions, apparently.

Solid: a new way to handle data on the web

Posted Oct 31, 2018 20:55 UTC (Wed) by madscientist (subscriber, #16861) [Link] (7 responses)

If the service can access comments etc. to display them then clearly they can cache them, either for long-term storage or for monetizing. There's no reason that LWN can't obtain a copy of your comment from your pod when you post it and cache it on its own servers forever.

My understanding is that Solid is mainly an attempt to avoid vendor lock-in like we have now with Facebook etc.: I own my comments and other works online and while other people, or companies, can read them (and cache them and monetize them just as they can now, assuming they have permissions), they can't bar me from accessing them or prevent me from easily distributing these same works to other places if I wanted to.

Solid: a new way to handle data on the web

Posted Oct 31, 2018 21:08 UTC (Wed) by droundy (subscriber, #4559) [Link] (6 responses)

I imagine the protection against long term storage of your data is copyright. If I retain copyright of my comments and that copyright claim is communicated via the Solid then LWN would be taking a risk by storing and publishing my comment.

I'm not saying I like this, but it's my guess as to how Solid would try to allow users to retain control of their data.

Solid: a new way to handle data on the web

Posted Oct 31, 2018 21:57 UTC (Wed) by madscientist (subscriber, #16861) [Link]

The original concern was that comments could be lost if users' pods were offline for whatever reason: that's easily solved by caching content on the server.

This comment is about something different: users using copyright to proactively remove content from a service. For services that do decide to cache long-term it is exactly the same situation we have today so Solid makes no difference.

There are other possibilities of course: for example the service could keep a copy of the content and check the pod when the content is to be displayed (not every time, but maybe if it's not been accessed in a day or week or month). If the pod couldn't be reached, or could be and the content is still available, then the cached version could be displayed. If the pod is reachable but the content is no longer available then the service would assume the content is no longer permitted and refuse to show the cached version. That would be up to the service I expect... could be a lot of overhead. Else you'd fall back to whatever you do now to ask that content be removed from a site.

It would be interesting to know whether services could attempt to offload copyright takedown requests somehow: if you find a photo you own published by someone else on Facebook and want it taken down, would Facebook be able to turn around and point you to the pod serving that image and tell you that you have to get them to stop serving it? Seems unlikely but it's interesting.

Solid: a new way to handle data on the web

Posted Nov 1, 2018 0:38 UTC (Thu) by zlynx (guest, #2285) [Link] (3 responses)

I'm not sure about LWN but many sites have a license agreement when you create a user account to post comments. As well as agreeing to not post anything illegal or abuse other users, you usually agree to grant a permanent, irrevocable license for the use of any user content that you post to the site.

And sites that don't have an agreement like that are probably OK because of the implied legal contract you entered into by posting a comment. It would make no sense to post a comment and at the same time deny a copyright use license on it. So obviously there was an implied agreement.

So storing copies is not a legal risk regarding copyright. That's already handled.

Solid: a new way to handle data on the web

Posted Nov 1, 2018 1:42 UTC (Thu) by droundy (subscriber, #4559) [Link] (2 responses)

It's currently already handled, but Solid is proposing a new way to do things and I expect that this is something they'd like to change. I don't see where the benefit with Solid would be if the same companies could do all the same things with your data, even though you had moved it to your open Solid server.

Solid: a new way to handle data on the web

Posted Nov 1, 2018 1:49 UTC (Thu) by zlynx (guest, #2285) [Link] (1 responses)

If I was running a web site that took user content, I wouldn't switch to Solid. I don't see any benefit to it.

What's the plus for the sites? Not having to host big things like video? I can see that. But on the down side you lose the information if the external site goes away. Many sites don't accept external IMG links anymore for that reason. Disk is cheap and if they host it themselves they'll never have a comments page full of holes. Or goat.se links.

Solid: a new way to handle data on the web

Posted Nov 1, 2018 11:36 UTC (Thu) by excors (subscriber, #95769) [Link]

And if the sites don't host the content themselves, privacy-conscious users will complain (rightly) that their browsing habits are being revealed to a thousand third-party servers every time they load a page that's full of user comments and profile pictures etc. Plus it'd be terrible for network performance, since that single page load would need a thousand DNS lookups and a thousand TCP connections.

(Nowadays you might have only a dozen tracking scripts per page, and there's some incentive for them to optimise their performance since site owners will prefer to choose ones that don't hurt page load times (as tracked by another script), so it's not nearly as bad as it could be.)

The sites would have to download and cache all the external content, so they can serve it themselves (for privacy) and can minimise latency on first page load, but that seems incompatible with the goals of Solid - read-write data is a mess if accesses don't all go through a centralised service to ensure consistency, and access control doesn't really work if these other sites are downloading and sharing your data themselves. I don't understand how it can be at all practical.

Solid: a new way to handle data on the web

Posted Nov 6, 2018 6:00 UTC (Tue) by dirtyepic (guest, #30178) [Link]

Publically made comments are generally be considered public discourse and therefore not eligible for copyright in and of themselves.

Solid: a new way to handle data on the web

Posted Nov 1, 2018 2:36 UTC (Thu) by bferrell (subscriber, #624) [Link]

...part of the Solid vision is that people would be responsible for hosting their own comments

Yeah... Right. Companies... Let alone, people, don't do ANYTHING for themselves.

Email (with web access) - CHECK O365
Document editing/word processing - CHECK O365
Number crunching (spreadsheets) - CHECK O365
maintaining customer records - CHECK SFDC
internal chat - CHECK SLACK/O365

All outsourced

And somehow Berners-Lee thinks that people who can't make the VCR (those that have them) stop blinking 1200. Will do this?

Or will centralized services rise to do it for them? Shifting the data from one set of behemoths to others?

Moving targets anyone?

Solid: a new way to handle data on the web

Posted Nov 1, 2018 4:08 UTC (Thu) by 07dosa (guest, #71402) [Link]

I think people are somewhat overreacting.

SOLID is more like distributed PaaS, where any SaaS application can build up upon. SaaS companies can provide only software, not storage and anything. This would make the life-cycle of SaaS applications much shorter, since companies only need to serve their application. This might turn web into an archive of applications.

But who knows what's gonna happen.

Solid: a new way to handle data on the web

Posted Nov 1, 2018 4:08 UTC (Thu) by flussence (guest, #85566) [Link] (6 responses)

I have very little faith in anything out of the W3C nowadays. As hinted at by the article, they're aren't all too cognisant of security and safety issues on the modern web.

Let standards like this be driven by the people who have to *implement* it, who'll be using it, and who'll be stuck pulling people out of the wreckage if it's engineered wrong — we don't need a repeat of ActivityPub.

Solid: a new way to handle data on the web

Posted Nov 1, 2018 12:23 UTC (Thu) by excors (subscriber, #95769) [Link] (5 responses)

I do like how their authentication system is seemingly based around <keygen>, which was known to be a badly designed and poorly supported feature since it was first documented (by reverse-engineering the implementations) in 2009 - see e.g. https://blog.whatwg.org/this-week-in-html5-episode-35, http://lists.w3.org/Archives/Public/public-html/2009Sep/0...

I think some groups in the W3C are doing useful stuff, but others (including Tim Berners-Lee) got distracted by the Semantic Web idea about twenty years ago and still haven't achieved anything significant with it.

Solid: a new way to handle data on the web

Posted Nov 1, 2018 14:31 UTC (Thu) by k8to (guest, #15413) [Link] (3 responses)

At this point I think the semantic web is just a bad idea. I've looked into it and tried to think about what you could do with the idea potentially, and I'm empty handed.

Solid: a new way to handle data on the web

Posted Nov 1, 2018 16:36 UTC (Thu) by burki99 (subscriber, #17149) [Link] (1 responses)

I don't know where for you starts and ends the Semantic Web. To me, https://query.wikidata.org/ is very useful (I can query for properties of persons and places through certain identifiers encoded into URIs).

Solid: a new way to handle data on the web

Posted Nov 1, 2018 18:17 UTC (Thu) by excors (subscriber, #95769) [Link]

I think that's missing the Web part of the Semantic Web - decentralisation seems to have always been a core part of the vision, but Wikidata looks like primarily a centralised database. The Semantic Web technologies (RDF etc) can still be useful for centralised databases, in the same way you can write an offline desktop application in HTML+JS, but then you suffer from all their technical compromises without gaining the benefits they were aiming for.

Solid: a new way to handle data on the web

Posted Nov 1, 2018 17:35 UTC (Thu) by bartoc (guest, #124262) [Link]

Yeah I keep looking into it and I don't find any good reference software. It seems to be a mess of huge data middle-ware "solutions" and is lacking stuff like a well-maintained library for doing just parsing or just querying.

Solid: a new way to handle data on the web

Posted Nov 2, 2018 12:13 UTC (Fri) by Lennie (subscriber, #49641) [Link]

"and still haven't achieved anything significant with it."

Well, this ties into that because it used Linked Data.

So if people do use it, then it will have done something significant.

Grognard alert

Posted Nov 1, 2018 23:13 UTC (Thu) by ecree (guest, #95790) [Link] (3 responses)

As long as it remains possible for an end-user to run their own web, and other Internet, servers, I will continue to have no sympathy for people who complain that the internet giants have too much power over their stuff. No-one's forcing you to put your photos on Instagram or imgur, your office on Google Docs or O365 and your social life on Facebook or Twitter. The old-fashioned technologies still work just fine; you have an IP address. Use it.

Grognard alert

Posted Nov 2, 2018 10:59 UTC (Fri) by spaetz (guest, #32870) [Link] (2 responses)

> As long as it remains possible for an end-user to run their own web, and other Internet, servers, I will continue to have no sympathy for people who complain that the internet giants have too much power over their stuff.

As soon as you are a parent and all other parents organize class stuff via WhatsApp, or you are a researcher having to learn about conferences on Facebook pages (exclusively), or your colleagues insist on sharing things with you on Dropbox, we can talk about this again. It is definitely possible to avoid these, but there are social and opportunity costs.

Grognard alert

Posted Nov 2, 2018 12:15 UTC (Fri) by Lennie (subscriber, #49641) [Link] (1 responses)

It's called network effect and more and more people are aware of the privacy, etc. implications of using Facebook/WhatsApp, etc. so it should become easier to convince people to use something else.

Grognard alert

Posted Nov 2, 2018 22:10 UTC (Fri) by NAR (subscriber, #1313) [Link]

I heard people moving from Facebook to vkontakt(?) because Facebook did not let them share nude pictures of themselves. They might be aware of the privacy implications, but definitely don't care...

Solid: a new way to handle data on the web

Posted Nov 2, 2018 12:18 UTC (Fri) by Lennie (subscriber, #49641) [Link]

This all sounds a lot like what unhosted/remotestorage was already doing, but maybe with his name he can give the idea more traction.

For those that don't know about it:

http://unhosted.org/

https://remotestorage.io/

Even work was done on creating a standard for it:

https://tools.ietf.org/html/draft-dejong-remotestorage-11

Solid: a new way to handle data on the web

Posted Nov 2, 2018 14:54 UTC (Fri) by MarcB (subscriber, #101804) [Link] (3 responses)

Somehow this misses the core problem. What makes Facebook, Google and others so powerful is not their storage of content, but their control over how/if/when the content can be found and is presented.

Facebook's, and even Youtube's, main value is not the storage of data, but the presentation of it, the social network around it, the search functions and last but not least the monetization options.
If Facebook were to support Solid as data storage, what exactly would change? If Google were to extend its search to PODs, what would change? They could still decide which content is displayed prominently, based on algorithms or payment, and they could still make content invisible if they do not like it or are forced to do so.

And what if Facebook and Google ignore PODs? Either someone else would take over the role of gatekeeper - perhaps initially a lot of competing gatekeepers - but eventually, we would observe the same "consolidation" (the network effect is strong, after all), and face the exact same problem.

In another comment someone mentioned Netflix' removal of reviews as an example of a central power removing conent. But this just proves my point: The reviews were never placed prominently by Netflix; they were even Desktop only, i.e. invisible to everyone using a mobile device or a TV-App. Presumably, that is ~85% of users.
In fact, the first time I even heard of them was when their deletion was announced. And this happened to a lot of people. For them, the reviews essentially never existed, even before they were removed.

Solid: a new way to handle data on the web

Posted Nov 4, 2018 19:35 UTC (Sun) by massimiliano (subscriber, #3048) [Link] (2 responses)

If Facebook were to support Solid as data storage, what exactly would change? If Google were to extend its search to PODs, what would change? They could still decide which content is displayed prominently, based on algorithms or payment, and they could still make content invisible if they do not like it or are forced to do so.

What would change is that somebody else could build an alternative search engine on the same content, with the same rights on the content as Facebook and Google.

Right now this is simply not possible: the data is locked up, and indexing from outside is infeasible.

Solid: a new way to handle data on the web

Posted Nov 5, 2018 8:36 UTC (Mon) by gfernandes (subscriber, #119910) [Link]

Well,I don't know.

Sure, the data is locked up. But the search only works _because_ the data is in the form and place it is in.

Change format and/or change location and suddenly indexing and aggregation become more complicated, fragile and error prone.

So, i don't know what moving to Pods would really give in terms of competitive search engines or providers.

Solid: a new way to handle data on the web

Posted Nov 5, 2018 14:35 UTC (Mon) by MarcB (subscriber, #101804) [Link]

> What would change is that somebody else could build an alternative search engine on the same content, with the same rights on the content as Facebook and Google.

Yes, and eventually one of those alternative search engines would "win" and become the most popular, so that only data shown by it would have prime visibility. Content creators will want ways to make money and said search engine would implement ways to make money.

We would be exactly where we are now, only data storage would be more decentralized (at least for some time, before the new search engine starts offering nice, all-inclusive solutions).

Centralized data storage is not the main problem of the current internet. It is merely a byproduct of the centralized data access portals that are the real problem. I do not see a technological solution for this. The network effect is incredibly strong and the legal as well as social frameworks make running such a service non-trivial.

Solid: a new way to handle data on the web

Posted Nov 8, 2018 23:05 UTC (Thu) by KjetilK (guest, #128335) [Link]

Thanks a lot for the review. I'm one of the core devs.

You can blame the curl | sudo on me, but I will pass the blame onto Nodesource. It is taken from here:
https://github.com/nodesource/distributions/blob/master/R...
I personally think it is really broken, I much prefer the Linux distro model, where security fixes are backported, and the platform remains stable for a long time, and only the updates you actually need and are able to grok are updated in between. The Node.js community doesn't play very nicely with that though, so that's what we've got. You are of course free to install Node by other means, including downloading the script first, look at what it does, and then run it with root privs.

I think the point of having encrypted PODs is important. I have made a longer comment in our forum:
https://forum.solidproject.org/t/encrypted-pod-is-solid-d...
but the TL;DR is that in the short term, you should start putting up your POD on hardware that you trust. Encrypted PODs are possible, but it involves some tradeoffs and quite a lot of work.

home server revolution

Posted Nov 16, 2018 9:30 UTC (Fri) by Garak (guest, #99377) [Link]

"Solid is how we evolve the web in order to restore balance — by giving every one of us complete control over data, personal or not, in a revolutionary way"

I've always thought that home f(l)oss-based servers were the obvious answer to this issue, reducing as much as possible unnecessary middlepersons from the equation. If we didn't have a situation where most people were prohibited from operating servers with their mainstream ISP contracts, I predict we would see the profiteers of huge centralized services profiteering much less from their access to and control over our data.