European funding available for interesting development projects

Nepomuk was part of a much larger project doing research in the whole 'semantic web' stuff that researchers continue to fawn over but which failed to provide enough tangible benefits from what I can tell. The concept is simple enough, if you're not familiar:
It's about relations with 3 elements, triplets: object1 - relation - object2. Think "Greg HusbandOf Pete". Normally, you'd just have relationship descriptions with 2 elements: "greg husband" but that obviously isn't helpful. Now when you search for Greg, you'd see Pete in the results.

So compared to plain full-text indexing, which is an entirely different thing, and would just show you documents with "Greg" in it when you search for that, there is a potential huge benefit of course. And they weren't too stupid - you are of course not going to tag all your files yourself. But you can automate extracting such data a lot: a file that came from an email can have these tags:
"file.txt ReceivedFrom Greg"
"file.txt AttachedTo [UniqueEmailID]"

So that when you look at the file you can open the email, or when you search for Greg you see the last mails you got from him and the files he send to you and that you saved locally.

In short, cool stuff.

So Nepomuk was developed to do that for KDE as a kind of proof-of-concept as part of the larger research project. It was using a powerful triplet-store-and-search engine. Which ate a lot of CPU cycles and was not so efficient.

But of course, you get most of what you need by just doing a full-text search. So they quickly decided they had to add basic full-text-search to the thing. Something that triplet-storage wasn't particularly good at, to say the least. So it got even slower and inefficient.

Now it did show the concept in reality. But it was never really fast enough to work well, and it was (sadly) never really integrated in other applications to be really useful - so the examples I gave above were never really implemented.

Generally, to get to that point, you have to reach a certain userbase and compelling-ness, and good API's etc etc - 1 guy being paid to work on a research project didn't get it to that point.

So, to summarize, Nepomuk was 1 big, powerful, triplet-capable database that was meant to hold everything: from full-text index to Amarok's music database to your email database to triplet relations to everything else. It was only good at those triplets, even though it never really took advantage of that capability in the first place, and it got too big and would corrupt sometimes and was slow and unstable...

So when the funding ran out, different ppl worked on it for a long time, trying to make it perform better. They got, well, somewhere, pretty much made the pig fly, but the tech was inherently too powerful to be efficient at the 'simple' use case it had to do most of the time: file name and full text search.

So Baloo was written in a very short time with a 100% different approach:
* 1 database for each purpose.
* Want triplets? have your triplet database, as app - store and access your stuff.
* Want SQLite for your contacts? Have your contacts database, store and access your stuff.
* Want a MySQL db for email? Have it...

So Baloo simplifies and separates things. I dunno how far it goes in offering apps ways to use it to store things in their own DB etc, being just an easy 'do your own database' back-end, or not - it might just focus on file indexing. It's been a while since I talked to any of its developers, but I know back then there were ambitions to do more. Anyhow, it is fast and nimble and never gave me any issues, so clearly the design is appropriate for its purpose. Which is to be expected, given the (quite brilliant) person who fought for 2-3 years to make Nepomuk fast (Vishesh Honda, hope I write his name correct from memory) is the one who designed Baloo. He knew the problem he was trying to solve ;-)

By the way, Akonadi went through a very comparable road, though sans the research part:
* have separate apps each with their own DB but a need to talk to each other (email, contacts, calendar etc) - requiring eg the contacts app to be running to see contacts in the email app, that's NOT efficient of course
* design a back-end that unifies them all so there is one huge database they can all talk to, super abstracted away so the db doesn't know what it is storing; managed by a daemon which has plugins for things like checking for mail, adding calendar items, inserting or retrieving blog posts etc
* discover that one HUGE database is not efficient at the amount and all those different types of data. Eg db doesn't know what it stores, so if you need a calendar applet that just wants to show the appointments of this month, you have to get all calendars: the db can't give you a date range as it doesn't know what a calendar item is.
* spend a few more years trying to optimize things, breaking down the clear separation for performance reasons and creating a complicated mess
* throw way that concept and start to write a new one where instead of one database and a daemon managing it that apps connect to, each type of data goes in its own database and the 'interface' is just a library apps load to manipulate the database (or retrieve new data like mail etc) which, itself, takes care of concurrency or the library does it with some IPC.

I'm simplifying a lot here, and I'm no developer - that's just what the devs talked about. But perhaps it helps clarify the baloo vs nepomuk thing, which wasn't really "it was EU funded and thrown away right after".