LWN.net Logo

The notmuch mail client

The notmuch mail client

Posted Nov 18, 2009 12:20 UTC (Wed) by endecotp (guest, #36428)
Parent article: The notmuch mail client

Nice to see that there is innovation going on in email, still.

A few years ago I had some spare time and decided to "scratch an itch" by implementing my own "ideal" email system. My requirements were:

- Accessible via a web client, at least as a secondary interface.
- No "lock in" to a particular front end.
- Fast search and message categorisation.

I ended up by putting all of the messages in a PostgreSQL database, with full text indexing, and then writing an IMAP interface to it. I have "virtual folders" that are implemented by SQL queries, with new ones created in response to IMAP actions. So in my IMAP client (any IMAP client) I can create a folder called "Keywords/Beer" and it immediately contains all messages containing that keyword, or I can create a folder called "People/Fred" and drag one message from Fred into it, and it immediately contains all messages from (and maybe to) Fred.

It works reasonably well with ~100,000 messages. The weak part is IMAP, which I found to be a rather nasty protocol. It is GPLed and available at http://decimail.org/ - however, it has never had much external interest and has suffered from bit rot, so I don't really encourage anyone to try to use it unless very keen.

Phil.


(Log in to post comments)

The notmuch mail client

Posted Nov 18, 2009 14:19 UTC (Wed) by nye (guest, #51576) [Link]

This look awesome.

I hope to find time to look at it properly at some point, thanks.

The notmuch mail client

Posted Nov 18, 2009 16:57 UTC (Wed) by jond (subscriber, #37669) [Link]

Thanks for pointing this out. I'm also going to have to have a proper look
:)

The notmuch mail client

Posted Nov 18, 2009 21:46 UTC (Wed) by jonabbey (subscriber, #2736) [Link]

Very neat! Did you ever talk with other IMAP server implementers about your experience with using a SQL back-end?

I had gathered from various comp.mail.imap discussions over the years that certain features of IMAP were rather difficult to do efficiently with a relational database, though I never went about implementing anything server-side with IMAP.

The notmuch mail client

Posted Nov 19, 2009 23:27 UTC (Thu) by endecotp (guest, #36428) [Link]

I was subscribed to an IMAP list at one time but didn't discuss this much; my opinion of IMAP is rather low but I'm too polite to jump into someone else's list and tell them what I really think...

There are certainly mismatches between IMAP and SQL. For example, anything that implements virtual folders on the server will have to try hard to make the IMAP message sequence numbers work right. To be honest, I have been using Decimail for about five years and it "just works" for me, and I have now forgotten (or blocked the memories!) of the implementation challenges.

Phil.

The notmuch mail client

Posted Nov 18, 2009 22:22 UTC (Wed) by raf (guest, #35151) [Link]

I'm doing something similar, using PostgreSQL at the back end, PXTL for a web-based front end, and Twisted's IMAP4 component as the basis of the IMAP server.

I wanted it in PostgreSQL because I want to link it up with my GnuCash data (whose beta versions support PostgreSQL again) and a personal wiki (currently MediaWiki).

Anyway, nothing worth releasing yet.

The notmuch mail client

Posted Nov 18, 2009 22:46 UTC (Wed) by aleXXX (subscriber, #2742) [Link]

Is that the same principle as what Akonadi does ?
They use also a database and IMAP is also involved AFAIK...

Alex

The notmuch mail client

Posted Nov 19, 2009 12:18 UTC (Thu) by krake (subscriber, #55996) [Link]

It is somewhat similar but then also quite different.

Akonadi's server access protocol is similar to IMAP in terms of syntax and
grammar. Contents of commands and responses can be different and there are
additional commands for things usually not found in IMAP.

As far as I can tell there is also a different usage pattern for the
database, i.e. permanent storage vs. transient storage (Akonadi uses its
database mainly for caching, potentially only of parts of items, e.g.
envelope of mails).

Copyright © 2013, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds