| |||||||||||||
Plain TextPlain TextPosted Feb 2, 2011 22:29 UTC (Wed) by jeremiah (subscriber, #1221)Parent article: LCA: Lessons from 30 years of Sendmail
I find his comment on using Plain Text interesting. I've got a project that would benefit greatly from storing it's files in a db as opposed to the file system. But I continue to resist the urge because damnit! Sometimes all you have is vi. I've toyed with the idea of a simple way of importing and exporting the data to and from a db, but again, who knows how things can break. I made the same choice with decision to the protocols, tokenized human readable text. So it's nice to see someone think that that decision was still worth it 30 years later.
Now I just need to get that O'Reilly book written. :)
(Log in to post comments)
Plain Text Posted Feb 2, 2011 22:49 UTC (Wed) by HelloWorld (guest, #56129) [Link] But I continue to resist the urge because damnit! Sometimes all you have is vi.Yes, sometimes. And in the remaining 95% of all cases, you have all the necessary tools at your disposal and cripple yourself by not using them. Sounds like a wonderful tradeoff to me.
Plain Text Posted Feb 2, 2011 23:13 UTC (Wed) by jeremiah (subscriber, #1221) [Link]
In this case, were talking remote Linux boxes on non-broad-band connections. So seriously vi is all we have sometimes. Doesn't mean I don't also have a nice GUI front end to things, when it's an option. Things are also written so that the data can be stored in a DB if the admin chooses. But the default is to assume we are in a worst case scenario, until proven otherwise. Also makes it lighter weight. Of course there is Berkley DB which is pretty light. Awe who knows. There's never a single correct solution for everyone, all you can do is make a lot of things optional, and spend a lot of time thinking about what the best defaults are for the majority of people.
Plain Text Posted Feb 3, 2011 17:34 UTC (Thu) by iabervon (subscriber, #722) [Link]
psql is probably even less latency-sensitive than vi (being based on readline and stdout), and is at least as good at making controlled modifications to structured data. It's not so good at changing one character in a long string in a single field, but it's great when the granularity of the database matches the granularity of the data.
Plain Text Posted Feb 3, 2011 5:06 UTC (Thu) by wahern (subscriber, #37304) [Link]
What tools? Are you going to go the Mailman route, creating a hundred useless command line tools alongside a custom web-based configuration engine, sitting atop a complex database schema, possibly using a relational database backend? Almost every other option effectively boils down to that, and it's horrible.
The limitations of an ASCII configuration file is its virtue! Often times they turn into domain specific languages. In which case, use sendmail.cf as your north star and head south.
Plain Text Posted Feb 3, 2011 13:14 UTC (Thu) by nix (subscriber, #2304) [Link]
If you have to put things in a DB, at least make it sqlite. The sqlite command-line tool can be used to manipulate that in all sorts of ways if your tool is broken.
Plain Text Posted Feb 5, 2011 15:07 UTC (Sat) by tzafrir (subscriber, #11501) [Link]
Though the sqlite command-line interface is no way close in friedlyness to that of pgsql (or even the mysql one).
Plain Text Posted Feb 7, 2011 16:49 UTC (Mon) by nix (subscriber, #2304) [Link]
Yeah, agreed. I was assuming 'zero dependencies', i.e. the sort of thing which might reasonably break in a situation in which 'all you have is vi'.
Plain Text Posted Feb 3, 2011 16:02 UTC (Thu) by dgm (subscriber, #49227) [Link]
It's not only about changing data with vi. Sometimes is about being able to recover critical data from a corrupt file with vi, rather than giving it up because the binary db file is f___ up, and someone forgot the change backup tapes for the last two months, and all you have is last night's backup of an already corrupt file.
Are you really sure a db will buy you something? Do you need ACID? Do you _really_ need an index to find your data? Be sure a database will buy you something before giving up reliability. For instance, have you thought that reading a 80 MB text file takes just a couple of seconds with any modern disk? That's a good million 80-byte records.
Plain Text Posted Feb 3, 2011 21:38 UTC (Thu) by dlang (✭ supporter ✭, #313) [Link]
also keep in mind the problems that firefox has where it can stall for significant amounts of time while it is doing an ACID update to it's bookmark file.
does that file _really_ need ACID protection? I don't think so, I think that just having a fairly recent backup is good enough.
Plain Text Posted Feb 4, 2011 1:53 UTC (Fri) by zlynx (subscriber, #2285) [Link]
The long lag is really a filesystem bug. Newer filesystems have fixed the problem. Firefox shouldn't lag on ext4, xfs or btrfs.
Plain Text Posted Feb 4, 2011 1:59 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]
the really long lag was a combination of an Ext3 bug and the fact that the browser was trying to get ACID protection of the file.
I still don't think that's appropriate.
Plain Text Posted Feb 26, 2011 13:24 UTC (Sat) by job (guest, #670) [Link]
True. For small projects (less than a couple of hundred thousand records perhaps), I just keep data in RAM and serialize to plain text periodically. RAM is cheap, even 100 MB of data structures is practically free, and crazy fast. I like to think lesser external dependencies make deployment easier and crashes won't hurt you all that bad.
|
|||||||||||||