LWN: Comments on "Elektrified X.org released"

Elektrified X.org released

aviram — 2005-03-02T04:52:21+00:00

Oh! There are brains around !
Finaly somebody pointed what Elektra is not !

Just to make it bold:
- Elektra is not a revolutionary lightweight new file format.
- Elektra, as an initiative, does not care about storage details.
- Elektra is not concerned about read/write transactions, just because current text configuration files aren't too.
- Elektra is not a network configuration system, just because current text configuration files aren't too. Go look for LDAP, SOAP, etc in the rare cases current Unix apps will need (or support) some.
- Elektra provides a way to vi/ed/sed/awk/OpenOffice.org/whatever keys-values only because Unix sysadmins (me included) are so addicted to this specialized practice. While average computer users (the entire human race) don't even know what "configuration" is, but they do know what are buttons, checkboxes and comboboxes.

Elektra is just what the 'dh' user above wrote. We are concerned about the problem that the concept of Elektra (but not only Elektra) can solve, more than the implementation details (which can evolve over time if the ecosystem grows -- yes, chicken and egg problem). If you don't like Elektra, we are sorry, but this was the way we found to solve the configuration superzoo issue in the most pragmactical way. Config4GNU, GConf, Hiveconf, UniConf, "etCeteraConf" currently can't do it, by design.

If you guys want to keep going deeper and deeper discussing implementation details of such a simple thing that Elektra is, fine. Just keep in mind that the general business world is way more practical than this discussion, and Linux doesn't have a real future without the interest, energy and investment of that world.

Avi Alkalay
Elektra Initiative

Hiveconf is another alternative

astrand — 2004-12-04T10:47:01+00:00

As the author of the competing configuration framework Hiveconf, I see several problems with the Elektra design. This mail summarizes my thoughts. While I believe that the Hiveconf principle is very good, the implementation needs a lot of work. Those of you that feel that Elektra isn't just right, feel free to take a look at Hiveconf instead.

Elektrified X.org released

evgeny — 2004-12-03T20:27:08+00:00

> > > Who should keep the whole XML file in the memory?

> Any software that needs to resolve IDs to nodes.

So do you say the kernel keeps the whole inode table/whatever in memory? And why memory requirements are different than that for similar task for a filesystem?

Elektrified X.org released

evgeny — 2004-12-03T19:49:54+00:00

> How can you configure the network using this daemon if the networked daemon needs the network configuration just to get going?

In exactly the same way I can run a LDAP server holding the user DB under a given account. Also, think e.g. of gcc bootstrapping, running of fsck on / before anything is mounted etc. We have a lot of similar chicken-or-egg problems.

Elektrified X.org released

evgeny — 2004-12-03T19:42:44+00:00

> OTOH, the strict typing of the data will get back at you when you try something like:
>
> grep -rl /home/joe /etc/

Sorry, I didn't get what you meant.

> Try rsync [etc etc etc]

I don't want. For the same reason that nobody in sane mind will pipe 'telnet 80 | html2txt | less' to browse the Web. Possible - yes. Great when nothing else is available - yes. But call this THE ultimate browser?!

Elektrified X.org released

tzafrir — 2004-12-02T23:15:50+00:00

> > Who should keep the whole XML file in the memory?

Any software that needs to resolve IDs to nodes.

Elektrified X.org released

sokol — 2004-12-02T23:09:21+00:00

> Using arbitrary whitespace to imply parsing semantics
> has almost always lead to problems (c.f., syslog.conf,
> /etc/fstab). Unless you intend for this format to only
> be parsed and written by automata, please don't do this.
I must confess that I thinked to propose a more
general format where three special chars (tab, new line
and backslash) would be "customizable". Indeed, whatever
three distinct ascii-7 chars would do the job. And you
are right to point that tab is the first candidate
to be replaced for less error prone human editing.
Unfortunatly, I didn't find an elegant way to indicate
in file what are special chars:
- should we treat systematicaly first three chars as
separator declarations?
- should we have three reserved keys whose values indicate
what separators became after these keys?
- ...?
I gave up and leave special chars fixed as they are.
Any other idea?

Serguei.

Elektrified X.org released

tzafrir — 2004-12-02T23:07:16+00:00

> Exactly. It, essentially, implements the
> day-before-yesterday functionality (INI configs)

No, it is not INI config. With INI config you have only thre-level deep tree of configuration (file, section, key). Electra does not have this silly limitation.

> yet requires facilities not present in all current filesystems.

The *required* facilities are avilable everywhere. On some systems it will be less efficient. But even ext3 now has support for a more scalable directory access . Try tune2fs -O dir_index

> Well, the keywords here are "on top" and
> "probably". ;-) With the same expected level
> of success as adding security on top of DOS was.

Rebuilding everything from scratch has even less expected chance of adding security.

> It just strikes me that a project with such
> an ambitious goal as converting _all_ the Unix
> configuration mess to a single API didn't bother
> thinking of even yesterday's requirements to
> system configuration!

Into a system that is easy to use by eveything and can be used anywhere.
How can you configure the network using this daemon if the networked daemon needs the network configuration just to get going?

Elektrified X.org released

tzafrir — 2004-12-02T22:55:03+00:00

> > But please name one thing a database provides that a
> > filesystem doesn't.
>
> Expressions (aka views). E.g. SELECT * FROM user WHERE
> user.id > 400 - to have a list of "real" users - used,
> for example, to send an announce email. Etc.

OTOH, the strict typing of the data will get back at you when you try something like:

grep -rl /home/joe /etc/

> Other things are replication,

Try rsync

> atomicity, true locking,

What about filesystem-level locking?

> notifications,

Change notification for a directory is available.

> rollbacks, versioning,

Put arch/subversion in there (cvs probably won't do as it does not support renames)

> transparent remote access,

Add a remote-access daemon. Quite simple to implement. You have to figure out the authentication method first. Currently practically no existing daemon intgrates well enough with the system.

> scalability (in _both_ directions).

Filesystem access is quite scalable

One unified config format is impossible

nix — 2004-12-02T17:13:34+00:00

The point is that they blur into each other: a clear distinction in the general case between `configuration state' and `commands which affect configuration' is as impossible as a clear distinction in the general case between data and code.

That's been known for as long as general-purpose computers have existed.

Elektrified X.org released

kfiles — 2004-12-02T17:03:06+00:00

> So, I came up with KVH format which uses the sipmliest parsing
> rules for handling key-values hierarchie I have ever seen. This
> format is program oriented but remains human readable and editable
> with plain text editor without any need of aditional APIs. APIs may
> come later for GUI editors and Co.

Argh. This falls victim to the same problems of the old syslog.conf: visual confusion in whitespace use. A naive user, editing such a config file, might not know that the indentation by \t (tab character) was mandatory for proper parsing. Substitution of spaces for tabs in syslog.conf provided no end of headaches for sysadmins trying to track down configuration problems. Some editors (like emacs) may even unhelpfully assist in creating this problems by substituting strings of spaces for tabs.

Using arbitrary whitespace to imply parsing semantics has almost always lead to problems (c.f., syslog.conf, /etc/fstab). Unless you intend for this format to only be parsed and written by automata, please don't do this.

I much prefer the gated.conf C-style self-descriptive hierarchical config blocks, which use curly-braces for closure, and do not require the pseudo-SGML-isms of httpd.conf to indicate hierarchy.

--kirby

Elektrified X.org released

sokol — 2004-12-02T13:06:02+00:00

I like the idea of elektra. It fixes a giving "itch" which happens to be quite general in Linux world. With a little bit of luck, elektra approach will generalize (with or without filesystem backend).

Talking about backend, I have my two cents to bring. Some time ago, I asked myself what was the simpliest file format to handle hierarchical key-value paires. Just this. No read-write permissions, no validators, no multiple data types. Just an hierarchie of key-values. If someone need mentioned features, he has to extend this simpliest format at cost of reserved keys, particular syntax for references and so on. Thus a user will pay only what it really uses.

So, I came up with KVH format which uses the sipmliest parsing rules for handling key-values hierarchie I have ever seen. This format is program oriented but remains human readable and editable with plain text editor without any need of aditional APIs. APIs may come later for GUI editors and Co.

So, if you are interested, have a walk to http://serguei.sokol.free.fr/kvh-format/

One unified config format is impossible

sokol — 2004-12-02T11:03:59+00:00

> And "real" config files are...?
It is in the text, files that describe
and declare things, not execute some code.

Elektrified X.org released

evgeny — 2004-12-02T10:28:04+00:00

> Don't forget to also return to the fact, that not every man and his dog
> want replication, versioning, transactions/rollbacks etc.
> And many of those who need, can do this with filesystems + there own
> scripts.

Continuing this nice logics, there is no need for a unified configuration system altogether - just as everyone has been adjusting dozens of different configs during the last twenty years of mainstream Unix usage, so we can do it further on. Right?

One unified config format is impossible

hppnq — 2004-12-02T10:24:47+00:00

And "real" config files are...?

Elektrified X.org released

szh — 2004-12-02T10:08:34+00:00

Don't forget to also return to the fact, that not every man and his dog
want replication, versioning, transactions/rollbacks etc.
And many of those who need, can do this with filesystems + there own
scripts. Just regular cron /etc [or parts of /etc] [incremental] backups
satisfy many men and dogs.
So the new system will give drawbacks only for those men and dogs.

One unified config format is impossible

sokol — 2004-12-01T22:43:17+00:00

To me, it sounds like a language abuse when you call a code
_configuration_ file. If we remain in the field
of _real_ config files which describe and declare
things rather than go down an operator chain, then
hierarchical key-value pairs do the job.

Elektrified X.org released

evgeny — 2004-12-01T21:37:35+00:00

> How do you create different permissions on different part of the database
> without a priviliged daemon (or something uglier, such as a SUID root binary) ?

Depends on the implementation. For a DB _service_ (a daemon running) it's solved like for any other similar problem (and there is no need for it to run under a privileged account - e.g. slapd on my server runs as an unprivileged user 'ldap', yet system-wide authentication works fine). For a simple file-based DB like SQLite it can be solved by splitting the whole data into a few files (1 - writable by root, readable by everyone; 2 - r/w by root ony, plus, similarly, two files per user account).

> But please name one thing a database provides that a filesystem doesn't.

Expressions (aka views). E.g. SELECT * FROM user WHERE user.id > 400 - to have a list of "real" users - used, for example, to send an announce email. Etc. Other things are replication, atomicity, true locking, notifications, rollbacks, versioning, transparent remote access, scalability (in _both_ directions). You get all these for free with a decent RDBMS, and NONE of them with an existing filesystem.

One unified config format is impossible

nix — 2004-12-01T20:25:46+00:00

The `make everything a key-value pair' thing works until you hit programs that are so complex that their configuration syntax is Turing-complete.

If you can turn my XEmacs configuration (here) into something Elektra-based, I'll be impressed.

But I really, really doubt it's possible.

You can map most configuration files two-ways into this format, but not all.

Elektrified X.org released

evgeny — 2004-12-01T20:22:29+00:00

> Who should keep the whole XML file in the memory?

1. Which is the "whole XML file"?

2. There is no need to keep the "whole XML" (whatever it means) in memory. There are alternative methods of parsing XML, e.g. SAX.

3. You probably should argue with another person anyway. My suggestion was to use an RDBMS, not XML file(s). I did say an XML-based DB (notice the word _database_ - I never meant plain XML files, especially not one huge file) could be used as well, in principle.

> Now what about caching those small changes?

Same here.

Elektrified X.org released

Ross — 2004-12-01T18:36:44+00:00

If you would read the context of most of the comments you deride you would
see that they are responding to the parent comment (such as the one which
said that they would rather see you utilizing a binary database), not to the
text of the article.

Elektrified X.org released

tzafrir — 2004-12-01T17:51:37+00:00

Who should keep the whole XML file in the memory?

Now what about caching those small changes?

Elektrified X.org released

tzafrir — 2004-12-01T16:27:49+00:00

This is achived by having all accesses to those files going through a database daemon. That daemon should guarantee that locks/transactions etc. are maintained. It also caches currently-used values.

In the filesystem level the kernel does basically the same things.

Elektrified X.org released

tzafrir — 2004-12-01T16:23:31+00:00

How do you create different permissions on different part of the database without a priviliged daemon (or something uglier, such as a SUID root binary) ?

This is an important requirement: user applications need not run as root to read configuration. But there should be a place for the secret data.

But please name one thing a database provides that a filesystem doesn't. What database exactly (daemon? file-based? which one?)

Elektrified X.org released

hppnq — 2004-12-01T15:05:44+00:00

Generally, it would ease the job of an admin tremendously if different packages could agree on some general syntax rules for config files, and Elektra tries to present a light-weight solution exactly for this purpose.

I'm not so sure. I can see that a simple, uniform configuration syntax helps automating configuration, but to me it looks like Elektrify will make hand-editing configuration files harder rather than simpler. I am not quite sure what problem it tries to solve, and whether the chosen path will bring any joy. Things might be entirely different for you, but my main problem with configuring software is usually choosing the correct values, not understanding the configuration file syntax.

Saying: "But now you can easily sort your key value pairs!" begs the question: do I need that? I don't think so.

I would appreciate a meta-configurator that has plugins for handling native formats like fstab and httpd.conf, so no patching would be needed for the filesystem utilities and Apache.

Elektrified X.org released

evgeny — 2004-12-01T11:33:37+00:00

> And that rule of Euclid, by being wrong, held back geometry for 2000 years.

Poor Newton. He could discover the general relativity, if not that stupid greek. Was it an attempt at troll?

> Real world data does NOT come in neat two-dimensionsal packages. Therefore real world data does NOT belong in relational databases.

Do some text-book reading before posting, please.

Elektrified X.org released

evgeny — 2004-12-01T11:18:37+00:00

> Tree-like information should live in tree and we already have one - filesystem tree.

Configurations are NOT trees. They're graphs (the symlinks used by Electra is a poor-man solution to this). Plus, they're _dynamic_ ones.

> Can I use vi over ssh to edit my config then ? No ? Then it's not contender.

Gosh. Why not `ed' then? There is a userfs plugin that allows to navigate/edit a postgres DB like a directory, if you insist on that. I also assume that you code exclusively in assembler and use `telnet 80 lwn.net' to read my reply.

> And even if there is you are still with the same problem: you do not have access control.

RDBMS have no access control?!

> You can add remote management, replication, versioning, transactions/rollbacks if you wish - you only need good enough filesystem.

U-hu. Just a tiny obstacle.

> Why not fix filesystems instead ?

Well, go fix them. IMHO, filesystem developers have more urgent tasks than tailoring their filesystems to an ambitios configuration project. So we can wait years and years - and still have a dumb (no logical expressions etc) configuration mechanism.

Elektrified X.org released

evgeny — 2004-12-01T10:47:05+00:00

> remote management etc. is beyond the scope of Elektra. [...] It just provides a simple method of accessing configuration.

Exactly. It, essentially, implements the day-before-yesterday functionality (INI configs) yet requires facilities not present in all current filesystems.

> Adding remote management, versioning on top of that is proably possible.

Well, the keywords here are "on top" and "probably". ;-) With the same expected level of success as adding security on top of DOS was. It just strikes me that a project with such an ambitious goal as converting _all_ the Unix configuration mess to a single API didn't bother thinking of even yesterday's requirements to system configuration!

Elektrified X.org released

dh — 2004-12-01T10:44:11+00:00

Hi,

I must admit that I can hardly remember the last time I read so much
rubbish as in this discussion here.

Guys, would you please bother reading and thinking precisely what Elektra
tries and how it archieves its goals before moaning and writing strange
things here? It is _not_ about XML, it is _not_ about not-vi-editable
configuration, it is _not_ about a single big file, it is _not_ about
Microsoft trying to subvert the poor penguin community. Why do you write
such absolutely unrelated things? And why do other people answer to
this???

Elektry tries to change configuration settings to be context-free by
making each line of the config file self-contained. This makes the file

- less error-prone in hand-editing and
- way better handleable for automated changes

So, generally it does _not_ take any freedom from you in editing with
whatever tool or editor you like. You are even supported in this as the
configuration semantics becomes much, much clearer. See: You can even
apply new tool chains sensibly to the config file: In the X case, if you
want to have all monitor settings, just type "cat configfile | grep
"/sys/monitors/"" or something similar. Wanna have the file sorted? "sort
configfile" does the job (ever tried this with httpd.conf or legacy
Xorg.conf?). Generally, it would ease the job of an admin tremendously if
different packages could agree on some general syntax rules for config
files, and Elektra tries to present a light-weight solution exactly for
this purpose.

Perhaps we could concentrate on these topics - and stop posting false
accusations. That project really deserves a chance IMHO.

Best regards,
Dirk
(not related to the Elektra project in any way)

Elektrified X.org released

evgeny — 2004-12-01T10:31:56+00:00

> If your filesystem is dumb - it's not possible. If your filesystem is smart - it can be done.

So we can return to this discussion when every man and his dog have such smart filesystems - including embeded devices etc.

Elektrified X.org released

evgeny — 2004-12-01T10:26:37+00:00

> You'll need application to deciper such references and use them. Doable - yes, convenient - no.

Expat and libxml can do this. And, BTW, what d'ya'think kernel is doing when resolving symlinks?

Elektrified X.org released

evgeny — 2004-12-01T10:03:39+00:00

> The filesystem is stable. It has to be.

I see no reason why a DB should inherently be less stable.

> However why replicate so many features that the file system already give you in a database?

Because FS will never provide me with all the advanced features one expects from a decent configuration system - not without stacking dozens of extra daemons/utilities/... on top of it, at least.

> Or in 2500 implementations of 50 different configuration formats.

Of course. I said it in the beginning, that I like the _idea_ itself - to replace the whole current zoo of config approaches with a single API. It's the _implementation_ that seems to me a dead end.

Elektrified X.org released

Wol — 2004-12-01T08:53:24+00:00

"Tree-like information should live in a tree". I couldn't agree more!

And I'm going to get shafted for this by the relational crew, but relational theory is CRUD! Look at C&D's first rule - "data is stored as rows and columns". What real-world data comes as rows and columns? NONE of it as far as I can see. Pretty much all of C&D's rules have as much mathematical basis in fact as Euclid's rule that "parallel lines never meet".

And that rule of Euclid, by being wrong, held back geometry for 2000 years. Real world data does NOT come in neat two-dimensionsal packages. Therefore real world data does NOT belong in relational databases.

That's not to say relational theory doesn't have its uses - it is very useful. But by trying to force everything into its own (faulty) mould, it does the rest of the world a major dis-service.

Cheers,
Wol

Elektrified X.org released

Wol — 2004-12-01T08:33:28+00:00

And why is changing multiple files any different to changing multiple places in one large file? The database I work with stores each "table" in a separate file, and yet it's quite happy with transactions and commits across multiple files.

And (certainly from the nix viewpoint) isn't the file system seen as just one large file? So, from that vantage point, updating multiple files IS just one change to one large file :-)

Cheers,
Wol

viewer tool

Duncan — 2004-12-01T08:21:50+00:00

Well.. make that ls, cat, echo, and shell redirection, to cover both the
read and write sides of the coin.

Duncan

Elektrified X.org released

bk — 2004-12-01T02:03:16+00:00

Ha. If you're still using Sendmail then you deserve the hell that is sendmail.cf :)

As a side note, it probably would be almost trivial to write a backend that writes the config back into the application's native format (ie, goes from the nested hierarchy -> xorg.conf), making this project a simple translation layer for those who prefer a registry-like layout. That seems like a better way of doing this, as opposed to shoving a completely un-Unix configuration system down the throat of the free software community.

Elektrified X.org released

sbergman27 — 2004-12-01T01:09:09+00:00

I don't think that this attitude really helps us. Certainly, MS's registry has flaws. In particular, it seems to be a binary, non human readable, put all your eggs in one basket, "solution". Just because something bears a superficial resemblance to the MS registry does not mean that it is not a good idea.

One of my pet peeves with Linux/Unix is the total balkanization of config file formats. I would really prefer to see all these moved into XML, though. But I really don't care, as long as there is some predictability and universality to it all.

And if someone could just XMLize or Elektrify my sendmail.cf, I'd be eternally grateful. I'd even spring to send you to the mental rehabilitation center of your choice after completing the task (if required). ;-)

Elektrified X.org released

tzafrir — 2004-12-01T00:45:34+00:00

remote management etc. is beyond the scope of Elektra. Electra is not aware of the semantics of the configuration. It just provides a simple method of accessing configuration.

Adding remote management, versioning on top of that is proably possible.

The main non-standard feature required for Elektra to work propely is symlinks. And even without it most things will work. Other than that all you need is basic file-system interface.

Elektrified X.org released

petegn — 2004-12-01T00:30:41+00:00

i See the microsofties are trying to get a hold by the back door now they cant beat us so they are tying to polute the system with there awfull fail all the time registery type thing . not on your flippin nellie go away .

Elektrified X.org released

emkey — 2004-12-01T00:30:05+00:00

>"8) Easy program interaction. I'll be thrown out of the *nix geek club for
sure for saying this but I *like* GUI configuration programs. I like things
to be easy."

As do most of us. The problem being GUI's don't for the most part scale worth a dang. In general GUI's are fine, but they should always be considered secondary to some sort of CLI interface. The problem is many vendors get it backwards, or worse yet don't implament any sort of CLI.

LWN: Comments on "Elektrified X.org released"

Elektrified X.org released

Hiveconf is another alternative

Elektrified X.org released

Elektrified X.org released

Elektrified X.org released

Elektrified X.org released

Elektrified X.org released

Elektrified X.org released

Elektrified X.org released

One unified config format is *impossible*

Elektrified X.org released

Elektrified X.org released

One unified config format is *impossible*

Elektrified X.org released

One unified config format is *impossible*

Elektrified X.org released

One unified config format is *impossible*

Elektrified X.org released

One unified config format is *impossible*

Elektrified X.org released

Elektrified X.org released

Elektrified X.org released

Elektrified X.org released

Elektrified X.org released

Elektrified X.org released

Elektrified X.org released

Elektrified X.org released

Elektrified X.org released

Elektrified X.org released

Elektrified X.org released

Elektrified X.org released

Elektrified X.org released

Elektrified X.org released

Elektrified X.org released

viewer tool

Elektrified X.org released

Elektrified X.org released

Elektrified X.org released

Elektrified X.org released

Elektrified X.org released

One unified config format is impossible

One unified config format is impossible

One unified config format is impossible

One unified config format is impossible

One unified config format is impossible