What If I Don't Actually Like My Users?

What If I Don't Actually Like My Users?

Posted Apr 7, 2008 1:00 UTC (Mon) by gdt (subscriber, #6284)
Parent article: What If I Don't Actually Like My Users?

I wrote this blog entry in reply to Rusty's first blog entry. I'l copy it here as my web server is on my home ADSL link.

The Linux kernel API, a user's view

Rusty's blog on API design is timely, as I'm struggling with the API for Linux's Netfilter. There's no shortage of HOWTOs on the topic and no shortage of production code to examine.

The problem is bit rot. The API to establish connection tracking has been deprecated, but the official HOWTO on the Netfilter website hasn't been changed. There's no documentation of the new nf_ct_expect_alloc() at all. A reasonable QA process would have rejected a code change which didn't update the official documentation.

The API to register a connection tracking helper has also silently changed. nf_conntrack_helper_register() no longer accepts a bitmask. Again the official documentation and sample code hasn't been updated. The entirety of the documentation is a set of obscure commit comments and a short NetDev list discussion. Without finding those and understanding their significance you can't understand why the production code works when it differs from the official sample code and the large collection of older code in Patch-o-matic.

The broader networking API also has a newish function: skb_header_pointer(). All of the original SKB manipulation functions have documentation in the header file. Somehow this new function appeared with no documentation in that header file.

I can read source. But let's not pretend that "Use the Source Luke" is ideal. The code for nf_conntrack_helper_register() is about adding entries to a list. The connection tracking magic doesn't happen until a packet arrives and that list is searched and acted upon, which is handled in other code in a galaxy far, far away.

Source code is also hard at explaining why. Why and when should skb_header_pointer() be used in preference to direct access to the SKB's data? The source won't tell you that unless you are already so immersed in the kernel that you half-know the answer anyway.

Source code can also mislead. For example, looking at existing Netfilter code in the kernel would give you the idea that a 64KB buffer is needed for parsing incoming packets in a Netfilter modules. That's not true at all, it's just that all of the modules which have been accepted into the kernel have needed a packet-sized buffer to parse for IETF-style protocol text or to decode ASN.1.

After struggling through all of this, I'll lay odds that posting the finished module will result in at least one put-down e-mail about some misuse of some Linux API.

A final thought. Is there a kernel API at all? Can something permanently partially obscured be said to exist? Or is the API like the Loch Ness Monster. In place of blurry photographs we have Linux device drivers, where even those who closely track the kernel API can be misled by poor design and worse documentation such as with in_atomic().

---

(Log in to post comments)

> I can read source. But let's not pretend that "Use the Source Luke" is
> ideal. The code for nf_conntrack_helper_register() is about adding entries
> to a list. The connection tracking magic doesn't happen until a packet
> arrives and that list is searched and acted upon, which is handled in other
> code in a galaxy far, far away.

Seems like you have working source code, isn't that enough? I mean,
somebody did that for you (and many many other/users). Maybe after that
any kind of documentation writing wasn't in IWANTNOW list of the author?

It's open source, many who use, few who contribute. So it can be boring and
upsetting for particular authors. Оthers can be outraged.

> After struggling through all of this, I'll lay odds that posting the
> finished module will result in at least one put-down e-mail about some
> misuse of some Linux API.

Maybe also a documentation patch and willingness to improve the kernel,
everyone needs? I'm sure original author will be happy.

> A final thought. Is there a kernel API at all?

I think it's just hard and boring to maintain. After some repetition
almost anything in programming can be automated. It's not harvesting or
fruit/mushroom collecting, which is by far manual-only work (making a
suitable robot is more complicated than rocket science).

Any repetition is boring for human or dumbing down.

Maybe tool set must be upgraded (diff+patch in any form is technology
of 1980th)?

I've started to work with text processing to make some automation for
maintaining big changes, i.e not just one-liners, which can be grep'ed.

With some input from coding-style policy department and developers making
tags/clues comments in hard to textually analyse cases, something can be
done, and i think quite useful.

http://kernelnewbies.org/olecom

prev. comment
http://lwn.net/Articles/275780/
_____