2.6.30 merge window, part I [LWN.net]

By Jonathan Corbet
April 1, 2009

As of this writing, almost 6200 non-merge changesets have been added to the mainline for the 2.6.30 release. So the merge window is well and truly open. There's a lot of stuff set up for 2.6.30 already, with more certainly to come. The user-visible changes merged so far include:

The relatime mount option is now the default; this means that file access times will only be updated if they are newer than the creation or modification times. Another change merged also causes the access time to be updated at least once per day. Users needing access times to be updated for every access can use the new "strictatime" mount option to get that behavior. See That massive filesystem thread for more information on this change.
At long last, the integrity management patches have been merged. Among other things, this code can use the trusted platform module (TPM) to ensure that the files on a system have not been tampered with and to do remote attestation.
Also at long last, TOMOYO Linux has been merged. TOMOYO is a pathname-based security module similar to (but significantly different from) AppArmor.
There is a new cpuinfo_transition_latency sysfs variable for CPU frequency governors; it serves to inform user space of the time it takes for the CPU to transition from one frequency to another.
There is now support for the new AES-NI cryptographic instructions being introduced into Intel processors; see this white paper [PDF] for details on AES-NI.
The x86_64 and SuperH architectures have gained kexec jump support.
There is a new guest debugging interface for KVM, allowing the host to do interactive debugging of guest systems. KVM has also gained support for PowerPC e500 processors.
There is a new user-space API for detailed control over the timestamping of network packets. See Documentation/networking/timestamping.txt for details.
The Reliable Datagram Sockets (RDS) protocol is now supported by the networking layer. See Documentation/networking/rds.txt for more information.
The x86 architecture now has an option to put a "canary" value at the end of the kernel stack; if that value ever changes, the stack has been (accidentally or maliciously) overrun.
The reiserfs filesystem has seen a burst of work which cleans up the code, improves SELinux support, and improves performance. This is likely to be the last set of updates for reiserfs.
The usual array of new drivers has been merged. They include:
- Block: PCI-Express SAS 6Gb/s host adapters.
- Graphics: AMD R6xx/R7xx GPUs (2D only for now).
- Networking: USB Qualcomm Serial modems, Marvell Libertas 8686 SPI 802.11b/g cards, Marvell 88w8xxx TOPDOG PCI/PCIe wireless cards, Prism54 SPI (stlc45xx) wireless adapters, Atmel at76c503/at76c505/at76c505a USB wireless adapters, OpenCores 10/100 Mbps Ethernet MAC devices, and Atheros "otus" 802.11n USB devices.
- Sound: Mitac MIO A701 phones, Wolfson Micro WM8400 and WM9705 codecs, Wolfson Microelectronics 1133-EV1 modules, Atmel Audio Bitstream DAC devices, Atmel AC97 controllers, Asaki-Kasei AK4104 S/PDIF transmitters, Echo Audio IndigoIOx and IndigoDJx cards, Turtle Beach Classic, Fiji and Pinnacle cards, and Asus Xonar Essence STX sound cards.
- Video/DVB: Mars-Semi MR97310A USB cameras, Freescale MC44S803 low power CMOS broadband tuners, SQ Technologies SQ905-based USB cameras, i.MX3x camera sensor interfaces, ST STV0900 satellite demodulators, ST STV6110 silicon tuners, SQ Technologies SQ905C-based USB cameras Zarlink ZL10036 silicon tuners, LG Electronics LGDT3305-based tuners, Hauppauge HD PVR USB devices, and Intel CE6230 DVB-T USB2.0 receivers.
- Processors and systems: SuperH SH7786, ESPT-Giga SH7763-based reference boards, SMSC reference platform with a SH7709S CPUs, Palm LifeDrive and Tungsten|T5 systems, Brivo Systems LLC ACS-5000 master boards, Dave/DENX QongEVB-LITE platforms, Marvell RD-78x00-mASA development boards, Marvell PXA168 and PXA910 processors, TI OMAP850 processors, OMAP 3430 SDP boards, Nokia RX-51 internet tablets, Teltonika 3G Router RUT100 systems, Faraday FA526 cores, Cortina Systems Gemini family SoCs, GE Fanuc SBC310 and PPC9A PowerPC boards, Freescale Media5200 boards, AMCC Redwood(460SX) systems, Phytec phyCORE-MPC5200B-IO (pcm032) boards, and Freescale MPC8544 ("Socrates") boards.
- Miscellaneous: AMCC PPC4xx crypto accelerators, Adrienne Electronics Corporation PCI time code devices, Symbol 6608 barcode scanners, E-Ink Broadsheet/Epson S1D13521 controllers, NXP Semiconductor PCA9665 i2c controllers, and Siemens Syleus and Hades sensor chips.
The "Phidgets" USB drivers have been removed; users should shift to the user-space drivers instead.

Changes visible to kernel developers include:

The adaptive spinning mutex patch has been merged. This change will cause mutexes to behave more like spinlocks in the contended case. If (and only if) the lock is held by code running on a different CPU, the mutex code will spin on the assumption that the lock will be released soon. This behavior results in significant performance improvements. Btrfs, which had its own spinning mutex implementation, has been converted to the new mutexes.
There is a new set of functions added to the crypto API which allow for piecewise compression and decompression of data.
The bus_id member of struct device is gone; code needing that information should use the dev_name() macro instead.
There is a new timer function:
```
    int mod_timer_pending(struct timer_list *timer, unsigned long expires);
```
It is like mod_timer() with the exception that it will not reactivate an already-expired timer.
There have been some changes around the fasync() function in struct file_operations. This function is now responsible for maintaining the FASYNC bit in struct file; it is also now called without the big kernel lock held. Finally, a positive return value from fasync() is mapped to zero, meaning that the return value from fasync_helper() can be returned directly by fasync(). (This is your editor's modest contribution to 2.6.30).
The SCSI layer has a new support library for object storage device support; see Documentation/scsi/osd.txt for details.
The x86 "subarchitecture" mechanism has been removed, now that no architectures actually use it. The Voyager architecture has been removed as a result of these changes.
x86 is also the first architecture to use a new per-CPU memory allocator merged for 2.6.30. This allocator changes little at the API level, but it will provide for more efficient and flexible per-CPU variable management.
Support for compressing the kernel with the bzip2 or lzma algorithms has been added. Support for the old zImage format has been removed.
The asynchronous function call infrastructure is now enabled by default.
The DMA operations debugging facility has been merged.
The owner field of struct proc_dir_entry has been removed, causing lots of changes throughout the tree.

If the usual two-week pattern holds, the merge window can be expected to remain open through about April 9. The rate at which changes flow into the mainline will likely be lower for the second half of the merge window - the alternative is for this development cycle to be far larger than any of its predecessors. But it is certain that more interesting changes will be merged for 2.6.30.

(Log in to post comments)

2.6.30 merge window, part I

Posted Apr 2, 2009 18:05 UTC (Thu) by nix (subscriber, #2304) [Link]

I've had a look at it, and, well, the core idea (process execution
history) is lovely, though the cost is high (a near-100% slowdown of
open() for instance), but the configuration file syntax and user
interface, ewwww. It renders it almost unusable in my eyes. You have
*numbered* 'profiles' corresponding to (non-POSIX) capability sets, so you
have to remember what each number corresponds to; backslashing of *all*
metacharacters, including *, combined with the absence of an 'all below'
option, leading to insanity like

/home/\*/\*
/home/\*/\*/\*
/home/\*/\*/\*/\*
/home/\*/\*/\*/\*/\*
/home/\*/\*/\*/\*/\*/\*
/home/\*/\*/\*/\*/\*/\*/\*

and hope your users don't create directories more than five deep under
their $HOME, and that you didn't make a typo in that appalling forest.
(What's worst about this last bit is that it can't be fixed by some
userspace component munging a configuration file in a better format and
echoing it to TOMOYO's /proc files. Oh, and did I mention the awful name
given to the directory those files are in? If I saw it on a random system
I'd have no idea it was related to TOMOYO at all.)

2.6.30 merge window, part I

Posted Apr 7, 2009 9:13 UTC (Tue) by haradats (guest, #44782) [Link]

> I've had a look at it, and, well, the core idea (process execution
> history) is lovely, though the cost is high (a near-100% slowdown of
> open() for instance),

Did you see the following?
http://tomoyo.sourceforge.jp/wiki-e/?WhatIs#benchmark
Yes, the cost for "open" is the worst, but others are not so bad
(read/write are not affected at all).

> but the configuration file syntax and user
> interface, ewwww. It renders it almost unusable in my eyes. You have
> *numbered* 'profiles' corresponding to (non-POSIX) capability sets, so you
> have to remember what each number corresponds to; backslashing of *all*
> metacharacters, including *, combined with the absence of an 'all below'
> option, leading to insanity like

I'm sorry for your eyes. ;-)

Profiles can be defined up to 256, but most cases you can live with
the following predefined ones.

profile 0: "disabled"
profile 1: "learning"
profile 2: "permissive"
profile 3: "enforcing"

Profile numbers have nothing with capabilities (and the just
merged version of TOMOYO Linux cannot control capabilities).
For more information, please take a look at the following documents.

http://tomoyo.sourceforge.jp/en/2.2.x/

> and hope your users don't create directories more than five deep under
> their $HOME, and that you didn't make a typo in that appalling forest.

Don't worry. You can define deeper directory patterns as you like.

Questions and suggestions are always welcome. Please visit our forum, too.
http://sourceforge.jp/forum/forum.php?forum_id=11352&...

Thanks

2.6.30 merge window, part I

Posted Apr 7, 2009 19:44 UTC (Tue) by nix (subscriber, #2304) [Link]

> Yes, the cost for "open" is the worst, but others are not so bad
> (read/write are not affected at all).

Indeed, that's nice... but IIRC the cost for, say, SELinux for open() is
much lower. And open() is a *common* syscall, optimized to hell and back.
IMNSHO this restricts TOMOYO to high-security or low-performance
environments :(((

But I'm confident that with more optimization work TOMOYO will overcome
this.

The backslashing of shell metacharacters and numbered rather than
named 'profiles' remains horrid (but can be overcome with a preprocessing
layer).

The consistent 'ccs' in the names of commands and /proc entries still has
no connection whatsoever that I can see to 'TOMOYO': why not rename or at
least provide links to more memorably-named commands?

I was under the impression that putting things like the TOMOYO ccs/
directories in /proc (or anything new at all not tightly related to
specific processes) was pretty seriously deprecated.

>> and hope your users don't create directories more than five deep under
>> their $HOME, and that you didn't make a typo in that appalling forest.
>
> Don't worry. You can define deeper directory patterns as you like.

But you can't say 'any depth', and without that you're just asking for
confusing failures. You can't sensibly assume any fixed depth for your
users' directory trees (one user of mine has directories *135* levels deep
for reasons known only to him and his god: do you want to make a policy
that caters for *that* with explicit .../\*/\*/...? No, neither do I.) And
Linux has *no* limit whatsoever on directory nesting depth, not PATH_MAX,
nothing.

So you need a 'recursive' thing (like AppArmor has).

AppArmor is much less flexible than TOMOYO, and lacks the lovely
process-execution-history idea, but boyoboy are its config files more
readable. It pretty much did everything right from a user-interface design
perspective. You could profitably copy syntax from it...

2.6.30 merge window, part I

Posted Apr 10, 2009 13:38 UTC (Fri) by haradats (guest, #44782) [Link]

>> Yes, the cost for "open" is the worst, but others are not so bad
>> (read/write are not affected at all).
>
> Indeed, that's nice... but IIRC the cost for, say, SELinux for open() is
> much lower. And open() is a *common* syscall, optimized to hell and back.
> IMNSHO this restricts TOMOYO to high-security or low-performance
> environments :(((
>
> But I'm confident that with more optimization work TOMOYO will overcome this.

For true high-security environments, I don't expect pathname-based MAC
can be the first candidate (and SELinux is there). Hopefully, TOMOYO and
other pathname-based MAC will be able to work with label-based MAC in the future.
In any case, it is clear we need to improve the performance as much as possible.

> The backslashing of shell metacharacters and numbered rather than
> named 'profiles' remains horrid (but can be overcome with a preprocessing
> layer).

I understand your points, but there's a compatibility issue, too.
Anyway thank you for your suggestions.

> The consistent 'ccs' in the names of commands and /proc entries still has
> no connection whatsoever that I can see to 'TOMOYO': why not rename or at
> least provide links to more memorably-named commands?
> I was under the impression that putting things like the TOMOYO ccs/
> directories in /proc (or anything new at all not tightly related to
> specific processes) was pretty seriously deprecated.

I can't agree with you more. :-)
Those directory were named by the architect of TOMOYO Linux.
The project has had several meetings and the conclusion was respect his choice.
(I think he has the right to name it and if we force him to change, his
motivation has been changed also ;-)

>>> and hope your users don't create directories more than five deep under
>>> their $HOME, and that you didn't make a typo in that appalling forest.
>>
> > Don't worry. You can define deeper directory patterns as you like.
>
> But you can't say 'any depth', and without that you're just asking for
> confusing failures. You can't sensibly assume any fixed depth for your
> users' directory trees (one user of mine has directories *135* levels deep
> for reasons known only to him and his god: do you want to make a policy
> that caters for *that* with explicit .../\*/\*/...? No, neither do I.) And
> Linux has *no* limit whatsoever on directory nesting depth, not PATH_MAX,
> nothing.

Well, IMHO, the fact that Linux has no limit does not mean you should do it. ;-)

In TOMOYO Linux, MAC can be controlled per domain basis.
(SELinux also has permissive domains)
So, if you want to give permission for certain domain to access 135 or
more deep, you just can put the mode in permissive or disable ( you don't have to
type /\*/\*/... Gee, it's really hard to type ;-)
If you do care the domain, I think it's worth to define /\*/*\... for 135 lines or more.
(This is just my opinion though)

> So you need a 'recursive' thing (like AppArmor has).
> AppArmor is much less flexible than TOMOYO, and lacks the lovely
> process-execution-history idea, but boyoboy are its config files more
> readable. It pretty much did everything right from a user-interface design
> perspective. You could profitably copy syntax from it...

Actually, you were not the first person that suggested recursive thing.
One of the reason we didn't take that was the simplicity in the kernel.
I think I am going to talk project members again.

Thank you. /\*/\*/\*/

2.6.30 merge window, part I

Posted Apr 17, 2009 20:21 UTC (Fri) by nix (subscriber, #2304) [Link]

As far as I can see, you don't need recursion to do the 'any directory
below' thing at all.

(And, er, you can fix the syntax by allowing two syntaxes, although that
probably isn't a job for the kernel.)

2.6.30 merge window, part I

Posted Nov 30, 2009 8:56 UTC (Mon) by haradats (guest, #44782) [Link]

nix,

> /home/\*/\*
> /home/\*/\*/\*
> /home/\*/\*/\*/\*
> /home/\*/\*/\*/\*/\*
> /home/\*/\*/\*/\*/\*/\*
> /home/\*/\*/\*/\*/\*/\*/\*
snip
> hope your users don't create directories more than five
> deep under their $HOME

After eight months of survey, it was turned out that some
of the users of TOMOYO has more than five deep under their $HOME.
So we decided to introduce recursive directory matching operators.
It should be available in 2.6.33.

http://lkml.org/lkml/2009/11/25/52

Thank you!

2.6.30 merge window, part I

Posted Dec 2, 2009 8:15 UTC (Wed) by nix (subscriber, #2304) [Link]

I noticed that fly past. Yay!

(but still, it was predictable ;P hell, I have directories eleven deep
down there now. part of that's because I need to reorganize...)