Straightening out ioctl() size confusion

Straightening out ioctl() size confusion

It is a historical mistake that the four-bytefield ioctl-CMD constant (type, number, size, direction) has found its way into the Linux kernel. It comes from the BSD-design, as contrasted to the AT&T design which uses a two-byte CMD-mechanism (type and number only). Berkeley introduced it because they wanted the "arg" argument, if it was of a pointer type, to be a pointer into kernel address space. In a BSD kernel a higher kernel part does the data transfer between user address space and (malloc'ed) kernel space v.v. and hence this higher part, which has no knowledge about what it is transporting at all. must be told the size and direction of the transfers. Think of this higher kernel part as being near the system call dispatcher (for data from user to kernel) and near the point where syscall return values are handed back (for data from driver to user). The benefit of this approach is that driver code can use the "arg" pointer in memcpy, strcpy or even cast it to a structure pointer. Linux adopted the AT&T style, where the "arg" argument serves as a pointer into user space and requires the driver to use the helper-routines copy_from_user and copy_to_user to do the transfer. The Linux-combination of BSD-style-CMD and AT&T-style-arg is the root of the evil.

In the AT&T-style the CMD-code only needs a "number" component to control the "switch" statement that every ioctl-driver-routine effectively is, and
it needs a "type" component to make sure that in a stack of drivers the right driver picks out the ioctl-call (or decides to pass it on downward in the stack; and yes, there has been a SysV-case where two drivers using the same type-character were put in a single stack and thus caused a soundcard to hiss when actually some mouse control was intended).

All relevant information re. the data involved in an ioctl should be conveyed between user and driver via the "arg" parameter exclusively, and not via CMD-components. Driver writers using the size-component in the CMD to draw conclusions actually pile a mistake of their own on top of a mistake in the design of the overall framework.

I understand the reasons for trying not to break existing driver code, but now the decision has been made to go for an incompatible change it is a missed chance that the cleanup has not been more thorough. The introduction of e.g. the _IOC_TYPECHECK(t) is still a continuation of the historical mistake and therefore most probably not a final end to all discussions. The size and direction should be dropped alltogether, and the type should be expanded into some better driver-signature-id for the stacking case.

Hendrik-Jan Thomassen <hjt@ATComputing.nl> (wrote my first UNIX driver in 1975...)