LZ4: vendoring in the kernel [LWN.net]

LZ4: vendoring in the kernel

Posted Feb 2, 2017 9:51 UTC (Thu) by karkhaz (subscriber, #99844)
Parent article: LZ4: vendoring in the kernel

> number of patches is small. Happily, they exist as separate patches, rather than having been quietly folded into the source when LZ4 was initially added to the kernel.

To clarify, are these "separate patches" a bunch of separate commits, or do they exist as individual patch files somewhere in the tree? I had a quick look through the tree but didn't find any patch files.

If they are separate commits, then indeed GKH's concern is valid, one must remember to rebase those commits back onto tip-of-tree every time the vendored code is updated. But there's a better way: if all the patches applied by the kernel are kept as separate files, and the `patch' command is used _as part of the build process, during every build_, then nobody has to remember anything. You keep the vendored code vanilla, and each patch is kept separately, and every time a patch gets upstreamed you remove it from the tree and update the vendored code to match.

This is how linux distros typically do things. See e.g. Arch Linux's source package for the kernel itself [0]. When building the package from source, the PKGBUILD file contains instructions on how to download the (vanilla) kernel, and also contains invocations to `patch' to correctly apply those (Arch Linux specific) patch files that you see in the directory. If I want to compile the kernel with _my own_ patches in addition to the Arch ones (so that it still works nicely on my Arch box) then I simply add one more patch file and run the build again.

[0] https://git.archlinux.org/svntogit/packages.git/tree/trun...

to post comments

LZ4: vendoring in the kernel

Posted Feb 2, 2017 10:07 UTC (Thu) by gregkh (subscriber, #8) [Link] (1 responses)

The changes were commits to the source code, the kernel tree does not keep patches that need to be applied when the build happens.

You can see them easily by running 'git log' on the specific files you are curious about.

LZ4: vendoring in the kernel

Posted Feb 6, 2017 3:57 UTC (Mon) by tterribe (guest, #66972) [Link]

We use the approach karkhaz describes in Firefox, also, with some amount of success: each vendored library has an update.sh script which copies the relevant files from a checkout of the relevant upstream version and applies all local patches (usually named after bug numbers that describe the issue and give steps to reproduce). If run, it should produce exactly what is in the tree.

Primarily, it serves as visible documentation so someone doesn't have to remember to run 'git log' on a bunch of files before patching/replacing them. The fact that you have to update it when you patch the vendored code is also a good reminder that someone else (like a future you) will have to deal with that patch on the next code import, and gives good incentive to move patches upstream if possible.