kdevops-ng [LWN.net]

kdevops-ng

Posted Jul 21, 2026 9:46 UTC (Tue) by dagmcr (subscriber, #143683)
Parent article: Topics in filesystem testing

Some of the topics discussed here came up recently on the kdevops list:

https://lore.kernel.org/all/9f64bee9-ecc3-4587-9645-21902...

The thread is about moving kdevops onto Nix, after one too many distro
updates broke my test host setup. Nix fixes the environment problem
across any distro, and together with systemd to run QEMU guests and
drive tests as plain services, the proposal allows kernel tests
frameworks or benchmark suites to be reproducible and portable across
machines and distros. For the orchestration on top I stumbled on
Windmill, a workflow engine that turned out to fit really well, and you
get a CLI, a web UI, scheduling/CI, dashboards and runners out of it
for free.

The idea is to keep what kdevops already does, but with other tools
that enable these things mentioned. Also, results that record exactly
which kernel and environment produced them, which is the "environments
make results hard to compare" problem. Bisection (wip) and more.
Still a PoC and very much under discussion. Docs and demo videos:
https://kdevops-ng.readthedocs.io/.

to post comments

kdevops-ng

Posted Jul 21, 2026 21:37 UTC (Tue) by koverstreet (subscriber, #4296) [Link] (2 responses)

Yeah, NixOS is perfect here. I haven't yet moved the core of my test infrastructure to Nix - the Rust migration has been higher priority - but it's been on the todo list for ages, and much of my server deployments are already converted.

For this kind of infrastructure, NixOS is just as big a deal as Rust is for kernel code.

kdevops-ng

Posted Jul 22, 2026 8:25 UTC (Wed) by dagmcr (subscriber, #143683) [Link] (1 responses)

> Yeah, NixOS is perfect here.

Quick clarification: you don't need full NixOS. Nix (the package manager) installs next to whatever distro you run, so there's no reinstall or infra migration. You keep your usual distro/userland. The tax is that you still have to learn the Nix way, which is probably the "yet another thing to learn" worry from the thread I linked. One payoff: the same /nix/store that boots the guests is what you run on baremetal too, so the two stop being separate setups to maintain.

But Nix is only one of the pieces. In the PoC, systemd runs the VM (and test suite) lifecycle, that let me cut the dependencies for running QEMU and
simplify both the process management and the QEMU incantations -> zero wrappers -> less things to maintain. Windmill ties all this together with a UI and a CLI, and your workflow steps can be in whatever language you like (Python, Bash, Rust, ...).

kdevops-ng

Posted Jul 23, 2026 6:19 UTC (Thu) by koverstreet (subscriber, #4296) [Link]

Yeah, I was referring to the test VM root filesystem.

For the uninitiated, NixOS has very slick stuff which completely removes the whole "generate a root image" step. Greatly simplifies deployment, and keeping everything updated and in sync :)