Continuous-integration testing for Intel graphics

Posted Oct 12, 2017 14:05 UTC (Thu) by dunlapg (guest, #57764)
In reply to: Continuous-integration testing for Intel graphics by blackwood
Parent article: Continuous-integration testing for Intel graphics

Too little (in terms of coverage, even on intel hardware), too late (in the patch merge process, abusing linux-next to get your stuff tested really isn't cool imo).

Remember that people submit things to Linux that the core developers themselves don't use and often don't care about. It would be completely infeasible for the core developers themselves to actually implement regression testing for every driver and every bit of functionality that anyone ever used.

Which means there are basically two options:

Refuse all features / drivers that you're not personally interested in (and therefore willing to test)
Leave it to the people who care about a feature to do the testing and make sure it still works

The second one is what they do, and I think it's probably the best: The people who care about a feature should be the ones testing it and making sure it doesn't bitrot.

The main thorny issue with this is when changes to your functionality break other people's functionality. It doesn't seem fair that (say) Intel should have to keep chasing down driver failures caused by bugs introduced by someone tweaking some TLB-flushing algorithm somewhere. But on the other hand, it also doesn't seem fair that someone who just wants to tweak a TLB-flushing algorithm should have to fix (or work around) dozens of buggy drivers whose authors disappeared years ago.

Continuous-integration testing for Intel graphics

Posted Oct 12, 2017 14:44 UTC (Thu) by blackwood (guest, #44174) [Link]

This is easy to fix, just set up specialized CI down in the leave maintainer nodes for this stuff. So intel gfx folks make sure intel gfx keeps working, x86 maintainer make sure x86 core stuff keeps working, networking people try to keep networking in working order and so on.

Of course you can't cover everything, and the occasional regression on an odd combo of hw gets through.

But that's far away from the current world where patches land in linux-next that just take out 2/3rd of our machines, and intel desktop/laptop chips are probably the most well-tested platforms there are. That's just pushing untested crap in.

We want to spend the time of our CI engineers improving intel gfx CI, no chasing around the regression-of-the-day from some random other place that just took out the lab. Of course we have defense-in-depth and try to run linux-next before we have to rely on it when it all this the merge window. But then it takes weeks (or even months) to get even simple regressions fixed, and often you get the 0 day garbage that lands in the merge window (or even later) without any kind of serious testing except maybe a day in linux-next - hey at least it compiles!