A kernel unit-testing framework
A kernel unit-testing framework
Posted Mar 3, 2019 20:06 UTC (Sun) by roc (subscriber, #30627)In reply to: A kernel unit-testing framework by k3ninho
Parent article: A kernel unit-testing framework
For example Firefox has very few true "unit tests", i.e. tests that test the functionality of one code module in isolation. Firefox tests are almost entirely "system tests", tests written in HTML, CSS or JS that test specific Web APIs, each test touching a lot of Firefox modules. There are good reasons to test this way. Those interfaces are public and therefore quite stable (especially after a feature has shipped), so tests need refactoring less often than tests that depend on internal interfaces. Also tests at this level can often be written to work on multiple different browsers, which is extremely useful. And testing modules in isolation often means mocking the interfaces of other modules which can be extremely expensive to build and maintain.
So while I think a policy of requiring tests with patches is important, I don't think they need to be *unit* tests.
Posted Mar 7, 2019 10:22 UTC (Thu)
by k3ninho (subscriber, #50375)
[Link]
The traditional sales pitch for predominant unit testing is lightweight, quick-to-evaluate method-level unit tests as the foundation of the Testing Pyramid thing: build trust in your program functions or methods, then build trust in your internal interfaces as you integrate components, then build trust in your external interfaces (user, API), finally trust you've installed it correctly with few fail-fast smoke tests.
I'm reconsidering whether that's right -- given the London School of Test-Driven design builds tests that reflect your design rather than (Chicago School's approach for) a test approach that evaluates every input (or class of inputs that get equivalent results) to the system via its component pieces. Somewhere, an automated framework reflects the full system in a second-system kind of way -- clearly not an acknowledged part of in-depth automated testing.
K3n.
A kernel unit-testing framework
Yeah, I railed on unit tests, when I also believe that there's a need, like security-in-depth, for testing-in-depth of different layers of the system in ways that are appropriate and convenient.