|
|
Subscribe / Log in / New account

No safeguards?

No safeguards?

Posted Apr 27, 2025 0:03 UTC (Sun) by gmaxwell (guest, #30048)
In reply to: No safeguards? by marcH
Parent article: Some __nonstring__ turbulence

I have a set of shell scripts that will go through source code and make varrious mutations one at a time such as change + to -, replace && with ||, replace 0 with ~0, change 1 to 2 or 2 to 1, swaps <, >, and =, inserts negations, blanks a line entirely, etc. Then it attempts to compile the code with optimizations. If it compiles, the script checks the sha256sum of the resulting binary against all the hashes it's seen before and if it's a new hash it runs the tests. If the tests pass the source is saved off for my manual inspection later.

The single point changes tend to not make errors which self cancel out, and usually if an error does cancel out or the change is in code that doesn't do anything the binary will not change. In code where tests have good condition/decision branch coverage most things this procedure catches are test omissions or bugs.

This approach is super slow and kludgy, I've been repeatedly surprised and frustrated that no one has made a C-syntax aware tool to do similar testing without wasting tons of time on stuff that won't compile or won't make a difference (e.g. mutating comments.. though sometimes I've addressed this by first running the code through something that removes all the comments).

But it's worked well enough for me and parsing C syntax is far enough away from the kind of programming I enjoy that I haven't bothered trying to close this gap myself.


to post comments

No safeguards?

Posted Apr 27, 2025 1:51 UTC (Sun) by roc (subscriber, #30627) [Link] (1 responses)

This is called mutation testing. There are a lot of existing tools for it, some of which are C-syntax-aware. Also there are mutation-testing tools that work by patching binary code.

No safeguards?

Posted Apr 28, 2025 14:49 UTC (Mon) by daroc (editor, #160859) [Link]

LWN covered one such tool for Rust code in October. I've tried it in some of my personal projects since then and found it somewhat useful for expanding my test suites.


Copyright © 2025, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds