the realtime thing is there already in my current codebase
right now what I do is (in summary)
if realtime => slack is 0
if nice, slack is 0.5% with a max of 100 msec
if not nice, slack is 0.1% with a max of 100 msec
if not rt and slack is less than the per thread setting, use the per thread setting