well, it worked

I see a lot of these hypotheticals which seem to imagine that employers are trying to figure out who they should hire out of Jamie Zawinski, Linus Torvalds or John Carmack. Oh dear, what if Jamie spends the whole time ranting about a bug in Emacs and forgets to close a parenthesis? What if Carmack mis-understands and optimises for a scenario where half precision incurs a performance penalty but doesn't consider cache effects? Wouldn't it be terrible if our process couldn't detect that these people are like Gods walking among us?

That's the wrong problem. The problem I've actually /seen/ is that applicants often don't know how to write a program.

Our pre-screening with Codility was put in place because interviewing applicants who can't write code was driving people nuts. As you describe, we had previously relied upon the arbitrarily discriminatory but effective method of hiring people who we personally knew could write programs. This had been effective when it was possible, but there are only a finite number of competent people you know who don't already have perfectly nice jobs, and as you get older very few of these people are looking for the sort of entry-level positions where "non-programmer applicants" are a big problem.

All we actually did with Codility was fire off the links, reject candidates who ignored them or stared at the problems without making a serious attempt to solve them, and interview everybody else. Codility gives a good programmer ample time to ace every problem, write their own unit tests, and try to squeeze out better performance - but we were deliberately not trusting a machine to screen for "good", just for "basically competent". We wanted to go into every interview knowing "this application for the programming job can write programs".

And that cut the flood to a trickle. It's really that easy.