"compiler benchmarks without optimization enabled"
Compiler flags are shown on the benchmark results themselves. Generally, the compiler flags used during testing are whatever the default flags that are set by the upstream program under test. The Phoronix benchmark philosophy is all about real-world, reproducible tests -- such as if most users obtaining the package manually are just doing ./configure; make; make install. The real-world tests are also targeted at end-users; I think this is where some of the disconnect comes in over testing methodology as it tries to be representative for a majority of the Linux end-users. This is also why Ubuntu is mostly used with Phoronix tests... There are, of course, also Phoronix benchmarks that look specifically at compiler tuning with different optimization levels, CPU targets, etc.
"a FreeBSD kernel with debug info compiled in"
This was one or two articles -- out of 2,000+ articles and 6,000+ news posts I've written -- quite a while ago when the FreeBSD camp didn't make clear all of the debugging options they had enabled within their kernel.
What Martin didn't know at the time of his blog posting that it was one of several articles being written on the subject... You can find many other window manager / DE benchmarks on Phoronix with a variety of GPUs/drivers/systems testing the "out of the box" performance.
I'm not saying that mistakes don't happen, as they obviously do when covering many different areas and single-handedly writing thousands of articles while also engaging in other full-time projects that consume 100+ hours total per week. And for those who get all bent out of shape over the occasional beer photo or "alcohol propaganda", it's completely silly, especially when those photos are used to subtly hint ('leak') at future details about a given product/topic if you read enough into the beer selected, photo details, etc.