as usual the truth is somewhere in between you two.
a very easy to use system that provides no security isn't worth even that small amount of effort.
on the other hand, the most secure system in the world does absolutly no good if nobody uses it.
SELinux is way out there towards the second extreme.
and before all the SELinux people jump in and say that millions of people do use it because it's default in Fedora, by use it I mean actually understanding what it does and being able to configure it.
it may provide a small amount of security in the default settings for people who just use the default setups, but the problem is that the people who are doing the things that most need security are not going to be using the defaults that the default SELinux policy is set for, and even those who do really need/want tighter security than that default provides (but a _different_ tighter setting than people who run some other set of servers, it should be tailored to the applications that that site is running)
SELinux falls on it's face in terms of being understandable to the sysadmin. On the other hand, if you have a multi-user system and really need to protect one local user from another, SELinux is the way to go.
But most of the servers running the services that most need to be secured aren't in this situation. On those servers the only local users are the administrators (and it's not unusual for the servers to be maintained via some tool so that even the sysadmins don't login to any particular box for months at a time, everything is done via a service account from a central admin tool)
In these cases, being able to easily understand the tool so that you can lock down the network accessible applications to limit what they can do will give you almost all the practical benefit of the more complex system, but since it's USED (i.e. tuned for a particular installation) while the more complex system isn't, the practical security value is much greater for the simpler version.