We are planning on using seccomp in QEMU specifically for mitigating against kernel bugs. This is in additional to use SELinux (via sVirt) to provide security (beyond that provided from running as a non-privileged user).
I agree that syscall filtering is strictly to reduce the kernel's attack surface. Access control should be done via an LSM module like SELinux.