Doing this for a firewall would get messy -- a firewall thinks about packets, not endpoints.
If the kernel had programmable policy for what tasks could listen, accept, and connect on which sockets to which endpoints, on the other hand, firewalls could (on non-routers, anyway) go away and everything would get simpler and faster.
And no, selinux doesn't count in my book. Try actually programming the policy.