Repository size is one issue. Just imagine if all of kernel.org, gnome.org and freedesktop.org (perhaps add in all of XFree86 too for fun) lived in a single repository. Oh and don't forget binary blobs that are popular to be added to corporate repos. Initial clone with --depth=MAX, anyone?
Sure you could split it up, but uh, all too tightly integrated. Should anything go git in a future, I would guess all repositories will start with a fresh slate.