The solution to that is to let one central server keep an up to date git svn clone around (run git svn rebase in a cron job).
Then you can use that to speed up clones for working on. I have a small script for cloning a git tree then adding the necessary information to sync against the svn server it was cloned from: http://nbd.name/gsc