Now you should not put (large) binaries into a VCS. VCS is for source code, not any form of compilation output. What you store in the VCS is a script which can pull the right, uniqely defined (version, sha1, etc.) binary from some server, or a build script, which produces the binary.
Where I work, we use subversion (and I use git svn :-). We have scripts which pulls the tar.gz files for various packages in specific versions from a server, unpacks, patches and crosscompiles them to our target. The only thing we have in subversion is the scripts and the files we have changed.
For the Linux kernel we tried to have the full thing in subversion, but it took way too much for subversion, so now we only have a makefile, which clones a git repository, when the source is needed.