SHA-1 can definitely be a bottleneck in some situations. The most extreme -- though I'm not sure whether git does this -- is verifying hashes on an initial clone. (The idea is to prevent one person's disk corruption on an old file or whatever from spreading throughout the network of clones.) Here the disk and network cost is proportional to the size of the delta-compressed repository, but the SHA-1 cost is proportional to the size of the uncompressed repository, which can easily be in the terabyte range.
It can also easily be the bottleneck on, say, committing a large merge (many modified files, all in cache because they were just written).