Huh? That shouldn't be the case at all. The memcheck tool should work fine regardless of what pointer you access the data through. Have an easily reproduced example? The fontconfig things I thought to try here are running clean in valgrind for me.
I googled around and found cases of people complaining about valgrind reports in older versions of it— but those cases sure appeared to be reads straddling distinct calls to malloc, which isn't valid (even if the memory happened to appear contiguous, IIRC you're not permitted to read across the boundary because it could be in separate segments).