Yes, but with "HStore as a cache", you can do your cache updates in a transaction. The biggest problem with external caching is keeping them in sync. (To keep it in perfect sync, you'd need to build your own 2-phase commit.)
I can also imagine non-cache uses. For example, "has this user seen this announcement?". Seems silly to create a new table for that.
HStore is just "one more tool in the toolbox". The future of data is all about "picking the right tool", which requires knowing the engineering tradeoffs. (Actually, today it's more like "figuring out the engineering tradeoffs", but it will get easier.)