Recently I did similiar tests for determining how well PostgreSQL would be able to deal with databases with potentially hundreds of thousands of tables. From what I found out, it's only limited by the file system's ability to work with that many files in a single directory.
So I tried that and put about 4.3 millions of files in a directory on my ext4 file system. Took quite a while to create and delete later on but file access times where impressive. It seems like accessing a file by it's name in such a directory takes a pretty much constant amount of time. Reading in the directory is quite fast as well, though that obviously takes longer the larger the directory is.