Dentry negativity

The fact that these two problems might have the same name is certainly delightful.
I would describe "the cache invalidation problem" as "choosing whether to invalidate something, and what, and when".

> The primary difficulty here is deciding how many negative dentries to allow.

I think that is a narrow formulation. If you look at the email that introduced the recent patches you will see two problems:
1/ speed of cleaning up negative dentries at unmount
2/ speed of freeing up memory used by negative dentries when memory is needed for something else.

i.e. the problem is speed. Reducing the number of negative dentries might help - certainly with 1, probably with 2 - but it is a secondary issue, not a primary one.
Problem 1 could be addressed with some sort of lazy-unmount protocol. We don't *need* to release all cache memory before reporting success for unmounting a filesystem. We just need to be sure it can be released (not currently in use) and that it cannot be accessed again.

Problem 2 could be addressed by optimizing the code (maybe - it's probably quite light weight already ... though I wonder if a negative dentry needs to be linked into the list from its parent - probably not if an alternate way could be provided to invalidate the ->parent pointer when the parent is freed) or by pruning negative dentries earlier (hence my comment about a shrinker with a different 'seeks' value) or by pruning some other cache - because when you need memory, you don't much care exactly what is sacrificed to provide it.

> Do you have any insight into how we might measure the effectiveness of the current cache?

Disable it and try to get some work done.
Any cache provides diminishing returns as it grows beyond the size of the working set, and as the "working set" changes over time, you can only hope for an approximate size. We have that by applying modest vm pressure.
The best heuristic for "should I provide more pressure on caches" is "am I being asked to apply lots of pressure". So the more often we need to find free memory, the more memory we should find each time (do we already do that?)
But that needs to be balanced against the risk of creating unnecessary pressure by pruning things that are needed again soon (thrashing). Maybe there is value in keeping a bloom filter recording identifies of recently pruned objects, and if the hit rate on that suggests we are re-populating objects that were recently removed, we could slow down the pruning. I suspect there are some circumstances were such a filter would help, but I doubt it would really be useful in general.

A really big problem with this stuff is that you cannot improve estimates of need without accounting of some sort, and any accounting has a cost - particularly global accounting on a multi-core multi-cache machine. As the primary problem is speed, you want to make sure that cost is (nearly) invisible.