Here are basically my thoughts about this:
- each VRAM page should be marked with the id of the GPU user that created it.
- the wiping process requires another attribute (being_wiped) on each page. Before scheduling a bunch of page wiping, the correspondent pages should be marked "being_wiped". When the wiping process is done, the pages should be marked wiped (not linked to a single user) and the being_wiped should be cleared.
- when allocating memory for a user, memory pages that were previously used by this user should be use for the allocation. When none is left, wiped memory pages should be used. If none are left but there are pages being wiped, then the allocator should wait for them to be wiped before using them. Otherwise, just return ENOMEM.
The wiping process should be run on the whole VRAM at boot time and then can be scheduled when the pool of wiped pages is getting low. This should lower the number of wiping and thus, lower the VRAM bandwidth usage.
I expect the performance-impact to be minimal in most cases. However, it makes the allocation more complicated and more memory-consuming.
On the other hand, if we want all buffers to be wiped at allocation time, then we can lower the allocation complexity at the expense of memory-bandwidth usage (more wiping are needed).