To clarify, my solution is all user space, no kernel changes. It's just changing the kernel from holding a 'strong reference' to a 'weak reference'. (You know what they say about adding a layer of indirection...)
When I did epoll, I used it in edge_triggered mode (and also not 'oneshot') and had a single thread processing the epoll events and scheduling workers. I had worker threads that just pumped data to and from the kernel. I put direct pointers in the epoll data (i.e. strong references) to my internal structures since I had only one thread calling epoll.
But if I had multiple threads calling epoll, I think the solution I outlined would work fine. I don't see what else I would need the cookie for... as long as its still doing its job, pointing to a real data structure of mine.
And the nice thing is, it works with all epoll modes. What's very distasteful about the kernel patch is that it requires ONESHOT (yuck!).
The epoll designer(s) had their thinking caps on with this api. Storing arbitrary cookies + edge triggered mode = Insanely good.