Advertisement
Free copy of The Founder's Checklist and The Founders Pitch Deck Template from M L Bittle - New York; Advisor/Coach.
| |||||||||||||
A kernel change breaks GlusterFSA kernel change breaks GlusterFSPosted Mar 27, 2013 22:06 UTC (Wed) by jengelh (subscriber, #33263)Parent article: A kernel change breaks GlusterFS
If glusterfs has no problem with 64-bit cookies that XFS uses, why would gluster suddenly have an issue with 64-bit cookies from ext4? Or is it that the gluster on-disk format is dependent upon the fstype?
(Log in to post comments)
A kernel change breaks GlusterFS Posted Mar 27, 2013 23:32 UTC (Wed) by bronson (subscriber, #4806) [Link]
I was wondering that myself. What singles ext4 out for breakage? Hoping someone knows.
A kernel change breaks GlusterFS Posted Mar 27, 2013 23:43 UTC (Wed) by bronson (subscriber, #4806) [Link]
Ha, I read the articles in the wrong order. Last paragraph of the companion article: https://lwn.net/Articles/544520/
Sounds like XFS also uses 32 bit cookies but uses some additional memory to ensure it doesn't suffer the collision problem.
A kernel change breaks GlusterFS Posted Mar 28, 2013 0:35 UTC (Thu) by bfields (subscriber, #19510) [Link] Traditionally the cookie is a byte-offset into some linear representation of the directory. If you do that, then in practice a "64 bit" cookie in practice probably isn't going to exceed 2^32 until you have millions of entries. ext4 is unusual in that it uses a hash of the filename as the cookie--so the 64-bit cookies are effectively random 64-bit sequences, and really do use the high bits. (Well, OK, actually they're limited to 63 bits, but you get the idea.)
A kernel change breaks GlusterFS Posted Mar 28, 2013 9:20 UTC (Thu) by paulj (subscriber, #341) [Link]
So really it'd be much better (ignoring legacy constraints) to use the filename as the handle for any readdir-like directory iteration operation?
A kernel change breaks GlusterFS Posted Mar 28, 2013 9:52 UTC (Thu) by dlang (✭ supporter ✭, #313) [Link]
Sure, except that there are those pesky legacy constraints.
The latest NFS protocol specifies that the token passed over the wire is a 64 bit value (earlier versions specified a 32 bit value)
Other network filesystems have similar specifications.
In fact, from a little googling, it looks like the POSIX spec says that these cookies are of type 'long'. this makes changing it to something like a variable length string like you would need to use the filename extremely hard.
A kernel change breaks GlusterFS Posted Mar 28, 2013 10:32 UTC (Thu) by paulj (subscriber, #341) [Link]
Sure. There's 2 trains of thought going through my head:
a) How to get a stable order over fixed-length IDs (dir size limited to the ID space then).
b) Assuming you can't get that, how would you design sane APIs that still allowed iteration of directories?
For b, it's names. However, I still don't understand why 'a' isn't possible. See my comment on the companion ext4 article about using a ring for the ID space.
A kernel change breaks GlusterFS Posted Mar 28, 2013 16:29 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]
Use "last returned filename" as a cookie. Easy and simple - Amazon does this for S3 and it works splendidly.
|
|||||||||||||