Common needs for Samba and NFS
Amir Goldstein led a discussion on things that the two major network filesystems for Linux, Samba and NFS, could cooperate on at the end of day one of the 2019 Linux Storage, Filesystem, and Memory-Management Summit. In particular, are there needs that both filesystems have that the kernel is not currently providing? He had some ideas of areas that might be tackled, but was looking for feedback from the assembled filesystem developers.
He has recently just started looking at the kernel NFS daemon (knfsd) as it is a lesser use case for the customers of his company's NAS device. Most use Samba (i.e. SMB). He would like to see both interoperate better with other operating systems, though.
![[Amir Goldstein]](https://static.lwn.net/images/2019/lsf-goldstein-sm.jpg "Amir Goldstein")
He noted that he had asked some Samba developers why it has been hard to get the features they need into the kernel. The "vibe" he got is that they were rather intimidated by the kernel community. But he believes that the last time they tried, "they did it wrong". They wanted Samba-specific features, such as preventing writes or reads to a specific file, to be enforced by the VFS. Those kinds of changes were not acceptable to the VFS maintainers, however.
In talking with some NFS developers, it was agreed that Samba and NFS should "talk amongst themselves" to find areas where both needed kernel support. There are likely things that either can only be done in the kernel or are far better done in the kernel. If there is a minimal set of infrastructure that the kernel could provide to help solve those problems, it may be possible to get it added.
He started with opportunistic locks (OpLocks). In SMB, these locks can be requested when the client takes a lease on a file. If granted, that means the client does not need to flush its changes to the server as long as it holds the lock. If another client is accessing the file, the server can revoke the OpLock. There are large performance gains that OpLocks provide, so it is important to be able to fully support them.
Samba will use OpLocks, but only if it is configured in the mode that it will be the only user of the filesystem it is serving. If, say, an NFS server could also be touching the filesystem, Samba does not use OpLocks at all. There are also two levels of OpLocks; Samba uses both when it is configured as the only thing touching the filesystem.
Steve French said there is more to it than that. OpLocks are identified with a key, which would allow level-1 locks to be upgraded to level 2, but doing an upgrade is not implemented for Linux. He was not sure whether NFS could upgrade its locks or not. Trond Myklebust said that the NFS protocol allows upgrading, but that it is not implemented for Linux.
So Goldstein wondered what Samba and NFS need in order to be able to implement level 2. He plans to go to the Samba conference (sambaXP) in June and will gather more information there. One thing he knows needs to be done is to be able to track clients through the filesystem operations in order to manage their leases, which is similar to what Bruce Fields described for NFS in an earlier talk. That would take care of file leases.
For directory leases, Goldstein has been working on a way to get notifications of directory changes. He has hooks for doing synchronous notifications, but there are sensible concerns about exporting those, he said. He thinks that perhaps exposing them in a way similar to leases, with a timeout in case there is no reply, might be possible.
One of the problems that NFS has with leases, Myklebust said, is that it doesn't detect that they have timed out. If you miss your notification window, ideally the file descriptor would be closed or something else would "fence off" access to the file. That would be an indication that some kind of recovery action needs to be taken. Goldstein said that he is not tied to any specific solution; he is just bringing up areas that might need to be addressed.
Next up was share modes that can be specified when a file is opened. Those modes can request that any other opens of the file, for say read, write, or both, will fail. There is a patch set from five years ago that implemented a flavor of mandatory lock to support these modes, but it took the approach of enforcing the modes in the VFS.
Goldstein thinks that Samba and NFS could cooperate on a new flavor of lock that only they would use. It would be sort of similar to the BSD flock() system call, he said. New open() flags would be added to request this exclusive access (e.g. O_DENYWRITE and O_DENYREAD), but they would only be implemented for filesystems that want to enforce that.
Ted Ts'o said that some kind of document outlining the various problems is going to be needed. He suggested that it would be helpful to explain what the downsides to not implementing those features would be. Samba works pretty well, so explaining why kernel developers should care about these additional features may help smooth their path.
Goldstein said that he was mostly just trying to inform the other filesystem developers of his plans. Overall, he does not expect all that much to come of this effort, but "maybe we can move the needle". His list is not comprehensive by any means; he does not know if such a thing exists. Mainly, he wants to start the conversation about these network filesystem needs.
| Index entries for this article | |
|---|---|
| Kernel | Filesystems/NFS |
| Kernel | Samba |
| Conference | Storage, Filesystem, and Memory-Management Summit/2019 |
Posted May 15, 2019 17:33 UTC (Wed)
by asn (subscriber, #62311)
[Link] (40 responses)
Posted May 15, 2019 17:45 UTC (Wed)
by jra (subscriber, #55261)
[Link] (39 responses)
Posted May 15, 2019 18:42 UTC (Wed)
by Cyberax (✭ supporter ✭, #52523)
[Link] (38 responses)
Posted May 15, 2019 19:03 UTC (Wed)
by jra (subscriber, #55261)
[Link] (37 responses)
It's the kind of attitude expressed above that gives Linux devs a bad name, unfortunately.
Posted May 15, 2019 19:16 UTC (Wed)
by Cyberax (✭ supporter ✭, #52523)
[Link] (4 responses)
Besides, there are things that really require kernel-level support like case insensitivity. ACLs do not, they can be done completely in user space. Even if you need NFS interoperability, you can still use user-level NFS servers and patch in support there.
Posted May 15, 2019 20:47 UTC (Wed)
by smfrench (subscriber, #124116)
[Link] (2 responses)
The NFS and SMB3 ACL models are reasonably similar so getting NFS and Samba server to agree on enforcement would help some of the cases, but note that this isn't just Windows clients that use (and expect to report to users and admins and tools) the 'RichACL' like format, Macs do as well, and presumably as government requirements get stricter (whether we like it or not), the presence of more complex ACLs with deny entries (one of the key differences between RichACL and the draft POSIX ACLs that Linux implements) is going to be hard to avoid, especially for systems accessed over the network.
Posted May 15, 2019 21:20 UTC (Wed)
by Cyberax (✭ supporter ✭, #52523)
[Link]
> the presence of more complex ACLs with deny entries (one of the key differences between RichACL and the draft POSIX ACLs that Linux implements) is going to be hard to avoid, especially for systems accessed over the network.
Posted May 23, 2019 3:35 UTC (Thu)
by Wol (subscriber, #4433)
[Link]
Please NO NO NO.
Deny ACLs are something I've never understood. Less complexity == better security.
Just don't mix and match user and group ACLs. If the user ACL over-rides the group ACL, why do you need a deny? If someone has permissions via one group, and denies via another, surely there's something wrong somewhere? And, looked at logically, why should deny over-ride permit?
Cheers,
Posted Jun 3, 2019 11:27 UTC (Mon)
by pgoetz (subscriber, #4931)
[Link]
I work with scientists at a university and have a real need for various RichACL features (so not just of importance to a handful of mastodon companies).
Posted May 15, 2019 22:58 UTC (Wed)
by nix (subscriber, #2304)
[Link] (28 responses)
Posted May 15, 2019 23:05 UTC (Wed)
by Cyberax (✭ supporter ✭, #52523)
[Link] (16 responses)
The problem with RichACLs is exactly that - they are way too rich and complicated. In particular, inheritance is a whole flaming garbage can.
Posted May 16, 2019 13:52 UTC (Thu)
by bfields (subscriber, #19510)
[Link] (9 responses)
Posted May 21, 2019 23:14 UTC (Tue)
by nix (subscriber, #2304)
[Link] (8 responses)
Posted May 22, 2019 16:18 UTC (Wed)
by bfields (subscriber, #19510)
[Link] (7 responses)
If Cyberax is thinking about the automatic inheritance stuff, that's described here: https://tools.ietf.org/html/rfc5661#section-6.4.3.2.
It's not really "automatic". It just provides a few extra flags that userspace could use to do a sort of inherited-ACL equivalent to chmod -R.
Basically the problem is that somebody creates a new directory, sets an inheritable ACL on the root, populates it, then realizes a month later, oops, we should have given the "sales" group read access. And they'd like to not only fix the inheritable ACL on the root, but also have the change propagate to the whole tree. And, yes, people that implement filesystems tend to run away screaming from that problem.
The easier solution is to teach user utilities to do the recursive traversal and fix up the tree. Stuff like "chmod -R" already does that.
But, one annoyance is that in the intervening month further ACLs may have been set on subtrees, and you may not want to clobber all those.
So all the "automatic inheritance" stuff does is add flags that userspace can use to determine which ACEs in a given ACL were inherited from somewhere else, so they can (if they want) use that information to decide how to modify the ACL while doing a recursive traversal.
Posted May 22, 2019 18:07 UTC (Wed)
by Cyberax (✭ supporter ✭, #52523)
[Link] (3 responses)
And a formerly fast atomic rename operations can take many minutes, and a crash in the middle of a re-ACL-ing can leave them in an indeterminate state.
Posted May 22, 2019 18:22 UTC (Wed)
by jra (subscriber, #55261)
[Link]
Only userspace tools change "automatic" inheritance bits on modification, not the kernel. So the comment about filesystems being left in an inconsistent state on rename makes no sense whatsoever.
I'm starting to understand that the objections here are coming from the fact the objectors don't actually understand RichACLs *at all*, or understand what is required.
Posted May 29, 2019 0:03 UTC (Wed)
by Wol (subscriber, #4433)
[Link] (1 responses)
Simple, reasonably efficient mechanism ...
First of all, don't "mix and match" ACLs - the rule should be "the first hit is the only hit". Which yes it means a not particularly efficient traversal up the path looking for an ACL, but you don't get things like wrongly updated trees because the tree isn't updated. Update the root ACL, all new accesses will find it straight away. How fast is an upwards traversal reading the ".." entry in the directory?
Everything I find a pain in Windows ACLs and others is all down to rules about how conflicting ACLs interact. Pr1me ACLs just worked on the basis of "the first hit applies". SOOO much simpler.
Cheers,
Posted May 29, 2019 15:10 UTC (Wed)
by bfields (subscriber, #19510)
[Link]
And, anyway, we're not really looking for a clever way to make automatic inheritance work, we're just trying to provide a little Windows compatibility. And the Windows folks (quite sensibly, in my opinion) chose to do this much simpler thing.
Posted May 23, 2019 12:44 UTC (Thu)
by nix (subscriber, #2304)
[Link] (2 responses)
Unlike Cyberax, I like the idea of this -- it just seems... really hard to do well.
Posted May 26, 2019 20:17 UTC (Sun)
by bfields (subscriber, #19510)
[Link] (1 responses)
Sure. Just as for chmod -R. You'd run setrichacl with a --recursive option, and you'd wait while setrichacl traversed the tree fixing things up.
It's a bit annoying, but it's a simple idea and I think people understand it. They know it'll take a while on a big tree, they know that the job's going to be left half-done if they interrupt it partway through, etc.
(Disclaimer: I don't remember whether Andreas actually got around to writing that UI, or exactly how he did it if so. And I don't know the Windows UI. But my understanding is that, again, it doesn't try to hide the fact that there's a recursive operation when you ask it to propagate a new inherited ACL.)
This is a really simple low-tech feature. "Automatic inheritance" was probably a bad name for it, at least if you want to sell it to filesystem developers. Filesystem people hear "automatic inheritance" and immediately jump to the assumption that you're trying to do something much more sophisticated, and start worrying about all the corner cases.
Posted May 27, 2019 17:48 UTC (Mon)
by nix (subscriber, #2304)
[Link]
Posted May 16, 2019 13:55 UTC (Thu)
by bfields (subscriber, #19510)
[Link] (3 responses)
Posted May 16, 2019 16:03 UTC (Thu)
by Cyberax (✭ supporter ✭, #52523)
[Link] (1 responses)
Posted May 16, 2019 16:45 UTC (Thu)
by bfields (subscriber, #19510)
[Link]
Posted May 23, 2019 0:54 UTC (Thu)
by neilbrown (subscriber, #359)
[Link]
I would say that the whole idea of per-file permissions is fundamentally broken and not what people really want, and I wouldn't let "POSIX" ACLs get past "draft" stage - I would nip it in the bud and "just say no".
There are two classes of permissions that cover most files: 0600 and 0644.
The next most common scenario is a group that wants shared access to files. People in group "foo" would see "/group/foo" as a read-write mount point. People not in group "foo" wouldn't see this mount point.
Note that this removes the current awkwardness with user and group quotas. The quota doesn't apply to a "user" or a "group", but to a fileset, which might be /home or might be /group/foo.
If you want any permissions more subtle than that, you have a daemon that mediates all accesses. In some cases this might be a database server with it's own arbitrarily complex permission scheme (which might have field granularity within records). In other cases it might be a more generic file server which uses access controls similar to those that Apache supports in .htaccess files, and which only allows whole-file reads and writes.
ACLs? Just Say NO!
Posted May 27, 2019 23:12 UTC (Mon)
by ssmith32 (subscriber, #72404)
[Link] (1 responses)
Isn't this the same argument being made above about RichACLs..? Namely, they exist in large Enterprises so they should be supported.
I suppose it all depends on where you "live".
Granted, I mostly "live" on cloud providers & home machines nowadays, so both posix & whatever windows is doing is fairly irrelevant.
It seems like, wherever I go, no one is happy with their auth scheme, but it's always better than whatever the other thing is. .. ¯\_(ツ)_/¯
Posted May 28, 2019 1:13 UTC (Tue)
by Cyberax (✭ supporter ✭, #52523)
[Link]
These enterprises can very well carry these patches in a custom kernel build.
Posted May 15, 2019 23:16 UTC (Wed)
by jra (subscriber, #55261)
[Link] (9 responses)
Posted May 15, 2019 23:33 UTC (Wed)
by Cyberax (✭ supporter ✭, #52523)
[Link] (8 responses)
Posted May 15, 2019 23:57 UTC (Wed)
by rahvin (guest, #16953)
[Link] (7 responses)
It's petty and silly to suggest the community shouldn't bother with it.
Posted May 16, 2019 1:18 UTC (Thu)
by Cyberax (✭ supporter ✭, #52523)
[Link] (6 responses)
Posted May 16, 2019 2:05 UTC (Thu)
by areilly (subscriber, #87829)
[Link] (5 responses)
If your answer is some sort of WebDav HTTP-based synchronization thing, then sure, those exist, but they all work orders of magnitude more slowly and generally worse than either SMB or NFS.
Posted May 16, 2019 2:10 UTC (Thu)
by Cyberax (✭ supporter ✭, #52523)
[Link] (4 responses)
The only place I've seen SMB was a mid-size factory that was using it for industrial automation.
Posted May 16, 2019 7:34 UTC (Thu)
by smurf (subscriber, #17840)
[Link] (1 responses)
Posted May 23, 2019 8:06 UTC (Thu)
by HelloWorld (guest, #56129)
[Link]
Posted May 26, 2019 6:48 UTC (Sun)
by cpitrat (subscriber, #116459)
[Link] (1 responses)
Wait, what?
Posted May 26, 2019 9:09 UTC (Sun)
by zdzichu (subscriber, #17118)
[Link]
Posted May 26, 2019 6:58 UTC (Sun)
by cpitrat (subscriber, #116459)
[Link]
They are complex? Well, I don't know any area which is not / does not require complexity when you look carefully at it.
They are more complex than POSIX ACLs? Well, I'd expect that, the main complain about POSIX ACLs is that they are too simple. Of course, in theory you can do whatever you want with them. In practice, having a different group per file is not manageable. I don't know what RichACL supports but it seems to me that being able to have multiple groups and users allowed a certain access to a file is a minimum to make users life easier.
Posted May 16, 2019 20:48 UTC (Thu)
by neilbrown (subscriber, #359)
[Link] (2 responses)
Posted May 16, 2019 21:40 UTC (Thu)
by jra (subscriber, #55261)
[Link]
Posted May 17, 2019 13:23 UTC (Fri)
by bfields (subscriber, #19510)
[Link]
- identification of users and groups by UIDs and GIDs vs. SIDs: not sure if we can do much about this, and in any case it seems like a separate issue.
In the end, I trust the Samba developers' judgement that richacls would solve a major problem for them.
Common needs for Samba and NFS
Common needs for Samba and NFS
Common needs for Samba and NFS
Common needs for Samba and NFS
Common needs for Samba and NFS
Well, they should just remain on Windows then. Why should Linux get all the braindeadness of other OSes for the sake of a handful of mastodon companies?
Common needs for Samba and NFS
Common needs for Samba and NFS
Yet this will require pretty much indefinite support of it in the mainline Linux, and deep inside VFS at that.
Oh, please. Naked file servers are pretty much a corner case right now.
Common needs for Samba and NFS
Wol
Common needs for Samba and NFS
Common needs for Samba and NFS
Common needs for Samba and NFS
What don't you like about inheritance?
Common needs for Samba and NFS
Common needs for Samba and NFS
Common needs for Samba and NFS
Common needs for Samba and NFS
Common needs for Samba and NFS
Common needs for Samba and NFS
Wol
Common needs for Samba and NFS
Common needs for Samba and NFS
So all the "automatic inheritance" stuff does is add flags that userspace can use to determine which ACEs in a given ACL were inherited from somewhere else, so they can (if they want) use that information to decide how to modify the ACL while doing a recursive traversal.
But doesn't that mean that there might be an arbitrarily long lag between your changing a root ACL and having the change propagate to children? That sort of thing seems... rather suboptimal for something with security implications (like permissions), though I suppose you could have synchronization mechanisms so you could wait until the userspace fixer-upper had propagated the change. (Heck, it's even suboptimal for something *without* security implications -- can you imagine if rename() worked that way? rename(a, b) and now we have to wait for a while before a vanishes and maybe longer before b appears?)
Common needs for Samba and NFS
Common needs for Samba and NFS
Filesystem people hear "automatic inheritance" and immediately jump to the assumption that you're trying to do something much more sophisticated, and start worrying about all the corner cases.
Guilty as charged, though I haven't been a filesystem developer for a long time :)
And while we're at it, how would you design "POSIX" ACLs differently?
Common needs for Samba and NFS
Common needs for Samba and NFS
I'm not sure what you mean, or what design problem you're solving.
Common needs for Samba and NFS
Common needs for Samba and NFS
(Having "execute" as a permission bit is a cheap-hack and should be deprecated. Your $PATH defines which files are to be treated as executable, not the permission bits).
Files that today have mode 0600 should appear in /home *only* for the owner. Other users shouldn't be able to see them.
Files that today have mode 0644 can appear in global namespace in a read-only part of the tree, and appear writable in /home/shared for the owner.
Common needs for Samba and NFS
Common needs for Samba and NFS
No. POSIX ACLs are already supported in Linux, with all of their mis-features. Adding more crap for the sake of a couple of large enterprises is not a good idea.
Common needs for Samba and NFS
Common needs for Samba and NFS
Not quite. It's "'enterprise' legacy storage for Windows clients".
Common needs for Samba and NFS
Common needs for Samba and NFS
Common needs for Samba and NFS
Common needs for Samba and NFS
Common needs for Samba and NFS
Common needs for Samba and NFS
Common needs for Samba and NFS
"The only place I've seen SMB ..."
Common needs for Samba and NFS
Common needs for Samba and NFS
- RichACLs are bad
- why?
- because I don't like them
- but why?
- because they are crap
- but why?
...
Common needs for Samba and NFS
If they do, then I can see a case being made for them.
If not, then I think they are best avoided.
Common needs for Samba and NFS
Common needs for Samba and NFS
- support for mode bits: we could simplify richacls if we just ignored mode bits, but that didn't seem like a good idea.