A rough start for ksmbd

Posted Oct 28, 2021 1:51 UTC (Thu) by neilbrown (subscriber, #359)
In reply to: A rough start for ksmbd by nybble41
Parent article: A rough start for ksmbd

> but rather performing all accesses as the particular user whose credentials were used to authenticate to the server

This doesn't mean anything for NFS. NFS doesn't authenticate a connection, it authenticates each request.

With NFSv4, there is a "first" request (EXCHANGE_ID I think in v4.1 and v4.2) and almost all other requests inherit a "state" from that. This is mostly used for clear ordering and exactly-once semantics.
You seem to be suggesting that the credentials used to authenticate all subsequent requests should be ignored, and the credentials of the "first" request should be used throughout.

If don't think that would be useful with any current NFS client, as they use "machine" credentials to authenticate the state management, and that doesn't necessarily map to any UID. Obviously you could change the NFS client to behave differently, but then you would just change it to send the credential you want the server to honour.

What precisely is it that you want to achieve. I'm in favour of making NFS useful for more use-cases, but we would need a clear description of what the use-case is.

A rough start for ksmbd

Posted Oct 28, 2021 17:19 UTC (Thu) by nybble41 (subscriber, #55106) [Link] (6 responses)

> NFS doesn't authenticate a connection, it authenticates each request. … You seem to be suggesting that the credentials used to authenticate all subsequent requests should be ignored, and the credentials of the "first" request should be used throughout.

Not exactly. The CIFS/SMB multiuser model is a closer fit, where the kernel maintains the credentials for each server in a per-user keyring. One would need to do something about the flaw that SMB multiuser mounts still require valid credentials for an account with access to the share at mount time[0], though perhaps an NFS-based equivalent wouldn't have that problem. It doesn't really matter whether there is a single connection or multiple connections as long the credentials are not tied to a specific shared UID or username between the client and the server and all access checks are enforced on the server (i.e. the client can be untrusted). And of course I'd rather have POSIX/Linux filesystem semantics like NFS as opposed to a protocol originally designed around the Windows VFS. The protocol would obviously need to be hardened and encrypted to be a suitable alternative to SSHFS (SFTP) over the Internet and not just LANs. Regarding authentication, I currently require public keys for all SSH logins on my server, and I'd rather not go back to passwords.

The full use case is basically this: Given any random Linux server which can be accessed through SSH, I would like to be able to mount a filesystem from this server from a separately-administered client machine using a kernel-based filesystem module, with the full POSIX semantics available from NFSv4 mounts and without the overhead and limitations of FUSE. The same mount point should be available to multiple users on the client, with each user accessing files on the server through their own existing SSH login credentials. In other words: Starting with SMB-style multiuser mounts, allow mounting without any default credentials, use the NFS protocol for the actual filesystem operations, and add public-key authentication and secure encryption akin to SSH.

(One option for the authentication would be to actually perform an SSH login in userspace when adding the credentials with a fixed command which, on success, registers a temporary session key which can be loaded into the client's keyring and used for all further requests. This seems like it would be fairly ergonomic and wouldn't require the kernel to implement all the different authentication types supported by SSH.)

The existing SMB3 support would probably be "good enough", though not ideal due to limited POSIX support, if it weren't for the issue of requiring mount-time credentials. I could even emulate SSH authentication by scripting a remote smbpasswd command with a temporary password, though that only allows one client machine at a time for each account and might involve running smbpasswd as root (with restricted options) to allow a new temporary password to be set without knowing the old one.

[0] https://lwn.net/Articles/874180/

A rough start for ksmbd

Posted Oct 28, 2021 22:17 UTC (Thu) by nix (subscriber, #2304) [Link] (4 responses)

Another way to emulate this stuff would be doing what sftp does: use SSH subsystems. An ordinary command that forks ssh -s $subsystem_name implements the client side, using FUSE; it serializes the requests, echoes them into the ssh -s process's stdin. The server side of this is an ordinary filter forked by sshd just as it forks sftp-server; this takes the serialized requests from the client on stdin, does... whatever on the filesystem (using ordinary fs ops: no need for special permission magic or fsuids because you are *already* the right user, properly authenticated by ssh), then serializes the results over its stdout. The client-side program receives them from its ssh -s invocation and hands them back via FUSE.

A place to start on the server side of this is already written in the form of the sftp subsystem, though it doesn't implement remotely enough operations and probably the serialization protocol should be rethought, since we are not at all wedded to the sftp protocol. The biggest problem is that by default this thing would be single-threaded, but a multithreaded version is perfectly possible that fires up multiple worker threads (possibly in an expanding-as-needed thread pool), kicks off separate ssh -s's for each one, and lets things rip accordingly.

Nobody has written any of this, but it's purely userspace coding, likely fairly humdrum, and the performance impact of FUSE is probably going to be ignorable compared to the unavoidable performance impact of, well, using SSH (and honestly for all but really big bulk ops or ops on machines with slow CPUs I think you won't even notice that).

... oh dammit I want to write this thing now. (Assuming nobody already has. I haven't even looked, but given the number of people who seem to be even *aware* of SSH subsystems, let alone how damn useful they are for things like this, I do strongly suspect that nothing like this exists.)

A rough start for ksmbd

Posted Oct 29, 2021 6:04 UTC (Fri) by nybble41 (subscriber, #55106) [Link] (3 responses)

I believe you're describing SSHFS[0], though perhaps with a richer subsystem then SFTP. SSHFS is great; I use it all the time. But it does tend to have some issues. FUSE filesystems are rarely as performant as their native equivalents. If nothing else you need several extra context switches for each operation (app -> kernel -> FUSE -> kernel -> app), and in my experience large file transfers without explicit bandwidth limits can make the rest of the filesystem non-responsive. The latter issue may be more of an implementation issue with SSHFS or SFTP rather than FUSE itself. It's not strictly single-threaded, so you can still access other files, but it doesn't seem to load-balance fairly. FUSE filesystems also run as ordinary users while servicing requests from the kernel, perhaps from other users or even root, which means they have some security issues to mitigate which may not apply to an in-kernel filesystem. And it would be difficult (though not impossible) to implement something like SMB3 multiuser mounts via FUSE where all local users see the same paths but access them with their own remote credentials.

An SSHFS equivalent using something like the NFS protocol (without any NFS authentication, just acting as the logged-in user) through an SSH tunnel instead of SFTP would be an interesting design, though it doesn't address my main design goal of migrating the filesystem away from FUSE and into the kernel.

[0] https://github.com/libfuse/sshfs

A rough start for ksmbd

Posted Oct 29, 2021 12:51 UTC (Fri) by nix (subscriber, #2304) [Link] (2 responses)

Sort of. To minimize installation difficulties (since subsystems have to be configured on the server side with one line in sshd_config), sshfs doesn't use the subsystem mechanism but implements its own transport, which means it has to encode everything passing over the wire and relies on the far side's shell being set up sanely and the like. But sshfs is probably a good place to start from!

A true multiuser permission-respecting filesystem... well, I guess if you ssh as root it could setfsuid as needed as requests came in. That's what the fsuid is for, after all.

A rough start for ksmbd

Posted Oct 29, 2021 14:54 UTC (Fri) by nybble41 (subscriber, #55106) [Link] (1 responses)

> sshfs doesn't use the subsystem mechanism but implements its own transport

The code in sshfs.c[0] appears to pass "-s sftp" to the SSH command by default (i.e. using the subsystem mechanism) unless the sftp_server option is set (with a path) or the SSHv1 protocol is selected.

> A true multiuser permission-respecting filesystem... well, I guess if you ssh as root it could setfsuid as needed as requests came in.

The kernel SMB3 implementation creates a separate connection for each user, and I'd probably do the same thing here. Many systems, my own included, don't allow direct root logins via SSH; ssh as root + setfsuid on the server would essentially mean trusting the client machine with root access to the server, and even with restrictions such as only allowing this one approved subsystem it could be used to bypass SSH login policies.

The FUSE filesystem would need to be set up by root on the client with the allow_other option to permit shared access. You could have an interface for users to link their ssh-agent to the FUSE filesystem so it can connect on their behalf (using keys), though I'm sure there would be all sorts of interesting security and UX implications.

[0] https://github.com/libfuse/sshfs/blob/master/sshfs.c

A rough start for ksmbd

Posted Oct 29, 2021 17:32 UTC (Fri) by nix (subscriber, #2304) [Link]

> The code in sshfs.c[0] appears to pass "-s sftp" to the SSH command by default (i.e. using the subsystem mechanism) unless the sftp_server option is set (with a path) or the SSHv1 protocol is selected.

OK I'm too tired to think then, or simply can't read. It really is there and really obvious :) I guess that shows I was thinking of the right design, since sshfs is already doing it!

OK, so the right thing to do is to soup up sftp-server until it can do everything FUSE can be asked for, then soup up sshfs to talk to it and add a thread pool etc to it :) if this doesn't work (rejected by upstream), sshfs could ship its own variant (under another name: sshfs-server) and use it if set up on a remote system.

A rough start for ksmbd

Posted Oct 29, 2021 4:14 UTC (Fri) by neilbrown (subscriber, #359) [Link]

Thanks for the extra context.

I interpret your problem description as "You want a key distribution protocol based on ssh rather than kerberos, and you want NFS to be able to work with the keys thus distributed".

NFS is designed to have pluggable authentication systems, but krb5 wrapped in rpcsec/gss is the only one that is actually implemented.
The kernel "knows" about krb5 certificates and encryption scheme, but out-sources to user-space for distributing those certificates and keys.

I wonder if it would be possible to use an ssh-based scheme to distribute keys. I have no knowledge of the internals of krb5 certificates, but my guess is that it isn't completely out of the question. You would need to modify or replace gssproxy on the server and rpc.gssd on the client.

An alternate possible direction involves NFS over TLS. This is a draft standard for this, and I think there is prototype code. Whether the standard allows the credential for the connection to be used for FS requests, I don't know. If it did, then this might be a direction that could be standards-complient and so more likely to be implemented widely.