File permissions with NFS have been a constant thorn in my side for years. Eventually I buckled down and ironed out as many issues with my setup as I could, and while I would still say the whole ordeal was a mess, I would like to share the things I’ve learned so that others may hopefully avoid the frustration I had. Without further ado, let’s get started.

Brief overview

Both my client and server systems are running Ubuntu 20.04. Keep this in mind when doing things yourself as other distros or operating systems may have varying levels of support for NFS features.

This is not intended to help you actually set up NFS. It is also not meant to set up authentication as that’s a whole other can of worms, and permissions on NFS are mostly unrelated to auth.

What we will be going over is UID/GID management, basic unix permissions, and ACLs.

A quick note about Samba

Samba is great. Making authentication and less controlled environments work with it is so much easier than NFS that I would seriously advise you to consider using it instead unless you really need the throughput of NFS, have a very tightly controlled environment, or are just looking to learn.

UIDs and GIDs

When you mount an NFS share, it works much like a local filesystem by default. When your client looks at a file, it can see the UID and GID of the owner exactly as it exists on the server, and that value may or may not align with any users present on your system. Likewise, any files you create on the shared folder will have the same UID and GID owner as if you had created them locally. The main exception here is the root user. NFS has an option called “root squash” that should be enabled by default, which will automatically translate the root user into an unprivileged user (usually nfsnobody:nfsnogroup) for security reasons.

So with this default behavior, you will need to ensure that any system interacting with files on the share has the relevant users and groups with the same UIDs and GIDs. If we have a user named Bob, and Bob wants to use his files on the NFS share from two different machines, both of those machines must have Bob’s user account with matching UID and GID, or his files will show up as being owned by some random number, or even another user if someone else has that particular ID on the machine.

Solution 1: all_squash

The most trivial way would be to enable the all_squash option, which translates any user and group on the client into a single user and group on the server. You can set which user this maps to as well, in case you want it to use something other than the default nfsnobody:nfsnogroup. Let’s take a look at an example export utilizing this.

/shared-folder *(rw,all_squash,anonuid=1100,anongid=1100)

In this example, we’re exporting /shared-folder and any clients reading and writing to that share to get mapped to 1100:1100 on the server. Do note, the server does not actually have to have a local user or group with those IDs unless you intend to interact with those files on the server. There is one caveat to this behavior later in the group permissions section

Solution 2: coordinate UIDs and GIDs

The second solution, and the one I would recommend assuming you have full control of all machines in the environment, is to simply ensure that your UIDs and GIDs are consistent across all the devices interacting with the shares, regardless of whether they are accessing the shared directory locall or remotely. The ways you can do this are endless, but the least headache inducing method for a small number of users and groups would probably be to use an orchestration tool like Ansible to set up the same users and groups on all your machines. This is especially handy if you have many VMs and containers that might need to work with shared folders like I do. For more users and groups, or just for fun, you can use a solution like OpenLDAP or FreeIPA, although this can be a project all on it’s own. Personally, I use FreeIPA as I run many different containers and VMs, and being able to have user and group changes take hold without any additional effort is convenient.

Solution 3: UID and GID mapping

NFSv4 provides a number of ways to map local UIDs and GIDs to remote ones, however none of them are what I would consider simple. Unfortunately, there’s no way I know of to do this at the moment without having a full fledged kerberos environment working, and as such it is not covered here. If you find yourself climbing this tree, I would once again advise you to consider Samba.

Permissions and masks

As you’re probably already aware, you can set your exports to be read-write, or read-only. If you set them to read-write however, you still have to deal with the actual unix permission systems. They work the same as they do locally, and if you aren’t familiar you may want to brush up on that. You have the same permission bits, and user:group ownership, and your local user has to match up correctly.

The most important thing to note here is the umask. You generally want to make sure this is consistent across all clients and any other machines accessing your data. Since all my systems are running Ubuntu, and therefore have the same umask (0002), it’s not much of an issue. If you have a more mixed environment however, you may want to manually set the umask when you mount your share. There’s nothing less fun than realizing one of your clients has group read masked out, breaking any sort of sharing. Which umask you choose depends on your use case.

Group permissions

Here’s where that caveat earlier takes effect. If you only care about the primary UID and GID, things will work as expected. If you have any secondary groups however, there are some restrictions here. The primary factor is a setting called --manage-gids. If this is enabled, rather than using the groups client side, a lookup for your UID is performed server side, which means that if your UID server side does not exist, you won’t have the secondary groups, and if it does exist, it will only have the GIDs defined server side, whether or not you have them client side. Why bother enabling this you ask? Unfortunately with this option disabled, NFS only supports sending up to 16 groups, which can cause frustrating problems when that limit is exceeded as the server will only receive a subset of the groups you have client side. The upside here is that your groups need not exist server side. Personally, I disable this as I never have more than 5 or 6 groups for a user in my home environment.

ACLs

ACLs are almost as obtuse as the UID:GID mappings. If you have any sort of collaborative storage, you’re probably going to be using ACLs somewhere, and they are not the most intuitive things when combined with NFS. In my case, my primary usage of them is having a shared media-write group for services and users that want to write to the media folders without them all having to run as the same user. First off, if you aren’t familiar with regular ACL usage on Linux, you’ll want to figure that out.

NFSv4 has it’s own ACL system which is actually considerably more powerful than the ACLs available on Linux systems. The big issue here is that ultimately, we’re exporting linux filesystems which only support POSIX ACLs and unless we had a secondary location to store the ACLs, we’re restricted to the ones locally supported on the server. The most important thing to note here is that all POSIX ACLs can map directly to NFSv4 ACLs, but not all NFSv4 ACLs can map to POSIX ACLs. In effect, this restricts us to POSIX ACLs when dealing with Linux systems.

Fortunately for us, POSIX ACLs will automatically get mapped to NFSv4 ACLs when exported, and when setting NFSv4 ACLs it will attempt to map them to POSIX ACLs.

ACLs on the server

When you’re on the server, you literally cannot set NFS ACLs unless you were to mount the share. If you’re working locally with the files, just use the regular getfacl and setfacl tools. You can see if a file has an ACL the same way you normally would by using stat or ls -l. Unless you screw with it, this side of things should “just work”

ACLs on the client

The client is the opposite. When you are working in a mounted NFS share, you won’t be able to use the regular getfacl and setfacl tools. Not only that, but running ls -l will not show the + indicating an ACL. Unfortunately there’s no real solution for that last bit, but for setting and viewing ACLs, you can use the nfs4_getfacl and nfs4_setfacl. The syntax and options available for nfs4_setacl are quite different, and you must be careful to not try and create ACLs that cannot be represented by plain POSIX ACLs. On that note, I personally avoid setting ACLs entirely on clients and on the rare occasion I need to create new ones, I just SSH into the file server and create them from there.

Conclusion

With that, I believe I have covered pretty much all the pitfalls I ran into while getting a sane NFS setup on my home network. I hope you enjoyed this article and that it provided some benefit to you. If you have any questions or spot an error, feel free to send an email.