feat: support NFS storage pools

[Foxboron]

WIP draft PR for the feature. So far it works, a bit slow on my machine due to the id remapping which should probably be investigated.

• Investigate the ID remapping
• I'm unsure about the different migration options
• Only support vers=4.2, should be fine?
• Error out on missing source
• Any specific options we should include?
• Info array needs some QA. Not sure I understand all of it.
• nfs can't support xattr, do we need to handle this in other places than migrate?

Fixes: #1311

[Foxboron]

@stgraber if you have any opinions or pointers to the checkboxes feel free to look over and I can investigate a bit.

I suspect some research needs to be done to figure out how we should interact with the uid/gid and squashing behavior of NFS mounts.

[bensmrs]

Hi! Is this PR stalled? Do you need help?

[Foxboron]

@bensmrs I think I need a bit of help to make sure that the Info struct is correct and that I'm not missing any details from the migration steps. I was hoping @stgraber had time to point in the correct direction on this part.

[stgraber]

So initially my thought was to only allow VolumeTypeCustom on there, so only allowing NFS to be used for additional storage to instances either stored on local storage or stored on a proper block shared storage solution (clustered LVM, Ceph or Linstor).

I think we should at least exclude VolumeTypeBucket and VolumeTypeImage.

Container and VM, we'd need to see how well they work. My gut feeling given NFS locking is "not well", the question is then whether we still allow it but document that it's really not recommended or we plain not allow it and only keep custom volumes on it.

[bensmrs]

Well it depends on the amount of work required. 4 hours of work that could potentially go to the trash is ok. 4 days, not so much.

[stgraber]

I think that should be set to false given above.

[stgraber]

Not sure about that one, does NFS handle O_DIRECT?

[stgraber]

Ah, I just left a few comments in the Info function now.

[bensmrs]

I was hoping @stgraber had time to point in the correct direction on this part.

Well now you’re served :þ

I can help review, fix stuff, and even write tests, don’t hesitate to ping me. I’d actually be happy to see it working pretty soon.

[Foxboron]

@bensmrs thanks!

I'll probably not touch this in July, and there is a hackercamp and work stuff happening in August on my end that might not allow me a lot of energy to pick this up after hours.

I'd appreciate some pointers on the id remapping incus does and how that should play with the ID squashing NFS does. I think there should be some guidance and testing there to make sure it behaves as expected. It also takes a bunch of time which might not be needed if nfs reassings UID/GID anyway.

If you have time to write tests and stuff I'd be happy to give you access to my fork.

[stgraber]

I don't believe that VFS idmap works on top of NFS at this point, so we'd be dealing with traditional shifting where Incus rewrites all the uid/gid as needed.

What you want to ensure is that NFS is mounted the same way on all machines and that no uid/gid squashing is performed, then that should work fine.

[Foxboron]

Should we call the driver nfs4 to make sure we don't end up in a weird situation down the line where we don't want to support a v5 along with the v4 code? Is that realistic?

[stgraber]

I think we should stick to nfs as nfs4 may give the impression that we don't support nfs3 whereas the featureset we really need is perfectly fine for NFS3, if anything some of the bits in NFS4 may be problematic (built-in uid/gid mapping and such).

Exactly what kind of server version and configuration we can handle is probably something that's best addressed through the documentation for the driver.

[symgryph]

NFS would be SO nice. Just an interested party!