Please add Alma Linux 10 images

[cquike]

Alma Linux 10 was released last week (https://wiki.almalinux.org/release-notes/10.0) so it would be nice to get incus images for it.

Thank you!

[stgraber]

We need someone who has a clue about RHEL 10 or Alma Linux 10 to do some testing, update the YAML and submit potential needed changes to distrobuilder.

We're yet to see a new major of RHEL and derivatives not needing some changes to the Go code on top of just tweaks to the YAML.

[MichaelStauber]

We need someone who has a clue about RHEL 10 or Alma Linux 10 to do some testing, update the YAML and submit potential needed changes to distrobuilder.

I should be able to help with that. I know RHEL (and derivates) inside out and I've used distrobuilder to build our BlueOnyx 5210R and 5211R Incus images (they're based on Alma 8 and 9), which I've been distributing through our own Incus image server.

I need an Alma 10 CT image like right now, so I came here looking for it and am willing to chip in some serious time to make it happen.

[stgraber]

That'd be great!

[MichaelStauber]

That'd be great!

No problem. I've just checked out the latest almalinux.yaml from here (mine was slightly older) and will start to add Alma 10 provisions to it and will see how it goes.

[MichaelStauber]

Ok, I see the problems:

1.) Distrobuilder's almalinux-httpd.go fails to verify the RPM signatures in the isoRunner() function during the installroot=/rootfs phase. Disabling the gpgcheck in the repo file or running yum/dnf with --nogpgcheck gets around that.

Am not sure if that would be considered an adequate work around. My build box for the image is an AlmaLinux 9.6 and the used RPM during the installroot phase seems to be the one from the OS of the build box. Which then is an issue, as AlmaLinux 10's RPM is newer and then croaks here:

Cannot find rpmkeys executable to verify signatures.

Like said: Not verifying the RPMs gets around that, but it isn't ideal.

There is another legacy issue in almalinux-httpd.go: For Almalinux 8 it checks if the mirror has the CHECKSUM file and for EL9 and EL10 it falls back to sha256sum.txt.asc - which doesn't exist on the mirrors. These days they all use a CHECKSUM file instead.

2.) almalinux.yaml: That's pretty straightforward except for one point: EL10 no longer has "dhclient". So that must be removed from the shared "package" section and moved into a separate "packages:" section for releases 8 and 9.

This may cause issues in the "trigger: post-files" section, as "dhcp=dhclient" is inserted into /etc/NetworkManager/conf.d/dhcp-client.conf

Other than that? Every existing "packages:" block for release 9 gets a "-10" line appended.

With that it builds (a cloud image in this case).

• Build log
• Distrobuilder's modified almalinux-httpd.go
• Modified almalinux.yaml

This still needs a bit more refining and I'll need to give the resulting images a proper test drive to see if everything checks out.

[stgraber]

Sounds promising!

[gregkuhnert]

Justified exception to build without signature checks during bootstrap if your distrobuilder is itself running on 9.x. Chicken and egg.

[MichaelStauber]

I concur. It's a hen and egg problem. Ideally you need a Alma 10 VM (or a physical box) to run Distrobuilder and to properly build images with GPG verification. As I'm short on x86_64v3 capable hardware it has to be an Incus VM for my building efforts.

So I'm now building an Alma 10 VM in Distrobuilder w/o GPG verification on an Alma 9.6. Once that VM is up and running I'll see that I can do a minimal-invasive almalinux.yaml in it that can be used to build AlmaLinux 8, 9 and 10 CTs and VMs.

CT creation is already sorted and VM creation is almost sorted. It's just post-files that needs a small fix for the grubby parameters.

I should have some usable results in a few hours.

[MichaelStauber]

I think I have something: Working CT and VM images for AlmaLinux 10. But these were built on an AlmaLinux 9.6 with the RPMs being pulled in with --nogpgcheck for the moment:

• CT image almalinux/10/cloud (build logfile)
• VM image almalinux/10/cloud (build logfile)
• Modified almalinux.yaml (Alma 8/9/10)
• Modified almalinux-http.go for Distrobuilder

The CT image was easy-peasy. No big deal. The VM image? That was problematic.

The key problems with the VM image were in the "post-files" stage. Which faced challenges with kernel/initramfs detection, root device and UUID retrieval, EFI partition assumptions, dracut execution, and GRUB configuration compatibility. All of which would lead to build failures or boot issues if the build environment or target system deviates from the script’s expectations.

With the resulting and now working almalinux/10/cloud VM I'll set up a dedicated Disctrobuilder environment which will allow to eliminate the --nogpgcheck provisions from almalinux-http.go

[MichaelStauber]

@stgraber: I tried to rebuild the CT/VM images on an AlmaLinux 10 VM to see if that would allow us to build images WITH verification of the RPM signatures. But the issue is basically the same as before:

The almalinux-http.go of distrobuilder sets up the chroot in a way that verification of RPM signatures fails. I could spend more time on it for rigging up a work around, but the Distrobuilder guys probably know better how to fix that issue for good.

In any case: If RPM signature verification is skipped, then the almalinux.yaml and modified almalinux-http.go linked in my previous message here work fine and as is.

[stgraber]

We're running all the builds on Debian 12 anyway so it sounds unlikely that the validation would ever work in our build environment.

[MichaelStauber]

We're running all the builds on Debian 12 anyway so it sounds unlikely that the validation would ever work in our build environment.

Yeah, that makes sense and keeps it the most portable.

[lnussbaum]

The almalinux-http.go of distrobuilder sets up the chroot in a way that verification of RPM signatures fails. I could spend more time on it for rigging up a work around, but the Distrobuilder guys probably know better how to fix that issue for good.

In any case: If RPM signature verification is skipped, then the almalinux.yaml and modified almalinux-http.go linked in my previous message here work fine and as is.

I think that the problem here is that the AlmaLinux ISO image does not include the rpm package in install.img (which is used as a chroot), which is needed for the rpmkeys binary used for GPG signature verification.

[stgraber]

@MichaelStauber can you open a PR on the distrobuilder side since that's the first thing we'll need to sort out here.