feat: Option to auto-upgrade CRDs with Helm

[denis-ryzhkov]

• Closes Option to auto-upgrade CRDs with Helm #2159
• Added crds.immutable with safe backward-compatible default value and detailed docstring.
• Moved CRDs from special crds/ directory to files/crds/ for templating.
• Updated Makefile to generate CRDs in the new location.
• Added conditional crds subchart with a special crds/ directory being a symlink to the files/crds/ in the main chart.
• Updated the docs to mention the new upgrade option.
• Tested few cases below in grafana-operator/deploy/helm/grafana-operator/ dir.
• Default values:

  cat values.yaml | yq .crds
  
    # -- Immutable CustomResourceDefinitions are installed only once using `crds/` directory
    # and require manual upgrade by `kubectl apply`.
    # Mutable CRDs are installed and upgraded together with the Helm chart using `templates/` directory
    # without manual `kubectl apply` step required.
    # Use `helm upgrade -i --take-ownership` when switching to mutable CRDs for the first time only.
    # Both types of CRDs are protected on the Helm chart uninstall to avoid cascading deletion.
    immutable: true
  
  kubectl get crd | grep graf | wc -l
  
    0
  
  helm upgrade -i grafana-operator .
  
    Release "grafana-operator" does not exist. Installing it now.
    walk.go:75: found symbolic link in path:
      REDACTED/helm/grafana-operator/charts/crds/crds resolves to
      REDACTED/helm/grafana-operator/files/crds. Contents of linked file included and used
    NAME: grafana-operator
    NAMESPACE: default
    STATUS: deployed
    REVISION: 1
  
  kubectl get crd | grep graf | wc -l
  
    12
  
  kubectl get crd grafanas.grafana.integreatly.org -o yaml | yq .metadata
  
    annotations:
      controller-gen.kubebuilder.io/version: v0.17.3
    ...
  
  kubectl get deploy/grafana-operator
  
    NAME               READY   UP-TO-DATE   AVAILABLE
    grafana-operator   1/1     1            1        
  
  kubectl logs deploy/grafana-operator | tail -1
  
    INFO	GrafanaReconciler	Grafana status sync complete
  
  helm uninstall grafana-operator
  
    release "grafana-operator" uninstalled
  
  kubectl get crd | grep graf | wc -l
  
    12
  

• Upgrade to mutable CRDs:

  helm upgrade -i --set crds.immutable=false grafana-operator .
  
    Error: Unable to continue with install:
    CustomResourceDefinition "grafanaalertrulegroups.grafana.integreatly.org"
    in namespace "" exists and cannot be imported into the current release: invalid ownership metadata;
    label validation error: missing key "app.kubernetes.io/managed-by": must be set to "Helm";
    annotation validation error: missing key "meta.helm.sh/release-name": must be set to "grafana-operator";
    annotation validation error: missing key "meta.helm.sh/release-namespace": must be set to "default"
  
  helm upgrade -i --take-ownership --set crds.immutable=false grafana-operator .
  
    Release "grafana-operator" has been upgraded. Happy Helming!
    ...
    STATUS: deployed
  
  kubectl get crd | grep graf | wc -l
  
    12
  
  kubectl get crd grafanas.grafana.integreatly.org -o yaml | yq .metadata
  
    annotations:
      controller-gen.kubebuilder.io/version: v0.17.3
      helm.sh/resource-policy: keep
      meta.helm.sh/release-name: grafana-operator
      meta.helm.sh/release-namespace: default
    labels:
      app.kubernetes.io/managed-by: Helm
    ...
  
  kubectl get deploy/grafana-operator
  
    NAME               READY   UP-TO-DATE   AVAILABLE
    grafana-operator   1/1     1            1        
  
  kubectl logs deploy/grafana-operator | tail -1
  
    INFO	GrafanaReconciler	Grafana status sync complete
  
  helm uninstall grafana-operator
  
    These resources were kept due to the resource policy:
    [CustomResourceDefinition] grafanaalertrulegroups.grafana.integreatly.org
    ...11 more...
  
    release "grafana-operator" uninstalled
  
  kubectl get crd | grep graf | wc -l
  
    12
  

• Clean install with mutable CRDs:

  kubectl get crd -o name | grep grafana | xargs kubectl delete
  
    customresourcedefinition.apiextensions.k8s.io "grafanaalertrulegroups.grafana.integreatly.org" deleted
    ...11 more...
  
  helm upgrade -i --set crds.immutable=false grafana-operator .
  
    Release "grafana-operator" does not exist. Installing it now.
    ...
    STATUS: deployed
  

  The rest of the tests showed the same output as in the "Upgrade to mutable CRDs" case.

[CLAassistant]

All committers have signed the CLA.