When using DependentResource, sometimes the workflow acts as if the resource needs to be created when instead it exists #1161
Description
I have a reconciler which is annotated with:
@Log
@RateLimited(maxReconciliations = 1, within = 20)
@GradualRetry
@Workflow(dependents = {
@Dependent(name = "associated-secret", type = SecretDependentResource.class)
})
@ControllerConfiguration(name = "externalSecretReconciler")
public class ExternalSecretReconciler
implements Reconciler<ExternalSecret>, Cleaner<ExternalSecret> {Where SecretDependentResource is:
@KubernetesDependent
@Log
public class SecretDependentResource
extends CRUDKubernetesDependentResource<Secret, ExternalSecret> {I've noticed that when built in native mode, on start sometimes reconciliation of the dependent Secret tries to create the secret instead of updating it, even if the secret is actually there in the cluster and correcty linked with the owner block. After a tiny bit, the situation stabilize by itself and the operator stops trying to create when the resoruce is there and switch to normal update.
I see this in 2 ways:
- Calling
context.getSecondaryResource(Secret.class)returns empty. - In its inner logic, a POST to create a new secret is called instead of updating existing one
After some digging, I suspect that this might be happening because (please forgive me if I understood things in a wrong way):
- On start, all resources are reconciled by design
- On start, event informers are attached to the watched resources. So In my case several (one for each namespace) for the ExternalSecret custom resource, and several for the Secret resource
- When a Secret is received from the Informer, it does populate a cache
- When an ExternalSecret is received from the Informer for the first time, it does trigger its reconciliation
- When this happen, the workflow is triggered and the dependent secret is reconciled
- But sometimes the secret is still not there inside the cache, and the cache does not try to fetch the Secret in this case, instead it returns like if the secret is not there
I was wondering if you think this could be a side-effect of building native. Or if I am doing something wrong and there is a way to wait until everything is ready and cached before starting to process things up?
This issue is especially a problem for me because when creating a Secret, SecretDependentResource does some operations which result in potentially disruptive side effects (API calls and similar things) that we want to avoid when not necessary.