Cache service ID list by namespace name

[astaticvoid]

Current Cloud Map client functionality makes SDK ListServices API call on every namespace reconciliation cycle. We need to cache these to scale the controller.

[astaticvoid]

Providing some context from multi-cluster testing leading to motivations of the changes. Primary goal is to reduce throttling on Cloud Map list API calls experienced with even a small scale multi-cluster test environment.

Why cache collections?

Since the Get API's do not support lookups by resource name, we need to use List Namespace/Service to check if resources in cluster have been exported to Cloud Map. If we locally cache each resource by name individually, and a resource is not in the cache, we cannot know whether the exported resource does not exist or the entry has simply expired. Cacheing the full collection from the list namespace call amortizes the cost across all namespaces in the cluster and enables a shorter cache entry TTL.

Why remove namespace negative cache

There are many internal namespaces and the user may have many cluster local services across multiple namespaces. We are tracking separating reconciliation cycles for endpointslices vs namespace/service as a future optimization. Our naive solution to prevent throttling on import reconciliation cycle (every 2 seconds), was to cache nil namespaces if they were not in Cloud Map. This only works with a longer cache TTL, which has the downside of longer worst case import time.

Given cluster-A exporting service example-svc from example-ns, and cluster-B importing the same:
If example-ns is created in cluster-B before cluster-A is deployed and exported, it takes around 2 minutes for cluster-b to import example-svc. This has caused confusion deploying our example setup.

Why cache maps instead of lists?

To prevent iterating over the collection for every namespace/service lookup in a reconciliation cycle.

[astaticvoid]

Prior to this change, every import reconcile cycle called ListServices every 2 seconds, for every matched namespace, for each importing cluster. 2 clusters with three namespaces meant 3 TPS, so cacheing the list would be necessary either way until reconcile is split for service/namepsace and endpoints. Using a single collection for all prevents cache/request redundancy.