Skip to content

Conversation

jhuliano
Copy link
Contributor

@jhuliano jhuliano commented Jul 31, 2025

This PR fixes a bug where Flagger failed to remove gateways and hosts from an Istio VirtualService when the Canary was updated to enable the delegation feature.

The Problem

When a user enables the istio route delegation by setting spec.service.delegation: true and removing gateways and hosts from the Canary spec, the reconciliation logic did not properly update the managed VirtualService.

See #1464 for more details.

The Solution

The reconciliation logic has been updated to explicitly trigger a update on the VS removing the gateways and hosts when delegation is enabled. This ensures the managed VirtualService is correctly synced.

I'm not sure if this is the best solution since at the core the problem seems to be that the reconciliation logic is doing a diff but because the virtualService.Spec (VS that is read from the cluster) is updated on the fly with empty ([]) hosts and gateways, the removal of hosts and gateways are never detected by the diff.

I thought the best option would be to simply NOT modify the virtualService variable, but this seemed to generate other bugs where a flagger.kubernetes.io/original-configuration annotation was added on the VS and the canary got stuck in progressing at 10% traffic, which I couldn't fully understand how to fix.

Testing

To validate the fix I have done the following:

  • A new unit test has been added to reproduce the original bug and confirm it is now resolved.
  • The existing istio delegation E2E test has been updated to cover this specific scenario, and I executed said tests on a minikube cluster.

Additional notes

This issue has some quirky workarounds that actually triggers the diff and in-turn causes the VS to be updated, for example, with the E2E I found out that setting portDiscovery: truecauses the diff logic to trigger and update the VS as expected.

Fixes #1464

@jhuliano jhuliano force-pushed the fix/istio-vs-delegation branch from 290ae8e to 38766d4 Compare July 31, 2025 08:14
@@ -68,8 +68,7 @@ spec:
service:
port: 80
targetPort: 9898
portDiscovery: true
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed portDiscovery: true because it was workaround for the reported issue.

@codecov-commenter
Copy link

Codecov Report

❌ Patch coverage is 66.66667% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 28.46%. Comparing base (12ee6cb) to head (38766d4).
⚠️ Report is 23 commits behind head on main.

Files with missing lines Patch % Lines
pkg/router/istio.go 66.66% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #1818       +/-   ##
===========================================
- Coverage   39.44%   28.46%   -10.98%     
===========================================
  Files         287      287               
  Lines       22706    22810      +104     
===========================================
- Hits         8956     6494     -2462     
- Misses      12777    15594     +2817     
+ Partials      973      722      -251     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jhuliano jhuliano changed the title Ensure VS is updated when delegation is enabled fix: ensure VS is updated when delegation is enabled Jul 31, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cannot delete gateways and hosts from VirtualService generated by Canary when enable Canary delegation
2 participants