Skip to content

Conversation

msbutler
Copy link
Collaborator

@msbutler msbutler commented Sep 9, 2025

crosscluster/physical: add reader tenant system table id offset setting

This patch adds the private
physical_cluster_replication.reader_system_table_id_offset setting, which a pcr
customer can set on the destination system tenant to some very large number,
like 1,000,000, which will bootstrap the reader tenant with dynamically
allocated system table ids to be offset+i. This setting can be set when the
reader tenant fails to start up because a source table id collides with a
system table id.

Informs #152909
Release note: none


catalog/bootstrap: add option to offset ids during system table creation

This patch adds an option to MakeSetadataSchema to offset the dynamically
allocated system table ids. So, if the caller sets this to say 1000, the vc
will still have ids 0-49, but then ids 1050,1051, etc.

A future patch will leverage this option during PCR reader tenant creation to
ensure that replicating user table ids never collide with the reader tenant's
system table ids.

Informs #152909

Release note: none

@msbutler msbutler self-assigned this Sep 9, 2025
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@msbutler msbutler force-pushed the butler-pcr-bump-id-v2 branch 2 times, most recently from ce3ce80 to d98bb07 Compare September 10, 2025 13:34
@msbutler msbutler changed the title catalog/bootstrap: add option to skip ids during system table creation crosscluster/physical: add reader tenant system table id offset setting Sep 10, 2025
@msbutler
Copy link
Collaborator Author

looks like tc stress run was interrupted. treating as flake.

@msbutler msbutler marked this pull request as ready for review September 10, 2025 18:08
@msbutler msbutler requested review from a team as code owners September 10, 2025 18:08
@msbutler msbutler requested review from jeffswenson, herkolategan, DarrylWong, dt and fqazi and removed request for a team, herkolategan and DarrylWong September 10, 2025 18:08
Copy link
Collaborator

@fqazi fqazi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

@fqazi reviewed 11 of 14 files at r1, 4 of 4 files at r3, 2 of 2 files at r4, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @dt and @jeffswenson)


pkg/server/node.go line 458 at r3 (raw file):

	defaultZoneConfig *zonepb.ZoneConfig, defaultSystemZoneConfig *zonepb.ZoneConfig,
) bootstrap.MetadataSchema {
	return bootstrap.MakeMetadataSchema(keys.SystemSQLCodec, defaultZoneConfig, defaultSystemZoneConfig, 0)

Nit: /* dynamicSystemTableIDOffset */

@msbutler msbutler force-pushed the butler-pcr-bump-id-v2 branch from d98bb07 to 7ccc91d Compare September 11, 2025 15:47
@dt
Copy link
Member

dt commented Sep 11, 2025

Nit: /* dynamicSystemTableIDOffset */

counter-nit: I hate those comments. I'd consider adding const NoOffset = 0 to bootstrap and using that in place of 0 in every caller; IMO a well named var is much more readable than an inline /* noisycommentThatIsntCheckedByCompiler */ in the middle of a func call.

@dt
Copy link
Member

dt commented Sep 11, 2025

alternatively: rename BootstrapTenant to BootstrapTenantWithOffset, and then add BootstrapTenant as a wrapper that passes 0 so we don't need to update all these other calls with an arg they don't care about. We could also make the 0 passed by the wrapper metamorphic to exercise this path more than just in readers, to get better coverage.

This patch adds an option to MakeSetadataSchema to offset the dynamically
allocated system table ids. So, if the caller sets this to say 1000, the vc
will still have ids 0-49, but then ids 1050,1051, etc.

A future patch will leverage this option during PCR reader tenant creation to
ensure that replicating user table ids never collide with the reader tenant's
system table ids.

Informs cockroachdb#152909

Release note: none
This patch adds the private
physical_cluster_replication.reader_system_table_id_offset setting, which a pcr
customer can set on the destination system tenant to some very large number,
like 1,000,000, which will bootstrap the reader tenant with dynamically
allocated system table ids to be offset+i.  This setting can be set when the
reader tenant fails to start up because a source table id collides with a
system table id.

Informs cockroachdb#152909
Release note: none
@msbutler msbutler force-pushed the butler-pcr-bump-id-v2 branch from 7ccc91d to 0429616 Compare September 11, 2025 20:16
@msbutler
Copy link
Collaborator Author

@dt i went with the const NoOffset approach. I decided to not create a new BootstrapTenant call since there are only 2 callers.

@msbutler
Copy link
Collaborator Author

We could also make the 0 passed by the wrapper metamorphic to exercise this path more than just in readers, to get better coverage.

I've modified the PCR roachtests to metamorphically bump the reader tenant system table Ids. i'm hesitant to bump the reader tenant ids in general for now, as i gotta backport this change and i don't want to introduce flakes.

@msbutler msbutler added backport-24.3.x Flags PRs that need to be backported to 24.3 backport-25.2.x Flags PRs that need to be backported to 25.2 backport-25.3.x labels Sep 11, 2025
@msbutler
Copy link
Collaborator Author

bors r=fqazi

@craig
Copy link
Contributor

craig bot commented Sep 11, 2025

@craig craig bot merged commit 96f9f14 into cockroachdb:master Sep 11, 2025
22 of 23 checks passed
Copy link

blathers-crl bot commented Sep 11, 2025

Encountered an error creating backports. Some common things that can go wrong:

  1. The backport branch might have already existed.
  2. There was a merge conflict.
  3. The backport branch contained merge commits.

You might need to create your backport manually using the backport tool.


💡 Consider backporting to the fork repo instead of the main repo. See instructions for more details.

error creating merge commit from 008a5f4 to blathers/backport-release-24.3-153284: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict []

you may need to manually resolve merge conflicts with the backport tool.

Backport to branch 24.3.x failed. See errors above.


💡 Consider backporting to the fork repo instead of the main repo. See instructions for more details.

error creating merge commit from 008a5f4 to blathers/backport-release-25.2-153284: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict []

you may need to manually resolve merge conflicts with the backport tool.

Backport to branch 25.2.x failed. See errors above.


💡 Consider backporting to the fork repo instead of the main repo. See instructions for more details.

error creating merge commit from 0429616 to blathers/backport-release-25.3-153284: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict []

you may need to manually resolve merge conflicts with the backport tool.

Backport to branch 25.3.x failed. See errors above.


🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-24.3.x Flags PRs that need to be backported to 24.3 backport-25.2.x Flags PRs that need to be backported to 25.2 backport-25.3.x backport-failed target-release-25.4.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants