Skip to content

Conversation

dentiny
Copy link
Contributor

@dentiny dentiny commented Jun 28, 2025

What changes are included in this PR?

Iceberg supports retry on commit: https://iceberg.apache.org/docs/latest/configuration/#table-behavior-properties
To implement retry, I would like to commit on a cloned transaction.

Old code without retry:

let action = txn.fast_append();
let action = action.add_data_files(data_file_import_result.new_iceberg_data_files);
txn = action.apply(txn)?;
txn.commit(catalog)

New code with retry:

// Apply all changes.
let action = txn.fast_append();
let action = action.add_data_files(data_file_import_result.new_iceberg_data_files);
txn = action.apply(txn)?;

// Perform commit with retry.
while count < max_retry && time < max_timeout {
  let txn_copy = txn.clone();
  txn.commit(catalog) // commit takes `mut self`, have to make a clone
}

Are these changes tested?

No op change, but compilation and existing tests all pass.

@dentiny dentiny force-pushed the hjiang/make-transaction-clonable branch from a993dc1 to c70f963 Compare June 28, 2025 20:30
@ZENOTME
Copy link
Contributor

ZENOTME commented Jun 29, 2025

I think the retry commit is implement in the commit internally. For now, looks like we can't guarantee that the transaction can be retry again directly in semantics.🤔

@liurenjie1024
Copy link
Contributor

For now, looks like we can't guarantee that the transaction can be retry again directly in semantics.🤔

Why? Do you mean the commit method consumed transaction?

Copy link
Contributor

@liurenjie1024 liurenjie1024 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @dentiny for this pr. I'm fine with the changes, but I want to claeify that tx commit is under development, and should happend in commit method internally.

@dentiny
Copy link
Contributor Author

dentiny commented Jun 30, 2025

Thanks @dentiny for this pr. I'm fine with the changes, but I want to claeify that tx commit is under development, and should happend in commit method internally.

Yeah I totally understand.
Thank you for the quick review!

@liurenjie1024 liurenjie1024 merged commit ec92bc5 into apache:main Jun 30, 2025
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants