[infra] publish rc to pypi as part of release process #1449
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kevinjqliu
commented
Jun 17, 2025
.github/workflows/publish.yml
Outdated
| @@ -20,7 +20,9 @@ name: Publish | |||
| on: | |||
| push: | |||
| tags: | |||
| - "*" | |||
| # Trigger this workflow when tag follows the versioning format: v<major>.<minor>.<patch>-rc.<release_candidate> | |||
There was a problem hiding this comment.
kevinjqliu
force-pushed
the
kevinjqliu/allow-pypi-prerelease
branch
from
9d00a95 to
63cd745
Compare
kevinjqliu
commented
Jun 23, 2025
| @@ -28,7 +28,7 @@ env: | |||
| rust_msrv: "1.85" | |||
|
|
|||
| concurrency: | |||
| group: ${{ github.workflow }}-${{ github.ref }}-${{ github.event_name }} | |||
| group: ${{ github.workflow }}-${{ github.event.workflow_run.head_branch }}-${{ github.event_name }} | |||
There was a problem hiding this comment.
after changing github.ref -> github.event.workflow_run.head_branch in #1444, this was missed.
just a minor improvement
|
Most important thing here is that for a RC tag push, i.e. For example, |
Fokko
reviewed
Jun 23, 2025
Comment on lines
+25
to
+26
| - "v[0-9]+.[0-9]+.[0-9]+" | ||
| - "v[0-9]+.[0-9]+.[0-9]+-rc.[0-9]+" |
There was a problem hiding this comment.
I consider myself a bit of a regex connoisseur.
Suggested change
| - "v[0-9]+.[0-9]+.[0-9]+" | |
| - "v[0-9]+.[0-9]+.[0-9]+-rc.[0-9]+" | |
| - "v\d+.\d+.\d+" | |
| - "v\d+.\d+.\d+-rc.\d+" |
There was a problem hiding this comment.
fun fact! the on.push.tags syntax here doesnt support regex, its a "filter pattern"
The branches, branches-ignore, tags, and tags-ignore keywords accept glob patterns that use characters like *, **, +, ?, ! and others to match more than one branch or tag name. If a name contains any of these characters and you want a literal match, you need to escape each of these special characters with \. For more information about glob patterns, see the [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#filter-pattern-cheat-sheet).
Fokko
approved these changes
Jun 23, 2025
12 tasks
kevinjqliu
added a commit
that referenced
this pull request
Jul 22, 2025
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123. --> - Closes #. ## What changes are included in this PR? See [comment](#1331 (comment)) for context Previously used `cargo-edit` but it does not allow setting version from `0.6.0` -> `0.6.0-rc1`. This PR changes the github action to use `toml-cli` to manually override and set the version in the `binding/python/Cargo.toml` so that we can build pre-release artifacts to push to pypi. <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> Yes, similar to #1449 Pushed this PR as the `main` branch in my fork Create a new tag and push to fork to trigger CI ``` git tag "v0.6.0-rc.2" -m "v0.6.0-rc.2" git push kevinjqliu v0.6.0-rc.2 ``` Verify that python artifacts are build correctly with pre-release tag, https://github.com/kevinjqliu/iceberg-rust/actions/runs/16436181005 - downloaded `wheels-sdist`, verified correct version in `PKG-INFO` but not in `Cargo.toml`. I think this is fine - downloaded `wheels-ubuntu-latest-armv7l` and verified the `METADATA` file
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
What changes are included in this PR?
Changes
publishworkflow only by matching specific tag names (v<major>.<minor>.<patch>orv<major>.<minor>.<patch>-rc.<release_candidate>) instead of all tagspypiworkflow to again validate the tag namepypiworkflow to override the Cargo version for RC. This allows us to upload to pypi as pre-releaseAre these changes tested?
Yes, tested against personal fork.
Tested
should-not-trigger. ✅ does not trigger thepublishjobv0.6.0-rc.1. ✅ triggerspublishjob andpypijobv0.6.0. ✅ triggerspublishjob andpypijobFor both
v0.6.0-rc.1andv0.6.0, verified the generated artifacts (wheels-sdistandwheels-ubuntu-latest-armv7l) locally for the correctly set python version number.