we should publish rc to pypi as part of the release process

[kevinjqliu]

Is your feature request related to a problem or challenge?

During the release process, we only upload rust artifacts to apache SVN https://rust.iceberg.apache.org/release.html#upload-artifacts-to-the-svn-dist-repo

it would be helpful to upload the RC python artifacts to pypi for testing. This is located at

iceberg-rust/.github/workflows/release_python.yml

Lines 98 to 124 in e5d3aab

	pypi-publish:
	name: Publish Python 🐍 distribution 📦 to Pypi
	needs: [sdist, wheels]
	runs-on: ubuntu-latest
	# Only publish to PyPi if the tag is not a pre-release OR if manually triggered
	if: ${{ (startsWith(github.ref, 'refs/tags/') && !contains(github.ref, '-')) || github.event_name == 'workflow_dispatch' }}
	environment:
	name: pypi
	url: https://pypi.org/p/pyiceberg-core
	permissions:
	id-token: write # IMPORTANT: mandatory for trusted publishing
	steps:
	- name: Download all the dists
	uses: actions/download-artifact@v4
	with:
	pattern: wheels-*
	merge-multiple: true
	path: bindings/python/dist
	- name: Publish to PyPI
	uses: pypa/gh-action-pypi-publish@release/v1
	with:
	skip-existing: true
	packages-dir: bindings/python/dist

We should absolutely make sure that the pypi artifacts are published as RC and not the actual release (0.9.0rc1 vs 0.9.0).

Describe the solution you'd like

No response

Willingness to contribute

None

[KrishnaSindhur]

@liurenjie1024 can I work on this?

[liurenjie1024]

@liurenjie1024 can I work on this?

Yes, thx!

[kevinjqliu]

hey @KrishnaSindhur thanks for volunteering to work on this. I've already started to work on this issue. Since it involves github CI, there's a lot of infrastructure set up to test the changes