Skip to content

Conversation

jhamman
Copy link
Member

@jhamman jhamman commented Sep 14, 2025

Adds support for uvloop as a faster alternative to Python's default IO loop.

closes #3451

TODO:

  • Add unit tests and/or doctests in docstrings
  • Add docstrings and API docs for any new/modified user-facing classes and functions
  • New/modified features documented in docs/user-guide/*.rst
  • Changes documented as a new file in changes/
  • GitHub Actions have all passed
  • Test coverage is 100% (Codecov passes)

Copy link

codecov bot commented Sep 14, 2025

Codecov Report

❌ Patch coverage is 82.35294% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 61.27%. Comparing base (62551c7) to head (2230848).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/zarr/core/sync.py 82.35% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3452      +/-   ##
==========================================
+ Coverage   61.24%   61.27%   +0.02%     
==========================================
  Files          83       83              
  Lines        9907     9922      +15     
==========================================
+ Hits         6068     6080      +12     
- Misses       3839     3842       +3     
Files with missing lines Coverage Δ
src/zarr/core/config.py 29.16% <ø> (ø)
src/zarr/core/sync.py 63.93% <82.35%> (+2.25%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@@ -107,7 +107,7 @@ def enable_gpu(self) -> ConfigSet:
"order": "C",
"write_empty_chunks": False,
},
"async": {"concurrency": 10, "timeout": None},
"async": {"concurrency": 10, "timeout": None, "use_uvloop": True},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is uvloop the only alternative event loop, or might there be others? If so, it might make more sense to have a field named "event_loop" which takes values "asyncio" | "uvloop"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I asked myself this exact question and concluded we are very unlikely to have another loop anytime soon. I also asked Claude:

Great question! Yes, there are a few other event loop implementations in the Python ecosystem, though uvloop is by far the most popular and mature:

Other Event Loop Implementations

  1. uvloop - The most popular, based on libuv (what we're implementing)
  2. asyncio-dgram - UDP-focused event loop
  3. winloop - Windows-specific attempt (less mature, not widely adopted)
  4. ProactorEventLoop - Windows-specific (already built into asyncio)
  5. SelectorEventLoop - Unix default (already built into asyncio)

However, looking at the ecosystem:

  • uvloop is the dominant alternative - 99% of users who want a replacement
    use uvloop
  • Other alternatives are niche - focused on specific use cases or platforms
  • Most are experimental or abandoned - uvloop has won the "alternative event
    loop" space

Config Design Considerations

Option 1: Boolean approach (current)

zarr.config.set({"async.use_uvloop": True})

Pros: Simple, clear intent, matches the reality that uvloop is the only practical alternative
Cons: Not extensible if other loops emerge

Option 2: String-based approach

zarr.config.set({"async.loop": "uvloop"}) # or "asyncio", "auto"
Pros: More extensible, could support future loop implementations
Cons: More complex, premature abstraction for a problem that may never exist

Option 3: Hybrid approach

zarr.config.set({"async.loop_policy": "uvloop"}) # or "default", "auto"

Recommendation: Stick with Boolean

I recommend keeping the current async.use_uvloop boolean approach for these
reasons:

  1. YAGNI principle - We shouldn't over-engineer for hypothetical future needs
  2. Ecosystem reality - uvloop has been the only serious asyncio alternative for 7+ years
  3. Clear semantics - Boolean is unambiguous about what it does
  4. Easy migration path - If other loops emerge, we could add async.loop_policy later and deprecate use_uvloop
  5. Precedent - Most libraries that integrate uvloop use similar boolean flags

The boolean approach is pragmatic and matches the current ecosystem reality.
If other event loop implementations gain traction in the future, we could
always add a more generic async.loop_policy setting while keeping use_uvloop
for backward compatibility.

@d-v-b
Copy link
Contributor

d-v-b commented Sep 14, 2025

is there a good way to demo the performance difference between uvloop and asyncio? If so, would it make sense to include that demo in the examples directory?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature Proposal: Add optional uvloop support for improved async performance
2 participants