-
-
Notifications
You must be signed in to change notification settings - Fork 358
Feature: support uvloop as a faster alternative to Python's default IO loop #3452
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3452 +/- ##
==========================================
+ Coverage 61.24% 61.27% +0.02%
==========================================
Files 83 83
Lines 9907 9922 +15
==========================================
+ Hits 6068 6080 +12
- Misses 3839 3842 +3
🚀 New features to boost your workflow:
|
@@ -107,7 +107,7 @@ def enable_gpu(self) -> ConfigSet: | |||
"order": "C", | |||
"write_empty_chunks": False, | |||
}, | |||
"async": {"concurrency": 10, "timeout": None}, | |||
"async": {"concurrency": 10, "timeout": None, "use_uvloop": True}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is uvloop the only alternative event loop, or might there be others? If so, it might make more sense to have a field named "event_loop"
which takes values "asyncio" | "uvloop"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I asked myself this exact question and concluded we are very unlikely to have another loop anytime soon. I also asked Claude:
Great question! Yes, there are a few other event loop implementations in the Python ecosystem, though uvloop is by far the most popular and mature:
Other Event Loop Implementations
- uvloop - The most popular, based on libuv (what we're implementing)
- asyncio-dgram - UDP-focused event loop
- winloop - Windows-specific attempt (less mature, not widely adopted)
- ProactorEventLoop - Windows-specific (already built into asyncio)
- SelectorEventLoop - Unix default (already built into asyncio)
However, looking at the ecosystem:
- uvloop is the dominant alternative - 99% of users who want a replacement
use uvloop- Other alternatives are niche - focused on specific use cases or platforms
- Most are experimental or abandoned - uvloop has won the "alternative event
loop" spaceConfig Design Considerations
Option 1: Boolean approach (current)
zarr.config.set({"async.use_uvloop": True})
Pros: Simple, clear intent, matches the reality that uvloop is the only practical alternative
Cons: Not extensible if other loops emergeOption 2: String-based approach
zarr.config.set({"async.loop": "uvloop"}) # or "asyncio", "auto"
Pros: More extensible, could support future loop implementations
Cons: More complex, premature abstraction for a problem that may never existOption 3: Hybrid approach
zarr.config.set({"async.loop_policy": "uvloop"}) # or "default", "auto"
Recommendation: Stick with Boolean
I recommend keeping the current async.use_uvloop boolean approach for these
reasons:
- YAGNI principle - We shouldn't over-engineer for hypothetical future needs
- Ecosystem reality - uvloop has been the only serious asyncio alternative for 7+ years
- Clear semantics - Boolean is unambiguous about what it does
- Easy migration path - If other loops emerge, we could add async.loop_policy later and deprecate use_uvloop
- Precedent - Most libraries that integrate uvloop use similar boolean flags
The boolean approach is pragmatic and matches the current ecosystem reality.
If other event loop implementations gain traction in the future, we could
always add a more generic async.loop_policy setting while keeping use_uvloop
for backward compatibility.
is there a good way to demo the performance difference between uvloop and asyncio? If so, would it make sense to include that demo in the |
Adds support for uvloop as a faster alternative to Python's default IO loop.
closes #3451
TODO:
docs/user-guide/*.rst
changes/