-
Notifications
You must be signed in to change notification settings - Fork 185
Add WebSocket transport support #781
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: GautamBytes <[email protected]>
Signed-off-by: GautamBytes <[email protected]>
cc @seetadev |
@GautamBytes : Thanks Gautam for submitting the PR. Appreciate your initiative and efforts. Please resolve the CI/CD issues. CCing @sumanjeet0012, @guha-rahul and @lla-dane for their feedback and pointers too. This module will be needed by a number of projects. Lets productionize it well. |
Signed-off-by: GautamBytes <[email protected]>
raise ValueError(f"No host protocol found in {maddr}") | ||
|
||
port = int(maddr.value_for_protocol("tcp")) | ||
uri = f"ws://{host}:{port}" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
are we leaving support for ipv6 for later on?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the catch!
I'll update the dial method to properly enclose IPv6 hosts in brackets.
…libp2p into add-ws-transport
@seetadev , @guha-rahul , i am trying to fix this failing workflows but facing difficulty . Its because of my test_js_ws_ping.py file not passing , error logs shows -
would really appreciate if someone can tell how to fix this! |
@GautamBytes : Thank you so much for taking the initiative to work on fixing the failing workflows. Your effort on test_js_ws_ping.py is really appreciated. The SwarmException error you’re seeing usually means that the WebSocket connection between the Python host and the JS peer isn’t being established — this could be due to a mismatch in multiaddr protocols, peer discovery issues, or the JS node not being ready when the test starts. @acul71, @guha-rahul , @varun-r-mallya — can you jump in to help debug this WebSocket issue with ? Maybe we can review the interop/test_js_ws_ping.py setup together and check: If the JS node is properly launched and listening on the expected /ws multiaddr. Whether dial_peer is using the correct transport and address. If any recent changes in the libp2p handshake or transport modules might have caused regressions. @GautamBytes, you might also try: Running the JS ping test manually with verbose logs (DEBUG=libp2p* pytest -s tests/interop/test_js_ws_ping.py) to see detailed handshake failures. Verifying that the JS node and Python host share the same set of supported protocols and transports. Testing if basic_host.connect(peer_info) works when you manually start a JS libp2p node with WebSocket enabled. Let’s collaborate and get this test passing soon. |
@GautamBytes @seetadev Work in progress....... |
…Add transport_registry.py for centralized transport management - Integrate WebSocket transport with new registry - Add comprehensive test suite for transport registry - Include WebSocket examples and demos - Update transport initialization and swarm integration
@GautamBytes can you give me access to push to your fork ? |
@GautamBytes : Hi Gautam. @acul71 is our amazing Py-libp2p developer, key maintainer to py-multiaddr, py-cid and core py-libp2p modules. Kindly give him push access so that he can collaborate with you on taking web socket to a production stage. Appreciate your efforts and initiative. |
@acul71 : Great, thanks Luca. Appreciate your initiative and support. I have asked Gautam to share access with you soon. He might be unavailable because of his engineering exams this week. Will be available soon in a day or two. |
@acul71 , i have invited u as collaborator , can u check the mail and confirm once? |
@acul71 : Can you confirm with @GautamBytes ? He did share an invite today. |
- Fix INotifee interface compliance in WebSocket demo - Fix handler function signatures to be async (THandler compatibility) - Fix is_closed method usage with proper type checking - Fix pytest.raises multiple exception type issue - Fix line length violations (E501) across multiple files - Add debugging logging to Noise security module for troubleshooting - Update WebSocket transport examples and tests - Improve transport registry error handling
@yashksaini-coder This scripts needs to be reviewed to see if they are well done (In the end make them work) Would you like to start investigating ? |
yes I was able to find out that during the interop js ping test, the security handshake fails, that could be a starting point, I will get js-libp2p module and start working on this. |
d0c0508
to
771b837
Compare
…plement proper error handling and cleanup in P2PWebSocketConnection. Update tests for improved connection handling.
…or improved readability. Clean up whitespace and enhance logging for connection management.
@yashksaini-coder and @acul71 : This PR is indeed coming along well. Appreciate your efforts. Wish if you could add a test suite and a newsfragment file. @acul71, @sumanjeet0012 and @lla-dane : Wish to have a collaborative review on this PR. We can take up js-libp2p interop in a separate PR if it is fully ready functionality wise. We are aiming for a new py-libp2p release and would like to get web socket transport protocol support in that release. |
@yashksaini-coder and @acul71 : Wish to also share that a number of CI/CD test cases are failing as well. Wish if you could resolve them. |
Got it sir, |
- Combined yashksaini-coder's flow control improvements with luca's WSS features - Preserved comprehensive WSS support, TLS configuration, and handshake timeout - Added production-ready buffer management and connection limits - Maintained backward compatibility with existing WebSocket functionality - Integrated both approaches for optimal WebSocket transport implementation
…ation - Added back debug_websocket_url.py for WebSocket URL testing - Added back test_websocket_client.py for standalone WebSocket testing - These tools complement the integrated WebSocket transport implementation
- Add TLS configuration support to new_host and new_swarm functions - Fix WebSocket transport tests (test_wss_host_pair_data_exchange, test_wss_listen_without_tls_config) - Integrate TLS configuration with transport registry for proper WebSocket WSS support - Move debug files to downloads directory for future reference - All 47 WebSocket tests now passing including WSS functionality - Maintain backward compatibility with existing code - Resolve all type checking and linting issues
Hi @seetadev |
SummaryThis last commits adds comprehensive WebSocket transport support to py-libp2p, including both WebSocket (WS) and WebSocket Secure (WSS) protocols with proper TLS configuration handling. Features Added
Key Changes
Test Results
Technical Details
Files Changed
Breaking ChangesNone - fully backward compatible Documentation
|
Thanks to @yashksaini-coder yashksaini-coder's ContributionsSummaryEnhanced WebSocket transport with production-grade features for py-libp2p. Key Features Added
Files Modified
Current Limitations
ImpactAdded production-ready infrastructure (flow control, error handling, connection limits) to WebSocket transport, but full integration with libp2p protocols and real-world usage scenarios need further validation. |
@acul71 thank you for the kind words, hope my contribution helped resolve this issue 👍🏼 |
@seetadev |
@acul71 : Hi Luca. I did run CI/CD pipeline. Sorry, my message was not posted earlier. There are 2 CI/CD issues remaining. Wish if we could arrive at a good conclusion on them. @acul71 and @yashksaini-coder : Thanks a lot for pushing this forward 🙌 Appreciate your initiative and great efforts. It’s great to see transport_registry shaping up in a way that can be reused by upcoming transports — that’s a really valuable step for the ecosystem. Appreciate you also flagging the WebSocket work. Even though it’s still marked as high-experimental, having the discussion (#937) and a dedicated issue (#938) already scoped out makes it much easier for contributors to jump in and build towards a complete implementation. Appreciate the thoroughness here — this PR will definitely help us move faster on transport expansion. |
- Change global registry from immediate to lazy initialization - Fix doctest failure caused by debug logging during MultiError import - Update all functions to use get_transport_registry() instead of direct access - Resolves CI/CD doctest failure in libp2p.rst
@seetadev |
@acul71 , @yashksaini-coder : Did run the CI/Cd test pipeline. Will share results soon. Hopefully, all test issues should get resolved. Appreciate your efforts and initiative. |
@acul71 , @yashksaini-coder : Indeed, all the test cases are passing. This PR is ready for final review + merge. Please now start working on the missing features at #938. This is in reference to our last maintainer's call. Thank you for your continued support and wonderful progress. @acul71: Would recommend you to collaborate with Paul on the upcoming py-libp2p release along with working on the above issue with @yashksaini-coder. Appreciate your efforts. |
Closes #585
This pull request introduces WebSocket transport support to
py-libp2p
, enabling interoperability with other major libp2p implementations likejs-libp2p
andgo-libp2p
that rely on/ws
multiaddrs.The implementation adds a new
libp2p.transport.websocket
module containing the necessary components to dial and listen on WebSocket connections, integrating seamlessly with the existingTransportUpgrader
andSwarm
.Key Features
WebsocketTransport
andWebsocketListener
: Implements the coreITransport
andIListener
interfaces for handling the WebSocket connection lifecycle./ws
Multiaddr Handling: Correctly parses/ip4/.../tcp/.../ws
multiaddrs for both dialing and listening. The listener also advertises the correct public/ws
address.trio-websocket
Integration: Usestrio-websocket
for native, non-blocking I/O within thepy-libp2p
trio
runtime.test_js_ws_ping.py
) that successfully establishes a connection and runs the/ipfs/ping/1.0.0
protocol with a livejs-libp2p
node, confirming bi-directional communication.TransportUpgrader
, allowing security protocols (e.g.,noise
) and stream multiplexers (e.g.,mplex
) to be layered on top.Scope
/ws
) transport./wss
) is deferred for future work as discussed in the issue. The transport will raise aNotImplementedError
if a/wss
address is provided.