WIP feat: Add py-libp2p to rust-libp2p interoperability tests suit

[paschal533]

What was wrong?

The current libp2p ecosystem lacks standardized interoperability tests between different language implementations. This creates uncertainty about cross-implementation compatibility and makes it difficult to identify protocol negotiation issues, encryption compatibility problems, and stream multiplexing incompatibilities.

Solution

This PR adds a complete test framework that validates:

🔌 Transport Layer Compatibility

• TCP connection establishment between py-libp2p and rust-libp2p
• Proper address resolution and connection handling

🔐 Security Layer Interoperability

• Noise encryption protocol handshake compatibility
• X25519 key exchange between implementations
• Secure channel establishment

🔀 Stream Multiplexing Verification

• Yamux multiplexer compatibility testing
• Stream creation and management across implementations
• Proper stream lifecycle handling

🤝 Protocol Negotiation Testing

• Multistream-select protocol compatibility
• Standard libp2p ping protocol (/ipfs/ping/1.0.0) negotiation
• Fallback protocol handling

📊 Performance Metrics

• Round-trip time (RTT) measurement
• Connection establishment timing
• Success/failure rate tracking
• Detailed logging for debugging

Files Added

├── py_node/
│   └── ping.py                          # Python libp2p ping client/server
├── rust_node/
│   ├── src/main.rs                      # Rust libp2p ping client/server  
│   └── Cargo.toml                       # Rust dependencies
├── scripts/
│   ├── run_py_to_rust_test.ps1         # Test: Python client → Rust server
│   └── run_rust_to_py_test.ps1         # Test: Rust client → Python server
└── README.md                            # Comprehensive test documentation

Key Features

🐍 Python Implementation (py_node/ping.py)

• Standards-compliant ping protocol implementation
• Support for multiple protocol IDs for compatibility
• Detailed debug logging and error handling
• Noise encryption with X25519 key exchange
• Yamux stream multiplexing
• Configurable ping count and timeout handling

🦀 Rust Implementation (rust_node/src/main.rs)

• Uses standard rust-libp2p ping behavior
• Tokio async runtime integration
• Comprehensive event logging
• Standard libp2p transport stack (TCP + Noise + Yamux + RSA)

🧪 Automated Test Scripts

• PowerShell scripts for Windows compatibility
• Automated server startup and peer discovery
• Error handling and cleanup
• Support for custom parameters (port, ping count, timeout)

Improved Error Handling

• Better connection failure diagnostics
• Detailed protocol negotiation logging
• Proper stream cleanup on errors
• Timeout handling for unresponsive peers

Testing

Manual Testing

# Test 1: Rust client → Python server
python py_node/ping.py server --port 8000
cargo run --manifest-path rust_node/Cargo.toml -- /ip4/127.0.0.1/tcp/8000/p2p/<PEER_ID>

# Test 2: Python client → Rust server  
cargo run --manifest-path rust_node/Cargo.toml
python py_node/ping.py client /ip4/127.0.0.1/tcp/<PORT>/p2p/<PEER_ID>

Current Test Status

✅ Working Components

• TCP connection establishment between py-libp2p and rust-libp2p
• Noise encryption handshake compatibility
• Initial stream multiplexer negotiation
• Peer ID exchange and connection setup

🔍 Active Issues Being Debugged

Yamux Frame Parsing Issues

Current test results show Yamux frame parsing problems:

DEBUG - Received header for peer 12D3KooWN2DmaiQaKpDaAKKZPZRC5FuaJsvTW2NRrmLedk4DvHxj:
type=2, flags=1, stream_id=0, length=3656394986
DEBUG - Received ping request with value 3656394986

Root Cause Analysis:

• Yamux frame headers are being parsed with corrupted length values (3656394986 instead of expected ~32 bytes)
• This suggests potential byte order (endianness) issues between implementations
• Frame corruption may be occurring during the Noise encryption/decryption process
• Stream multiplexing protocol version mismatch between py-libp2p and rust-libp2p

Current Status:

• Connection establishment: ✅ Working
• Noise encryption: ✅ Working
• Yamux negotiation: ✅ Working
• Yamux frame parsing: ❌ Under Investigation
• Ping protocol: ❌ Blocked by frame parsing issues

Expected Results (Once Fixed)

[INFO] Connection established!
[INFO] Stream opened with protocol: /ipfs/ping/1.0.0
[PING 1] Successful! RTT: 1.23ms
[PING 2] Successful! RTT: 0.89ms
...
[STATS] Packets: Sent=5, Received=5, Lost=0
[STATS] Loss rate: 0.0%
[STATS] RTT: min=0.87ms, avg=1.05ms, max=1.23ms

Verification Checklist

• TCP transport connection establishment
• Noise encryption handshake compatibility
• Yamux stream multiplexing interoperability
• Multistream protocol negotiation
• Standard ping protocol payload exchange (32 bytes)
• RTT measurement and statistics
• Proper connection cleanup
• Error handling and timeout management
• Comprehensive logging for debugging
• Documentation and usage examples

To-Do

• Clean up commit history
• Add or update documentation related to these changes
• Add entry to the release notes

Cute Animal Picture

[paschal533]

Hi @lla-dane and @varun-r-mallya here is the interop test suite for Rust-libp2p and Py-libp2p. Let me know if you have any questions. follow the manual setup guild. the automatic scripts are not completed yet.

[seetadev]

@paschal533 : Kindly don't open new pull requests. Please use git rebase or squash and work on the original pull request.

Adding new pull requests will make it very difficult for rust-libp2p team to track anything.

Closing the new pull request.

[paschal533]

@paschal533 : Kindly don't open new pull requests. Please use git rebase or squash and work on the original pull request.

Adding new pull requests will make it very difficult for rust-libp2p team to track anything.

Closing the new pull request.

Alright Manu. I had no idea that you have shared the Previous PR with the Rust team. I will Rebase and continue from there. The new PLDG cohort members are having some difficulties trying to follow up with the previous PR because a lot of changes have been done since I created it, that was why I decided to create a fresh new PR with recent updates.

[seetadev]

@paschal533 : The link to #620 was shared twice in community calls both by Dave and me, and also at the rust maintainer calls as we required help on rust-libp2p front. Thought you were aware about it as we do see you at these calls.

On the same note, I re-opened your pull request as we will keep 2 versions of interop testing: one, which will be submitted for test plans repo and the other, which will be used for internal testing (think of it like dev branch).

Interop testing has to be handled with deep care. Learned it recently from an experience from another libp2p's module, whose build image got updated leading to failure of all the interop tests.

We will not submit to test plans repo unless we are thorough on all fronts.

[paschal533]

Alright @seetadev, I have taken note of that. I will update the new cohort members on what to do. Thank you.

[varun-r-mallya]

Let me try testing with the changes made in this PR in that case and will reflect these on #620.

[paschal533]

Let me try testing with the changes made in this PR in that case and will reflect these on #620.

Alright
Good idea
Keep me posted with your updates