MESSAGE_TOO_LARGE in MirrorMaker2 #11829
Unanswered
bingkunyangvungle
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I was trying to use MM2 to sync data between Kafka, but got error:
2025-09-05 03:18:04 WARN [kafka-producer-network-thread | connector-producer-mm2-source->mm2-target.MirrorSourceConnector-3] Sender:672 - [Producer clientId=connector-producer-mm2-source->mm2-target.MirrorSourceConnector-3] Got error produce response in correlation id 2077 on topic-partition topic, splitting and retrying (2147483647 attempts left). Error: MESSAGE_TOO_LARGE
I actually checked the msg in the original cluster and manually sent on by one and it has no issue of syncing to the target cluster.
Could someone help to identify what cause this issue?
Here is the config:
sourceConnector: tasksMax: 10 config: producer: batchSize: 16777216 lingerMs: 50 bufferMemory: 536870912 # 512MB maxRequestSize: 52428800 # 50MB consumer: fetchMaxBytes: 104857600 # 100MB maxPartitionFetchBytes: 10485760 # 10MB per partition maxPollRecords: 5000 fetchMinBytes: 10485760 # 10MB autoOffsetReset: "latest" replicationPolicyClass: "org.apache.kafka.connect.mirror.IdentityReplicationPolicy"
Beta Was this translation helpful? Give feedback.
All reactions