Kafka gateway: protocol parity tracking

[hubcio]

Umbrella tracking issue for the Kafka-wire-protocol gateway (gateways/), letting existing Kafka clients talk to Iggy unmodified.

Seed design + TCP listener lands in #3421.

Goal: implement enough of the Kafka wire protocol that standard Kafka producers/consumers (and admin tools) work against Iggy via a bridge, then document migration off Kafka.

Sub-issues

Foundation:

• Kafka gateway: Build Gateway to Iggy for existing Kafka users - Create a TCP Listener for Kafka users #3421 - TCP listener for Kafka clients (seed)
• Kafka gateway: Kafka gateway IggyBridge core — SDK client and stream/topic mapping #3533 - IggyBridge core: SDK client + stream/topic mapping

Phase 1 - core wire APIs + tooling:

• Kafka gateway: Kafka Metadata API (key 3) — broker and topic discovery via Iggy #3534 - Metadata API (key 3): broker/topic discovery
• Kafka gateway: Kafka Produce API (key 0) — bridge to Iggy send_messages #3535 - Produce API (key 0): bridge to send_messages
• Kafka gateway: Kafka Fetch API (key 1) — bridge to Iggy offset-based polling #3536 - Fetch API (key 1): bridge to offset-based polling
• Kafka gateway: Kafka ListOffsets API (key 2) — earliest/latest via Iggy high watermark #3537 - ListOffsets API (key 2): earliest/latest via high watermark
• Kafka gateway: Kafka CreateTopics API (key 19) — Iggy stream/topic provisioning #3538 - CreateTopics API (key 19): stream/topic provisioning
• Kafka gateway: Kafka gateway Phase 1 — docker-compose, E2E tests, and quick-start docs #3539 - Phase 1: docker-compose, E2E tests, quick-start docs

Phase 2 - consumer groups + offsets:

• Kafka gateway: Design Consumer offset persistence strategy for Kafka gateway Phase 2 #3540 - design: consumer offset persistence strategy
• Kafka gateway: Kafka FindCoordinator, JoinGroup, SyncGroup, and Heartbeat APIs #3541 - FindCoordinator, JoinGroup, SyncGroup, Heartbeat
• Kafka gateway: Kafka OffsetCommit and OffsetFetch with persisted consumer offsets #3542 - OffsetCommit + OffsetFetch with persisted offsets
• Kafka gateway: Kafka LeaveGroup and consumer group rebalance semantics #3543 - LeaveGroup + rebalance semantics
• Kafka gateway: Kafka gateway Phase 2 — E2E tests with consumer groups #3544 - Phase 2: E2E tests with consumer groups

Admin + extended APIs:

• Kafka gateway: Kafka transactional APIs — UNSUPPORTED_VERSION stub and idempotence docs #3545 - transactional APIs: UNSUPPORTED_VERSION stub + idempotence docs
• Kafka gateway: Kafka DeleteTopics, DescribeConfigs, and AlterConfigs admin APIs #3546 - DeleteTopics, DescribeConfigs, AlterConfigs
• Kafka gateway: Kafka DeleteRecords — trim topic to offset via Iggy retention #3547 - DeleteRecords: trim topic to offset via retention
• Kafka gateway: Kafka DescribeGroups and ListGroups admin APIs #3548 - DescribeGroups + ListGroups

Auth + migration:

• Kafka gateway: Kafka SASL authentication bridge to Iggy users #3549 - Kafka SASL auth bridge to Iggy users
• Kafka gateway: Kafka-to-Iggy migration guide — dual-write, cutover, decommission #3550 - migration guide: dual-write, cutover, decommission

Status

Stays open until boxes above are checked or de-scoped.