[krishvishal]

Which issue does this PR address?

Relates to #4515

Rationale

Since #4662, every gateway lists the others, but one gateway still holds every consumer group in its memory. FindCoordinator names the gateway with the lowest node id. While that gateway is down, every group waits for it. When it restarts, it has lost every member, so every group rebalances.

For N gateways that cover for each other, the gateways must share the groups. Any gateway must take over the groups of a gateway that stops, with their members, generations and offsets. Kafka keeps this state in __consumer_offsets, and a new coordinator replays it. This PR keeps it in an Iggy topic in the same way.

What changed?

The gateways share their groups through a coordinator log: the Iggy topic coordinators, with 16 partitions, in the stream <default stream>.meta. A hash of the group id picks the partition. Every gateway joins the Iggy consumer group kafka-gateways on that topic, and Iggy spreads the partitions over the gateways. No gateway polls through that group, so Iggy hands a partition over at once. A gateway that Iggy gives a partition reads it from its last checkpoint. Then it appends a claim, numbered above every claim before it. While its claim is the newest, it coordinates the groups of that partition.

When the leader's SyncGroup completes a generation, the coordinator stores the whole group. The members get their assignments only after the gateway reads that record back. The coordinator also appends each offset commit, with its metadata and leader epoch, and each deletion. It reads each append back. If a newer claim comes before the record, the record does not count. The coordinator then answers NOT_COORDINATOR (16) and stops serving the partition. So a gateway that lost a partition cannot change it. Each owner follows its partitions with the waiting read of #4629. It writes a checkpoint after 10,000 records, and at least once a day. It announces itself on the topic coordinator_owners, which every gateway reads.

With that:

• FindCoordinator names the coordinator of each group, or answers 15 while no gateway is known to own it. The other gateways answer 16 to the requests of that group. ListGroups, DescribeGroups, DeleteGroups and OffsetDelete answer only for the groups of the gateway that gets them.
• On SIGTERM, a gateway leaves the Iggy group before its drain. The others take its groups over at once, and the members keep their generation.
• OffsetFetch answers from the log, with the commit metadata. A partition that the log does not hold is read from Iggy, which holds the offsets from before the log.
• Metadata lists the live gateways. It names a leader for each partition by a hash of the topic and the index. The owner of log partition 0 is the controller. The other gateways answer NOT_CONTROLLER (41) to CreateTopics, DeleteTopics and CreatePartitions.
• The bridge follows the metadata change log of feat(metadata)!: let clients follow metadata changes without re-listing #4625 to find the groups that hold offsets, in place of one scan per topic. If the bridge user lacks manage_servers, it scans as before. The change log also updates the permissions of open connections. A connection whose Iggy user is deleted now closes.

The SDK gets ConsumerGroupClient::get_consumer_group_assignment, a public read of the caller's own assignment. It sends the existing SyncConsumerGroup command, so the wire format and the server do not change. Over HTTP it returns FeatureUnavailable. It is a separate commit.

IGGY_KAFKA_COORDINATOR_LOG turns the log on. It is on by default with IGGY_KAFKA_BROKERS and the bridge. With the log off, the gateway works as before.

[codecov]

Codecov Report

❌ Patch coverage is 89.28571% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.99%. Comparing base (fb7f727) to head (ae6d2f1).

Files with missing lines	Patch %	Lines
core/common/src/traits/consumer_group_client.rs	0.00%	2 Missing ⚠️
core/common/src/traits/binary_impls/messages.rs	92.30%	1 Missing ⚠️

❌ Your patch check has failed because the patch coverage (42.85%) is below the target coverage (50.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff            @@
##             master    #4681   +/-   ##
=========================================
  Coverage     90.99%   90.99%           
+ Complexity     3862     3857    -5     
=========================================
  Files          1555     1556    +1     
  Lines        400903   400926   +23     
  Branches     334849   334881   +32     
=========================================
+ Hits         364798   364819   +21     
+ Misses        28799    28780   -19     
- Partials       7306     7327   +21     

Components	Coverage Δ
Rust Core	92.24% <89.28%> (-0.01%)	⬇️
Java SDK	79.34% <ø> (-0.04%)	⬇️
C# SDK	84.32% <ø> (+0.03%)	⬆️
Python SDK	86.50% <ø> (ø)
PHP SDK	92.33% <ø> (ø)
Node SDK	96.60% <ø> (+0.01%)	⬆️
Go SDK	71.75% <ø> (+0.18%)	⬆️
C++ SDK	86.84% <ø> (ø)

Files with missing lines	Coverage Δ
.../common/src/traits/binary_impls/consumer_groups.rs	100.00% <100.00%> (ø)
core/kafka_wire/src/api.rs	100.00% <ø> (ø)
...rc/client_wrappers/binary_consumer_group_client.rs	87.50% <100.00%> (+1.78%)	⬆️
core/sdk/src/clients/binary_consumer_group.rs	87.50% <100.00%> (+1.78%)	⬆️
core/common/src/traits/binary_impls/messages.rs	96.16% <92.30%> (+0.40%)	⬆️
core/common/src/traits/consumer_group_client.rs	0.00% <0.00%> (ø)

... and 47 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[krishvishal]

/skill team-review

[github-actions]

Summary: The review found five defects in the new group restore and coordinator log paths, plus documentation and API surface nits. They cover the restored member protocol list, unbounded offset deletion markers, a per-batch checkpoint scan, an unvalidated coordinator log topic, and topic deletions lost on a broadcast lag.

Counts: critical 0, warning 5, nit 2, simplification 2

---

This review was generated by Claude Code 2.1.310 on deepseek-flash[1m]. Review the output before you act on it.