Skip to content

docs(moe): report shared expert banks across replicas - #451

Open
kuangrenaigc-stack wants to merge 1 commit into
FlashML-org:mainfrom
kuangrenaigc-stack:docs/dual-replica-shared-banks
Open

kuangrenaigc-stack wants to merge 1 commit into
FlashML-org:mainfrom
kuangrenaigc-stack:docs/dual-replica-shared-banks

Conversation

@kuangrenaigc-stack

Copy link
Copy Markdown

Independent MoE replicas normally allocate one full host expert-bank set per process. On a 96 GB workstation, that prevents two Qwen3.8-Flash-Next replicas from serving one 262K lane per GPU even though the GPU-side state fits.

This documentation-only change reports a tested owner/follower design that maps one immutable 63.457 GiB NVFP4 expert-bank set into two TP1 FreeToken processes. It records the protocol, normalized command, physical-sharing evidence, concurrent short and near-256K results, and the failed prefix-reuse, port-allocation, and shutdown checks. It explicitly does not present the external shim as a supported feature or tensor parallelism.

The report is intended as implementation evidence for the multi-GPU roadmap in #79. All local paths, usernames, hostnames, device identifiers, ports, workload content, and service configuration outside FreeToken were removed.

Validation:

  • git diff --cached --check
  • Markdown fence, tab, and trailing-whitespace checks
  • privacy scan over the complete committed diff
  • roadmap link returned HTTP 200
  • attachment rechecked on 953565667f3141c90d0f0eb469bb2655d2407140; benchmark numbers remain attributed to the earlier tested base stated in the document

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant