[trcwebdesign]

Summary

This PR contains a broad set of changes focused on improving Windows support for large Qwen and Gemma models in FreeToken, covering:

NVFP4 and GGUF checkpoint loading
MoE execution and offloading
Improved checkpoint compatibility
Additional groundwork for direct Hugging Face model support

I acknowledge that this exceeds the preferred "one change per PR" contribution guideline and combines multiple areas of work that would normally be submitted separately.
The reason for opening it in this form is that these changes were developed in a private fork over an extended period.
Rather than leaving the work isolated, I wanted to make it available to the project maintainers and community.
Even if the PR is ultimately split or only parts are accepted, I believe several of the changes may be useful as a reference
or as a basis for future support of out-of-the-box Hugging Face models loading (see below).

These specific uncensored model builds and the Windows launch workflows documented here are supported by this fork, maybe not by the upstream/origin FreeToken repository. Upstream does not include all of this fork's checkpoint compatibility fixes and Windows-specific features, so do not expect these models to run there with the commands or configuration shown here.

Local model folder	Hugging Face checkpoint / source	Upstream base
gemma-4-26B-A4B-it-abliterated-Q4_0-GGUF	huihui-ai/Huihui-gemma-4-26B-A4B-it-abliterated (local Q4_0 GGUF)	google/gemma-4-26B-A4B-it
gemma-4-26B-A4B-it-ultra-uncensored-heretic-BF16-nvfp4-with-metadata	llmfan46/gemma-4-26B-A4B-it-ultra-uncensored-heretic (local NVFP4 export)	google/gemma-4-26B-A4B-it
gemma-4-26B-A4B-it-ultra-uncensored-heretic-GGUF	mradermacher/gemma-4-26B-A4B-it-ultra-uncensored-heretic-GGUF; source safetensors: llmfan46/gemma-4-26B-A4B-it-ultra-uncensored-heretic	google/gemma-4-26B-A4B-it
Gemma-4-26B-A4B-it-Uncensored-EXPERTS-NVFP4	Local mixed GGUF: routed expert weights are NVFP4 and other tensors remain BF16; metadata marks the HF-NVFP4 layout.	Gemma 4 26B A4B Instruct family: google/gemma-4-26B-A4B-it
Gemma-4-26B-A4B-it-Uncensored-FP8	orcarouter/Gemma-4-26B-A4B-it-Uncensored-FP8	google/gemma-4-26B-A4B-it
Gemma-4-26B-A4B-it-Uncensored-TRC-nvfp4	Local mixed NVFP4/BF16 GGUF (11,921 NVFP4 and 12,277 BF16 tensors in the local file).	Gemma 4 26B A4B Instruct family: google/gemma-4-26B-A4B-it
Ornith-1.5-35B-A3B-FULL-OBLITERATED-NVFP4	jhone888/Ornith-1.5-35B-A3B-FULL-OBLITERATED-NVFP4, derived from jhone888/Ornith-1.5-35B-A3B-FULLY-OBLITERATED	huihui-ai/Huihui-Ornith-1.5-35B-A3B-abliterated, from ornith-ai/Ornith-1.5-35B-A3B
Ornith-1.5-35B-A3B-uncensored-NVFP4	ressl/Ornith-1.5-35B-A3B-uncensored-NVFP4	ornith-ai/Ornith-1.5-35B-A3B
Qwen3.6-Fallen-Fabulist-35B-A3B-heretic	Cyclone-Labs/Qwen3.6-Fallen-Fabulist-35B-A3B-heretic	Merge of Qwen/Qwen3.6-35B-A3B, Gryphe/WorldSim-Opus-3.6-35B-A3B, ReadyArt/Melody1437-35B-A3B, and ornith-ai/Ornith-1.5-35B-A3B
Qwen3.6-Fallen-Fabulist-35B-A3B-heretic-GGUF	mradermacher/Qwen3.6-Fallen-Fabulist-35B-A3B-heretic-GGUF; source: Cyclone-Labs/Qwen3.6-Fallen-Fabulist-35B-A3B-heretic	Same merged bases as the safetensors checkpoint above
Qwen3.8-Flash-Next-ABLITERATED-NVFP4	dealignai/Qwen3.8-Flash-Next-ABLITERATED-NVFP4	Qwen/Qwen3.8-Flash-Next
Qwen3.8-Flash-Next-Uncensored-FP8	orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8	Qwen/Qwen3.8-Flash-Next
Qwen3.8-Flash-Next-Uncensored-NVFP4	orcarouter/Qwen3.8-Flash-Next-Uncensored-NVFP4	Qwen/Qwen3.8-Flash-Next
supergemma4-e4b-abliterated-i1-GGUF	mradermacher/supergemma4-e4b-abliterated-i1-GGUF; source: Jiunsong/supergemma4-e4b-abliterated	google/gemma-4-E4B-it

What changed

• Add Windows disk-tier MoE offload for native NVFP4 checkpoints. Routed expert rows can be fetched from safetensors while keeping a configurable subset in host RAM.
• Improve Qwen3.8 and related Qwen loaders for NVFP4, FP8 GDN, BF16 vision weights, hyperconnection tensors, and Qwen4Exp PLE handling.
• Expand Gemma 4 GGUF compatibility, including Q8_0 embeddings/projections/experts, tokenizer behavior, and mixed-quantization checkpoint support.
• Add GGUF support and execution paths for quantized weights, including shared GGML CUDA kernels, NVFP4 dequantization, and Q4_0 CPU MoE execution.
• Add checkpoint repair and conversion utilities for Gemma and Qwen metadata, tokenizers, sampling configuration, and mixed GGUF creation.
• Update CLI/model documentation and add regression coverage for the new loader, GGUF, MoE, and Windows paths.

Performance notes

Up to 100 tokens/s on some Blackwell/model combinations and about 10 tokens/s for Qwen3.8-Flash-Next NVFP4 in the reported setup. Throughput depends on hardware, checkpoint, context length, and serving configuration.

Please include the tested GPU, CPU, driver, exact checkpoint ID, exact command, and comparable main versus branch results before presenting these figures as benchmark results.

Review notes

The branch includes two archived site-library ZIPs. Please confirm they are intended upstream deliverables; otherwise remove them from the PR before submission.