perf(marlin): cache scale permutation tensors and use index_select

[Qubitium]

Summary

marlin_permute_scales and marlin_permute_bias re-created the Marlin scale/bias permutation index tensors from Python lists on every call. For MoE models with tens of thousands of experts (e.g. Laguna-S-2.1-GPTQ-FIXED, 36,432 quant modules) this added ~4.4s during GPTQModel.load.

What Changed

• gptqmodel/utils/marlin.py

  • Added a per-device cache _SCALE_PERM_TENSOR_CACHE protected by a threading.Lock and helper _get_scale_perm_tensors().
  • marlin_permute_scales and marlin_permute_bias now use pre-materialized torch.long index tensors and torch.index_select instead of Python-list advanced indexing.

• tests/test_marlin_permute.py

  • Added correctness tests for marlin_permute_scales and marlin_permute_bias covering Laguna attention/MLP/MoE shapes.
  • Tests dynamically select cuda:0 when available and fall back to cpu, and use float32 to avoid crashing on CPU-only hosts.

Tests

• I added a new simple/fast unit test for this change.
• I ran the new targeted test locally before opening this PR.
• I ran any other directly relevant local tests.

cd /root/repos/GPTQModel
/root/vm314t/bin/python -m ruff check gptqmodel/utils/marlin.py tests/test_marlin_permute.py
/root/vm314t/bin/python -m pytest tests/test_marlin_permute.py -q

Result: 6 passed, 16 warnings in 6.73s and ruff clean.

Notes

Benchmarked on Laguna-S-2.1-GPTQ-FIXED with BACKEND.GPTQ_MARLIN on GPU 7:

• gptq_marlin_repack remained ~1.9s (already fast CUDA kernel).
• marlin_permute_scales dropped from ~4.4s to ~2.4s for all 36,432 modules.
• Total packing/repacking overhead is ~6.3s out of a ~96-140s load, so the remaining load-time bottleneck is elsewhere (safetensors weight I/O and per-module Python/validation overhead).

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring