Skip to content
#

tensor-parallel

Star

Here are 3 public repositories matching this topic...

Language: All
Filter by language
All 3 Python 2 Shell 1

theogravity / dual-rtx-6000-blackwell-Gemma-4-31B-IT-NVFP4

Sponsor
Star 5

Optimized vLLM setup for Gemma 4 31B NVFP4 with MTP on dual RTX PRO 6000 Blackwell using vllm and docker: native FP4 Tensor Cores, Multi-Token Prediction (96.5% acceptance rate), and prefix caching. Includes benchmark results and replication scripts.

docker amd cuda gemma blackwell vllm llm-inference am5 speculative-decoding fp4 prefix-caching multi-token-prediction nvfp4 rtx-6000 gemma4 tensor-parallel
  • Updated May 10, 2026
  • Shell

Improve this page

Add a description, image, and links to the tensor-parallel topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the tensor-parallel topic, visit your repo's landing page and select "manage topics."

Learn more