vsl.cuda is a high-performance GPU compute backend for VSL backed by
NVIDIA CUDA (cuBLAS + cuDNN).
CUDA backend — cuBLAS/cuDNN bindings active when CUDA Toolkit + cuDNN are available at build time (
-d cuda). Operations use GPU kernels where implemented; CPU fallback when CUDA/cuDNN is unavailable.VTL integration (opt-in CUDA Linear/Conv2D): merged in vlang/vtl#93 (issues #89–#91 closed).
| Operation | GPU (CUDA+cuDNN) | Fallback | Tracker |
|---|---|---|---|
gemm |
✅ cublasDgemm |
CPU col-major | — |
gemv |
✅ cublasDgemv |
CPU | — |
relu / sigmoid / tanh |
✅ cuDNN activation | CPU | — |
add_vec |
✅ cublasDaxpy |
CPU | — |
mul_vec |
✅ cublasDdgmm |
CPU | — |
add_scalar / mul_scalar |
✅ cuBLAS/cuDNN path | CPU | — |
softmax |
✅ cudnnSoftmaxForward |
CPU | — |
layernorm |
optional -d cudnn_layernorm with cuDNN 9.1+ |
CPU when the GPU implementation is unavailable | #550 |
conv2d |
✅ cudnnConvolutionForward |
CPU | — |
conv2d_backward |
✅ cudnnConvolutionBackward* |
CPU | — |
mul_vec uses legacy cublasDdgmm (SIDE_RIGHT, 1×n row layout;
cublasDdgmm_v2 absent on some distros). Layer norm GPU: build with
-d cudnn_layernorm when libcudnn exports cudnnLayerNormForward (9.1+).
CUDABackend.supports('layernorm') reports whether the selected device and
build can execute the GPU implementation. If it is available but execution
fails, the error is propagated; only an unavailable GPU implementation uses
the CPU fallback. Strict compute contexts reject that fallback. Numerical
parity tests: cuda/compute/numerical_validation_test.v
(#281).
vsl.cuda
├── backend.v # CUDABackend (ComputeBackend interface impl)
├── compute/
│ ├── elementwise.v # Activation functions (relu, sigmoid, tanh, ...)
│ ├── gemm.v # Public GEMM wrapper (row↔col conversion)
│ └── gemm_impl.v # Internal GEMM/GEMV implementation + CPU fallbacks
└── v.mod
Memory layout: cuBLAS is column-major (same as VCL/Vulkan). The
CUDABackend.to_internal() / from_internal() methods handle row↔column-major
conversion at the dispatch boundary.
import vsl.compute
// Use CUDA backend automatically when available
ctx := compute.new_context(.cuda)
// All compute operations dispatch to CUDA when the backend is set
a := []f64{len: 6}
b := []f64{len: 6}
// ... fill a, b ...
result := compute.add_vec(ctx, a, b)!Or directly via the backend:
import vsl.cuda
mut dev := cuda.get_default_device()!
mut backend := cuda.new_cuda_backend()
result := backend.relu(my_data)!- NVIDIA GPU with compute capability ≥ 5.0 (Maxwell or newer)
- NVIDIA Driver ≥ 525.60 (for CUDA 12.x)
- CUDA Toolkit ≥ 11.8
- cuDNN ≥ 8.0
nvcc(NVIDIA C compiler) in$PATH- CUDA Toolkit headers (
cuda.h,cublas.h,cudnn.h) - cuDNN headers
# NVIDIA driver (verify with nvidia-smi)
sudo pacman -S nvidia nvidia-utils
# CUDA Toolkit (includes cuBLAS)
sudo pacman -S cuda
# cuDNN (must match CUDA version)
sudo pacman -S cudnn
# Verify
nvcc --version # should show 12.x or 13.x
nvidia-smi # should show your GPU# NVIDIA driver
sudo apt-get install nvidia-driver-535
# CUDA Toolkit
wget https://developer.downloads.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update
sudo apt-get install cuda-toolkit-12-2
# cuDNN
sudo apt-get install libcudnn8 libcudnn8-devCUDA Toolkit is no longer supported on macOS arm64 (Apple Silicon). For GPU acceleration on Apple Silicon, use the VCL/OpenCL backend instead.
VSL uses conditional compilation ($if cuda ?) to include CUDA code:
# Compile with CUDA backend
v -d cuda run your_app.v
# Run tests with CUDA
v -d cuda test .
# Lint with CUDA
v -d cuda vet .Set CUDNN_PATH if cuDNN is not in a standard location:
CUDNN_PATH=/opt/cuda v -d cuda run your_app.v- GEMM (matrix-matrix) — the highest-impact primitive. cuBLAS
dgemmis heavily optimized for NVIDIA Tensor Cores on Ampere+ GPUs. - cuDNN activations — fused kernels for ReLU, Sigmoid, Tanh provide ~2-5x speedup over CPU for large tensors.
- Memory layout — cuBLAS uses column-major; VSL uses row-major. The
to_internal()/from_internal()conversion adds overhead for small tensors but is negligible for large matrices (≥ 256×256).
See the cuda/examples/ directory:
# Run CUDA backend smoke test
v -d cuda run cuda/examples/relu_example.v| Phase | Description |
|---|---|
| ✅ Phase B | Infrastructure ready; CPU fallback path |
| ✅ Phase C | cuBLAS/cuDNN kernels for GEMM/GEMV, activations, softmax, Conv2D, LayerNorm |
| 🏗️ Phase D | Device discovery, multi-GPU support |
| 🧠 Phase E | GPU memory management (avoid CPU↔GPU copies) |
| ✅ Phase F | Numerical validation vs reference for key kernels |
- VSL Documentation
- cuBLAS Documentation
- cuDNN Documentation
- CUDA Toolkit Download
- VSL ADR-001: Multi-backend GPU compute
- VTL CUDA example
Accelerate your scientific computing with NVIDIA GPUs! 🚀