Skip to content

Latest commit

 

History

History

cuda

Folders and files

README.md

CUDA Compute Backend for VSL 🖥️

vsl.cuda is a high-performance GPU compute backend for VSL backed by NVIDIA CUDA (cuBLAS + cuDNN).

🚀 Status

CUDA backend — cuBLAS/cuDNN bindings active when CUDA Toolkit + cuDNN are available at build time (-d cuda). Operations use GPU kernels where implemented; CPU fallback when CUDA/cuDNN is unavailable.

VTL integration (opt-in CUDA Linear/Conv2D): merged in vlang/vtl#93 (issues #89–#91 closed).

Operation GPU (CUDA+cuDNN) Fallback Tracker
gemm ✅ cublasDgemm CPU col-major —
gemv ✅ cublasDgemv CPU —
relu / sigmoid / tanh ✅ cuDNN activation CPU —
add_vec ✅ cublasDaxpy CPU —
mul_vec ✅ cublasDdgmm CPU —
add_scalar / mul_scalar ✅ cuBLAS/cuDNN path CPU —
softmax ✅ cudnnSoftmaxForward CPU —
layernorm optional -d cudnn_layernorm with cuDNN 9.1+ CPU when the GPU implementation is unavailable #550
conv2d ✅ cudnnConvolutionForward CPU —
conv2d_backward ✅ cudnnConvolutionBackward* CPU —

mul_vec uses legacy cublasDdgmm (SIDE_RIGHT, 1×n row layout; cublasDdgmm_v2 absent on some distros). Layer norm GPU: build with -d cudnn_layernorm when libcudnn exports cudnnLayerNormForward (9.1+). CUDABackend.supports('layernorm') reports whether the selected device and build can execute the GPU implementation. If it is available but execution fails, the error is propagated; only an unavailable GPU implementation uses the CPU fallback. Strict compute contexts reject that fallback. Numerical parity tests: cuda/compute/numerical_validation_test.v (#281).

📁 Architecture

vsl.cuda
├── backend.v           # CUDABackend (ComputeBackend interface impl)
├── compute/
│   ├── elementwise.v   # Activation functions (relu, sigmoid, tanh, ...)
│   ├── gemm.v          # Public GEMM wrapper (row↔col conversion)
│   └── gemm_impl.v     # Internal GEMM/GEMV implementation + CPU fallbacks
└── v.mod

Memory layout: cuBLAS is column-major (same as VCL/Vulkan). The CUDABackend.to_internal() / from_internal() methods handle row↔column-major conversion at the dispatch boundary.

🎯 Quick Start

import vsl.compute

// Use CUDA backend automatically when available
ctx := compute.new_context(.cuda)

// All compute operations dispatch to CUDA when the backend is set
a := []f64{len: 6}
b := []f64{len: 6}
// ... fill a, b ...

result := compute.add_vec(ctx, a, b)!

Or directly via the backend:

import vsl.cuda

mut dev := cuda.get_default_device()!
mut backend := cuda.new_cuda_backend()

result := backend.relu(my_data)!

🔧 Requirements

Runtime (at runtime)

  • NVIDIA GPU with compute capability ≥ 5.0 (Maxwell or newer)
  • NVIDIA Driver ≥ 525.60 (for CUDA 12.x)
  • CUDA Toolkit ≥ 11.8
  • cuDNN ≥ 8.0

Build time (for CUDA GPU builds)

  • nvcc (NVIDIA C compiler) in $PATH
  • CUDA Toolkit headers (cuda.h, cublas.h, cudnn.h)
  • cuDNN headers

📦 Installation

Arch Linux

# NVIDIA driver (verify with nvidia-smi)
sudo pacman -S nvidia nvidia-utils

# CUDA Toolkit (includes cuBLAS)
sudo pacman -S cuda

# cuDNN (must match CUDA version)
sudo pacman -S cudnn

# Verify
nvcc --version        # should show 12.x or 13.x
nvidia-smi            # should show your GPU

Ubuntu / Debian

# NVIDIA driver
sudo apt-get install nvidia-driver-535

# CUDA Toolkit
wget https://developer.downloads.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update
sudo apt-get install cuda-toolkit-12-2

# cuDNN
sudo apt-get install libcudnn8 libcudnn8-dev

macOS

CUDA Toolkit is no longer supported on macOS arm64 (Apple Silicon). For GPU acceleration on Apple Silicon, use the VCL/OpenCL backend instead.

🏗️ Compiling with CUDA

VSL uses conditional compilation ($if cuda ?) to include CUDA code:

# Compile with CUDA backend
v -d cuda run your_app.v

# Run tests with CUDA
v -d cuda test .

# Lint with CUDA
v -d cuda vet .

Set CUDNN_PATH if cuDNN is not in a standard location:

CUDNN_PATH=/opt/cuda v -d cuda run your_app.v

⚡ Performance Notes

  • GEMM (matrix-matrix) — the highest-impact primitive. cuBLAS dgemm is heavily optimized for NVIDIA Tensor Cores on Ampere+ GPUs.
  • cuDNN activations — fused kernels for ReLU, Sigmoid, Tanh provide ~2-5x speedup over CPU for large tensors.
  • Memory layout — cuBLAS uses column-major; VSL uses row-major. The to_internal() / from_internal() conversion adds overhead for small tensors but is negligible for large matrices (≥ 256×256).

🧪 Examples

See the cuda/examples/ directory:

# Run CUDA backend smoke test
v -d cuda run cuda/examples/relu_example.v

🗺️ Roadmap

Phase Description
✅ Phase B Infrastructure ready; CPU fallback path
✅ Phase C cuBLAS/cuDNN kernels for GEMM/GEMV, activations, softmax, Conv2D, LayerNorm
🏗️ Phase D Device discovery, multi-GPU support
🧠 Phase E GPU memory management (avoid CPU↔GPU copies)
✅ Phase F Numerical validation vs reference for key kernels

🔗 Resources


Accelerate your scientific computing with NVIDIA GPUs! 🚀