Diffusers

diffusers is a Ruby-first experiment for running diffusion pipelines with torch-rb.

The current MVP target is intentionally narrow:

• SDXL latent denoising
• Euler ancestral sampling
• safetensors weight loading
• latent output only for now, with no VAE decode requirement

Installation

Add the gem to your application:

gem "diffusers"

Or install it directly:

gem install diffusers

Dependencies

The MVP expects these runtime gems:

• torch-rb
• safetensors
• numo-narray

Usage

require "diffusers"

model = Diffusers::Model::Sdxl.new(
  backend: lambda do |latents:, timestep:, **|
    latents.map { |value| value.is_a?(Numeric) ? value * 0.1 : value }
  end,
  state_dict: { "unet" => :placeholder }
)

scheduler = Diffusers.scheduler(:euler_ancestral)
pipeline = Diffusers.pipeline(:text_to_image, model: model, scheduler: scheduler)

result = pipeline.call(
  prompt: "cinematic product photo of a ruby gem on velvet",
  num_inference_steps: 30,
  guidance_scale: 6.5,
  seed: 1234
)

pp result.latents
pp result.metadata

Status

This repo now provides the foundation for:

• runtime dependency wiring
• scheduler registration
• SDXL pipeline orchestration
• model loading hooks for safetensors-backed weights

It does not yet provide:

• real SDXL UNet graph construction in torch-rb
• dual text encoders and tokenizer parity
• VAE decode
• prompt weighting, LoRA, ControlNet, or Hub download support

Development

Run:

bundle install
rake test