Skip to content

audio8_asr.md

Latest commit

 

History

History
104 lines (86 loc) · 5.06 KB

audio8_asr.md

File metadata and controls

104 lines (86 loc) · 5.06 KB
Raw

Audio8-ASR-0.1B in audio.cpp

Audio8-ASR-0.1B is a compact multilingual autoregressive ASR model (en / zh / yue / ja / ko / fr / de): a Qwen3-ASR audio encoder adapted by an MLP tower into an 8-layer Qwen2-style decoder with only ~103M language-model parameters (324M end-to-end). The checkpoint ships under CC-BY-NC-4.0, so audio.cpp loads it from locally converted weights only; the converted GGUF must not be redistributed.

Architecture

  • Audio frontend: 16 kHz Whisper log-mel, 128 bins, hop 160, n_fft 400 (shared with the qwen3_asr family). The reference processor emits bfloat16 mel values; the audio8_asr frontend rounds to bfloat16 before encoding to match.
  • Audio encoder: Qwen3-ASR audio tower (d_model 896, 18 layers, 14 heads, FFN 3584, 128 mel bins, output dim 1024) — bit-for-bit the same architecture as Qwen/Qwen3-ASR-0.6B, loaded through the shared qwen3_asr encoder implementation.
  • Adapter: 4 pre-norm residual MLP blocks (1024 → 4096, erf GELU), a final LayerNorm, then an adaptive average pool (merge factor 4 against the mel-frame count) followed by LayerNorm + Linear(1024 → 512).
  • Decoder: Qwen2-style causal LM, 8 layers, hidden 512, 8 heads, head_dim 64, SwiGLU FFN 1408, tied embeddings over a 151,936 Qwen BPE vocabulary, RoPE theta 1e6, RMSNorm eps 1e-6.
  • Prompt: <|user|><|begin_of_audio|><|audio|>×N<|end_of_audio|>Please transcribe this audio.<|assistant|> where N = max(floor(floor((floor(samples / hop) + 1) / 2) / merge_factor), 1) (all divisions integer).

Usage

# Convert locally (requires the audiocpp_gguf tool and a self-downloaded
# checkpoint from the Audio8/Audio8-ASR-0.1B HF repository):
python tools/community_models/audio8_asr/convert_audio8_asr.py \
    --checkpoint models/Audio8-ASR-0.1B-hf \
    --converter build/debug/bin/audiocpp_gguf \
    --type q8_0 \
    --output models/Audio8-ASR-0.1B-GGUF/audio8-asr-0.1b-q8_0.gguf

# Transcribe
audiocpp_cli --task asr --family audio8_asr \
    --model models/Audio8-ASR-0.1B-GGUF/audio8-asr-0.1b-q8_0.gguf \
    --audio sample.wav

# The safetensors package loads directly, no conversion required:
audiocpp_cli --task asr --family audio8_asr \
    --model models/Audio8-ASR-0.1B-hf --audio sample.wav

Parity

tools/community_models/audio8_asr/audio8_asr_reference.py runs the Hugging Face trust_remote_code reference (torch CPU, fp32) and tools/community_models/audio8_asr/audio8_asr_stages.py captures staged tensors (mel, encoder output, projected audio embeddings) for comparison.

Greedy transcription on the repo test clips matched the fp32 reference exactly with the Q8_0 GGUF on both the Metal and CPU backends:

Audio Reference (fp32) audio.cpp (Q8_0 GGUF, Metal + CPU)
assets/resources/a.wav (5.95 s) "This little work was finished in the year eighteen o three, and intended for immediate publication." identical
assets/resources/sample_16k.wav (14.07 s) "Some call me nature. Others call me Mother Nature. I've been here for over four point five billion years, twenty-two thousand five hundred times longer than you." identical

A 61-second clip transcribed through rate-correct 30-second windows matched the per-window reference transcripts (also verified by an independent review pass). test_audio8_asr_golden_transcription asserts the first row end-to-end — run it manually once weights exist (it is not part of ctest, matching the granite5asr golden-test convention); test_audio8_asr_units covers the token-count formula and bfloat16 rounding and runs under ctest.

Measured performance

Release build (-DCMAKE_BUILD_TYPE=Release), Apple M4, Metal backend, Q8_0 GGUF (345 MB weights):

Audio Session wall (session.wall_ms) Effective RTF CLI wall (incl. ~2.2 s process + load)
5.95 s ~0.75 s ~8x realtime 2.2 s
14.07 s 757 ms 18.6x realtime 3.8 s
61 s (3 windows) 3.46 s 17.6x realtime 5.8 s

The audio encoder dominates (~64% of session time; its Metal shaders are insensitive to build type, so Debug and Release measure within 3%). Peak RSS is ~1.0 GB for a single clip and ~1.3 GB for a three-window transcription (Metal buffers + per-window-shape graph pools on top of the 345 MB of weights; CPU backend measures the same ~1.0 GB).

Known limitations

  • Offline only: no streaming mode, no word timestamps, no language-id output.
  • 30-second windows: audio longer than 30 s is transcribed in fixed windows sized at the input sample rate (0.5 s minimum tail folds into the previous window) and space-joined, without VAD segmentation. Unlike the single-pass reference, each window is peak-normalized independently, so relative loudness across a window boundary can shift.
  • CC-BY-NC-4.0: non-commercial use only; convert locally, do not redistribute the converted GGUF. There is no release GGUF package and no WebUI catalog entry (the package manager cannot download unsupported-license packages).
  • Hotword logit boosting from the reference implementation is not ported.