Repository navigation
Conversation
…tamps
The ElevenLabs adapter hard-coded eleven_multilingual_v2 through the Python
SDK and then ran a Whisper pass for word timings. It now calls
/v1/text-to-speech/{voice}/with-timestamps directly, defaults to eleven_v4
(override with --tts-model, tts_model or ELEVENLABS_MODEL_ID), sends the
line's language as language_code, and builds voices[].words from the
character alignment with audio tags left out. ElevenLabs TTS no longer needs
Python or the elevenlabs package.
kumanday
force-pushed
the
fix/media-use-elevenlabs-v4-timestamps
branch
from
October 16, 2026 17:46
4f1d5a2 to
fa123fa
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The media-use ElevenLabs TTS adapter now calls
POST /v1/text-to-speech/{voice_id}/with-timestampsdirectly:eleven_v4by default, overridable with--tts-model,tts_modelin the request, or$ELEVENLABS_MODEL_ID.pip install elevenlabs.Why
skills/media-use/audio/scripts/lib/tts.mjshard-codedmodel_id="eleven_multilingual_v2"in the Python helper, with no option to change it. Every workflow that voices through the shared audio engine therefore got the older model:faceless-explainer,product-launch-video,pr-to-videoand others.The provider table in
references/tts.mdlisted ElevenLabs as having no word timestamps. Because of that, every ElevenLabs line was transcribed again with Whisper, which costs time and gives estimated timings./with-timestampsreturns exact character timings with the audio.Related work
tts.cleanup.test.mjsnow cover Kokoro only, since ElevenLabs no longer writes a temp text file.lib/tts.mjs. The two changes touch different branches ofsynthesizeOne.How
synthesizeElevenLabs: a new function inlib/tts.mjs, following thesynthesizeHeygenpattern: injectablefetchandtranscodeToWav, never throws, and returns{ ok, words, error }with the reason on failure.language_codefrom the line'slang. If a model rejects it with a 400 that mentions language, the call is retried once without it..mp3output.elevenlabsWords: turns the alignment into[{text,start,end}]and skips bracketed audio tags ([warm],[slightly wry]), so tags never reach captions.elevenlabsAvailable()now checks only for$ELEVENLABS_API_KEY, since no Python import is involved.references/tts.md):eleven_multilingual_v2voice. It now says the pin applies only to docs-site films. Switching that house voice is a separate call for maintainers.references/tts-to-captions.mdandreferences/requirements.mdreflect the new path.skills-manifest.json: regenerated.Test plan
lib/tts.test.mjscoversmodel_idandxi-api-keyheader;language_coderetry;.mp3output into a new directory;node --testoverskills/media-use/**/*.test.mjs,skills/faceless-explainer/**/*.test.mjsandskills/product-launch-video/**/*.test.mjswith thepackages/cli/scripts/test-home.mjsimport: 366 pass.bun packages/cli/scripts/gen-skills-manifest.ts --check,bun scripts/lint-skills.ts,oxfmt --checkandoxlintare clean on the changed files.synthesizeOne({ provider: "elevenlabs", lang: "es", text: "[curious] ¿Y si el grafo aprende solo?" })returned a 2.08 s, 44.1 kHz mono wav and six words with timings. The[curious]tag was excluded.