Hyli

Hyli is a voice agent platform that connects any telephony provider (Plivo, Twilio) to any conversational AI provider (ElevenLabs, OpenAI Realtime) with a neural memory layer that persists emotional state, intent, and conversation context across calls.

Most voice agents forget callers the moment the call ends. Hyli remembers. Not with a database lookup or a GPT summarization call, but with Helix, a phase-space neural architecture that encodes emotional state as a persistent tensor and restores it at the start of the next call.

What it does

• Routes inbound and outbound calls through a WebSocket bridge to your ConvAI provider.
• Runs a Helix subprocess that tracks caller affect (frustrated, urgent, confused, positive, neutral) and intent (complaint, escalation, general inquiry, etc.) token by token, in real time.
• Exports the full call state (phase tensor, affect history, transcript summary) to SQLite at call end.
• On the caller's next call, restores that state before connecting to the AI agent so the agent already knows who they are, how they felt last time, and what the call was about.
• Does all of this at zero marginal cost. No extra LLM API calls for memory.

Architecture

Plivo / Twilio
      |
      | webhook
      v
  Hyli server (Go)
      |
      |-- /ws/stream --> bridge --> ElevenLabs / OpenAI Realtime
      |
      |-- Helix subprocess (Python, stdin/stdout JSON)
      |       |-- HelixCell (phase-space neural memory)
      |       |-- AffectiveEncoder (Russell Circumplex emotional state)
      |       |-- intent head + affect head (trained or keyword fallback)
      |
      |-- SQLite (calls + caller profiles with helix state)

Requirements

• Go 1.22+
• Python 3.10+ with torch installed
• The Helix repo cloned alongside this one
• A Plivo or Twilio account
• An ElevenLabs or OpenAI account

Setup

cp .env.example .env
# fill in your keys
go run ./cmd/server

For the Helix subprocess to work, the helix repo must be importable from Python. Set HELIX_PATH in .env to point at it.

To train the neural intent and affect heads:

python scripts/train_helix.py

This generates models/helix_call.pt. Without it, Hyli falls back to keyword-based detection, which still passes all tests.

To run the test suite:

python scripts/test_hyli.py

Environment variables

See .env.example for the full list. The critical ones:

Variable	Purpose
ELEVENLABS_API_KEY	ElevenLabs credentials
ELEVENLABS_AGENT_ID	Your ConvAI agent
PLIVO_AUTH_ID / PLIVO_AUTH_TOKEN	Plivo credentials
PUBLIC_BASE_URL	Publicly reachable URL for webhooks
API_SECRET_KEY	Bearer token for REST API auth
STREAM_API_KEY	Token for WebSocket stream auth
HELIX_PATH	Path to the Helix repo
HELIX_ENABLED	Set to true to enable neural memory

Provider swap

To switch from ElevenLabs to OpenAI Realtime, set CONVAI_PROVIDER=openai and provide OPENAI_API_KEY. The internal/convai package defines a provider interface. Both implementations are included.

Security

Webhook signatures are validated for both Plivo (HMAC-SHA256) and Twilio (HMAC-SHA1). All /api/* routes require a Bearer token. The WebSocket stream endpoint requires a token passed at connection time. Outbound calls are rate-limited to 10 per minute.