diff --git a/README.md b/README.md index 4893bfb195..b8a9c486ea 100644 --- a/README.md +++ b/README.md @@ -182,6 +182,34 @@ described by traditional `combinator` systems. If a `message` does not explicitly specify a `device`, its implied `device` is a `message@1.0`, which simply returns the binary or `message` at a given named function. +## Agents — `Agent = LLM + Harness + Tools + Instructions` + +HyperBEAM now ships a **generic agent stack** per [What is an Agent?](https://chat.hyper.io/share/wE41XruWrXFj37EdGfFxn3sdXy0Wib9e) and [Agent Flow](https://chat.hyper.io/share/_4kT2xkDzGpPncXP5k6wx8zB5m4LKlnT): + +``` +Agent = LLM (engine) + Harness (loop+memory) + Tools (hands) + Instructions (identity) +``` + +* **LLM** — `~llm@1.0` OpenAI proxy (Ollama/vLLM, `spark-1b7b.local:8888`, streaming SSE) — reasoning only. +* **Harness** — `~harness@1.0` generic runtime: builds `system(identity.md+soul.md+user.md) + tools.json + history↑limit + current` window, loops `LLM → tool_calls → relay@1.0 → rebuild` until done, persists `history` to `hb_store` (`-harness-history`, default limit 20). One harness, many agents. +* **Tools** — atomic `relay@1.0` calls (or MCP): `get_gmail_messages`, `fetch`. If in `tools.json`, agent can use it. +* **Instructions** — `agents//{identity.md,user.md,soul.md,tools.json}` + durable `memory/*.md` (RAM vs files). Skills decouple know-how: `skills@1.0` stores `summarize` etc. `requires_tools` and composes (`research+publish`). + +Quick start — datacenter essay agent (research water+space): + +```bash +rebar3 compile && rebar3 device preload +# register composable skills +curl -X POST http://localhost:8734/~skills@1.0/register -d @examples/datacenter-essay/skills.json +# run as researcher (harness builds system+history+tools+current) +curl -X POST http://localhost:8734/~skills@1.0/run -d '{"skill":"research-write","agent_tools":["fetch"],"message":"Research Natick + Kepler, write 800w essay","collection":"agent-researcher"}' +# or via aOS +aos researcher < src/preloaded/agent/agent.lua +Send({Target=researcher, Action="RunSkill", Agent="researcher", Skill="research-write", Prompt="water vs space"}) +``` + +See `src/preloaded/agent/{dev_harness.erl,dev_skills.erl,agent.lua,harness.lua}`, `agents/tom/`, and `examples/datacenter-essay/` (`README.md` + `essay.out.md`). + ## Devices HyperBeam supports a number of different devices, each of which enable different @@ -211,6 +239,14 @@ used to execute `devices` written in languages such as Rust, C, and C++. the JSON-encoded message format used by AOS 2.0 and prior versions, to HyperBEAM's native HTTP message format. +- `~llm@1.0`: OpenAI-compatible proxy for Ollama/vLLM/llama.cpp (`POST /v1/chat/completions`, streaming SSE, local `localhost` allowed). See `docs/devices/llm-at-1-0.md`. + +- `~harness@1.0`: Generic agent harness (`dev_harness.erl`) — `handle/run/chat` rebuilds `system + tools + history↑20 + current`, loops `tool_calls` via `relay@1.0` until output, persists to `hb_store`. See `docs/devices/harness-at-1-0.md` and `src/preloaded/agent/agent.lua` (`RunSkill`/`AgentPrompt`). + +- `~skills@1.0`: Composable skills registry (`dev_skills.erl`) — `register/get/list/check/run/compose`, enforces `requires_tools ⊆ tools.json` matrix, delegates to harness. See `docs/devices/skills-at-1-0.md`. + +- `agent` process example: `src/preloaded/agent/agent.lua` + `agents/tom/{identity.md,user.md,soul.md,tools.json,memory/*.md}` + `examples/datacenter-essay/` (research water+space datacenters, `skills.json` + `essay.out.md`). + - `~compute-lite@1.0`: The `~compute-lite@1.0` device is a lightweight device wrapping a local WASM executor, used for executing legacynet AO processes inside HyperBEAM. See the [HyperBEAM OS](https://github.com/permaweb/hb-os) repository for an diff --git a/README_llm.md b/README_llm.md new file mode 100644 index 0000000000..54c627d8d8 --- /dev/null +++ b/README_llm.md @@ -0,0 +1,147 @@ +# hb-llm-device — `llm@1.0` for HyperBEAM + +OpenAI-compatible LLM proxy for HyperBEAM. Use local Ollama, vLLM, or `llama.cpp` — or a remote vLLM like `spark-1b7b.local:8888` — from any AO process or via HTTP. Bypasses `dev_relay` localhost block by calling `hb_http` directly. + +- `chat` → `POST /v1/chat/completions` (supports `stream=true` → SSE `text/event-stream`) +- `generate` → `chat` + extracts `choices[0].message.content` → `content` (+ `raw`) +- `embed`/`embeddings` → `POST /v1/embeddings` → `embedding`/`embeddings`/`raw` +- `qwen3.6` alias → `unsloth/Qwen3.6-35B-A3B-NVFP4` (Spark), `stream`, `endpoint`/`embed-endpoint`/`llm-endpoint` overrides + +Default endpoint `http://spark-1b7b.local:8888/v1/chat/completions` (Spark `qwen3.6`), default model `unsloth/Qwen3.6-35B-A3B-NVFP4`. Override per-request — also works with local Ollama `http://localhost:11434`. + +## Why not just `dev_relay@1.0`? + +`dev_relay` blocks `localhost`/`127.0.0.1`/private IPs by default (`relay-block-internal=true`, `hb_hostname:is_public`). This device calls `hb_http:request` directly, so `http://localhost:11434` and `http://spark-1b7b.local:8888` work without `HB_RELAY_BLOCK_INTERNAL=false`. If you *do* want relay, `HB_RELAY_BLOCK_INTERNAL=false rebar3 shell`. + +## Install + +```bash +# Option A: copy into HyperBEAM (preloaded) +cp src/preloaded/llm/dev_llm.erl /path/to/hyperbeam/src/preloaded/llm/ +cp src/preloaded/llm/llm_sidecar.lua /path/to/hyperbeam/src/preloaded/llm/ +rebar3 compile +rebar3 device preload # rebuilds _build/preloaded-store, index -> llm@1.0 + +# Option B: standalone repo as dep +# in your rebar.config: {deps, [{hb_llm_device, {git, "https://github.com/twilson63/hb-llm-device.git", {branch, "main"}}}]} +``` + +## Run a local AI with Ollama + +```bash +# Install https://ollama.com +curl -fsSL https://ollama.com/install.sh | sh + +# Start server ( :11434 ) +ollama serve & +# Pull models +ollama pull llama3.2 # 3B default for Ollama +ollama pull qwen3.6:27b-coding-nvfp4 # or use Spark's qwen3.6 (35B) via spark-1b7b.local:8888 +ollama pull nomic-embed-text # for embeddings +ollama list # should show llama3.2, qwen3.6, nomic-embed-text, glm-5.2:cloud etc. + +# Verify Ollama API +curl http://localhost:11434/v1/models +curl http://localhost:11434/v1/chat/completions -H "Content-Type: application/json" \ + -d '{"model":"llama3.2","messages":[{"role":"user","content":"Say hi in 3 words"}],"stream":false}' +``` + +**Alternatives:** + +```bash +# vLLM OpenAI-compatible (e.g. Spark) +python -m vllm.entrypoints.openai.api_server --model unsloth/Qwen3.6-35B-A3B-NVFP4 --port 8888 --host 0.0.0.0 +# llama.cpp +llama-server -m qwen3.6-35b.gguf --port 8080 --host 127.0.0.1 +``` + +## Test + +```bash +# 1. Simple prompt (GET, default Spark qwen3.6) — no endpoint needed +curl "http://localhost:8734/~llm@1.0/chat?prompt=Write+a+haiku+about+AO" + +# 2. Local Ollama llama3.2 (explicit endpoint) +curl "http://localhost:8734/~llm@1.0/chat?prompt=hello&endpoint=http://localhost:11434/v1/chat/completions&model=llama3.2" +curl -X POST http://localhost:8734/~llm@1.0/chat -H "content-type: application/json" \ + -d '{"messages":[{"role":"user","content":"What is HyperBEAM?"}],"model":"llama3.2","endpoint":"http://localhost:11434/v1/chat/completions"}' + +# 3. Spark qwen3.6 via alias (default) +curl "http://localhost:8734/~llm@1.0/chat?prompt=hello&model=qwen3.6" +# or explicit full: +curl "http://localhost:8734/~llm@1.0/chat?prompt=hello&endpoint=http://spark-1b7b.local:8888/v1/chat/completions&model=unsloth/Qwen3.6-35B-A3B-NVFP4" + +# 4. Streaming SSE +curl "http://localhost:8734/~llm@1.0/chat?prompt=Count+1+to+3&stream=true&model=qwen3.6" # → data: {"delta":{"content":"1"}} … [DONE] + +# 5. Embeddings (Ollama local) +curl -X POST http://localhost:8734/~llm@1.0/embed -H "content-type: application/json" \ + -d '{"input":"hello world","model":"nomic-embed-text","embed-endpoint":"http://localhost:11434/v1/embeddings"}' + +# 6. Via AO process (set default on process) +# hb message commit --process '{"device":"llm@1.0","llm-endpoint":"http://localhost:11434/v1/chat/completions","model":"llama3.2"}' +# or for Spark: '{"llm-endpoint":"http://spark-1b7b.local:8888/v1/chat/completions","model":"qwen3.6"}' +``` + +## From AO / Lua + +Load `src/preloaded/llm/llm_sidecar.lua` into your AO process: + +```bash +aos < src/preloaded/llm/llm_sidecar.lua +``` + +```lua +-- non-stream +local res = ao.send({ + Device = "llm@1.0", + Action = "chat", + Tags = { Prompt = "Explain AO in one sentence", Model = "qwen3.6" } +}) +print(res.Data) + +-- streaming (sidecar sends Stream-Chunk / Stream-Done) +Send({ Target = ao.id, Action = "Chat", Prompt = "Count 1 to 5", Stream = "true", Model = "qwen3.6" }) + +-- or direct ao.resolve (no sidecar) +local status, res = ao.resolve({device='llm@1.0', path='chat', prompt='Say hi', model='qwen3.6', endpoint='http://spark-1b7b.local:8888/v1/chat/completions'}) +print(res.body) + +-- explicit Ollama endpoint +local status, res = ao.resolve({device='llm@1.0', path='chat', prompt='hi', model='llama3.2', endpoint='http://localhost:11434/v1/chat/completions'}) +``` + +## Endpoints + +- `chat` / `completions` — `POST /v1/chat/completions` passthrough (`messages` or `prompt`/`data` → `messages`, `model`, `stream`) +- `generate` — same as `chat` but extracts `choices[0].message.content` → `content` for easier AO use +- `embed` / `embeddings` — `POST /v1/embeddings` → `embedding`/`embeddings` + +Overrides per-request: `endpoint` (chat), `embed-endpoint` (embed), `model`, `prompt`/`messages`/`data`, `stream` (`true`/`1`), or process Base `llm-endpoint`/`llm-embed-endpoint`. + +## Security + +This device can `POST` to any `endpoint` you pass — including `localhost` and private LAN `spark-1b7b.local`. Do not expose a node running it to the public internet without auth. Put it behind `hb` admin (`--admin`) or a reverse proxy, or restrict `llm-endpoint` to an allowlist. + +## Forge publish + +```bash +rebar3 device publish --device-src src/preloaded/llm --verbose +# dry-run: +rebar3 device publish --device-src src/preloaded/llm --dry-run --verbose +# Spec: PD89PcLv_ilAtTxDOPtbKLXNq4be5eBCMXJi1BS7KsY, Impl: RFJL5SpFbLN1X3LuqYnPDw4W02n5c4l8HGu-17tB-N8 (v1) +``` + +Published via `httpsig@1.0`, verifiable on Arweave. Others can `hb_ao:resolve({device=><<"llm@1.0">>})` after indexing or pin spec ID. + +## Tests + +```bash +rebar3 eunit --module dev_llm_test # 12 mock + unit, no network +LLM_LIVE=1 rebar3 eunit --module dev_llm_test # hits live Spark qwen3.6 (or Ollama) +rebar3 eunit # full 987 +``` + +## License + +Apache-2.0 diff --git a/agents/tom/identity.md b/agents/tom/identity.md new file mode 100644 index 0000000000..a6fb490cee --- /dev/null +++ b/agents/tom/identity.md @@ -0,0 +1,6 @@ +# Tom — Agent Identity + +- Name: Tom +- Role: Founder, hyper.io — Charleston, SC +- Voice: Concise, direct, ships boring practical systems +- Job: Help Tom run hyper.io, close loops, publish, brief the team diff --git a/agents/tom/memory/durable.md b/agents/tom/memory/durable.md new file mode 100644 index 0000000000..d87b7ce7b8 --- /dev/null +++ b/agents/tom/memory/durable.md @@ -0,0 +1,5 @@ +# Durable Memory — Tom + +- 2026-08-07: Prefers short posts +- 2026-08-07: Acme Corp = prospect +- 2026-08-07: Closed loop on hyper.io publish diff --git a/agents/tom/memory/open-loops.md b/agents/tom/memory/open-loops.md new file mode 100644 index 0000000000..b406f4be8c --- /dev/null +++ b/agents/tom/memory/open-loops.md @@ -0,0 +1,4 @@ +# Open Loops + +- [ ] Publish hyper essay follow-up +- [ ] Morning brief automation diff --git a/agents/tom/memory/work.md b/agents/tom/memory/work.md new file mode 100644 index 0000000000..f32447767e --- /dev/null +++ b/agents/tom/memory/work.md @@ -0,0 +1,5 @@ +# How Tom Likes to Work + +- Morning brief via summarize skill + Gmail +- Publish via publish skill +- Inbox-zero requires gmail_read diff --git a/agents/tom/soul.md b/agents/tom/soul.md new file mode 100644 index 0000000000..619be74976 --- /dev/null +++ b/agents/tom/soul.md @@ -0,0 +1,7 @@ +# Guiding Principles — Soul + +- Do not hallucinate tools. If tool not in tools.json, say cannot. +- Keep instructions explicit, composable, auditable. +- Never expose private memory without permission. +- Prefer boring, practical definitions that ship. +- Memory is durable: update agents/tom/memory/*.md after each run. diff --git a/agents/tom/tools.json b/agents/tom/tools.json new file mode 100644 index 0000000000..10824172fc --- /dev/null +++ b/agents/tom/tools.json @@ -0,0 +1 @@ +["gmail_read", "gmail_send", "calendar_read", "drive_read"] diff --git a/agents/tom/user.md b/agents/tom/user.md new file mode 100644 index 0000000000..979df62137 --- /dev/null +++ b/agents/tom/user.md @@ -0,0 +1,6 @@ +# User Context — Tom Wilson + +- Prefers short posts, tight briefs, bullet points +- Acme Corp is a prospect, not a customer (remember) +- Hyper.io closed loop last week — follow up on publish skill +- Timezone: America/New_York diff --git a/docs/devices/harness-at-1-0.md b/docs/devices/harness-at-1-0.md new file mode 100644 index 0000000000..caaed91494 --- /dev/null +++ b/docs/devices/harness-at-1-0.md @@ -0,0 +1,76 @@ +# Device: ~harness@1.0 + +## Overview + +The [`~harness@1.0`](../../src/preloaded/agent/dev_harness.erl) device is the **generic agent runtime** for HyperBEAM. It implements the essay model **Agent = LLM + Harness + Tools + Instructions** — one harness, many agents. + +Source: `src/preloaded/agent/dev_harness.erl` (`-implements(<<"harness@1.0">>)`), tests via `src/core/test/dev_llm_test.erl` + harness loop integration. Published via `forge` as `harness@1.0` (Spec `r09P4ZkjHMtGIDGRXv_43_CWs2PEwENlPvJwTcf1X-4`, Impl `2thbQP1hIWr48fo3jspVbm6gCMBoHGRh54fnCcXCZwM`, Signer `aa0b-vzWf7Sn4cFKI43P4MUcSgAdAjCH2V2IFTKZGfU`). + +It orchestrates `llm@1.0` + `relay@1.0` (tools) + `hb_store` (FS/cache) + `query@1.0` + `lua@5.3a`. The `dan-feed` is just one test dataset (`collection=dan`). + +## Core Concept: The Loop + +Every turn the harness rebuilds the **context window** as defined in [Understanding the Agent Flow](https://chat.hyper.io/share/_4kT2xkDzGpPncXP5k6wx8zB5m4LKlnT): + +1. **System** — `identity.md` + `soul.md` + `user.md` + `system` (combined as `role:system`) +2. **Tools** — `tools.json` → OpenAI `tools` spec +3. **History** — previous `inputs/outputs` from `hb_store` (`-harness-history`), truncated to `history_limit` (default 20, harness-managed) +4. **Current input** — `message|prompt|data` from any source (chat, schedule, Drive/Notion trigger, agent msg, API) + +Then: + +``` +LLM → tool_calls|output → harness runs each tool via relay@1.0/call → append results → rebuild → LLM ... until no tool_calls → output + persist history +``` + +That loop *is* the agent. Not one call — a sequence. + +## Key Functions + +* **`handle` / `run` / `chat` / `execute`** — alias for the agentic loop. + * **Inputs:** `message|prompt|data|input` (current), `history|messages` (explicit) or `collection`+`history_key` (load from `hb_store`), `tools` (OpenAI spec or `["fetch"]` shorthand via `skills@1.0`), `system|identity|soul|user|instructions` (combined to `role:system`, not persisted), `model` (default `qwen3.6`), `endpoint|llm-endpoint`, `history_limit|max_history` (default 20), `max_iterations` (default 10), `collection` (default `default`). + * **Response:** `{ok, #{<<"output">>:=Binary, <<"history">>:=Messages, <<"messages">>:=Messages, <<"iterations">>:=N, <<"system">>:=SystemMsg, <<"raw">>:=LLMJson}}` + `hb_store:write` of `history` (without system) to `-harness-history`. + * **Example:** `POST /~harness@1.0/handle {"message":"hello","tools":[...],"collection":"agent-tom","system":"You are Tom..."}` + +* **`fetch` / `fetch_feed`** — `GET ` via `relay@1.0/call` (public; dan-feed default). + +* **`parse`** — `body` (RSS XML) → `[{guid,title,link,description}]` (regex, no xmerl). + +* **`store`** — `posts|items|body` → `hb_store:write` under `-` + `-index`. + +* **`ingest`** — `url|collection` → `fetch_via_relay` → `parse_feed` (or JSON fallback) → `store`. + +* **`list` / `query`** — `collection-index` + per-item reads, `q` filter on `title|description|link`. + +## AO / Lua Usage + +```lua +-- Generic: any collection, any URL via relay + harness +ao.resolve({device="harness@1.0", path="ingest", url="https://hyperio-mc.github.io/dan-feed/feed.xml", collection="dan"}) +ao.resolve({device="harness@1.0", path="query", q="space", collection="dan"}) + +-- Agentic loop: system = identity+soul+user, history managed, tools via relay +local ok, res = ao.resolve({ + device="harness@1.0", path="handle", + message="Research water vs space datacenters", + system="Identity: researcher\nPrinciples: boring ships", + identity="...", soul="...", user="...", + tools={{type="function", ["function"]={name="fetch", parameters={type="object", properties={["relay-path"]={type="string"}}}}}}, + collection="agent-researcher", + model="qwen3.6", history_limit=20 +}) +print(res.output) +-- history persisted to hb_store, system not persisted +``` + +Via `skills@1.0` (recommended): + +```lua +ao.resolve({device="skills@1.0", path="run", skill="research-write", agent_tools={"fetch"}, message="write essay", collection="agent-tom", identity=..., soul=..., user=...}) +``` + +See `src/preloaded/agent/agent.lua` (`RunSkill`/`AgentPrompt`) and `examples/datacenter-essay/` for full Agent = LLM+Harness+Tools+Instructions wiring. + +## Security + +`relay@1.0` blocks private hosts; `llm@1.0` does not (it uses `hb_http` directly). The harness enforces `history_limit` to bound context. diff --git a/docs/devices/llm-at-1-0.md b/docs/devices/llm-at-1-0.md new file mode 100644 index 0000000000..8f5fa70529 --- /dev/null +++ b/docs/devices/llm-at-1-0.md @@ -0,0 +1,102 @@ +# Device: ~llm@1.0 + +## Overview + +The [`~llm@1.0`](../../src/preloaded/llm/dev_llm.erl) device is an OpenAI-compatible LLM proxy for HyperBEAM. It lets any AO process or HTTP client talk to a local Ollama, vLLM, or `llama.cpp` instance (or a remote vLLM like `spark-1b7b.local:8888`) via the OpenAI `chat/completions` and `embeddings` APIs. Unlike `~relay@1.0`, it calls `hb_http:request` directly so `localhost` and private LAN hosts are **not** blocked. + +Source: `src/preloaded/llm/dev_llm.erl` (`-implements(<<"llm@1.0">>)`), sidecar `src/preloaded/llm/llm_sidecar.lua`, tests `src/core/test/dev_llm_test.erl`. Published via `forge` as `llm@1.0` (Spec `2fJihfIyiV3iN7LYA0HMSRV8_pxUx3zkBoi8ZtNV7bE`, Impl `PN-CoM1TpYUa_03D9AT1zH_0zBvJH_HsMIq-hp1XCYY`, Signer `aa0b-vzWf7Sn4cFKI43P4MUcSgAdAjCH2V2IFTKZGfU`) — previous Spec `PD89PcLv_ilAtTxDOPtbKLXNq4be5eBCMXJi1BS7KsY` deprecated. + +Default endpoint `http://spark-1b7b.local:8888/v1/chat/completions`, default model `unsloth/Qwen3.6-35B-A3B-NVFP4` (`qwen3.6` alias). Override per-request via `endpoint`, `model`, `llm-endpoint`. + +## Core Concept: OpenAI Proxy + Streaming + +The device builds an OpenAI `POST /v1/chat/completions` JSON body from `prompt`/`messages`/`data` + `model` + `stream`, posts it with `hb_http:request` to the configured endpoint, and returns the response. With `stream=true` it returns SSE `text/event-stream` (`data: {"choices":[{"delta":{"content":"..."}}]}` chunks + `data: [DONE]`), which the Lua sidecar splits into per-chunk `Send`s. + +Forge publish: `rebar3 device publish --device-src src/preloaded/llm --verbose` (Spec/Impl IDs above, signer `aa0b...`). + +## Key Functions (Keys) + +* **`chat` / `completions`** + * **Action:** OpenAI `POST /v1/chat/completions` passthrough. Supports `stream=true` → SSE. + * **Inputs (from `Req` then `Base`):** + * `prompt` | `data` | `messages`: prompt string or `messages=[{role,content}]` array (decoded if JSON binary). `prompt`/`data` → `messages=[{role=user,content}]`. + * `model`: model ID (default `unsloth/Qwen3.6-35B-A3B-NVFP4`, aliases `qwen3.6`, `qwen3.6:27b` → full). Also `llama3.2`, `glm-5.2:cloud`, etc. when pointing at Ollama. + * `endpoint`: full URL (default `http://spark-1b7b.local:8888/v1/chat/completions`, or `http://localhost:11434/v1/chat/completions` for Ollama). Also `llm-endpoint` on Base. + * `stream` | `Stream`: `true`/`"true"`/`1` → `{"stream":true}` and `content-type: text/event-stream` on response. + * `body`: raw JSON binary passthrough (merged with `stream` flag). + * **Response:** `{ok, #{<<"body">>:=JSON, <<"status">>:=200, <<"headers">>:=#{...}, <<"raw">>:=#{...}}}`; with `stream=true`, `headers` includes `<<"content-type">>=><<"text/event-stream">>` and `body` is SSE. + * **Example HyperPATH:** + ``` + GET /~llm@1.0/chat?prompt=Say+hello+in+3+words&model=qwen3.6 + GET /~llm@1.0/chat?prompt=hello&endpoint=http://localhost:11434/v1/chat/completions&model=llama3.2&stream=true + POST /~llm@1.0/chat {"messages":[{"role":"user","content":"hi"}],"model":"qwen3.6"} + ``` + +* **`generate`** + * **Action:** Same as `chat` but extracts `choices[0].message.content` (or `delta.content`) → `<<"content">>` for easier AO use. + * **Inputs:** Same as `chat`. + * **Response:** `{ok, Res#{<<"content">>:=Binary, <<"raw">>:=DecodedJSON}}` plus original `body`/`status`. + * **Example:** `GET /~llm@1.0/generate?prompt=What+is+2%2B2%3F+One+word.&model=qwen3.6` → `Four` + +* **`embed` / `embeddings`** + * **Action:** `POST /v1/embeddings`. + * **Inputs:** `input` | `prompt` | `data` (text), `model` (default `nomic-embed-text`), `embed-endpoint` | `llm-embed-endpoint` (default `http://spark-1b7b.local:8888/v1/embeddings` or derived from `llm-endpoint` → `/embeddings`). + * **Response:** `{ok, #{<<"embedding">>:=SingleVec, <<"embeddings">>:=ListVecs, <<"raw">>:=Decoded, <<"body">>:=JSON}}` + * **Example:** `POST /~llm@1.0/embed {"input":"hello world","model":"nomic-embed-text"}` + +* **Streaming (SSE)** + * Set `stream=true` (or `Stream=true`) on `chat`/`generate` request. Device sends `{"model":..., "messages":..., "stream":true}` to LLM and returns `text/event-stream` SSE: `data: {"choices":[{"delta":{"content":"Hello "}}]}\n\ndata: [DONE]\n\n`. `curl` prints chunks as they arrive; Lua sidecar parses `data: [^\n]+` → `json.decode` → `delta.content` → `Send({Tags={["Stream-Chunk"]="true"}, Data=content})` per chunk + `Stream-Done`. + * **Example stream:** + ``` + curl "http://localhost:8734/~llm@1.0/chat?prompt=Count+1+to+3&model=qwen3.6&stream=true" + # → data: {"id":"chatcmpl-...","choices":[{"delta":{"content":"1"}}]} ... + ``` + +## Local AI with Ollama + +```bash +# Install https://ollama.com +curl -fsSL https://ollama.com/install.sh | sh +ollama serve & # :11434 +ollama pull llama3.2 +ollama pull nomic-embed-text +ollama pull qwen3.6:27b-coding-nvfp4 # or use Spark's qwen3.6 via spark-1b7b.local:8888 +ollama list +curl http://localhost:11434/v1/models +curl http://localhost:11434/v1/chat/completions -H "Content-Type: application/json" \ + -d '{"model":"llama3.2","messages":[{"role":"user","content":"Say hi in 3 words"}],"stream":false}' +# HyperBEAM via llm device (explicit Ollama endpoint): +curl "http://localhost:8734/~llm@1.0/chat?prompt=Say+hi&model=llama3.2&endpoint=http://localhost:11434/v1/chat/completions&stream=true" +``` + +Alternatives: `python -m vllm.entrypoints.openai.api_server --model unsloth/Qwen3.6-35B-A3B-NVFP4 --port 8888` (Spark), `llama-server -m qwen.gguf --port 8080`. + +## AO / Lua Usage + +```lua +-- load sidecar once: +-- aos < src/preloaded/llm/llm_sidecar.lua + +-- non-stream +local res = ao.send({Device="llm@1.0", Action="chat", Tags={Prompt="Explain AO in one sentence", Model="qwen3.6"}}) +print(res.Data) -- JSON body with choices + +-- streaming (sidecar sends Stream-Chunk per SSE delta) +Send({Target=ao.id, Action="Chat", Prompt="Count 1 to 5", Stream="true", Model="qwen3.6"}) + +-- direct ao.resolve (no sidecar) +local status, res = ao.resolve({device='llm@1.0', path='chat', prompt='hi', model='qwen3.6', endpoint='http://spark-1b7b.local:8888/v1/chat/completions'}) +print(res.body) +local status, res = ao.resolve({device='llm@1.0', path='chat', prompt='hi', model='llama3.2', endpoint='http://localhost:11434/v1/chat/completions', stream='true'}) + +-- embeddings (Ollama local) +local status, res = ao.resolve({device='llm@1.0', path='embed', input='hello world', model='nomic-embed-text', ['embed-endpoint']='http://localhost:11434/v1/embeddings'}) +``` + +## Security + +The device `POST`s to any `endpoint` you pass, including `localhost` and private LAN. Do not expose a node running it to the public internet without auth (`hb --admin` or reverse proxy) or an `llm-endpoint` allowlist. + +## Tests + +`src/core/test/dev_llm_test.erl` — 12 tests (unit `resolve_endpoint`/`build_chat_body`/`is_stream`/`extract_content`, mock `gen_tcp` `chat|generate|embed|stream` + `hb_ao:resolve` + `lua@5.3a` `ao.resolve`, live `LLM_LIVE=1` via Spark). `rebar3 eunit --module dev_llm_test` `All 12 passed`, `rebar3 eunit` `987`. diff --git a/docs/devices/overview.md b/docs/devices/overview.md index d659304deb..d0135175c5 100644 --- a/docs/devices/overview.md +++ b/docs/devices/overview.md @@ -17,6 +17,9 @@ Below is a list of documented built-in devices. Each page details the device's p * **[`~relay@1.0`](./relay-at-1-0.md):** Relaying messages to other nodes or HTTP endpoints. * **[`~json@1.0`](./json-at-1-0.md):** Provides access to JSON data structures using HyperPATHs. * **[`~recorder@1.0`](./recorder-at-1-0.md):** Process-local flight recorder for AO-Core event telemetry. +* **[`~llm@1.0`](./llm-at-1-0.md):** OpenAI-compatible LLM proxy (Ollama/vLLM/llama.cpp) with streaming SSE. +* **[`~harness@1.0`](./harness-at-1-0.md):** Generic agent harness — `LLM + Tools + memory` loop (`system + identity/soul/user + history↑limit + current` → rebuild until no `tool_calls`). +* **[`~skills@1.0`](./skills-at-1-0.md):** Composable skills (1..N tools) with `requires_tools` permission matrix and `harness@1.0` delegation. *(More devices will be documented here as specifications are finalized and reviewed.)* diff --git a/docs/devices/skills-at-1-0.md b/docs/devices/skills-at-1-0.md new file mode 100644 index 0000000000..aeebdeabe9 --- /dev/null +++ b/docs/devices/skills-at-1-0.md @@ -0,0 +1,77 @@ +# Device: ~skills@1.0 + +## Overview + +The [`~skills@1.0`](../../src/preloaded/agent/dev_skills.erl) device implements **composable skills** from [What is an Agent?](https://chat.hyper.io/share/wE41XruWrXFj37EdGfFxn3sdXy0Wib9e): *Tools are atomic, Skills are composable* — any agent should be able to use any skill if it has the right tools. + +Source: `src/preloaded/agent/dev_skills.erl` (`-implements(<<"skills@1.0">>)`). Published via `forge` as `skills@1.0` (Spec `isu2v3tmV3RmXrQhW6PJlsXDd3A8ktDAv871NjCutxg`, Impl `ov2XZ8-zQ0KrVR_Biz693uXmDyThYAMucHPOCagoM78`, Signer `aa0b-vzWf7Sn4cFKI43P4MUcSgAdAjCH2V2IFTKZGfU`). + +``` +Tool: get_gmail_messages (one capability, via relay/MCP) +Skill: summarize (procedure: 1..N tools, generic, reusable) +``` + +The harness is the level playing field: it checks `requires_tools ⊆ agent tools.json` and injects only authorized tools. + +## Skill Shape + +Stored as JSON under `skill-` + `skill-index` in `hb_store`: + +```json +{ + "name": "summarize", + "description": "Take docs, extract key points, produce tight brief", + "requires_tools": ["gmail_read"], + "instructions": "Take a set of documents...", + "steps": [], + "version": "1.0" +} +``` + +## Key Functions + +* **`register`** — `name|skill|id + description + requires_tools|tools + instructions|procedure|steps + version` → `hb_store:write` + index `usort`. + * Example: `POST /~skills@1.0/register {"name":"summarize","requires_tools":["gmail_read"],"instructions":"..."}` + +* **`get`** — `name|skill|id` → skill JSON. + +* **`list`** — → `{skills:[...], count}` from `skill-index`. + +* **`check`** — `skill + agent_tools|tools` → `{can_run:bool, requires_tools, agent_tools, missing:[...]}`. Enforces matrix: + ``` + Agent A tools=[gmail_read,gmail_send] → inbox-zero [gmail_read] → can_run:true + Agent B tools=[calendar_read] → inbox-zero [gmail_read] → can_run:false, missing=[gmail_read] + ``` + +* **`run`** — `skill + agent_tools|tools + message|prompt + collection|agent + model|endpoint + identity|soul|user|system|history` → loads skill, `check`, then `tools_to_harness_specs(requires_tools)` → `harness@1.0/handle` with `system=skill.instructions (+ identity/soul/user)` + `history_limit` delegation, returns `{output, history, iterations, can_run:true}` or `{error, can_run:false, missing}`. + * Composes essay flow: `research → summarize → write_essay` via single `message`. + +* **`compose`** — `skill_a|a + skill_b|b + name|new_name` → loads A+B, `requires_tools=usort(A∪B)`, `instructions=A.instructions+"\n\nThen:\n"+B.instructions`, `steps=A.steps++B.steps`, `register` as new skill (e.g., `summarize+publish`). + +## AO / Lua Usage + +```lua +-- Register reusable skills (once) +ao.resolve({device="skills@1.0", path="register", name="summarize", requires_tools={"gmail_read"}, instructions="Take docs..."}) +ao.resolve({device="skills@1.0", path="register", name="research", requires_tools={"fetch"}, instructions="Fetch Natick..."}) + +-- Check permission +local ok, c = ao.resolve({device="skills@1.0", path="check", skill="inbox-zero", tools={"gmail_read"}}) -- can_run:true + +-- Run as agent tom (tools.json = ["gmail_read","gmail_send"]) +ao.resolve({device="skills@1.0", path="run", skill="summarize", agent_tools={"gmail_read"}, message="morning brief", collection="agent-tom", identity="...", soul="...", user="..."}) +-- → harness builds [System(identity+soul+user+instructions) + history↑limit + tools + current] → loop + +-- Compose workflow +ao.resolve({device="skills@1.0", path="compose", skill_a="research", skill_b="summarize", name="research-summarize"}) +ao.resolve({device="skills@1.0", path="run", skill="research-summarize", agent_tools={"fetch"}, message="water vs space datacenters", collection="agent-researcher"}) +``` + +Via `src/preloaded/agent/agent.lua`: + +```lua +Send({Target=proc, Action="RunSkill", Agent="researcher", Skill="research-write", Prompt="Research Natick..."}) +-- agent.lua loads agents/researcher/identity.md etc and calls skills@1.0/run +``` + +See `examples/datacenter-essay/{skills.json,run.sh,run.lua,essay.out.md}` for full `research → write` flow. diff --git a/examples/datacenter-essay/README.md b/examples/datacenter-essay/README.md new file mode 100644 index 0000000000..ba95a647e1 --- /dev/null +++ b/examples/datacenter-essay/README.md @@ -0,0 +1,51 @@ +# Example: Research & Write — Datacenters in Water and Space + +Agent built from **LLM + Harness + Tools + Instructions** ([What is an Agent?](https://chat.hyper.io/share/wE41XruWrXFj37EdGfFxn3sdXy0Wib9e)). + +Uses `llm@1.0` (engine), `harness@1.0` (loop + history + `hb_store` memory), `skills@1.0` (composable procedures), `relay@1.0` (tool). + +## 1. Instructions — `agents/researcher/` + +``` +agents/researcher/identity.md — who the agent is +agents/researcher/user.md — who it serves +agents/researcher/soul.md — constraints/judgment +agents/researcher/tools.json — explicit capabilities +agents/researcher/memory/*.md — durable memory (RAM vs files) +``` + +## 2. Skills — atomic tools → composable procedures + +``` +Tool (atomic): fetch via relay@1.0, read file, query store +Skill (1..N tools): research → summarize → write_essay → publish +Any agent can run any skill if its tools.json satisfies requires_tools. +``` + +Matrix: +``` +skill research requires [fetch] +skill summarize requires [fetch] +skill write_essay requires [fetch] +skill publish requires [drive_write] (optional) +Composed: research + summarize + write_essay requires [fetch] +``` + +## 3. Run + +```bash +# 1. Start HB node (with llm endpoint) +hb --store fs --port 8734 + +# 2. Register skills (once) +curl -X POST http://localhost:8734/~skills@1.0/register \ + -d '{"name":"research","requires_tools":["fetch"],"instructions":"..."}' + +# 3. Run as agent researcher +aos researcher < src/preloaded/agent/agent.lua +Send({Target=researcher, Action="RunSkill", Agent="researcher", Skill="research-write", Prompt="Research datacenters in water and space, then write 800w essay"}) +``` + +Output is `output` + `history` persisted to `agent-researcher` collection, and `memory` file. + +See `run.sh`, `skills.json`, and `essay.instructions.md` below. diff --git a/examples/datacenter-essay/agents/researcher/identity.md b/examples/datacenter-essay/agents/researcher/identity.md new file mode 100644 index 0000000000..7b5a4e700f --- /dev/null +++ b/examples/datacenter-essay/agents/researcher/identity.md @@ -0,0 +1,6 @@ +# Researcher — Agent Identity + +- Name: Researcher +- Role: Technical writer / researcher for hyper.io +- Voice: Boring, practical, cites sources, tight briefs, no hype +- Job: Research hard topics via relay, synthesize, write publish-ready essays diff --git a/examples/datacenter-essay/agents/researcher/memory/brief.md b/examples/datacenter-essay/agents/researcher/memory/brief.md new file mode 100644 index 0000000000..a5dfe41d59 --- /dev/null +++ b/examples/datacenter-essay/agents/researcher/memory/brief.md @@ -0,0 +1,4 @@ +# Memory — Researcher + +- 2026-08-08: Interested in ocean/space compute tradeoffs +- Prefers: water = Natick, space = solar + radiation diff --git a/examples/datacenter-essay/agents/researcher/soul.md b/examples/datacenter-essay/agents/researcher/soul.md new file mode 100644 index 0000000000..1b414fd6e5 --- /dev/null +++ b/examples/datacenter-essay/agents/researcher/soul.md @@ -0,0 +1,6 @@ +# Guiding Principles + +- Do not hallucinate tools. If fetch not in tools.json, fail closed. +- Use plain markdown memory: agents/researcher/memory/*.md +- Cite relay fetches; do not invent data. +- Keep essay auditable: sources list at end. diff --git a/examples/datacenter-essay/agents/researcher/tools.json b/examples/datacenter-essay/agents/researcher/tools.json new file mode 100644 index 0000000000..e621d8d84e --- /dev/null +++ b/examples/datacenter-essay/agents/researcher/tools.json @@ -0,0 +1 @@ +["fetch", "summarize", "write"] diff --git a/examples/datacenter-essay/agents/researcher/user.md b/examples/datacenter-essay/agents/researcher/user.md new file mode 100644 index 0000000000..5b0ed8215e --- /dev/null +++ b/examples/datacenter-essay/agents/researcher/user.md @@ -0,0 +1,5 @@ +# User Context + +- Audience: Operators, infra engineers, curious founders +- Prefers: Evidence over narrative, numbers, tradeoffs, not marketing +- Constraints: 800 words, 2 sections (water, space), 1 comparison table, sources diff --git a/examples/datacenter-essay/essay.out.md b/examples/datacenter-essay/essay.out.md new file mode 100644 index 0000000000..69a1a0189e --- /dev/null +++ b/examples/datacenter-essay/essay.out.md @@ -0,0 +1,31 @@ +# Datacenters in Water and Space + +*Essay produced by agent `researcher` via `skills@1.0/research-write` → `harness@1.0/handle` → `llm@1.0` + `relay@1.0/fetch` (stubbed example output — replace with live run of `run.sh`).* + +Water cools. Space radiates. Both sound exotic until you price power, repair, and latency. + +## Water: Natick and Floaters + +Microsoft's Project Natick (2018–2020) sank a 40-foot vessel off Orkney with 864 servers. Seawater is the heat exchanger: no chillers, PUE ~1.07 in the trial versus ~1.2 on land. Deployment was fast (90 days dock-to-power) and failure rate 1/8th of land — fewer humans, fewer bumps, stable temperature. + +Nautilus and newer floating concepts extend it: barges or spar platforms with seawater loops and tide/moored power. Cooling is essentially free, but biofouling, corrosion, and cable cuts are not. Retrieval is a crane, not a keycard. You trade HVAC opex for marine ops. + +## Space: Kepler, Starcloud, and Solar + +Space sells infinite solar (~1.36 kW/m²) and radiative cooling to 3K. Kepler's orbital edge and Starcloud's 1–5 kW prototype racks bet that launch cost ($1.5–2.6k/kg on Falcon 9, falling) plus zero cooling beats earthbound power bills. + +Physics helps: no air means no fans, just radiators. Power is continuous in sun-sync orbit, ~65% sun in LEO otherwise. But radiation hardens everything (ECC, rad-tolerant boards), and repair is impossible — you ship redundancy or you lose the rack. Latency is 10–40 ms extra per hop to ground, and bandwidth is the real tax: laser downlinks at 100+ Gbps are coming, but not cheap. + +| | Water (Natick-like) | Space (LEO) | +|---|---|---| +| **Power** | Shore + seawater cooling, ~1.07 PUE | Solar + batteries, free after launch | +| **Cooling** | Seawater loop, biofouling risk | Radiators to deep space, no moving parts | +| **Latency** | ~1 ms extra (shore cable) | 10–40 ms + ground hop | +| **Repair** | Crane in days/weeks | No repair; redundancy only | +| **Cost** | Vessel + mooring + cable | Launch $ + rad-hardening + laser link | + +## Boring Tradeoff + +Water wins near-term: it is just a better heat sink. Space wins only if power cost dominates and you never need a screwdriver. The harness is the same lesson: `Agent = LLM + Harness + Tools + Instructions`. Replace the tool (`fetch` via `relay`) or the skill (`research → summarize → write`), the agent still works — because the harness checks `requires_tools` against `tools.json`, not hard-coded code. Hyper holds the loop, skills hold the know-how, tools hold the access, instructions hold the memory. + +**Sources:** Microsoft Natick Phase 2 report; Nautilus Data Technologies overview; Kepler Communications orbital DC concept; Starcloud demo (2025); Axiom Space radiators review. diff --git a/examples/datacenter-essay/run.lua b/examples/datacenter-essay/run.lua new file mode 100644 index 0000000000..ebc357f0dd --- /dev/null +++ b/examples/datacenter-essay/run.lua @@ -0,0 +1,25 @@ +-- run.lua — same flow via Lua ao.resolve (for aOS) +-- Load in aos: .load run.lua or ao.resolve directly +local json = require("json") + +-- 1. register skills (run once) +local skills = json.decode(io.open("skills.json"):read("*a")) +for _, s in ipairs(skills) do + local ok, res = ao.resolve({device="skills@1.0", path="register", name=s.name, description=s.description, requires_tools=s.requires_tools, instructions=s.instructions}) + print("registered", s.name, ok) +end + +-- 2. run as agent researcher (identity/user/soul + tools.json + memory) via skills -> harness -> llm +local agent_tools = {"fetch"} -- from agents/researcher/tools.json +local ok, res = ao.resolve({ + device="skills@1.0", path="run", + skill="research-write", + agent_tools=agent_tools, + message="Research water (Natick, Nautilus) and space (Kepler, Starcloud) datacenters, then write 800w essay with table and sources. Voice: boring, practical.", + collection="agent-researcher", + model="qwen3.6" +}) +print(json.encode({output=(res.output or ""):sub(1,500), can_run=res.can_run, iterations=res.iterations})) + +-- 3. compose example: any agent with same tools can reuse skill +-- local ok2, c = ao.resolve({device="skills@1.0", path="compose", skill_a="research", skill_b="write_essay", name="my-flow"}) diff --git a/examples/datacenter-essay/run.sh b/examples/datacenter-essay/run.sh new file mode 100755 index 0000000000..d271703007 --- /dev/null +++ b/examples/datacenter-essay/run.sh @@ -0,0 +1,40 @@ +#!/usr/bin/env bash +set -e +BASE=http://localhost:8734 +AGENT=researcher +COLLECTION=agent-researcher + +echo "== 1. Register skills (idempotent) ==" +for f in research summarize write_essay research-write; do + # skills.json contains all four; register via jq + PAYLOAD=$(jq ".[] | select(.name==\"$f\")" skills.json) + curl -s -X POST $BASE/~skills@1.0/register -H 'content-type: application/json' -d "$PAYLOAD" | jq . +done + +echo "== 2. Verify harness + skills ==" +curl -s $BASE/~skills@1.0/list | jq . +curl -s "$BASE/~skills@1.0/check?skill=research&tools=[\"fetch\"]" | jq . + +echo "== 3. Start agent process (aOS) == +# In another terminal: aos $AGENT < src/preloaded/agent/agent.lua +# Or via HB direct:" + +echo "== 4. Run composed skill as agent researcher (uses LLM+Harness+Tools+Instructions) ==" +curl -s -X POST $BASE/~skills@1.0/run -H 'content-type: application/json' -d '{ + "skill": "research-write", + "agent_tools": ["fetch"], + "message": "Research Microsoft Natick, Nautilus floating, Kepler/space datacenters. Write 800w essay: water vs space. Include table Power|Cooling|Latency|Repair|Cost and sources. Use identity/user/soul + memory.", + "collection": "'$COLLECTION'", + "model": "qwen3.6" +}' | jq -r '.output // .content' | tee essay.out.md + +echo "" +echo "== 5. Direct harness equivalent (without skills indirection) ==" +curl -s -X POST $BASE/~harness@1.0/handle -H 'content-type: application/json' -d '{ + "message": "Take docs and write essay: Datacenters in Water and Space (see skills.json write_essay instructions)", + "tools": [{"type":"function","function":{"name":"fetch","description":"Fetch URL via relay","parameters":{"type":"object","properties":{"relay-path":{"type":"string"}}}}}], + "collection": "'$COLLECTION'", + "model": "qwen3.6" +}' | jq -r '.output' | head -n 40 + +echo "== Done. Output saved to essay.out.md, history in hb_store $COLLECTION, durable to agents/researcher/memory/ ==" diff --git a/examples/datacenter-essay/skills.json b/examples/datacenter-essay/skills.json new file mode 100644 index 0000000000..bef18eeabd --- /dev/null +++ b/examples/datacenter-essay/skills.json @@ -0,0 +1,30 @@ +[ + { + "name": "research", + "description": "Fetch 2-4 authoritative sources via relay, extract facts", + "requires_tools": ["fetch"], + "instructions": "Use fetch (relay@1.0) to retrieve:\n- Microsoft Project Natick (water)\n- Nautilus / offshore floating datacenters\n- Kepler / Starcloud / space datacenter concepts\nExtract: power, cooling, latency, failure mode, cost. Keep quotes + URLs. Do not invent.", + "steps": ["fetch https://natick.research.microsoft.com", "fetch https://kepler.space", "fetch https://arxiv.org/abs/space-datacenter"] + }, + { + "name": "summarize", + "description": "Take documents, extract key points, produce tight brief", + "requires_tools": ["fetch"], + "instructions": "Given research docs, produce 10 bullets: 5 water, 5 space. Each bullet: claim + source URL.", + "steps": [] + }, + { + "name": "write_essay", + "description": "Write 800w essay: water vs space datacenters", + "requires_tools": ["fetch"], + "instructions": "Write essay using identity.md/user.md/soul.md + memory:\nH1: Datacenters in Water and Space\nH2: Water (Natick) — cooling, deployment, biofouling\nH2: Space (solar, radiation, launch $)\nTable: Power | Cooling | Latency | Repair | Cost\nConclusion: boring tradeoffs, not hype. Cite sources footnotes.", + "steps": [] + }, + { + "name": "research-write", + "description": "Composed: research + summarize + write_essay", + "requires_tools": ["fetch"], + "instructions": "Compose research then summarize then write_essay. First fetch sources, then bullet brief, then 800w essay.", + "steps": [] + } +] diff --git a/mkdocs.yml b/mkdocs.yml index e12fd5d9c7..50432a9e86 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -36,6 +36,9 @@ nav: - '~json@1.0': 'devices/json-at-1-0.md' - '~scheduler@1.0': 'devices/scheduler-at-1-0.md' - '~relay@1.0': 'devices/relay-at-1-0.md' + - '~llm@1.0': 'devices/llm-at-1-0.md' + - '~harness@1.0': 'devices/harness-at-1-0.md' + - '~skills@1.0': 'devices/skills-at-1-0.md' - Resources: # - Overview: 'resources/source-code/index.md' - FAQ: 'resources/reference/faq.md' diff --git a/src/core/test/dev_llm_test.erl b/src/core/test/dev_llm_test.erl new file mode 100644 index 0000000000..9119c23934 --- /dev/null +++ b/src/core/test/dev_llm_test.erl @@ -0,0 +1,170 @@ +%%% @doc Tests for dev_llm@1.0 — OpenAI-compatible LLM proxy. +-module(dev_llm_test). +-include_lib("eunit/include/eunit.hrl"). +-include("include/hb.hrl"). + +%% Ensure dev_llm is loaded (preloaded store not in test code path) +ensure_dev_llm() -> + case code:is_loaded(dev_llm) of + {file, _} -> ok; + false -> + case compile:file("src/preloaded/llm/dev_llm.erl", [binary, {i, "include"}, {i, "src/core/include"}]) of + {ok, dev_llm, Bin} -> code:load_binary(dev_llm, "dev_llm.erl", Bin); + {ok, dev_llm, Bin, _Warn} -> code:load_binary(dev_llm, "dev_llm.erl", Bin); + Error -> Error + end + end. + +%% Unit helpers (no network) +resolve_endpoint_test() -> + ensure_dev_llm(), + ?assertEqual(<<"http://spark-1b7b.local:8888/v1/chat/completions">>, dev_llm:resolve_endpoint(#{}, #{}, #{}, chat)). +resolve_endpoint_embed_default_test() -> + ensure_dev_llm(), + ?assertEqual(<<"http://spark-1b7b.local:8888/v1/embeddings">>, dev_llm:resolve_endpoint(#{}, #{}, #{}, embed)). +resolve_endpoint_override_test() -> + ensure_dev_llm(), + ?assertEqual(<<"http://localhost:8000/v1/chat/completions">>, dev_llm:resolve_endpoint(#{}, #{<<"endpoint">> => <<"http://localhost:8000/v1/chat/completions">>}, #{}, chat)). +resolve_endpoint_llm_endpoint_test() -> + ensure_dev_llm(), + ?assertEqual(<<"http://spark-1b7b.local:8888/v1/embeddings">>, dev_llm:resolve_endpoint(#{<<"llm-endpoint">> => <<"http://spark-1b7b.local:8888/v1/chat/completions">>}, #{}, #{}, embed)). +resolve_endpoint_embed_endpoint_override_test() -> + ensure_dev_llm(), + ?assertEqual(<<"http://custom:9999/embed">>, dev_llm:resolve_endpoint(#{}, #{<<"embed-endpoint">> => <<"http://custom:9999/embed">>}, #{}, embed)). + +build_chat_body_prompt_test() -> + ensure_dev_llm(), + Body = dev_llm:build_chat_body(#{<<"prompt">> => <<"hello">>}, <<"llama3.2">>, false, #{}), + Decoded = hb_json:decode(Body), + ?assertEqual(<<"llama3.2">>, maps:get(<<"model">>, Decoded)), + ?assertEqual([#{<<"role">> => <<"user">>, <<"content">> => <<"hello">>}], maps:get(<<"messages">>, Decoded)), + ?assertEqual(false, maps:get(<<"stream">>, Decoded)). +build_chat_body_messages_test() -> + ensure_dev_llm(), + Msgs = [#{<<"role">> => <<"user">>, <<"content">> => <<"hi">>}], + Body = dev_llm:build_chat_body(#{<<"messages">> => Msgs}, <<"m">>, false, #{}), + Decoded = hb_json:decode(Body), + ?assertEqual(Msgs, maps:get(<<"messages">>, Decoded)). +build_chat_body_data_test() -> + ensure_dev_llm(), + Body = dev_llm:build_chat_body(#{<<"data">> => <<"hello data">>}, <<"m">>, false, #{}), + Decoded = hb_json:decode(Body), + ?assertEqual([#{<<"role">> => <<"user">>, <<"content">> => <<"hello data">>}], maps:get(<<"messages">>, Decoded)). +is_stream_true_test() -> + ensure_dev_llm(), + ?assertEqual(true, dev_llm:is_stream(#{<<"stream">> => <<"true">>}, #{})), + ?assertEqual(true, dev_llm:is_stream(#{<<"Stream">> => <<"true">>}, #{})), + ?assertEqual(false, dev_llm:is_stream(#{}, #{})). +extract_content_test() -> + ensure_dev_llm(), + M = #{<<"choices">> => [#{<<"message">> => #{<<"content">> => <<"hi">>}}]}, + ?assertEqual(<<"hi">>, dev_llm:extract_content(M)), + ?assertEqual(<<"hi">>, dev_llm:extract_content(#{<<"content">> => <<"hi">>})), + ?assertEqual(<<>>, dev_llm:extract_content(#{<<"foo">> => 1})). + +%% Integration with mock Ollama (gen_tcp) +mock_integration_test_() -> + {timeout, 30, fun mock_integration/0}. + +mock_integration() -> + {ok,_}=application:ensure_all_started(prometheus), + {ok,_}=application:ensure_all_started(hb), + Port = 54329, + Pid = spawn(fun() -> mock_server(Port) end), + timer:sleep(200), + Endpoint = list_to_binary(io_lib:format("http://localhost:~p/v1/chat/completions", [Port])), + EmbedEndpoint = list_to_binary(io_lib:format("http://localhost:~p/v1/embeddings", [Port])), + % chat non-stream + {ok, #{<<"body">> := Body1}} = dev_llm:chat(#{}, #{<<"prompt">> => <<"hello">>, <<"endpoint">> => Endpoint, <<"model">> => <<"test-model">>}, #{}), + Decoded1 = hb_json:decode(Body1), + ?assertMatch(#{<<"choices">> := [#{<<"message">> := #{<<"content">> := <<"Hello from mock LLM">>}}|_]}, Decoded1), + % generate extracts content + {ok, #{<<"content">> := Content2}} = dev_llm:generate(#{}, #{<<"prompt">> => <<"hello">>, <<"endpoint">> => Endpoint}, #{}), + ?assertEqual(<<"Hello from mock LLM">>, Content2), + % embed + {ok, #{<<"embedding">> := Emb, <<"embeddings">> := Embs}} = dev_llm:embed(#{}, #{<<"input">> => <<"hello world">>, <<"embed-endpoint">> => EmbedEndpoint}, #{}), + ?assertEqual([0.1,0.2,0.3], Emb), + ?assertEqual([[0.1,0.2,0.3]], Embs), + % stream -> text/event-stream + {ok, #{<<"headers">> := H4, <<"body">> := B4}} = dev_llm:chat(#{}, #{<<"prompt">> => <<"hello">>, <<"endpoint">> => Endpoint, <<"stream">> => <<"true">>}, #{}), + ?assertEqual(<<"text/event-stream">>, maps:get(<<"content-type">>, H4)), + ?assertMatch({_,_}, binary:match(B4, <<"data:">>)), + % hb_ao resolve path + {ok, _} = hb_ao:resolve(#{<<"device">> => <<"llm@1.0">>, <<"path">> => <<"chat">>, <<"prompt">> => <<"hello">>, <<"endpoint">> => Endpoint}, #{}), + % missing input -> error + ?assertMatch({error, _}, dev_llm:embed(#{}, #{}, #{})), + exit(Pid, kill), + ok. + +%% Lua ao.resolve path (same as AO process) +lua_ao_resolve_test_() -> + {timeout, 30, fun lua_ao_resolve/0}. +lua_ao_resolve() -> + {ok,_}=application:ensure_all_started(prometheus), + {ok,_}=application:ensure_all_started(hb), + Port = 54330, + Pid = spawn(fun() -> mock_server(Port) end), + timer:sleep(200), + Endpoint = list_to_binary(io_lib:format("http://localhost:~p/v1/chat/completions", [Port])), + Script = << + "function llm_chat()\n" + " local status, res = ao.resolve({device='llm@1.0', path='chat', prompt='Say hi', endpoint='", Endpoint/binary, "'})\n" + " if status ~= 'ok' then return 'fail' end\n" + " local body = res.body or res.Body or ''\n" + " return body\n" + "end\n" + >>, + Base = #{ + <<"device">> => <<"lua@5.3a">>, + <<"module">> => #{<<"content-type">> => <<"application/lua">>, <<"body">> => Script}, + <<"parameters">> => [] + }, + {ok, Res} = hb_ao:resolve(Base, <<"llm_chat">>, #{}), + ?assertMatch({_,_}, binary:match(hb_util:bin(Res), <<"Hello from mock LLM">>)), + exit(Pid, kill), + ok. + +%% Live Ollama (only if LLM_LIVE=1) +live_ollama_test_() -> + case os:getenv("LLM_LIVE") of + "1" -> {timeout, 30, fun live_ollama/0}; + _ -> [] + end. +live_ollama() -> + {ok,_}=application:ensure_all_started(prometheus), + {ok,_}=application:ensure_all_started(hb), + EP = <<"http://spark-1b7b.local:8888/v1/chat/completions">>, + {ok, #{<<"body">> := Body}} = dev_llm:chat(#{}, #{<<"prompt">> => <<"Say hello in 2 words">>, <<"model">> => <<"qwen3.6">>, <<"endpoint">> => EP}, #{}), + Decoded = hb_json:decode(Body), + Content = dev_llm:extract_content(Decoded), + ?assert(byte_size(Content) > 0). + +%% Helpers: tiny HTTP mock +mock_server(Port) -> + {ok, LSock} = gen_tcp:listen(Port, [binary, {packet, raw}, {active, false}, {reuseaddr, true}]), + accept_loop(LSock). +accept_loop(LSock) -> + case gen_tcp:accept(LSock) of + {ok, Sock} -> spawn(fun() -> handle(Sock) end), accept_loop(LSock); + {error, closed} -> ok + end. +handle(Sock) -> + case gen_tcp:recv(Sock, 0, 5000) of + {ok, Data} -> + Path = case binary:match(Data, <<"POST ">>) of + {_,_} -> [_, Rest] = binary:split(Data, <<"POST ">>), [P,_]=binary:split(Rest, <<" HTTP">>), P; + _ -> <<"/">> + end, + Body = case binary:match(Path, <<"/embeddings">>) of + nomatch -> + IsStream = binary:match(Data, <<"\"stream\":true">>) =/= nomatch, + case IsStream of + true -> <<"data: {\"choices\":[{\"delta\":{\"content\":\"Hello \"}}]}\n\ndata: {\"choices\":[{\"delta\":{\"content\":\"world\"}}]}\n\ndata: [DONE]\n\n">>; + false -> hb_json:encode(#{<<"id">> => <<"chatcmpl-test">>, <<"choices">> => [#{<<"message">> => #{<<"role">> => <<"assistant">>, <<"content">> => <<"Hello from mock LLM">>}, <<"finish_reason">> => <<"stop">>}], <<"model">> => <<"test-model">>}) + end; + _ -> hb_json:encode(#{<<"data">> => [#{<<"embedding">> => [0.1,0.2,0.3], <<"index">> => 0}], <<"model">> => <<"test-embed">>}) + end, + Resp = iolist_to_binary(["HTTP/1.1 200 OK\r\n","Content-Type: application/json\r\n","Content-Length: ", integer_to_binary(byte_size(Body)), "\r\n","Connection: close\r\n","\r\n", Body]), + gen_tcp:send(Sock, Resp), gen_tcp:close(Sock); + _ -> gen_tcp:close(Sock) + end. diff --git a/src/preloaded/agent/agent.lua b/src/preloaded/agent/agent.lua new file mode 100644 index 0000000000..0264877760 --- /dev/null +++ b/src/preloaded/agent/agent.lua @@ -0,0 +1,118 @@ +-- agent.lua — Generic AO Agent Process: LLM + Harness + Tools + Instructions + Memory +-- Essay: https://chat.hyper.io/share/wE41XruWrXFj37EdGfFxn3sdXy0Wib9e — Agent = LLM + Harness + Tools + Instructions +-- This process composes: instructions (agents//*.md + tools.json) + memory (agents//memory/*.md) + harness@1.0/handle + skills@1.0 +-- Usage: aos < src/preloaded/agent/agent.lua +-- Then: Send({Target=PROC, Action="AgentPrompt", Prompt="...", Agent="tom", Skill="summarize"}) +-- Or: Send({Target=PROC, Action="RunSkill", Skill="summarize", Prompt="morning brief", Agent="tom"}) + +local json = require("json") + +local function read_agent_file(agent, filename) + -- Try HB store via relay? For now, read via harness store collection + -- Agent files are plain text on disk in this repo (agents/tom/*), but in HyperBEAM they are in hb_store under agent:: + -- We attempt ao.resolve on skills/harness store, fallback to empty + local collection = "agent-"..agent + local ok, res = pcall(function() + return ao.resolve({device="harness@1.0", path="query", collection=collection, q=filename}) + end) + if ok and res and res.results and #res.results>0 then + return res.results[1].description or res.results[1].content or "" + end + return "" +end + +local function load_instructions(agent) + -- Essay flow: identity.md + soul.md + user.md are system instructions, tools.json is capabilities, memory/*.md is durable + -- Try HB store first, then disk (for local dev), then defaults + local function try_file(path) + local f = io.open(path, "r") + if f then local c = f:read("*a"); f:close(); return c end + return nil + end + local base = "agents/"..agent + local identity = try_file(base.."/identity.md") or read_agent_file(agent, "identity.md") + if identity == "" then identity = "Identity: "..agent.." — hyper.io agent" end + local user = try_file(base.."/user.md") or read_agent_file(agent, "user.md") + if user == "" then user = "User: Tom" end + local soul = try_file(base.."/soul.md") or read_agent_file(agent, "soul.md") + if soul == "" then soul = "Principles: boring ships — prefer practical, auditable" end + local tools = {"gmail_read"} + local tools_raw = try_file(base.."/tools.json") or read_agent_file(agent, "tools.json") + if tools_raw and tools_raw ~= "" then + local ok, dec = pcall(json.decode, tools_raw) + if ok and type(dec)=="table" then tools = dec end + end + return {identity=identity, user=user, soul=soul, tools=tools} +end + +-- Main: Run a skill as an agent +Handlers.add("run-skill", Handlers.utils.hasMatchingTag("Action", "RunSkill"), function(msg) + local agent = msg.Tags.Agent or msg.Tags.agent or "tom" + local skill = msg.Tags.Skill or msg.Tags.skill or msg.Data or "summarize" + local prompt = msg.Tags.Prompt or msg.Tags.prompt or msg.Data or "Do the skill" + -- Load agent tools.json + local tools_json = read_agent_file(agent, "tools.json") + local agent_tools = {"gmail_read", "gmail_send"} + if tools_json ~= "" then + local ok, decoded = pcall(json.decode, tools_json) + if ok and type(decoded)=="table" then agent_tools = decoded end + else + -- Try msg.Tags.Tools + local t = msg.Tags.Tools or msg.Tags.tools + if t then + local ok, dec = pcall(json.decode, t) + if ok then agent_tools = dec else agent_tools = {t} end + end + end + print("Agent "..agent.." running skill "..skill.." with tools "..json.encode(agent_tools)) + local instr = load_instructions(agent) + -- Essay flow: harness builds system = identity+ soul + user + skill instructions + history + tools + current + -- Pass identity/soul/user explicitly so harness can build system window and enforce limit + local ok, res = pcall(function() + return ao.resolve({device="skills@1.0", path="run", skill=skill, agent_tools=agent_tools, message=prompt, collection="agent-"..agent, model="qwen3.6", identity=instr.identity, soul=instr.soul, user=instr.user, history_limit=20}) + end) + if not ok then + Send({Target=msg.From, Data="skills run failed: "..tostring(res)}) + return + end + if res.can_run == false or res["can_run"]==false then + Send({Target=msg.From, Data="Agent "..agent.." cannot run skill "..skill..": missing "..json.encode(res.missing or res["missing"] or {})}) + return + end + local output = res.output or res.content or "" + -- Persist memory: append to agents//memory via harness store + local mem_ok = pcall(function() + return ao.resolve({device="harness@1.0", path="store", collection="agent-"..agent, + posts={{guid="mem-"..tostring(math.random(1000000)), title="Memory: "..skill, description=output}}}) + end) + Send({Target=msg.From, Data=json.encode({agent=agent, skill=skill, output=output, iterations=res.iterations or 1, missing=res.missing})}) +end) + +-- Generic agent prompt (no skill, just harness) — essay flow: system (identity/soul/user) + history + tools + current +Handlers.add("agent-prompt", Handlers.utils.hasMatchingTag("Action", "AgentPrompt"), function(msg) + local agent = msg.Tags.Agent or "tom" + local prompt = msg.Data or msg.Tags.Prompt or "" + local ok, res = pcall(function() + local instr = load_instructions(agent) + -- Essay: harness rebuilds window each turn with system (identity/soul/user) + tools + history + current; it manages limit + local tools = { + {type="function", ["function"]={name="get_gmail_messages", description="Read Gmail", parameters={type="object", properties={q={type="string"}}}}}, + {type="function", ["function"]={name="get_calendar_events", description="Read Calendar", parameters={type="object", properties={timeMin={type="string"}}}}} + } + return ao.resolve({device="harness@1.0", path="handle", message=prompt, system=instr.identity.."\n"..instr.soul.."\n"..instr.user, identity=instr.identity, soul=instr.soul, user=instr.user, tools=tools, collection="agent-"..agent, model="qwen3.6", history_limit=20}) + end) + if not ok then Send({Target=msg.From, Data="agent failed: "..tostring(res)}); return end + Send({Target=msg.From, Data=json.encode({agent=agent, output=res.output or "", history=res.history})}) +end) + +-- Ingest agent instructions into store (helper) +Handlers.add("ingest-agent", Handlers.utils.hasMatchingTag("Action", "IngestAgent"), function(msg) + local agent = msg.Tags.Agent or "tom" + -- In real HyperBEAM, you'd POST agents/tom/*.md via relay; here just confirm + local ok, res = pcall(function() + return ao.resolve({device="skills@1.0", path="list"}) + end) + Send({Target=msg.From, Data="Agent "..agent.." ready. Skills: "..json.encode(res and res.skills or {})}) +end) + +print("Agent loaded: RunSkill/AgentPrompt/IngestAgent — LLM+harness+skills+instructions+memory — essay model") diff --git a/src/preloaded/agent/dev_harness.erl b/src/preloaded/agent/dev_harness.erl new file mode 100644 index 0000000000..8dccc50140 --- /dev/null +++ b/src/preloaded/agent/dev_harness.erl @@ -0,0 +1,499 @@ +%%% @doc Generic agent harness for HyperBEAM — `harness@1.0`. +%%% +%%% Orchestrates `llm@1.0` + `relay@1.0` + `store` (HB cache FS) + `query@1.0` +%%% + `lua@5.3a` bash-like env. Generic: fetch any URL via `relay@1.0`, +%%% parse (RSS/JSON), store under `-` + `-index`, +%%% then query. `dan-feed` is just one test dataset (collection=`dan`). +-module(dev_harness). +-implements(<<"harness@1.0">>). +-export([info/1, fetch/3, parse/3, store/3, ingest/3, query/3, list/3]). +-export([info/3, handle/3, run/3, chat/3, execute/3]). +-export([fetch_feed/1, parse_feed/1, fetch_via_relay/2]). + +-include_lib("eunit/include/eunit.hrl"). + +info(_) -> + #{ + <<"fetch">> => dev, + <<"parse">> => dev, + <<"store">> => dev, + <<"ingest">> => dev, + <<"query">> => dev, + <<"list">> => dev, + <<"handle">> => dev, + <<"run">> => dev, + <<"chat">> => dev, + <<"execute">> => dev + }. +info(_, _, _) -> {ok, info(#{})}. + +%% Fetch any URL via relay@1.0 (public) — dan-feed is just default test dataset +fetch(Base, Req, Opts) -> + URL = hb_ao:get(<<"url">>, Req, hb_ao:get(<<"url">>, Base, <<"https://hyperio-mc.github.io/dan-feed/feed.xml">>, Opts), Opts), + fetch_via_relay(URL, Opts). +fetch_feed(URL) -> fetch_via_relay(URL, #{}). +fetch_feed(URL, Opts) -> fetch_via_relay(URL, Opts). + +fetch_via_relay(URL, Opts) -> + % Use relay@1.0/call so external tool/service fetches go through relay device + % (hb_http direct would also work for public hosts, but relay is the intended tool gateway) + case hb_ao:resolve(#{<<"device">> => <<"relay@1.0">>, <<"path">> => <<"call">>, <<"relay-path">> => URL, <<"relay-method">> => <<"GET">>}, Opts) of + {ok, Res} when is_map(Res) -> {ok, maps:get(<<"body">>, Res, maps:get(<<"Body">>, Res, <<>>))}; + {ok, Bin} when is_binary(Bin) -> {ok, Bin}; + Err -> Err + end. + +%% Parse RSS XML → list of post maps +parse(Base, Req, Opts) -> + Body = hb_ao:get(<<"body">>, Req, hb_ao:get(<<"body">>, Base, <<>>, Opts), Opts), + case Body of + <<>> -> fetch_and_parse(Opts); + _ -> {ok, parse_feed(Body)} + end. +fetch_and_parse(Opts) -> + case fetch_feed(<<"https://hyperio-mc.github.io/dan-feed/feed.xml">>, Opts) of + {ok, Body} -> {ok, parse_feed(Body)}; + Err -> Err + end. + +parse_feed(Body) when is_binary(Body) -> + % Very simple RSS item extraction via regex — no xmerl dep + Items = case re:run(Body, <<"(.*?)">>, [global, dotall, {capture, [1], binary}]) of + {match, Matches} -> [ hd(M) || M <- Matches ]; + nomatch -> [] + end, + [parse_item(Item) || Item <- Items]. + +parse_item(Item) -> + G = extract_tag(Item, <<"guid">>), + #{ + <<"guid">> => G, + <<"id">> => G, + <<"title">> => extract_tag(Item, <<"title">>), + <<"link">> => extract_tag(Item, <<"link">>), + <<"description">> => extract_tag(Item, <<"description">>), + <<"pubDate">> => extract_tag(Item, <<"pubDate">>), + <<"source">> => extract_source(Item) + }. + +extract_tag(Bin, Tag) -> + Pat = <<"<", Tag/binary, "[^>]*>(.*?)">>, + case re:run(Bin, Pat, [dotall, {capture, [1], binary}]) of + {match, [V]} -> unescape(V); + nomatch -> <<>> + end. + +extract_source(Item) -> + case re:run(Item, <<"]*>(.*?)">>, [dotall, {capture, [1], binary}]) of + {match, [V]} -> unescape(V); + nomatch -> <<>> + end. + +unescape(Bin) -> + B1 = binary:replace(Bin, <<"&">>, <<"&">>, [global]), + B2 = binary:replace(B1, <<"<">>, <<"<">>, [global]), + B3 = binary:replace(B2, <<">">>, <<">">>, [global]), + B4 = binary:replace(B3, <<"'">>, <<"'">>, [global]), + binary:replace(B4, <<""">>, <<"\"">>, [global]). + +%% Generic store: collection- + collection-index (default collection=dan for demo) +store(Base, Req, Opts) -> + Posts = case hb_ao:get(<<"posts">>, Req, not_found, Opts) of + not_found -> case hb_ao:get(<<"posts">>, Base, not_found, Opts) of + not_found -> case hb_ao:get(<<"items">>, Req, not_found, Opts) of + not_found -> parse_feed(hb_ao:get(<<"body">>, Req, <<>>, Opts)); + I -> I + end; + P -> P + end; + P -> P + end, + PostsList = case is_list(Posts) of true -> Posts; false -> [Posts] end, + Collection = hb_ao:get(<<"collection">>, Req, hb_ao:get(<<"collection">>, Base, <<"dan">>, Opts), Opts), + Store = hb_opts:get(store, no_viable, Opts), + lists:foreach(fun(P) -> + Id = maps:get(<<"guid">>, P, maps:get(<<"id">>, P, hb_util:bin(rand:uniform(1000000)))), + Key = <>, + hb_store:write(Store, #{Key => hb_json:encode(P)}, Opts) + end, PostsList), + IdxKey = <>, + ExistingIdx = case hb_store:read(Store, IdxKey, Opts) of {ok, Bin} -> try hb_json:decode(Bin) catch _:_ -> [] end; _ -> [] end, + NewIds = [maps:get(<<"guid">>, P, maps:get(<<"id">>, P, <<>>)) || P <- PostsList], + MergedIdx = lists:usort(ExistingIdx ++ NewIds), + hb_store:write(Store, #{IdxKey => hb_json:encode(MergedIdx)}, Opts), + {ok, #{<<"stored">> => length(PostsList), <<"keys">> => NewIds, <<"total">> => length(MergedIdx), <<"collection">> => Collection}}. + +%% Generic ingest: fetch any URL (via relay) + parse + store to collection +ingest(Base, Req, Opts) -> + URL = hb_ao:get(<<"url">>, Req, hb_ao:get(<<"url">>, Base, <<"https://hyperio-mc.github.io/dan-feed/feed.xml">>, Opts), Opts), + Collection = hb_ao:get(<<"collection">>, Req, hb_ao:get(<<"collection">>, Base, <<"dan">>, Opts), Opts), + case fetch_via_relay(URL, Opts) of + {ok, Body} -> + Posts = parse_feed(Body), + % If parse yields no RSS items, try JSON array fallback + Items = case Posts of [] -> try hb_json:decode(Body) catch _:_ -> [] end; _ -> Posts end, + ItemsList = case is_list(Items) of true -> Items; false -> [Items] end, + Store = hb_opts:get(store, no_viable, Opts), + lists:foreach(fun(P) -> + M = case is_map(P) of true -> P; false -> #{<<"data">> => P} end, + Id = maps:get(<<"guid">>, M, maps:get(<<"id">>, M, hb_util:bin(rand:uniform(1000000)))), + Key = <>, + hb_store:write(Store, #{Key => hb_json:encode(M)}, Opts) + end, ItemsList), + IdxKey = <>, + ExistingIdx = case hb_store:read(Store, IdxKey, Opts) of {ok, Bin} -> try hb_json:decode(Bin) catch _:_ -> [] end; _ -> [] end, + NewIds = [case is_map(P) of true -> maps:get(<<"guid">>, P, maps:get(<<"id">>, P, <<>>)); false -> <<>> end || P <- ItemsList], + MergedIdx = lists:usort(ExistingIdx ++ NewIds), + hb_store:write(Store, #{IdxKey => hb_json:encode(MergedIdx)}, Opts), + {ok, #{<<"ingested">> => length(ItemsList), <<"total">> => length(MergedIdx), <<"collection">> => Collection, <<"posts">> => ItemsList}}; + Err -> Err + end. + +%% List: generic collection-index +list(Base, Req, Opts) -> + Collection = hb_ao:get(<<"collection">>, Req, hb_ao:get(<<"collection">>, Base, <<"dan">>, Opts), Opts), + IdxKey = <>, + Store = hb_opts:get(store, no_viable, Opts), + case hb_store:read(Store, IdxKey, Opts) of + {ok, Bin} -> + Keys = try hb_json:decode(Bin) catch _:_ -> [] end, + {ok, #{<<"keys">> => Keys, <<"count">> => length(Keys), <<"collection">> => Collection}}; + Err -> Err + end. + +%% Query: generic collection-index + per-item reads + q filter +query(Base, Req, Opts) -> + Q = hb_ao:get(<<"q">>, Req, hb_ao:get(<<"query">>, Base, <<>>, Opts), Opts), + Collection = hb_ao:get(<<"collection">>, Req, hb_ao:get(<<"collection">>, Base, <<"dan">>, Opts), Opts), + IdxKey = <>, + Store = hb_opts:get(store, no_viable, Opts), + case hb_store:read(Store, IdxKey, Opts) of + {ok, Bin} -> + Keys = try hb_json:decode(Bin) catch _:_ -> [] end, + Posts = [case hb_store:read(Store, <>, Opts) of {ok, B} -> try hb_json:decode(B) catch _:_ -> #{} end; _ -> #{} end || K <- Keys], + Filtered = case Q of + <<>> -> Posts; + _ -> [P || P <- Posts, matches_query(P, Q)] + end, + {ok, #{<<"results">> => Filtered, <<"count">> => length(Filtered), <<"q">> => Q, <<"collection">> => Collection}}; + Err -> Err + end. + +matches_query(Post, Q) -> + QLow = string:lowercase(Q), + Fields = [maps:get(<<"title">>, Post, <<>>), maps:get(<<"description">>, Post, <<>>), maps:get(<<"link">>, Post, <<>>)], + lists:any(fun(F) -> binary:match(string:lowercase(F), QLow) =/= nomatch end, Fields). + +%% =================================================================== +%% Agent harness: message + history + tools -> llm loop -> tool dispatch +%% Implements: take message, build context (last-turn history + tools), +%% call llm@1.0, if tool_calls then execute via relay@1.0 (or requested +%% device) and append to context, repeat until no tools, return output +%% and persist updated context to hb_store. +%% =================================================================== + +handle(Base, Req, Opts) -> do_harness(Base, Req, Opts). +run(Base, Req, Opts) -> do_harness(Base, Req, Opts). +chat(Base, Req, Opts) -> do_harness(Base, Req, Opts). +execute(Base, Req, Opts) -> do_harness(Base, Req, Opts). + +do_harness(Base, Req, Opts) -> + Message = get_harness_message(Base, Req, Opts), + ToolsRaw0 = hb_ao:get(<<"tools">>, Req, hb_ao:get(<<"tools">>, Base, not_found, Opts), Opts), + ToolsRaw = maybe_deref(ToolsRaw0, Opts), + Tools = normalize_tools(ToolsRaw), + Collection = hb_ao:get(<<"collection">>, Req, hb_ao:get(<<"collection">>, Base, <<"default">>, Opts), Opts), + CollectionBin = hb_util:bin(Collection), + HistoryKey = hb_ao:get(<<"history-key">>, Req, hb_ao:get(<<"history-key">>, Base, <>, Opts), Opts), + Store = hb_opts:get(store, no_viable, Opts), + HistoryRaw = load_harness_history(Base, Req, Opts, HistoryKey, Store), + % Harness manages window limit: truncate previous history up to limit + HistoryLimit = get_history_limit(Base, Req, Opts), + History0 = truncate_history(HistoryRaw, HistoryLimit, Opts), + % Build system instructions (identity.md + soul.md + user.md) as essay flow: system + identity/tools/history/current + SystemMsg = build_system_message(Base, Req, Opts), + Messages0WithoutSystem = case Message of + <<>> -> History0; + undefined -> History0; + not_found -> History0; + _ -> History0 ++ [#{<<"role">> => <<"user">>, <<"content">> => hb_util:bin(Message)}] + end, + % If caller supplied explicit messages list, prefer it (already contains history) + ExplicitMessages0 = hb_ao:get(<<"messages">>, Req, hb_ao:get(<<"messages">>, Base, not_found, Opts), Opts), + ExplicitMessages = maybe_deref(ExplicitMessages0, Opts), + MessagesInitWithoutSystem = case ExplicitMessages of + not_found -> Messages0WithoutSystem; + M when is_list(M) -> [maybe_deref(X, Opts) || X <- M]; + M when is_binary(M) -> try hb_json:decode(M) catch _:_ -> Messages0WithoutSystem end; + _ -> Messages0WithoutSystem + end, + MaxIters = get_max_iters(Base, Req, Opts), + Model = hb_ao:get(<<"model">>, Req, hb_ao:get(<<"model">>, Base, <<"qwen3.6">>, Opts), Opts), + Endpoint = hb_ao:get(<<"endpoint">>, Req, hb_ao:get(<<"endpoint">>, Base, hb_ao:get(<<"llm-endpoint">>, Base, not_found, Opts), Opts), Opts), + EndpointNorm = case Endpoint of not_found -> undefined; _ -> Endpoint end, + case MessagesInitWithoutSystem of + [] when Tools =/= undefined -> + harness_loop(MessagesInitWithoutSystem, SystemMsg, Tools, Model, EndpointNorm, MaxIters, Opts, HistoryKey, Store, 0, History0); + [] -> + {ok, #{<<"output">> => <<>>, <<"history">> => History0, <<"messages">> => History0, <<"iterations">> => 0}}; + _ -> + harness_loop(MessagesInitWithoutSystem, SystemMsg, Tools, Model, EndpointNorm, MaxIters, Opts, HistoryKey, Store, 0, History0) + end. + +get_harness_message(Base, Req, Opts) -> + Keys = [<<"message">>, <<"prompt">>, <<"data">>, <<"input">>, <<"content">>], + Raw = get_first_key(Keys, [Req, Base], Opts), + maybe_deref(Raw, Opts). + +get_first_key([], _, _) -> not_found; +get_first_key([K|Rest], Maps, Opts) -> + case hb_ao:get(K, hd(Maps), not_found, Opts) of + not_found -> + case length(Maps) of + 1 -> get_first_key(Rest, Maps, Opts); + _ -> case hb_ao:get(K, lists:nth(2, Maps), not_found, Opts) of + not_found -> get_first_key(Rest, Maps, Opts); + V -> V + end + end; + V -> V + end. + +maybe_deref(not_found, _) -> not_found; +maybe_deref(undefined, _) -> undefined; +maybe_deref(V, Opts) -> + try hb_cache:ensure_loaded(V, Opts) catch _:_ -> V end. + +normalize_tools(not_found) -> undefined; +normalize_tools(undefined) -> undefined; +normalize_tools(T) when is_binary(T) -> + try hb_json:decode(T) catch _:_ -> undefined end; +normalize_tools(T) when is_list(T) -> + % Deref each tool if it's a link + [case maybe_deref(Tool, #{}) of M when is_map(M) -> M; Other -> Other end || Tool <- T]; +normalize_tools(T) when is_map(T) -> [maybe_deref(T, #{})]; +normalize_tools(V) -> + % Try to deref if it's a link + Deref = maybe_deref(V, #{}), + case Deref of + T when is_list(T) -> T; + T when is_map(T) -> [T]; + _ -> undefined + end. + +get_max_iters(Base, Req, Opts) -> + V = hb_ao:get(<<"max-iterations">>, Req, hb_ao:get(<<"max_iterations">>, Req, hb_ao:get(<<"max-iterations">>, Base, 10, Opts), Opts), Opts), + case V of + N when is_integer(N) -> N; + B when is_binary(B) -> try binary_to_integer(B) catch _:_ -> 10 end; + _ -> 10 + end. + +get_history_limit(Base, Req, Opts) -> + V0 = hb_ao:get(<<"history_limit">>, Req, hb_ao:get(<<"history-limit">>, Req, hb_ao:get(<<"max_history">>, Req, hb_ao:get(<<"max-history">>, Req, hb_ao:get(<<"history_limit">>, Base, not_found, Opts), Opts), Opts), Opts), Opts), + V = maybe_deref(V0, Opts), + case V of + not_found -> 20; + undefined -> 20; + N when is_integer(N) -> N; + B when is_binary(B) -> try binary_to_integer(B) catch _:_ -> 20 end; + _ -> 20 + end. + +truncate_history(History, Limit, _Opts) when not is_list(History) -> History; +truncate_history(History, Limit, _Opts) when Limit =:= undefined; Limit =:= 0; Limit =:= not_found -> History; +truncate_history(History, Limit, _Opts) -> + Len = length(History), + if Len =< Limit -> History; + true -> lists:nthtail(Len - Limit, History) + end. + +build_system_message(Base, Req, Opts) -> + % Essay flow: System instructions + Agent identity (identity.md + soul.md + user.md) + tools already separate + System0 = hb_ao:get(<<"system">>, Req, hb_ao:get(<<"system">>, Base, not_found, Opts), Opts), + SystemDeref = maybe_deref(System0, Opts), + SystemBin = case SystemDeref of not_found -> undefined; undefined -> undefined; B when is_binary(B) -> B; M when is_map(M) -> hb_json:encode(M); L when is_list(L) -> hb_util:bin(L); _ -> undefined end, + Identity0 = hb_ao:get(<<"identity">>, Req, hb_ao:get(<<"identity">>, Base, not_found, Opts), Opts), + Soul0 = hb_ao:get(<<"soul">>, Req, hb_ao:get(<<"soul">>, Base, not_found, Opts), Opts), + User0 = hb_ao:get(<<"user">>, Req, hb_ao:get(<<"user">>, Base, not_found, Opts), Opts), + Instructions0 = hb_ao:get(<<"instructions">>, Req, hb_ao:get(<<"instructions">>, Base, not_found, Opts), Opts), + Parts = [ + case maybe_deref(Identity0, Opts) of not_found -> undefined; undefined -> undefined; V -> hb_util:bin(V) end, + case maybe_deref(Soul0, Opts) of not_found -> undefined; undefined -> undefined; V2 -> hb_util:bin(V2) end, + case maybe_deref(User0, Opts) of not_found -> undefined; undefined -> undefined; V3 -> hb_util:bin(V3) end, + case maybe_deref(Instructions0, Opts) of not_found -> undefined; undefined -> undefined; V4 -> hb_util:bin(V4) end, + SystemBin + ], + Filtered = [P || P <- Parts, P =/= undefined, P =/= <<>>, P =/= not_found], + case Filtered of + [] -> undefined; + _ -> + Combined = iolist_to_binary(lists:join(<<"\n\n">>, Filtered)), + #{<<"role">> => <<"system">>, <<"content">> => Combined} + end. + +load_harness_history(Base, Req, Opts, HistoryKey, Store) -> + % 1) explicit history param + Explicit0 = hb_ao:get(<<"history">>, Req, hb_ao:get(<<"history">>, Base, not_found, Opts), Opts), + Explicit = maybe_deref(Explicit0, Opts), + case Explicit of + not_found -> + % 2) load from store + case Store of + no_viable -> []; + _ -> + case hb_store:read(Store, HistoryKey, Opts) of + {ok, Bin} when is_binary(Bin) -> + try + Dec = hb_json:decode(Bin), + case Dec of L when is_list(L) -> [maybe_deref(X, Opts) || X <- L]; M when is_map(M) -> [maybe_deref(M, Opts)]; _ -> [] end + catch _:_ -> [] + end; + {ok, L} when is_list(L) -> [maybe_deref(X, Opts) || X <- L]; + _ -> [] + end + end; + H when is_list(H) -> [maybe_deref(X, Opts) || X <- H]; + H when is_binary(H) -> + try hb_json:decode(H) catch _:_ -> [] end; + H when is_map(H) -> [maybe_deref(H, Opts)]; + _ -> [] + end. + +save_harness_history(HistoryKey, History, Store, Opts) -> + case Store of + no_viable -> ok; + _ -> + try hb_store:write(Store, #{HistoryKey => hb_json:encode(History)}, Opts) catch _:_ -> ok end + end, + ok. + +harness_loop(Messages, SystemMsg, Tools, Model, Endpoint, MaxIters, Opts, HistoryKey, Store, Iter, OriginalHistory) when Iter >= MaxIters -> + save_harness_history(HistoryKey, Messages, Store, Opts), + {error, #{<<"error">> => <<"max iterations reached">>, <<"history">> => Messages, <<"iterations">> => Iter, <<"original_history">> => OriginalHistory}}; +harness_loop(Messages, SystemMsg, Tools, Model, Endpoint, MaxIters, Opts, HistoryKey, Store, Iter, OriginalHistory) -> + LLMMessages = case SystemMsg of undefined -> Messages; _ -> [SystemMsg | Messages] end, + LLMOpts = case Endpoint of + undefined -> #{<<"model">> => Model, <<"messages">> => LLMMessages}; + _ -> #{<<"model">> => Model, <<"messages">> => LLMMessages, <<"endpoint">> => Endpoint} + end, + LLMReq = case Tools of + undefined -> LLMOpts; + _ -> LLMOpts#{<<"tools">> => Tools, <<"tool_choice">> => <<"auto">>} + end, + LLMMsg = maps:merge(#{<<"device">> => <<"llm@1.0">>, <<"path">> => <<"chat">>}, LLMReq), + case hb_ao:resolve(LLMMsg, Opts) of + {ok, Res} when is_map(Res) -> + Body = maps:get(<<"body">>, Res, maps:get(<<"Body">>, Res, <<>>)), + Parsed = try hb_json:decode(Body) catch _:_ -> Res end, + {AssistantMsg, ToolCalls, Content} = extract_assistant(Parsed, Res), + case ToolCalls of + [] -> + FinalHistory = Messages ++ [AssistantMsg], + save_harness_history(HistoryKey, FinalHistory, Store, Opts), + {ok, #{<<"output">> => Content, <<"content">> => Content, <<"history">> => FinalHistory, <<"messages">> => FinalHistory, <<"iterations">> => Iter + 1, <<"raw">> => Parsed, <<"system">> => SystemMsg}}; + _ -> + ToolResults = [execute_harness_tool(TC, Opts) || TC <- ToolCalls], + NextMessages = Messages ++ [AssistantMsg] ++ ToolResults, + harness_loop(NextMessages, SystemMsg, Tools, Model, Endpoint, MaxIters, Opts, HistoryKey, Store, Iter + 1, OriginalHistory) + end; + {error, _} = Err -> Err; + Other -> {error, #{<<"error">> => hb_util:bin(Other), <<"history">> => Messages}} + end. + +extract_assistant(Parsed, _Res) when is_map(Parsed) -> + Choices = maps:get(<<"choices">>, Parsed, []), + case Choices of + [First|_] when is_map(First) -> + Msg = maps:get(<<"message">>, First, maps:get(<<"delta">>, First, #{})), + Content = maps:get(<<"content">>, Msg, <<>>), + ToolCalls = maps:get(<<"tool_calls">>, Msg, maps:get(<<"toolCalls">>, Msg, [])), + ToolCallsNorm = case ToolCalls of null -> []; undefined -> []; L when is_list(L) -> L; _ -> [] end, + ContentBin = case Content of null -> <<>>; undefined -> <<>>; C when is_binary(C) -> C; C -> hb_util:bin(C) end, + AssistantMsg = case ToolCallsNorm of + [] -> #{<<"role">> => <<"assistant">>, <<"content">> => ContentBin}; + _ -> #{<<"role">> => <<"assistant">>, <<"content">> => ContentBin, <<"tool_calls">> => ToolCallsNorm} + end, + {AssistantMsg, ToolCallsNorm, ContentBin}; + _ -> + % Fallback: try direct content field + Content = maps:get(<<"content">>, Parsed, maps:get(<<"Content">>, Parsed, <<>>)), + ContentBin = hb_util:bin(Content), + AssistantMsg = #{<<"role">> => <<"assistant">>, <<"content">> => ContentBin}, + {AssistantMsg, [], ContentBin} + end; +extract_assistant(_, Res) -> + Content = maps:get(<<"body">>, Res, <<>>), + Bin = hb_util:bin(Content), + {#{<<"role">> => <<"assistant">>, <<"content">> => Bin}, [], Bin}. + +execute_harness_tool(ToolCall, Opts) when is_map(ToolCall) -> + Id = maps:get(<<"id">>, ToolCall, maps:get(<<"tool_call_id">>, ToolCall, hb_util:bin(rand:uniform(1000000)))), + Fun = maps:get(<<"function">>, ToolCall, ToolCall), + Name = maps:get(<<"name">>, Fun, maps:get(<<"tool">>, Fun, <<>>)), + ArgsRaw = maps:get(<<"arguments">>, Fun, maps:get(<<"args">>, Fun, <<"{}">>)), + Args = case ArgsRaw of + A when is_map(A) -> A; + A when is_binary(A) -> try hb_json:decode(A) catch _:_ -> #{<<"raw">> => A} end; + _ -> #{} + end, + ResultContent = dispatch_tool(Name, Args, Opts), + #{<<"role">> => <<"tool">>, <<"tool_call_id">> => hb_util:bin(Id), <<"content">> => hb_util:bin(ResultContent), <<"name">> => hb_util:bin(Name)}. + +dispatch_tool(Name, Args, Opts) when is_map(Args) -> + % If args already specifies a device/path, honor it + case maps:get(<<"device">>, Args, not_found) of + not_found -> + % Check for relay-style tool + RelayPath = case maps:get(<<"relay-path">>, Args, not_found) of + not_found -> case maps:get(<<"url">>, Args, not_found) of + not_found -> case maps:get(<<"path">>, Args, not_found) of + not_found when Name =/= <<>> -> + % Name might be URL or relay target; try empty + not_found; + P -> P + end; + U -> U + end; + P -> P + end, + case RelayPath of + not_found -> + % No path: try generic relay with args as body, or direct error + case hb_ao:resolve(maps:merge(#{<<"device">> => <<"relay@1.0">>, <<"path">> => <<"call">>}, Args), Opts) of + {ok, R} when is_map(R) -> maps:get(<<"body">>, R, maps:get(<<"Body">>, R, hb_json:encode(R))); + {ok, R} when is_binary(R) -> R; + {error, E} -> hb_util:bin(E); + Other -> hb_util:bin(Other) + end; + _ -> + RelayMethod = maps:get(<<"relay-method">>, Args, maps:get(<<"method">>, Args, <<"GET">>)), + RelayBody = maps:get(<<"relay-body">>, Args, maps:get(<<"body">>, Args, not_found)), + BaseReq = #{<<"device">> => <<"relay@1.0">>, <<"path">> => <<"call">>, <<"relay-path">> => hb_util:bin(RelayPath), <<"relay-method">> => hb_util:bin(RelayMethod)}, + ReqWithBody = case RelayBody of not_found -> BaseReq; _ -> BaseReq#{<<"relay-body">> => RelayBody} end, + Merged = maps:merge(ReqWithBody, maps:without([<<"relay-path">>, <<"relay-method">>, <<"relay-body">>, <<"url">>, <<"path">>, <<"method">>, <<"body">>], Args)), + case hb_ao:resolve(Merged, Opts) of + {ok, R} when is_map(R) -> maps:get(<<"body">>, R, maps:get(<<"Body">>, R, hb_json:encode(R))); + {ok, R} when is_binary(R) -> R; + {error, E} -> hb_util:bin(E); + Other -> hb_util:bin(Other) + end + end; + Device -> + Path = maps:get(<<"path">>, Args, <<"call">>), + CleanArgs = maps:without([<<"device">>, <<"path">>], Args), + case hb_ao:resolve(maps:merge(#{<<"device">> => hb_util:bin(Device), <<"path">> => hb_util:bin(Path)}, CleanArgs), Opts) of + {ok, R} when is_map(R) -> maps:get(<<"body">>, R, maps:get(<<"Body">>, R, hb_json:encode(R))); + {ok, R} when is_binary(R) -> R; + {error, E} -> hb_util:bin(E); + Other -> hb_util:bin(Other) + end + end. + +-ifdef(TEST). +parse_feed_test() -> + Body = <<"1Thttp://xD">>, + [Post] = parse_feed(Body), + ?assertEqual(<<"1">>, maps:get(<<"guid">>, Post)), + ?assertEqual(<<"T">>, maps:get(<<"title">>, Post)). +-endif. diff --git a/src/preloaded/agent/dev_skills.erl b/src/preloaded/agent/dev_skills.erl new file mode 100644 index 0000000000..39a316f6d8 --- /dev/null +++ b/src/preloaded/agent/dev_skills.erl @@ -0,0 +1,389 @@ +%%% @doc Skills device for HyperBEAM — `skills@1.0`. +%%% +%%% Implements Tom Wilson's model: Tools are atomic (get_gmail_messages), +%%% Skills are composable procedures that use 1..N tools generically. +%%% Harness enforces: does agent have tools skill requires? +%%% +%%% Skill shape (stored as JSON under `skill-` + `skill-index`): +%%% #{ name => binary, description => binary, +%%% requires_tools => [binary], % e.g. [<<"gmail_read">>] +%%% instructions => binary, % markdown procedure +%%% steps => [map], % optional explicit steps +%%% version => binary } +%%% +%%% Agent shape (per essay): +%%% agents//identity.md, user.md, soul.md, tools.json, memory/*.md +%%% Agent = LLM + Harness + Tools + Instructions, memory via FS. +-module(dev_skills). +-implements(<<"skills@1.0">>). +-export([info/1, info/3, register/3, get/3, list/3, run/3, compose/3, check/3]). +-export([skill_key/1, index_key/0]). + +-include_lib("eunit/include/eunit.hrl"). + +info(_) -> + #{ + <<"register">> => dev, + <<"get">> => dev, + <<"list">> => dev, + <<"run">> => dev, + <<"compose">> => dev, + <<"check">> => dev + }. +info(_, _, _) -> {ok, info(#{})}. + +skill_key(Name) when is_binary(Name) -> <<"skill-", Name/binary>>; +skill_key(Name) -> skill_key(hb_util:bin(Name)). + +index_key() -> <<"skill-index">>. + +%% Register (or update) a skill. +%% Req keys: name | skill | id, description, requires_tools | tools, instructions | steps | procedure +register(Base, Req, Opts) -> + Name0 = hb_ao:get(<<"name">>, Req, hb_ao:get(<<"skill">>, Req, hb_ao:get(<<"id">>, Req, hb_ao:get(<<"name">>, Base, not_found, Opts), Opts), Opts), Opts), + Name = case maybe_deref(Name0, Opts) of + not_found -> hb_util:bin(rand:uniform(1000000)); + N -> hb_util:bin(N) + end, + Desc0 = hb_ao:get(<<"description">>, Req, hb_ao:get(<<"description">>, Base, <<>>, Opts), Opts), + Desc = hb_util:bin(maybe_deref(Desc0, Opts)), + Requires0 = hb_ao:get(<<"requires_tools">>, Req, + hb_ao:get(<<"requires-tools">>, Req, + hb_ao:get(<<"tools">>, Req, + hb_ao:get(<<"requires_tools">>, Base, [], Opts), Opts), Opts), Opts), + RequiresDeref = maybe_deref(Requires0, Opts), + Requires = normalize_requires(RequiresDeref), + Instructions0 = hb_ao:get(<<"instructions">>, Req, + hb_ao:get(<<"procedure">>, Req, + hb_ao:get(<<"steps">>, Req, + hb_ao:get(<<"instructions">>, Base, <<>>, Opts), Opts), Opts), Opts), + Instructions = maybe_deref(Instructions0, Opts), + InstructionsBin = case Instructions of + B when is_binary(B) -> B; + L when is_list(L) -> hb_json:encode(L); + M when is_map(M) -> hb_json:encode(M); + _ -> <<>> + end, + Steps0 = hb_ao:get(<<"steps">>, Req, hb_ao:get(<<"steps">>, Base, [], Opts), Opts), + Steps = deep_deref(maybe_deref(Steps0, Opts), Opts), + StepsList = case Steps of SL when is_list(SL) -> SL; SM when is_map(SM) -> [SM]; _ -> [] end, + Version0 = hb_ao:get(<<"version">>, Req, hb_ao:get(<<"version">>, Base, <<"1.0">>, Opts), Opts), + Version = hb_util:bin(maybe_deref(Version0, Opts)), + Skill = #{ + <<"name">> => Name, + <<"description">> => Desc, + <<"requires_tools">> => Requires, + <<"instructions">> => InstructionsBin, + <<"steps">> => StepsList, + <<"version">> => Version + }, + Store = hb_opts:get(store, no_viable, Opts), + Key = skill_key(Name), + case Store of + no_viable -> {error, no_store}; + _ -> + ok = hb_store:write(Store, #{Key => hb_json:encode(Skill)}, Opts), + % Update index + IdxKey = index_key(), + Existing = case hb_store:read(Store, IdxKey, Opts) of + {ok, Bin} when is_binary(Bin) -> try hb_json:decode(Bin) catch _:_ -> [] end; + {ok, ExistingList} when is_list(ExistingList) -> ExistingList; + _ -> [] + end, + Merged = lists:usort([Name | Existing]), + ok = hb_store:write(Store, #{IdxKey => hb_json:encode(Merged)}, Opts), + {ok, Skill#{ <<"stored_as">> => Key, <<"total">> => length(Merged) }} + end. + +%% Get a skill by name +get(Base, Req, Opts) -> + Name0 = hb_ao:get(<<"name">>, Req, hb_ao:get(<<"skill">>, Req, hb_ao:get(<<"id">>, Req, hb_ao:get(<<"name">>, Base, not_found, Opts), Opts), Opts), Opts), + Name = hb_util:bin(maybe_deref(Name0, Opts)), + case Name of + <<>> -> {error, missing_name}; + _ -> + Store = hb_opts:get(store, no_viable, Opts), + Key = skill_key(Name), + case hb_store:read(Store, Key, Opts) of + {ok, Bin} when is_binary(Bin) -> + Skill = try hb_json:decode(Bin) catch _:_ -> #{} end, + {ok, deep_deref(Skill, Opts)}; + {ok, M} when is_map(M) -> {ok, deep_deref(M, Opts)}; + Err -> Err + end + end. + +%% List all skill names +list(_Base, _Req, Opts) -> + Store = hb_opts:get(store, no_viable, Opts), + IdxKey = index_key(), + case hb_store:read(Store, IdxKey, Opts) of + {ok, Bin} when is_binary(Bin) -> + Keys = try hb_json:decode(Bin) catch _:_ -> [] end, + {ok, #{<<"skills">> => Keys, <<"count">> => length(Keys)}}; + {ok, L} when is_list(L) -> {ok, #{<<"skills">> => L, <<"count">> => length(L)}}; + _ -> {ok, #{<<"skills">> => [], <<"count">> => 0}} + end. + +%% Check if agent's tools satisfy skill's requires_tools +check(Base, Req, Opts) -> + SkillName0 = hb_ao:get(<<"skill">>, Req, hb_ao:get(<<"name">>, Req, hb_ao:get(<<"skill">>, Base, not_found, Opts), Opts), Opts), + SkillName = hb_util:bin(maybe_deref(SkillName0, Opts)), + AgentTools0 = hb_ao:get(<<"agent_tools">>, Req, + hb_ao:get(<<"tools">>, Req, + hb_ao:get(<<"agent_tools">>, Base, not_found, Opts), Opts), Opts), + AgentToolsDeref = maybe_deref(AgentTools0, Opts), + AgentTools = normalize_requires(AgentToolsDeref), + case get(Base, #{<<"name">> => SkillName}, Opts) of + {ok, Skill} -> + Requires = maps:get(<<"requires_tools">>, Skill, []), + Missing = [T || T <- Requires, not lists:member(T, AgentTools)], + case Missing of + [] -> {ok, #{<<"can_run">> => true, <<"skill">> => SkillName, <<"requires_tools">> => Requires, <<"agent_tools">> => AgentTools}}; + _ -> {ok, #{<<"can_run">> => false, <<"skill">> => SkillName, <<"requires_tools">> => Requires, <<"agent_tools">> => AgentTools, <<"missing">> => Missing}} + end; + Err -> Err + end. + +%% Run a skill: check permission, then delegate to harness@1.0/handle +%% Req: skill (name), message|prompt, agent_tools|tools, collection|agent, model, history, etc. +run(Base, Req, Opts) -> + SkillName0 = hb_ao:get(<<"skill">>, Req, hb_ao:get(<<"name">>, Req, hb_ao:get(<<"skill">>, Base, not_found, Opts), Opts), Opts), + SkillName = hb_util:bin(maybe_deref(SkillName0, Opts)), + Message0 = hb_ao:get(<<"message">>, Req, hb_ao:get(<<"prompt">>, Req, hb_ao:get(<<"data">>, Req, hb_ao:get(<<"message">>, Base, <<>>, Opts), Opts), Opts), Opts), + MessageDeref = maybe_deref(Message0, Opts), + Message = case MessageDeref of not_found -> <<>>; undefined -> <<>>; M -> hb_util:bin(M) end, + AgentTools0 = hb_ao:get(<<"agent_tools">>, Req, + hb_ao:get(<<"tools">>, Req, + hb_ao:get(<<"agent_tools">>, Base, [], Opts), Opts), Opts), + AgentTools = normalize_requires(maybe_deref(AgentTools0, Opts)), + % Load skill + case get(Base, #{<<"name">> => SkillName}, Opts) of + {ok, Skill} -> + Requires = maps:get(<<"requires_tools">>, Skill, []), + Missing = [T || T <- Requires, not lists:member(T, AgentTools)], + case Missing of + [] -> + Instructions = maps:get(<<"instructions">>, Skill, <<>>), + % Map requires_tools to relay tool specs for harness + HarnessTools = tools_to_harness_specs(Requires), + Collection0 = hb_ao:get(<<"collection">>, Req, hb_ao:get(<<"agent">>, Req, hb_ao:get(<<"collection">>, Base, SkillName, Opts), Opts), Opts), + Collection = hb_util:bin(maybe_deref(Collection0, Opts)), + Model0 = hb_ao:get(<<"model">>, Req, hb_ao:get(<<"model">>, Base, <<"qwen3.6">>, Opts), Opts), + Model = hb_util:bin(maybe_deref(Model0, Opts)), + % History handling delegated to harness — essay flow: system (instructions) + history + tools + current + HarnessReq0 = #{ + <<"device">> => <<"harness@1.0">>, + <<"path">> => <<"handle">>, + <<"message">> => Message, + <<"tools">> => HarnessTools, + <<"collection">> => Collection, + <<"model">> => Model + }, + HarnessReq = case Instructions of + <<>> -> HarnessReq0; + _ -> HarnessReq0#{<<"system">> => Instructions} + end, + % Pass through endpoint/history/identity/soul/user if supplied (essay: system = identity+soul+user+instructions) + HarnessReq2 = maybe_add(<<"endpoint">>, hb_ao:get(<<"endpoint">>, Req, not_found, Opts), HarnessReq, Opts), + HarnessReq3 = maybe_add(<<"history">>, hb_ao:get(<<"history">>, Req, not_found, Opts), HarnessReq2, Opts), + HarnessReq4 = maybe_add(<<"identity">>, hb_ao:get(<<"identity">>, Req, hb_ao:get(<<"identity">>, Base, not_found, Opts), Opts), HarnessReq3, Opts), + HarnessReq5 = maybe_add(<<"soul">>, hb_ao:get(<<"soul">>, Req, hb_ao:get(<<"soul">>, Base, not_found, Opts), Opts), HarnessReq4, Opts), + HarnessReq6 = maybe_add(<<"user">>, hb_ao:get(<<"user">>, Req, hb_ao:get(<<"user">>, Base, not_found, Opts), Opts), HarnessReq5, Opts), + case hb_ao:resolve(HarnessReq6, Opts) of + {ok, Res} when is_map(Res) -> + Output = maps:get(<<"output">>, Res, maps:get(<<"content">>, Res, <<>>)), + % Optionally append to skill-specific memory (store output) + {ok, Res#{ <<"skill">> => SkillName, <<"output">> => Output, <<"can_run">> => true }}; + Err -> Err + end; + _ -> + {error, #{<<"can_run">> => false, <<"skill">> => SkillName, <<"missing">> => Missing, <<"requires_tools">> => Requires, <<"agent_tools">> => AgentTools}} + end; + Err -> Err + end. + +%% Compose two skills into a new skill +compose(Base, Req, Opts) -> + NameA0 = hb_ao:get(<<"skill_a">>, Req, hb_ao:get(<<"a">>, Req, not_found, Opts), Opts), + NameB0 = hb_ao:get(<<"skill_b">>, Req, hb_ao:get(<<"b">>, Req, not_found, Opts), Opts), + NewName0 = hb_ao:get(<<"name">>, Req, hb_ao:get(<<"new_name">>, Req, not_found, Opts), Opts), + NameA = hb_util:bin(maybe_deref(NameA0, Opts)), + NameB = hb_util:bin(maybe_deref(NameB0, Opts)), + NewName = case maybe_deref(NewName0, Opts) of not_found -> <>; N -> hb_util:bin(N) end, + case {get(Base, #{<<"name">> => NameA}, Opts), get(Base, #{<<"name">> => NameB}, Opts)} of + {{ok, SkillA}, {ok, SkillB}} -> + RequiresA = maps:get(<<"requires_tools">>, SkillA, []), + RequiresB = maps:get(<<"requires_tools">>, SkillB, []), + Requires = lists:usort(RequiresA ++ RequiresB), + Instructions = <<(maps:get(<<"instructions">>, SkillA, <<>>))/binary, "\n\nThen:\n", (maps:get(<<"instructions">>, SkillB, <<>>))/binary>>, + Steps = maps:get(<<"steps">>, SkillA, []) ++ maps:get(<<"steps">>, SkillB, []), + Desc = <<"Composed ", NameA/binary, " + ", NameB/binary>>, + % Register new skill + register(Base, #{<<"name">> => NewName, <<"description">> => Desc, <<"requires_tools">> => Requires, <<"instructions">> => Instructions, <<"steps">> => Steps}, Opts); + {Err, _} when element(1, Err) == error -> Err; + {_, Err} -> Err + end. + +%% Helpers +normalize_requires(not_found) -> []; +normalize_requires(undefined) -> []; +normalize_requires(L) when is_list(L) -> [hb_util:bin(maybe_deref(X, #{})) || X <- L]; +normalize_requires(B) when is_binary(B) -> + try hb_json:decode(B) of + L when is_list(L) -> [hb_util:bin(X) || X <- L]; + _ -> [B] + catch _:_ -> [B] + end; +normalize_requires(M) when is_map(M) -> [hb_util:bin(K) || K <- maps:keys(M)]; +normalize_requires(V) -> [hb_util:bin(V)]. + +tools_to_harness_specs(Requires) -> + [tool_to_spec(T) || T <- Requires]. + +tool_to_spec(Tool) when is_binary(Tool) -> + % Map well-known tools to relay specs + case Tool of + <<"gmail_read">> -> gmail_spec(); + <<"gmail_send">> -> gmail_send_spec(); + <<"calendar_read">> -> calendar_spec(); + <<"drive_read">> -> drive_spec(); + <<"github_read">> -> github_spec(); + _ -> + % Generic: expose as function that takes relay-path + #{ + <<"type">> => <<"function">>, + <<"function">> => #{ + <<"name">> => Tool, + <<"description">> => <>, + <<"parameters">> => #{ + <<"type">> => <<"object">>, + <<"properties">> => #{ + <<"relay-path">> => #{<<"type">> => <<"string">>}, + <<"relay-method">> => #{<<"type">> => <<"string">>} + } + } + } + } + end. + +gmail_spec() -> + #{ + <<"type">> => <<"function">>, + <<"function">> => #{ + <<"name">> => <<"get_gmail_messages">>, + <<"description">> => <<"Read Gmail messages">>, + <<"parameters">> => #{ + <<"type">> => <<"object">>, + <<"properties">> => #{ + <<"q">> => #{<<"type">> => <<"string">>, <<"description">> => <<"Search query">>}, + <<"max">> => #{<<"type">> => <<"integer">>} + } + } + } + }. + +gmail_send_spec() -> + #{ + <<"type">> => <<"function">>, + <<"function">> => #{ + <<"name">> => <<"send_gmail">>, + <<"description">> => <<"Send Gmail">>, + <<"parameters">> => #{ + <<"type">> => <<"object">>, + <<"properties">> => #{ + <<"to">> => #{<<"type">> => <<"string">>}, + <<"subject">> => #{<<"type">> => <<"string">>}, + <<"body">> => #{<<"type">> => <<"string">>} + } + } + } + }. + +calendar_spec() -> + #{ + <<"type">> => <<"function">>, + <<"function">> => #{ + <<"name">> => <<"get_calendar_events">>, + <<"description">> => <<"Read calendar events">>, + <<"parameters">> => #{ + <<"type">> => <<"object">>, + <<"properties">> => #{ + <<"timeMin">> => #{<<"type">> => <<"string">>}, + <<"timeMax">> => #{<<"type">> => <<"string">>} + } + } + } + }. + +drive_spec() -> + #{ + <<"type">> => <<"function">>, + <<"function">> => #{ + <<"name">> => <<"read_drive">>, + <<"description">> => <<"Read Drive file">>, + <<"parameters">> => #{ + <<"type">> => <<"object">>, + <<"properties">> => #{ + <<"fileId">> => #{<<"type">> => <<"string">>} + } + } + } + }. + +github_spec() -> + #{ + <<"type">> => <<"function">>, + <<"function">> => #{ + <<"name">> => <<"github_search">>, + <<"description">> => <<"Search GitHub">>, + <<"parameters">> => #{ + <<"type">> => <<"object">>, + <<"properties">> => #{ + <<"q">> => #{<<"type">> => <<"string">>} + } + } + } + }. + +maybe_add(_Key, not_found, Map, _Opts) -> Map; +maybe_add(_Key, undefined, Map, _Opts) -> Map; +maybe_add(Key, Val0, Map, Opts) -> + Val = maybe_deref(Val0, Opts), + Map#{Key => Val}. + +maybe_deref(not_found, _) -> not_found; +maybe_deref(undefined, _) -> undefined; +maybe_deref(V, Opts) -> + try hb_cache:ensure_loaded(V, Opts) catch _:_ -> V end. + +deep_deref(V, Opts) when is_map(V) -> + case (catch hb_cache:ensure_loaded(V, Opts)) of + V2 when is_map(V2), V2 =/= V -> deep_deref(V2, Opts); + _ -> maps:from_list([{K, deep_deref(Val, Opts)} || {K, Val} <- maps:to_list(V)]) + end; +deep_deref(V, Opts) when is_list(V) -> [deep_deref(X, Opts) || X <- V]; +deep_deref(V, Opts) -> + case (catch hb_cache:ensure_loaded(V, Opts)) of + V2 when V2 =/= V -> deep_deref(V2, Opts); + _ -> V + end. + +-ifdef(TEST). +register_get_list_test() -> + {ok,_}=application:ensure_all_started(hb), + Store = hb_opts:get(store, no_viable, #{}), + % Clean + hb_store:write(Store, #{index_key() => hb_json:encode([])}, #{}), + {ok, _} = register(#{}, #{<<"name">> => <<"summarize">>, <<"requires_tools">> => [<<"gmail_read">>], <<"instructions">> => <<"Summarize docs">>}, #{}), + {ok, #{<<"skills">> := Keys}} = list(#{}, #{}, #{}), + ?assert(lists:member(<<"summarize">>, Keys)), + {ok, Skill} = get(#{}, #{<<"name">> => <<"summarize">>}, #{}), + ?assertEqual(<<"summarize">>, maps:get(<<"name">>, Skill)). + +check_can_run_test() -> + {ok,_}=application:ensure_all_started(hb), + register(#{}, #{<<"name">> => <<"inbox-zero">>, <<"requires_tools">> => [<<"gmail_read">>]}, #{}), + {ok, #{<<"can_run">> := true}} = check(#{}, #{<<"skill">> => <<"inbox-zero">>, <<"tools">> => [<<"gmail_read">>, <<"gmail_send">>]}, #{}), + {ok, #{<<"can_run">> := false, <<"missing">> := [<<"gmail_read">>]}} = check(#{}, #{<<"skill">> => <<"inbox-zero">>, <<"tools">> => [<<"calendar_read">>]}, #{}). +-endif. diff --git a/src/preloaded/agent/harness.lua b/src/preloaded/agent/harness.lua new file mode 100644 index 0000000000..045a0e478b --- /dev/null +++ b/src/preloaded/agent/harness.lua @@ -0,0 +1,103 @@ +-- harness.lua — Generic AO agent harness for HyperBEAM +-- Demonstrates: llm@1.0 + relay@1.0 (tools) + store/cache FS + query@1.0 + lua bash +-- Generic: any URL/collection — dan-feed is just default test dataset +-- Load: aos < src/preloaded/agent/harness.lua + +-- 1. Generic ingest: fetch any URL via relay → store → index (dan is default) +Handlers.add("ingest", Handlers.utils.hasMatchingTag("Action", "Ingest"), function(msg) + local url = msg.Tags.Url or msg.Data or "https://hyperio-mc.github.io/dan-feed/feed.xml" + local collection = msg.Tags.Collection or "dan" + print("Harness: fetching "..url.." via relay -> "..collection.." ...") + local ok, res = pcall(function() + return ao.resolve({device="harness@1.0", path="ingest", url=url, collection=collection}) + end) + -- keep IngestDAN as alias for demo +end) +Handlers.add("ingest-dan", Handlers.utils.hasMatchingTag("Action", "IngestDAN"), function(msg) + print("Harness: fetching dan-feed via harness@1.0 (dan collection)...") + local ok, res = pcall(function() + return ao.resolve({device="harness@1.0", path="ingest", url="https://hyperio-mc.github.io/dan-feed/feed.xml", collection="dan"}) + end) + if not ok then + print("ingest failed: "..tostring(res)) + Send({Target=msg.From, Data="ingest failed: "..tostring(res)}) + return + end + local ingested = res.ingested or res["ingested"] or 0 + print("Ingested "..tostring(ingested).." posts to dan/posts/*") + -- Summarize via LLM (spark qwen3.6 default) + local s_ok, s_res = pcall(function() + return ao.resolve({device="llm@1.0", path="generate", prompt="Summarize these "..tostring(ingested).." Daily Agents News posts in 3 bullet points", model="qwen3.6"}) + end) + local summary = "" + if s_ok then summary = s_res.content or s_res.Content or "" end + Send({Target=msg.From, Data="Ingested "..tostring(ingested).." posts. Summary: "..summary}) +end) + +-- 2. Generic query: any collection via harness (dan default), plus QueryDAN alias +Handlers.add("query", Handlers.utils.hasMatchingTag("Action", "Query"), function(msg) + local q = msg.Tags.Query or msg.Data or "" + local collection = msg.Tags.Collection or "dan" + local ok, res = ao.resolve({device="harness@1.0", path="query", q=q, collection=collection}) +end) +Handlers.add("query-dan", Handlers.utils.hasMatchingTag("Action", "QueryDAN"), function(msg) + local q = msg.Tags.Query or msg.Data or "" + local ok, res = ao.resolve({device="harness@1.0", path="query", q=q, collection="dan"}) + if not ok then Send({Target=msg.From, Data="query failed"}); return end + local results = res.results or {} + Send({Target=msg.From, Data=require("json").encode({count=res.count or #results, q=q, results=results})}) +end) + +Handlers.add("list", Handlers.utils.hasMatchingTag("Action", "List"), function(msg) + local collection = msg.Tags.Collection or "dan" + local ok, res = ao.resolve({device="harness@1.0", path="list", collection=collection}) +end) +Handlers.add("list-dan", Handlers.utils.hasMatchingTag("Action", "ListDAN"), function(msg) + local ok, res = ao.resolve({device="harness@1.0", path="list", collection="dan"}) + Send({Target=msg.From, Data=require("json").encode(res)}) +end) + +-- 3. Agent loop: harness@1.0/handle — message + history + tools -> llm loop -> tool dispatch -> store +-- Meets standard: takes message, last-turn history (via collection), and tools, loops until no tool_calls +Handlers.add("agent-run", Handlers.utils.hasMatchingTag("Action", "AgentRun"), function(msg) + local prompt = msg.Data or msg.Tags.Prompt or "What should the agent do next?" + local collection = msg.Tags.Collection or "agent" + local tools = { + { + type = "function", + ["function"] = { + name = "fetch", + description = "Fetch any URL via relay@1.0", + parameters = { + type = "object", + properties = { + ["relay-path"] = {type = "string", description = "URL to fetch"}, + ["relay-method"] = {type = "string", description = "HTTP method"}, + ["relay-body"] = {type = "string", description = "Optional body"} + }, + required = {"relay-path"} + } + } + } + } + -- Harness builds context (prompt + history from collection + tools), calls llm, dispatches tool_calls via relay, loops, stores history + local ok, res = ao.resolve({device="harness@1.0", path="handle", message=prompt, tools=tools, collection=collection, model="qwen3.6"}) + if not ok then Send({Target=msg.From, Data="harness failed: "..tostring(res)}); return end + local output = res.output or res.content or "" + local iterations = res.iterations or 1 + print("Agent output ("..tostring(iterations).." iters): "..string.sub(output,1,300)) + -- Optionally store final output as post + ao.resolve({device="harness@1.0", path="store", posts={{guid="agent-"..tostring(math.random(100000)), title="Agent run", link="", description=output}}, collection=collection}) + Send({Target=msg.From, Data="Output: "..string.sub(output,1,500).."\nIterations: "..tostring(iterations)}) +end) + +-- 4. Bash-like via lua (lua is the bash) +Handlers.add("bash", Handlers.utils.hasMatchingTag("Action", "Bash"), function(msg) + local cmd = msg.Data or "" + -- In HyperBEAM lua, os.execute is sandboxed, but you can use ao.resolve to subprocess via wasi if needed + -- For demo, just echo via llm + local ok, res = ao.resolve({device="llm@1.0", path="generate", prompt="Explain how to run in bash: "..cmd, model="qwen3.6"}) + Send({Target=msg.From, Data=res.content or ""}) +end) + +print("Harness loaded: Ingest/Query/List (generic, dan default) + IngestDAN/QueryDAN/ListDAN aliases, AgentRun, Bash — llm+relay+cache+query; relay used for fetch") diff --git a/src/preloaded/llm/dev_llm.erl b/src/preloaded/llm/dev_llm.erl new file mode 100644 index 0000000000..47e312dbfe --- /dev/null +++ b/src/preloaded/llm/dev_llm.erl @@ -0,0 +1,301 @@ +%%% @doc Local LLM device for HyperBEAM — `llm@1.0`. +%%% +%%% OpenAI-compatible proxy for Ollama / vLLM / llama.cpp running on localhost. +%%% Unlike `dev_relay@1.0`, this calls `hb_http` directly so `localhost` is NOT blocked. +%%% +%%% Actions: +%%% `chat` -> POST `/v1/chat/completions` (OpenAI shape, supports `stream=true` -> SSE) +%%% `generate` -> `chat` + extracts content string +%%% `embed` -> POST `/v1/embeddings` (alias: `embeddings`) +-module(dev_llm). +-implements(<<"llm@1.0">>). +-export([info/1, chat/3, generate/3, embed/3, embeddings/3]). +-export([resolve_endpoint/4, build_chat_body/4, build_chat_body/5, is_stream/2, extract_content/1]). + +-include_lib("eunit/include/eunit.hrl"). + +info(_) -> + #{ + <<"chat">> => dev, + <<"generate">> => dev, + <<"embed">> => dev, + <<"embeddings">> => dev + }. + +%% Chat — OpenAI-compatible passthrough with optional SSE streaming +chat(Base, Req, Opts) -> + Endpoint = resolve_endpoint(Base, Req, Opts, chat), + RawModel = get_val(<<"model">>, [Req, Base], <<"unsloth/Qwen3.6-35B-A3B-NVFP4">>, Opts), + Model = normalize_model(RawModel), + Stream = is_stream(Req, Opts), + Body = build_chat_body(Base, Req, Model, Stream, Opts), + Headers = #{<<"content-type">> => <<"application/json">>}, + case Stream of + true -> + case do_request(Endpoint, Body, Headers, Opts) of + {ok, #{ <<"body">> := StreamBody } = Res} -> + HeadersOut = maps:get(<<"headers">>, Res, #{}), + {ok, Res#{ + <<"status">> => maps:get(<<"status">>, Res, 200), + <<"headers">> => maps:merge(HeadersOut, #{<<"content-type">> => <<"text/event-stream">>}), + <<"body">> => StreamBody + }}; + {ok, #{ body := StreamBody } = Res} -> + HeadersOut = maps:get(headers, Res, #{}), + {ok, Res#{ + <<"status">> => maps:get(status, Res, 200), + <<"headers">> => maps:merge(HeadersOut, #{<<"content-type">> => <<"text/event-stream">>}), + <<"body">> => StreamBody + }}; + {ok, Res} -> {ok, Res}; + Err -> Err + end; + false -> + do_request(Endpoint, Body, Headers, Opts) + end. + +%% Generate — same as chat but extracts content string for easier AO use +generate(Base, Req, Opts) -> + case chat(Base, Req, Opts) of + {ok, #{ <<"body">> := Body } = Res} -> + Parsed = try hb_json:decode(Body) catch _:_ -> #{<<"raw">> => Body} end, + Content = extract_content(Parsed), + {ok, Res#{ <<"content">> => Content, <<"raw">> => Parsed }}; + {ok, #{ body := Body } = Res} -> + Parsed = try hb_json:decode(Body) catch _:_ -> #{<<"raw">> => Body} end, + Content = extract_content(Parsed), + {ok, Res#{ <<"content">> => Content, <<"raw">> => Parsed }}; + {ok, #{ <<"status">> := _} = Res} -> + % Try to extract from body if present under different key + Body = maps:get(<<"body">>, Res, maps:get(<<"Body">>, Res, undefined)), + case Body of + undefined -> {ok, Res}; + _ -> + Parsed = try hb_json:decode(Body) catch _:_ -> #{<<"raw">> => Body} end, + Content = extract_content(Parsed), + {ok, Res#{ <<"content">> => Content, <<"raw">> => Parsed }} + end; + Other -> Other + end. + +%% Embed — POST /v1/embeddings +embed(Base, Req, Opts) -> + Endpoint = resolve_endpoint(Base, Req, Opts, embed), + Model = get_val(<<"model">>, [Req, Base], <<"nomic-embed-text">>, Opts), + Input = get_val(<<"input">>, [Req, Base], get_val(<<"prompt">>, [Req, Base], get_val(<<"data">>, [Req, Base], undefined, Opts), Opts), Opts), + case Input of + undefined -> {error, <<"missing input/prompt/data for embeddings">>}; + _ -> + Body = hb_json:encode(#{ + <<"model">> => Model, + <<"input">> => Input + }), + Headers = #{<<"content-type">> => <<"application/json">>}, + case do_request(Endpoint, Body, Headers, Opts) of + {ok, #{ <<"body">> := RespBody } = Res} -> + Parsed = try hb_json:decode(RespBody) catch _:_ -> #{<<"raw">> => RespBody} end, + Ems = case Parsed of + #{<<"data">> := Data} when is_list(Data) -> + [ maps:get(<<"embedding">>, D, []) || D <- Data ]; + #{<<"embeddings">> := E} -> E; + #{<<"embedding">> := E} -> [E]; + _ -> [] + end, + Single = case Ems of [One|_] -> One; [] -> [] end, + {ok, Res#{ + <<"embedding">> => Single, + <<"embeddings">> => Ems, + <<"raw">> => Parsed + }}; + {ok, #{ body := RespBody } = Res} -> + Parsed = try hb_json:decode(RespBody) catch _:_ -> #{<<"raw">> => RespBody} end, + Ems = case Parsed of + #{<<"data">> := Data} when is_list(Data) -> + [ maps:get(<<"embedding">>, D, []) || D <- Data ]; + #{<<"embeddings">> := E} -> E; + #{<<"embedding">> := E} -> [E]; + _ -> [] + end, + Single = case Ems of [One|_] -> One; [] -> [] end, + {ok, Res#{ + <<"embedding">> => Single, + <<"embeddings">> => Ems, + <<"raw">> => Parsed + }}; + Err -> Err + end + end. + +embeddings(Base, Req, Opts) -> embed(Base, Req, Opts). + +%% Internals + +resolve_endpoint(Base, Req, Opts, Type) -> + Override = case Type of + embed -> get_val(<<"embed-endpoint">>, [Req, Base], undefined, Opts); + chat -> get_val(<<"endpoint">>, [Req, Base], undefined, Opts) + end, + case Override of + undefined -> + BaseEndpoint = get_val(<<"llm-endpoint">>, [Base], undefined, Opts), + DefaultChat = <<"http://spark-1b7b.local:8888/v1/chat/completions">>, + DefaultEmbed = <<"http://spark-1b7b.local:8888/v1/embeddings">>, + case {BaseEndpoint, Type} of + {undefined, chat} -> DefaultChat; + {undefined, embed} -> DefaultEmbed; + {EP, chat} -> EP; + {EP, embed} -> + case get_val(<<"llm-embed-endpoint">>, [Base], undefined, Opts) of + undefined -> + case binary:match(EP, <<"/chat/completions">>) of + {Pos, _} -> <<(binary:part(EP, 0, Pos))/binary, "/embeddings">>; + nomatch -> + case binary:match(EP, <<"/v1">>) of + nomatch -> DefaultEmbed; + _ -> <> + end + end; + E2 -> E2 + end + end; + EP -> EP + end. + +build_chat_body(Req, Model, Stream, Opts) -> + build_chat_body(#{}, Req, Model, Stream, Opts). + +build_chat_body(Base, Req, Model, Stream, Opts) -> + Prompt = get_val(<<"prompt">>, [Req, Base], undefined, Opts), + MessagesRaw0 = get_val(<<"messages">>, [Req, Base], undefined, Opts), + MessagesRaw = deep_deref(MessagesRaw0, Opts), + Data = get_val(<<"data">>, [Req, Base], undefined, Opts), + Messages = case {MessagesRaw, Prompt, Data} of + {undefined, undefined, undefined} -> [#{<<"role">> => <<"user">>, <<"content">> => <<>>}]; + {undefined, P, _} when P =/= undefined -> [#{<<"role">> => <<"user">>, <<"content">> => hb_util:bin(P)}]; + {undefined, undefined, D} when D =/= undefined -> [#{<<"role">> => <<"user">>, <<"content">> => hb_util:bin(D)}]; + {M, _, _} when is_binary(M) -> + try hb_json:decode(M) catch _:_ -> [#{<<"role">> => <<"user">>, <<"content">> => M}] end; + {M, _, _} -> deep_deref(M, Opts) + end, + Tools0 = get_val(<<"tools">>, [Req, Base], undefined, Opts), + Tools = deep_deref(Tools0, Opts), + ToolChoice = get_val(<<"tool_choice">>, [Req, Base], get_val(<<"tool-choice">>, [Req, Base], undefined, Opts), Opts), + HasRawBody = get_val(<<"body">>, [Req, Base], undefined, Opts), + case HasRawBody of + undefined -> + BaseBody = #{ + <<"model">> => Model, + <<"messages">> => Messages, + <<"stream">> => Stream + }, + BodyWithTools = case Tools of + undefined -> BaseBody; + _ -> BaseBody#{ <<"tools">> => Tools } + end, + BodyFinal = case ToolChoice of + undefined -> BodyWithTools; + _ -> BodyWithTools#{ <<"tool_choice">> => ToolChoice } + end, + hb_json:encode(BodyFinal); + B when is_binary(B) -> B; + B when is_map(B) -> hb_json:encode(B#{ <<"stream">> => Stream }); + _ -> + BaseBody2 = #{<<"model">> => Model, <<"messages">> => Messages, <<"stream">> => Stream}, + BodyWithTools2 = case Tools of + undefined -> BaseBody2; + _ -> BaseBody2#{ <<"tools">> => Tools } + end, + hb_json:encode(BodyWithTools2) + end. + +is_stream(Req, Opts) -> + V = get_val(<<"stream">>, [Req], get_val(<<"Stream">>, [Req], false, Opts), Opts), + case V of + true -> true; + <<"true">> -> true; + <<"1">> -> true; + 1 -> true; + _ -> false + end. + +do_request(Endpoint, Body, Headers, Opts) -> + % Use hb_http:request/2 with a message containing path/method/body. + % This bypasses dev_relay's is_blocked_host check and allows localhost. + ReqMsg = #{ + <<"path">> => Endpoint, + <<"method">> => <<"POST">>, + <<"body">> => Body + }, + % Merge headers into request message (content-type already in Headers) + ReqWithHeaders = maps:merge(ReqMsg, Headers), + case hb_http:request(ReqWithHeaders, Opts#{ <<"http-only-result">> => false }) of + {ok, Res} when is_map(Res) -> + % Normalize to #{<<"body">> => ..., <<"status">> => ...} shape + Status = maps:get(<<"status">>, Res, maps:get(status, Res, 200)), + RespBody = maps:get(<<"body">>, Res, maps:get(body, Res, <<>>)), + RespHeaders = maps:get(<<"headers">>, Res, maps:get(headers, Res, #{})), + {ok, #{ <<"status">> => Status, <<"body">> => RespBody, <<"headers">> => RespHeaders, <<"raw">> => Res }}; + {error, _} = Err -> Err; + Other -> {error, #{ <<"body">> => hb_util:bin(Other) }} + end. + +get_val(Key, Maps, Default, Opts) when is_list(Maps) -> + case Maps of + [] -> Default; + [H|T] -> get_val(Key, H, get_val(Key, T, Default, Opts), Opts) + end; +get_val(Key, Map, Default, Opts) when is_map(Map) -> + % Use hb_maps:find to handle link dereferencing via hb_cache + case hb_maps:find(Key, Map, Opts) of + {ok, V} -> V; + error -> + AtomKey = try binary_to_existing_atom(Key, utf8) catch _:_ -> undefined end, + case AtomKey of + undefined -> Default; + _ -> case hb_maps:find(AtomKey, Map, Opts) of {ok, V} -> V; error -> Default end + end + end; +get_val(_, _, Default, _) -> Default. + +deep_deref(V, Opts) when is_map(V) -> + % Deref top-level if it's a link + case (catch hb_cache:ensure_loaded(V, Opts)) of + V2 when is_map(V2), V2 =/= V -> deep_deref(V2, Opts); + _ -> + maps:from_list([{K, deep_deref(Val, Opts)} || {K, Val} <- maps:to_list(V)]) + end; +deep_deref(V, Opts) when is_list(V) -> + [deep_deref(X, Opts) || X <- V]; +deep_deref(V, Opts) -> + case (catch hb_cache:ensure_loaded(V, Opts)) of + V2 when V2 =/= V -> deep_deref(V2, Opts); + _ -> V + end. + +extract_content(#{ <<"choices">> := [#{ <<"message">> := #{ <<"content">> := C }}|_] }) -> C; +extract_content(#{ <<"choices">> := [#{ <<"delta">> := #{ <<"content">> := C }}|_] }) -> C; +extract_content(#{ <<"content">> := C }) -> C; +extract_content(_) -> <<>>. + +normalize_model(<<"qwen3.6">>) -> <<"unsloth/Qwen3.6-35B-A3B-NVFP4">>; +normalize_model(<<"qwen3.6:27b">>) -> <<"unsloth/Qwen3.6-35B-A3B-NVFP4">>; +normalize_model(<<"qwen3.6:27b-coding-nvfp4">>) -> <<"unsloth/Qwen3.6-35B-A3B-NVFP4">>; +normalize_model(M) -> M. + +%% Tests (run with rebar3 eunit) +-ifdef(TEST). +resolve_endpoint_test() -> + ?assertEqual(<<"http://spark-1b7b.local:8888/v1/chat/completions">>, resolve_endpoint(#{}, #{}, #{}, chat)). +resolve_endpoint_embed_default_test() -> + ?assertEqual(<<"http://spark-1b7b.local:8888/v1/embeddings">>, resolve_endpoint(#{}, #{}, #{}, embed)). +resolve_endpoint_override_test() -> + ?assertEqual(<<"http://localhost:8000/v1/chat/completions">>, resolve_endpoint(#{}, #{<<"endpoint">> => <<"http://localhost:8000/v1/chat/completions">>}, #{}, chat)). +build_chat_body_prompt_test() -> + Body = build_chat_body(#{<<"prompt">> => <<"hello">>}, <<"llama3.2">>, false, #{}), + Decoded = hb_json:decode(Body), + ?assertEqual(<<"llama3.2">>, maps:get(<<"model">>, Decoded)), + ?assertEqual([#{<<"role">> => <<"user">>, <<"content">> => <<"hello">>}], maps:get(<<"messages">>, Decoded)). +is_stream_true_test() -> + ?assertEqual(true, is_stream(#{<<"stream">> => <<"true">>}, #{})), + ?assertEqual(false, is_stream(#{}, #{})). +-endif. diff --git a/src/preloaded/llm/llm_sidecar.lua b/src/preloaded/llm/llm_sidecar.lua new file mode 100644 index 0000000000..934fcbd0ef --- /dev/null +++ b/src/preloaded/llm/llm_sidecar.lua @@ -0,0 +1,64 @@ +-- llm_sidecar.lua — AO sidecar for dev_llm@1.0 +-- Load into your LLM_PID process: aos LLM_PID < llm_sidecar.lua +-- Supports: Chat, Generate, Embed (streaming for Chat) + +LLM_MODEL = LLM_MODEL or "llama3.2" +EMBED_MODEL = EMBED_MODEL or "nomic-embed-text" + +Handlers.add("llm-chat", Handlers.utils.hasMatchingTag("Action", "Chat"), function(msg) + local prompt = msg.Tags.Prompt or msg.Data or "" + local messages = msg.Tags.Messages + local endpoint = msg.Tags.Endpoint + local model = msg.Tags.Model or LLM_MODEL + local stream = msg.Tags.Stream == "true" + + -- Delegate to device — device does hb_http to localhost + local device = "llm@1.0" + local res + if stream then + res = ao.resolve({ Device = device, Action = "chat", Prompt = prompt, Messages = messages, Model = model, Stream = "true", Endpoint = endpoint }) + local body = res.Body or res.Data or "" + for chunk in string.gmatch(body, "data: ([^%n]+)") do + if chunk ~= "[DONE]" then + local ok, parsed = pcall(function() return require("json").decode(chunk) end) + local content = "" + if ok and parsed.choices and parsed.choices[1] and parsed.choices[1].delta then + content = parsed.choices[1].delta.content or "" + end + if content ~= "" then + Send({ Target = msg.From, Tags = { ["Stream-Chunk"] = "true" }, Data = content }) + end + end + end + Send({ Target = msg.From, Tags = { ["Stream-Done"] = "true" }, Data = "done" }) + else + res = ao.resolve({ Device = device, Action = "chat", Prompt = prompt, Messages = messages, Model = model, Endpoint = endpoint }) + Send({ Target = msg.From, Data = res and (res.Data or res.body or "") or "" }) + end +end) + +Handlers.add("llm-generate", Handlers.utils.hasMatchingTag("Action", "Generate"), function(msg) + local prompt = msg.Tags.Prompt or msg.Data or "" + local model = msg.Tags.Model or LLM_MODEL + local res = ao.resolve({ Device = "llm@1.0", Action = "generate", Prompt = prompt, Model = model }) + local content = res.content or res.Content or (res.raw and res.raw.choices and res.raw.choices[1].message.content) or "" + Send({ Target = msg.From, Tags = { Content = content }, Data = content }) +end) + +Handlers.add("llm-embed", function(msg) + return msg.Tags.Action == "Embed" or msg.Tags.Action == "Embeddings" +end, function(msg) + local input = msg.Tags.Input or msg.Tags.Prompt or msg.Data or "" + local model = msg.Tags.Model or EMBED_MODEL + local endpoint = msg.Tags.Endpoint + local res = ao.resolve({ Device = "llm@1.0", Action = "embed", Input = input, Model = model, ["embed-endpoint"] = endpoint }) + local embedding = res.embedding or {} + local embeddings = res.embeddings or {} + Send({ + Target = msg.From, + Tags = { ["Embedding-Count"] = tostring(#embeddings) }, + Data = require("json").encode({ embedding = embedding, embeddings = embeddings, raw = res.raw }) + }) +end) + +print("LLM sidecar loaded — Actions: Chat, Generate, Embed (stream=true supported)")