Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,34 @@ described by traditional `combinator` systems.
If a `message` does not explicitly specify a `device`, its implied `device` is a
`message@1.0`, which simply returns the binary or `message` at a given named function.

## Agents — `Agent = LLM + Harness + Tools + Instructions`

HyperBEAM now ships a **generic agent stack** per [What is an Agent?](https://chat.hyper.io/share/wE41XruWrXFj37EdGfFxn3sdXy0Wib9e) and [Agent Flow](https://chat.hyper.io/share/_4kT2xkDzGpPncXP5k6wx8zB5m4LKlnT):

```
Agent = LLM (engine) + Harness (loop+memory) + Tools (hands) + Instructions (identity)
```

* **LLM** — `~llm@1.0` OpenAI proxy (Ollama/vLLM, `spark-1b7b.local:8888`, streaming SSE) — reasoning only.
* **Harness** — `~harness@1.0` generic runtime: builds `system(identity.md+soul.md+user.md) + tools.json + history↑limit + current` window, loops `LLM → tool_calls → relay@1.0 → rebuild` until done, persists `history` to `hb_store` (`<collection>-harness-history`, default limit 20). One harness, many agents.
* **Tools** — atomic `relay@1.0` calls (or MCP): `get_gmail_messages`, `fetch`. If in `tools.json`, agent can use it.
* **Instructions** — `agents/<id>/{identity.md,user.md,soul.md,tools.json}` + durable `memory/*.md` (RAM vs files). Skills decouple know-how: `skills@1.0` stores `summarize` etc. `requires_tools` and composes (`research+publish`).

Quick start — datacenter essay agent (research water+space):

```bash
rebar3 compile && rebar3 device preload
# register composable skills
curl -X POST http://localhost:8734/~skills@1.0/register -d @examples/datacenter-essay/skills.json
# run as researcher (harness builds system+history+tools+current)
curl -X POST http://localhost:8734/~skills@1.0/run -d '{"skill":"research-write","agent_tools":["fetch"],"message":"Research Natick + Kepler, write 800w essay","collection":"agent-researcher"}'
# or via aOS
aos researcher < src/preloaded/agent/agent.lua
Send({Target=researcher, Action="RunSkill", Agent="researcher", Skill="research-write", Prompt="water vs space"})
```

See `src/preloaded/agent/{dev_harness.erl,dev_skills.erl,agent.lua,harness.lua}`, `agents/tom/`, and `examples/datacenter-essay/` (`README.md` + `essay.out.md`).

## Devices

HyperBeam supports a number of different devices, each of which enable different
Expand Down Expand Up @@ -211,6 +239,14 @@ used to execute `devices` written in languages such as Rust, C, and C++.
the JSON-encoded message format used by AOS 2.0 and prior versions, to HyperBEAM's
native HTTP message format.

- `~llm@1.0`: OpenAI-compatible proxy for Ollama/vLLM/llama.cpp (`POST /v1/chat/completions`, streaming SSE, local `localhost` allowed). See `docs/devices/llm-at-1-0.md`.

- `~harness@1.0`: Generic agent harness (`dev_harness.erl`) — `handle/run/chat` rebuilds `system + tools + history↑20 + current`, loops `tool_calls` via `relay@1.0` until output, persists to `hb_store`. See `docs/devices/harness-at-1-0.md` and `src/preloaded/agent/agent.lua` (`RunSkill`/`AgentPrompt`).

- `~skills@1.0`: Composable skills registry (`dev_skills.erl`) — `register/get/list/check/run/compose`, enforces `requires_tools ⊆ tools.json` matrix, delegates to harness. See `docs/devices/skills-at-1-0.md`.

- `agent` process example: `src/preloaded/agent/agent.lua` + `agents/tom/{identity.md,user.md,soul.md,tools.json,memory/*.md}` + `examples/datacenter-essay/` (research water+space datacenters, `skills.json` + `essay.out.md`).

- `~compute-lite@1.0`: The `~compute-lite@1.0` device is a lightweight device wrapping
a local WASM executor, used for executing legacynet AO processes inside HyperBEAM.
See the [HyperBEAM OS](https://github.com/permaweb/hb-os) repository for an
Expand Down
147 changes: 147 additions & 0 deletions README_llm.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
# hb-llm-device — `llm@1.0` for HyperBEAM

OpenAI-compatible LLM proxy for HyperBEAM. Use local Ollama, vLLM, or `llama.cpp` — or a remote vLLM like `spark-1b7b.local:8888` — from any AO process or via HTTP. Bypasses `dev_relay` localhost block by calling `hb_http` directly.

- `chat` → `POST /v1/chat/completions` (supports `stream=true` → SSE `text/event-stream`)
- `generate` → `chat` + extracts `choices[0].message.content` → `content` (+ `raw`)
- `embed`/`embeddings` → `POST /v1/embeddings` → `embedding`/`embeddings`/`raw`
- `qwen3.6` alias → `unsloth/Qwen3.6-35B-A3B-NVFP4` (Spark), `stream`, `endpoint`/`embed-endpoint`/`llm-endpoint` overrides

Default endpoint `http://spark-1b7b.local:8888/v1/chat/completions` (Spark `qwen3.6`), default model `unsloth/Qwen3.6-35B-A3B-NVFP4`. Override per-request — also works with local Ollama `http://localhost:11434`.

## Why not just `dev_relay@1.0`?

`dev_relay` blocks `localhost`/`127.0.0.1`/private IPs by default (`relay-block-internal=true`, `hb_hostname:is_public`). This device calls `hb_http:request` directly, so `http://localhost:11434` and `http://spark-1b7b.local:8888` work without `HB_RELAY_BLOCK_INTERNAL=false`. If you *do* want relay, `HB_RELAY_BLOCK_INTERNAL=false rebar3 shell`.

## Install

```bash
# Option A: copy into HyperBEAM (preloaded)
cp src/preloaded/llm/dev_llm.erl /path/to/hyperbeam/src/preloaded/llm/
cp src/preloaded/llm/llm_sidecar.lua /path/to/hyperbeam/src/preloaded/llm/
rebar3 compile
rebar3 device preload # rebuilds _build/preloaded-store, index -> llm@1.0

# Option B: standalone repo as dep
# in your rebar.config: {deps, [{hb_llm_device, {git, "https://github.com/twilson63/hb-llm-device.git", {branch, "main"}}}]}
```

## Run a local AI with Ollama

```bash
# Install https://ollama.com
curl -fsSL https://ollama.com/install.sh | sh

# Start server ( :11434 )
ollama serve &
# Pull models
ollama pull llama3.2 # 3B default for Ollama
ollama pull qwen3.6:27b-coding-nvfp4 # or use Spark's qwen3.6 (35B) via spark-1b7b.local:8888
ollama pull nomic-embed-text # for embeddings
ollama list # should show llama3.2, qwen3.6, nomic-embed-text, glm-5.2:cloud etc.

# Verify Ollama API
curl http://localhost:11434/v1/models
curl http://localhost:11434/v1/chat/completions -H "Content-Type: application/json" \
-d '{"model":"llama3.2","messages":[{"role":"user","content":"Say hi in 3 words"}],"stream":false}'
```

**Alternatives:**

```bash
# vLLM OpenAI-compatible (e.g. Spark)
python -m vllm.entrypoints.openai.api_server --model unsloth/Qwen3.6-35B-A3B-NVFP4 --port 8888 --host 0.0.0.0
# llama.cpp
llama-server -m qwen3.6-35b.gguf --port 8080 --host 127.0.0.1
```

## Test

```bash
# 1. Simple prompt (GET, default Spark qwen3.6) — no endpoint needed
curl "http://localhost:8734/~llm@1.0/chat?prompt=Write+a+haiku+about+AO"

# 2. Local Ollama llama3.2 (explicit endpoint)
curl "http://localhost:8734/~llm@1.0/chat?prompt=hello&endpoint=http://localhost:11434/v1/chat/completions&model=llama3.2"
curl -X POST http://localhost:8734/~llm@1.0/chat -H "content-type: application/json" \
-d '{"messages":[{"role":"user","content":"What is HyperBEAM?"}],"model":"llama3.2","endpoint":"http://localhost:11434/v1/chat/completions"}'

# 3. Spark qwen3.6 via alias (default)
curl "http://localhost:8734/~llm@1.0/chat?prompt=hello&model=qwen3.6"
# or explicit full:
curl "http://localhost:8734/~llm@1.0/chat?prompt=hello&endpoint=http://spark-1b7b.local:8888/v1/chat/completions&model=unsloth/Qwen3.6-35B-A3B-NVFP4"

# 4. Streaming SSE
curl "http://localhost:8734/~llm@1.0/chat?prompt=Count+1+to+3&stream=true&model=qwen3.6" # → data: {"delta":{"content":"1"}} … [DONE]

# 5. Embeddings (Ollama local)
curl -X POST http://localhost:8734/~llm@1.0/embed -H "content-type: application/json" \
-d '{"input":"hello world","model":"nomic-embed-text","embed-endpoint":"http://localhost:11434/v1/embeddings"}'

# 6. Via AO process (set default on process)
# hb message commit --process <PROC> '{"device":"llm@1.0","llm-endpoint":"http://localhost:11434/v1/chat/completions","model":"llama3.2"}'
# or for Spark: '{"llm-endpoint":"http://spark-1b7b.local:8888/v1/chat/completions","model":"qwen3.6"}'
```

## From AO / Lua

Load `src/preloaded/llm/llm_sidecar.lua` into your AO process:

```bash
aos <PROC> < src/preloaded/llm/llm_sidecar.lua
```

```lua
-- non-stream
local res = ao.send({
Device = "llm@1.0",
Action = "chat",
Tags = { Prompt = "Explain AO in one sentence", Model = "qwen3.6" }
})
print(res.Data)

-- streaming (sidecar sends Stream-Chunk / Stream-Done)
Send({ Target = ao.id, Action = "Chat", Prompt = "Count 1 to 5", Stream = "true", Model = "qwen3.6" })

-- or direct ao.resolve (no sidecar)
local status, res = ao.resolve({device='llm@1.0', path='chat', prompt='Say hi', model='qwen3.6', endpoint='http://spark-1b7b.local:8888/v1/chat/completions'})
print(res.body)

-- explicit Ollama endpoint
local status, res = ao.resolve({device='llm@1.0', path='chat', prompt='hi', model='llama3.2', endpoint='http://localhost:11434/v1/chat/completions'})
```

## Endpoints

- `chat` / `completions` — `POST /v1/chat/completions` passthrough (`messages` or `prompt`/`data` → `messages`, `model`, `stream`)
- `generate` — same as `chat` but extracts `choices[0].message.content` → `content` for easier AO use
- `embed` / `embeddings` — `POST /v1/embeddings` → `embedding`/`embeddings`

Overrides per-request: `endpoint` (chat), `embed-endpoint` (embed), `model`, `prompt`/`messages`/`data`, `stream` (`true`/`1`), or process Base `llm-endpoint`/`llm-embed-endpoint`.

## Security

This device can `POST` to any `endpoint` you pass — including `localhost` and private LAN `spark-1b7b.local`. Do not expose a node running it to the public internet without auth. Put it behind `hb` admin (`--admin`) or a reverse proxy, or restrict `llm-endpoint` to an allowlist.

## Forge publish

```bash
rebar3 device publish --device-src src/preloaded/llm --verbose
# dry-run:
rebar3 device publish --device-src src/preloaded/llm --dry-run --verbose
# Spec: PD89PcLv_ilAtTxDOPtbKLXNq4be5eBCMXJi1BS7KsY, Impl: RFJL5SpFbLN1X3LuqYnPDw4W02n5c4l8HGu-17tB-N8 (v1)
```

Published via `httpsig@1.0`, verifiable on Arweave. Others can `hb_ao:resolve({device=><<"llm@1.0">>})` after indexing or pin spec ID.

## Tests

```bash
rebar3 eunit --module dev_llm_test # 12 mock + unit, no network
LLM_LIVE=1 rebar3 eunit --module dev_llm_test # hits live Spark qwen3.6 (or Ollama)
rebar3 eunit # full 987
```

## License

Apache-2.0
6 changes: 6 additions & 0 deletions agents/tom/identity.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Tom — Agent Identity

- Name: Tom
- Role: Founder, hyper.io — Charleston, SC
- Voice: Concise, direct, ships boring practical systems
- Job: Help Tom run hyper.io, close loops, publish, brief the team
5 changes: 5 additions & 0 deletions agents/tom/memory/durable.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Durable Memory — Tom

- 2026-08-07: Prefers short posts
- 2026-08-07: Acme Corp = prospect
- 2026-08-07: Closed loop on hyper.io publish
4 changes: 4 additions & 0 deletions agents/tom/memory/open-loops.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Open Loops

- [ ] Publish hyper essay follow-up
- [ ] Morning brief automation
5 changes: 5 additions & 0 deletions agents/tom/memory/work.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# How Tom Likes to Work

- Morning brief via summarize skill + Gmail
- Publish via publish skill
- Inbox-zero requires gmail_read
7 changes: 7 additions & 0 deletions agents/tom/soul.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Guiding Principles — Soul

- Do not hallucinate tools. If tool not in tools.json, say cannot.
- Keep instructions explicit, composable, auditable.
- Never expose private memory without permission.
- Prefer boring, practical definitions that ship.
- Memory is durable: update agents/tom/memory/*.md after each run.
1 change: 1 addition & 0 deletions agents/tom/tools.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
["gmail_read", "gmail_send", "calendar_read", "drive_read"]
6 changes: 6 additions & 0 deletions agents/tom/user.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# User Context — Tom Wilson

- Prefers short posts, tight briefs, bullet points
- Acme Corp is a prospect, not a customer (remember)
- Hyper.io closed loop last week — follow up on publish skill
- Timezone: America/New_York
76 changes: 76 additions & 0 deletions docs/devices/harness-at-1-0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# Device: ~harness@1.0

## Overview

The [`~harness@1.0`](../../src/preloaded/agent/dev_harness.erl) device is the **generic agent runtime** for HyperBEAM. It implements the essay model **Agent = LLM + Harness + Tools + Instructions** — one harness, many agents.

Source: `src/preloaded/agent/dev_harness.erl` (`-implements(<<"harness@1.0">>)`), tests via `src/core/test/dev_llm_test.erl` + harness loop integration. Published via `forge` as `harness@1.0` (Spec `r09P4ZkjHMtGIDGRXv_43_CWs2PEwENlPvJwTcf1X-4`, Impl `2thbQP1hIWr48fo3jspVbm6gCMBoHGRh54fnCcXCZwM`, Signer `aa0b-vzWf7Sn4cFKI43P4MUcSgAdAjCH2V2IFTKZGfU`).

It orchestrates `llm@1.0` + `relay@1.0` (tools) + `hb_store` (FS/cache) + `query@1.0` + `lua@5.3a`. The `dan-feed` is just one test dataset (`collection=dan`).

## Core Concept: The Loop

Every turn the harness rebuilds the **context window** as defined in [Understanding the Agent Flow](https://chat.hyper.io/share/_4kT2xkDzGpPncXP5k6wx8zB5m4LKlnT):

1. **System** — `identity.md` + `soul.md` + `user.md` + `system` (combined as `role:system`)
2. **Tools** — `tools.json` → OpenAI `tools` spec
3. **History** — previous `inputs/outputs` from `hb_store` (`<collection>-harness-history`), truncated to `history_limit` (default 20, harness-managed)
4. **Current input** — `message|prompt|data` from any source (chat, schedule, Drive/Notion trigger, agent msg, API)

Then:

```
LLM → tool_calls|output → harness runs each tool via relay@1.0/call → append results → rebuild → LLM ... until no tool_calls → output + persist history
```

That loop *is* the agent. Not one call — a sequence.

## Key Functions

* **`handle` / `run` / `chat` / `execute`** — alias for the agentic loop.
* **Inputs:** `message|prompt|data|input` (current), `history|messages` (explicit) or `collection`+`history_key` (load from `hb_store`), `tools` (OpenAI spec or `["fetch"]` shorthand via `skills@1.0`), `system|identity|soul|user|instructions` (combined to `role:system`, not persisted), `model` (default `qwen3.6`), `endpoint|llm-endpoint`, `history_limit|max_history` (default 20), `max_iterations` (default 10), `collection` (default `default`).
* **Response:** `{ok, #{<<"output">>:=Binary, <<"history">>:=Messages, <<"messages">>:=Messages, <<"iterations">>:=N, <<"system">>:=SystemMsg, <<"raw">>:=LLMJson}}` + `hb_store:write` of `history` (without system) to `<collection>-harness-history`.
* **Example:** `POST /~harness@1.0/handle {"message":"hello","tools":[...],"collection":"agent-tom","system":"You are Tom..."}`

* **`fetch` / `fetch_feed`** — `GET <url>` via `relay@1.0/call` (public; dan-feed default).

* **`parse`** — `body` (RSS XML) → `[{guid,title,link,description}]` (regex, no xmerl).

* **`store`** — `posts|items|body` → `hb_store:write` under `<collection>-<id>` + `<collection>-index`.

* **`ingest`** — `url|collection` → `fetch_via_relay` → `parse_feed` (or JSON fallback) → `store`.

* **`list` / `query`** — `collection-index` + per-item reads, `q` filter on `title|description|link`.

## AO / Lua Usage

```lua
-- Generic: any collection, any URL via relay + harness
ao.resolve({device="harness@1.0", path="ingest", url="https://hyperio-mc.github.io/dan-feed/feed.xml", collection="dan"})
ao.resolve({device="harness@1.0", path="query", q="space", collection="dan"})

-- Agentic loop: system = identity+soul+user, history managed, tools via relay
local ok, res = ao.resolve({
device="harness@1.0", path="handle",
message="Research water vs space datacenters",
system="Identity: researcher\nPrinciples: boring ships",
identity="...", soul="...", user="...",
tools={{type="function", ["function"]={name="fetch", parameters={type="object", properties={["relay-path"]={type="string"}}}}}},
collection="agent-researcher",
model="qwen3.6", history_limit=20
})
print(res.output)
-- history persisted to hb_store, system not persisted
```

Via `skills@1.0` (recommended):

```lua
ao.resolve({device="skills@1.0", path="run", skill="research-write", agent_tools={"fetch"}, message="write essay", collection="agent-tom", identity=..., soul=..., user=...})
```

See `src/preloaded/agent/agent.lua` (`RunSkill`/`AgentPrompt`) and `examples/datacenter-essay/` for full Agent = LLM+Harness+Tools+Instructions wiring.

## Security

`relay@1.0` blocks private hosts; `llm@1.0` does not (it uses `hb_http` directly). The harness enforces `history_limit` to bound context.
Loading