Skip to content

fix: encode subresolve directives in dev_lua - #1063

Open
jim-toth wants to merge 1 commit into
permaweb:edgefrom
anyone-protocol:fix/dev-lua-encode-subresolve
Open

fix: encode subresolve directives in dev_lua#1063
jim-toth wants to merge 1 commit into
permaweb:edgefrom
anyone-protocol:fix/dev-lua-encode-subresolve

Conversation

@jim-toth

@jim-toth jim-toth commented Aug 7, 2026

Copy link
Copy Markdown

Summary

dev_lua:do_encode/2 passes any term it does not recognize to luerl:encode/2, whose final clause is error({badarg, Term}). An {as, Device, Msg} subresolve directive is such a term, and dev_meta puts one in the body of every hook invocation whose request path names a device. So lua@5.3a cannot serve as an on/request or on/response hook for any path using the ~Device syntax.

Reproduction

The existing lua_http_hook_test setup, varying only the path:

path first parsed element edge
/hello?hello=world #{<<"hello">> => <<"world">>} ok
/~meta@1.0/info {as,<<"meta@1.0">>,#{}} badarg
/~message@1.0&hello=world/hello {as,<<"message@1.0">>,...} badarg
/hello~message@1.0 {as,<<"message@1.0">>,...} badarg
status 500
body   Erlang error while running Lua: {badarg,{as,<<"meta@1.0">>,...}}
trace  luerl:encode/2

A hook exists to inspect requests before they resolve, and the requests it cannot see are exactly the ones addressed to a device.

This is what makes it worth fixing rather than working around. A request hook is where a node authenticates or prices a request before it resolves, and it has what it needs to do so: on a signed request the script can read req.request.commitments, each carrying its commitment-device, committed keys and committer address. What it cannot do is run on ~Device paths at all, which is most of the addressable surface of a node.

The fix

Two clauses ahead of the catch-all, encoding the directive as hb_ao:subresolve/4 interprets it: the device set upon the message, and undefined meaning "leave the device alone", mirroring subresolve/4's own undefined -> Base. hb_format:do_term/3 already carries this same pair.

The device is applied with hb_ao:set/4, the call subresolve/4 itself makes, rather than by direct map syntax, leaving dev_message:set/3 to decide what becomes of any commitments rather than assuming there are none. To be clear about what that is and is not doing: instrumenting the encoder across the full suite shows the directive always carries a plain message here, commitments travelling on the hook's request key rather than on the parsed sequence, so this is not correcting an observed invalidation. The directive is also rare enough for the call to cost nothing measurable — four reach the encoder across 3,508 tests. dev_lua:info/1 lists set in its excludes, so it cannot re-enter the Lua device.

Rendering any unencodable tuple as a string also clears the crash, but it would turn the {error, {device_not_loadable, _}} that a device-sandbox denial produces into an ordinary string the script reads and returns successfully. ao_core_sandbox_test catches this, so these clauses are deliberately narrow.

Per CONTRIBUTING rule 2, there is no application-layer fix available: the failure happens while encoding the arguments, before any Lua runs.

Scope

encode/2 has a single call site, supplying the arguments to luerl:call_function_dec/3. Every {as, ...} that reached the encoder therefore reached luerl:encode/2 and raised, so this can only alter paths that raised before it.

It also cannot affect a process's recorded history. The directive is an in-memory resolution artifact that no message codec carries: converting to structured@1.0 or httpsig@1.0 drops the key, signing leaves it out of committed, and a cache write and read-back returns the message without it. It cannot be part of a signed assignment, and dev_process passes the execution device the assignment rather than the parsed path, so it does not reach compute.

Testing

lua_http_hook_subresolve_test drives GET /~meta@1.0/info against a real node and asserts the hook script received meta@1.0, covering the encoding rather than just the absence of a crash. It fails on unmodified edge with the 500 above.

rebar3 device test --devices dev_lua: 39/39, including ao_core_sandbox_test.

rebar3 eunit-all against edge at 2208327f: 3,502 passed, 5 failed. This branch: 3,503 passed, 5 failed — the same 5 (four scheduler@1.0 http_get_legacy_*, plus push@1.0: test_push_prompts_encoding_change), the extra pass being the test added here.

On rule 1, since it names flakes: a second full-suite run of this branch showed a sixth failure, bundler@1.0: await_bundle_test_parallel. It is nondeterministic on unmodified edgerebar3 device test --devices dev_bundler four times against 2208327f gave one failure and three clean runs, and twice against this branch gave two clean runs. Test totals are stable across every run: edge 3,507, this branch 3,508.

@jim-toth
jim-toth force-pushed the fix/dev-lua-encode-subresolve branch from 9ad60d3 to ae7dc5c Compare August 10, 2026 23:41
`do_encode/2` passed any term it did not recognize straight to `luerl:encode/2`,
whose final clause is `error({badarg, Term})`. An `{as, Device, Msg}` subresolve
directive is such a term, and `dev_meta` places one in the `body` of every hook
invocation whose request path names a device. So `lua@5.3a` could not be used as
a device on `on/request` or `on/response` for any request using the `~Device`
path syntax. `/~meta@1.0/info` is among the most basic URLs HyperBEAM serves and
it took the hook down with a 500 before any Lua ran. A hook exists to inspect
requests before they resolve, and the requests it could not see were exactly the
ones addressed to a device.

Two clauses ahead of the catch-all encode the directive the way
`hb_ao:subresolve/4` interprets it: `undefined` leaves the device alone,
mirroring its own `undefined -> Base`, and a named device is set upon the
message. A script then receives an ordinary message carrying a device, which is
what `hb_singleton` documents `Part~Device` to expand to. Overwriting an
existing `device` is the semantics rather than a collision.

The device is applied with `hb_ao:set/4`, the call `subresolve/4` itself makes,
rather than by direct map syntax. This is not correcting an observed commitment
invalidation: instrumenting the encoder across the full suite shows the
directive always carries a plain message here, commitments travelling on the
hook's `request` key rather than on the parsed sequence. It leaves the question
to `dev_message:set/3` instead of assuming the answer, and the directive is rare
enough for that to cost nothing measurable -- four reach the encoder across
3,508 tests. `dev_lua:info/1` lists `set` in its `excludes`, so the call cannot
re-enter the Lua device.

Guarded on `is_map` because that is the only shape produced, and
`hb_singleton:do_build/4` guards on it too, so anything else falls through to
the existing clause and behaves as it did before.

Rendering any unencodable tuple as a string also clears the crash and is the
more obvious fix, but it is not taken here: it converts the
`{error, {device_not_loadable, _}}` that a `device-sandbox` denial produces from
a hard failure into an ordinary string that the script reads and returns
successfully. `ao_core_sandbox_test` catches this. Softening a sandbox denial is
a security-relevant change that this fix does not need to make, so the clauses
are deliberately narrow.

This reaches only hooks and other request-time devices, which is where these
terms live. It cannot affect a process's recorded history: the directive is an
in-memory resolution artifact and no message codec carries it. Converting to
`structured@1.0` or `httpsig@1.0` drops the key, signing leaves it out of
`committed`, and a cache write and read-back returns the message without it.
Since it cannot be part of a signed assignment, and `dev_process` passes the
execution device the assignment rather than the parsed path, it does not reach
`compute`.

`lua_http_hook_subresolve_test` covers the encoding rather than the absence of a
crash: the hook script returns the `device` it saw on the first parsed message,
driven by `GET /~meta@1.0/info`, and the test asserts it received `meta@1.0`.
@jim-toth
jim-toth force-pushed the fix/dev-lua-encode-subresolve branch from ae7dc5c to c2c36e5 Compare August 11, 2026 00:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant