fix: encode subresolve directives in dev_lua - #1063
Open
jim-toth wants to merge 1 commit into
Open
Conversation
jim-toth
force-pushed
the
fix/dev-lua-encode-subresolve
branch
from
August 10, 2026 23:41
9ad60d3 to
ae7dc5c
Compare
`do_encode/2` passed any term it did not recognize straight to `luerl:encode/2`,
whose final clause is `error({badarg, Term})`. An `{as, Device, Msg}` subresolve
directive is such a term, and `dev_meta` places one in the `body` of every hook
invocation whose request path names a device. So `lua@5.3a` could not be used as
a device on `on/request` or `on/response` for any request using the `~Device`
path syntax. `/~meta@1.0/info` is among the most basic URLs HyperBEAM serves and
it took the hook down with a 500 before any Lua ran. A hook exists to inspect
requests before they resolve, and the requests it could not see were exactly the
ones addressed to a device.
Two clauses ahead of the catch-all encode the directive the way
`hb_ao:subresolve/4` interprets it: `undefined` leaves the device alone,
mirroring its own `undefined -> Base`, and a named device is set upon the
message. A script then receives an ordinary message carrying a device, which is
what `hb_singleton` documents `Part~Device` to expand to. Overwriting an
existing `device` is the semantics rather than a collision.
The device is applied with `hb_ao:set/4`, the call `subresolve/4` itself makes,
rather than by direct map syntax. This is not correcting an observed commitment
invalidation: instrumenting the encoder across the full suite shows the
directive always carries a plain message here, commitments travelling on the
hook's `request` key rather than on the parsed sequence. It leaves the question
to `dev_message:set/3` instead of assuming the answer, and the directive is rare
enough for that to cost nothing measurable -- four reach the encoder across
3,508 tests. `dev_lua:info/1` lists `set` in its `excludes`, so the call cannot
re-enter the Lua device.
Guarded on `is_map` because that is the only shape produced, and
`hb_singleton:do_build/4` guards on it too, so anything else falls through to
the existing clause and behaves as it did before.
Rendering any unencodable tuple as a string also clears the crash and is the
more obvious fix, but it is not taken here: it converts the
`{error, {device_not_loadable, _}}` that a `device-sandbox` denial produces from
a hard failure into an ordinary string that the script reads and returns
successfully. `ao_core_sandbox_test` catches this. Softening a sandbox denial is
a security-relevant change that this fix does not need to make, so the clauses
are deliberately narrow.
This reaches only hooks and other request-time devices, which is where these
terms live. It cannot affect a process's recorded history: the directive is an
in-memory resolution artifact and no message codec carries it. Converting to
`structured@1.0` or `httpsig@1.0` drops the key, signing leaves it out of
`committed`, and a cache write and read-back returns the message without it.
Since it cannot be part of a signed assignment, and `dev_process` passes the
execution device the assignment rather than the parsed path, it does not reach
`compute`.
`lua_http_hook_subresolve_test` covers the encoding rather than the absence of a
crash: the hook script returns the `device` it saw on the first parsed message,
driven by `GET /~meta@1.0/info`, and the test asserts it received `meta@1.0`.
jim-toth
force-pushed
the
fix/dev-lua-encode-subresolve
branch
from
August 11, 2026 00:35
ae7dc5c to
c2c36e5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
dev_lua:do_encode/2passes any term it does not recognize toluerl:encode/2, whose final clause iserror({badarg, Term}). An{as, Device, Msg}subresolve directive is such a term, anddev_metaputs one in thebodyof every hook invocation whose request path names a device. Solua@5.3acannot serve as anon/requestoron/responsehook for any path using the~Devicesyntax.Reproduction
The existing
lua_http_hook_testsetup, varying only the path:edge/hello?hello=world#{<<"hello">> => <<"world">>}/~meta@1.0/info{as,<<"meta@1.0">>,#{}}badarg/~message@1.0&hello=world/hello{as,<<"message@1.0">>,...}badarg/hello~message@1.0{as,<<"message@1.0">>,...}badargA hook exists to inspect requests before they resolve, and the requests it cannot see are exactly the ones addressed to a device.
This is what makes it worth fixing rather than working around. A request hook is where a node authenticates or prices a request before it resolves, and it has what it needs to do so: on a signed request the script can read
req.request.commitments, each carrying itscommitment-device,committedkeys andcommitteraddress. What it cannot do is run on~Devicepaths at all, which is most of the addressable surface of a node.The fix
Two clauses ahead of the catch-all, encoding the directive as
hb_ao:subresolve/4interprets it: the device set upon the message, andundefinedmeaning "leave the device alone", mirroringsubresolve/4's ownundefined -> Base.hb_format:do_term/3already carries this same pair.The device is applied with
hb_ao:set/4, the callsubresolve/4itself makes, rather than by direct map syntax, leavingdev_message:set/3to decide what becomes of any commitments rather than assuming there are none. To be clear about what that is and is not doing: instrumenting the encoder across the full suite shows the directive always carries a plain message here, commitments travelling on the hook'srequestkey rather than on the parsed sequence, so this is not correcting an observed invalidation. The directive is also rare enough for the call to cost nothing measurable — four reach the encoder across 3,508 tests.dev_lua:info/1listssetin itsexcludes, so it cannot re-enter the Lua device.Rendering any unencodable tuple as a string also clears the crash, but it would turn the
{error, {device_not_loadable, _}}that adevice-sandboxdenial produces into an ordinary string the script reads and returns successfully.ao_core_sandbox_testcatches this, so these clauses are deliberately narrow.Per CONTRIBUTING rule 2, there is no application-layer fix available: the failure happens while encoding the arguments, before any Lua runs.
Scope
encode/2has a single call site, supplying the arguments toluerl:call_function_dec/3. Every{as, ...}that reached the encoder therefore reachedluerl:encode/2and raised, so this can only alter paths that raised before it.It also cannot affect a process's recorded history. The directive is an in-memory resolution artifact that no message codec carries: converting to
structured@1.0orhttpsig@1.0drops the key, signing leaves it out ofcommitted, and a cache write and read-back returns the message without it. It cannot be part of a signed assignment, anddev_processpasses the execution device the assignment rather than the parsed path, so it does not reachcompute.Testing
lua_http_hook_subresolve_testdrivesGET /~meta@1.0/infoagainst a real node and asserts the hook script receivedmeta@1.0, covering the encoding rather than just the absence of a crash. It fails on unmodifiededgewith the 500 above.rebar3 device test --devices dev_lua: 39/39, includingao_core_sandbox_test.rebar3 eunit-allagainstedgeat2208327f: 3,502 passed, 5 failed. This branch: 3,503 passed, 5 failed — the same 5 (fourscheduler@1.0http_get_legacy_*, pluspush@1.0: test_push_prompts_encoding_change), the extra pass being the test added here.On rule 1, since it names flakes: a second full-suite run of this branch showed a sixth failure,
bundler@1.0: await_bundle_test_parallel. It is nondeterministic on unmodifiededge—rebar3 device test --devices dev_bundlerfour times against2208327fgave one failure and three clean runs, and twice against this branch gave two clean runs. Test totals are stable across every run:edge3,507, this branch 3,508.