Skip to content

Add serial transport - #232

Merged
trond-snekvik merged 1 commit into
mainfrom
transport/serial
Sep 2, 2026
Merged

Add serial transport#232
trond-snekvik merged 1 commit into
mainfrom
transport/serial

Conversation

@trond-snekvik

Copy link
Copy Markdown
Collaborator

Adds the serial transport layer, as outlined in golioth/architecture#85. This does not include a real transport adapter, but includes three test suites that test the serial device, serial broker and a test that covers both together, each using mock serial adapters and endpoints.

There's some cleanup and some hacking in the kconfig menusystem going into this PR. The intent is to just kinda make serial build on both the gateway and the device without making our menuconfig worse. I'll make a separate PR with some more deliberate restructuring.

@trond-snekvik
trond-snekvik force-pushed the transport/serial branch 5 times, most recently from 53bfd2d to b945e26 Compare June 11, 2026 10:11
Comment thread port/zephyr/transport/Kconfig
Comment thread port/zephyr/transport/Kconfig
Comment thread port/zephyr/transport/Kconfig Outdated
Comment thread src/transport/serial/broker.c
Comment thread src/transport/serial/broker.c Outdated
Comment thread src/transport/serial/broker.c
Comment thread src/transport/serial/channel.c
Comment thread src/transport/serial/channel.c Outdated
Comment thread src/transport/serial/device.c
@trond-snekvik
trond-snekvik force-pushed the transport/serial branch 5 times, most recently from b95ce87 to 79ad75f Compare July 6, 2026 14:33
Comment thread src/transport/CMakeLists.txt Outdated
Comment thread include/pouch/transport/serial/broker.h Outdated
Comment thread src/transport/serial/channel.c Outdated
Comment thread src/transport/serial/channel.c
beriberikix added a commit that referenced this pull request Jul 23, 2026
Add an interrupt-driven UART device adapter on the serial transport core for
development and native_sim end-to-end testing. rpmsg is not available on
native_sim, so the same serial device core is exercised over a host pty (the
native-tty UART driver) with the broker running as a host process. A minimal
length-delimited framing (SOF + 2-byte length) recovers frame boundaries on
the byte stream; the serial frame's own 1-byte header carries the channel.

Also add the examples/zephyr/rpmsg_device sample: a Pouch device that reports
telemetry and handles a setting over the serial transport. It builds for
native_sim (build_only) so CI compiles the serial-core + adapter path. The
same device code runs over the rpmsg adapter on real MPU+MCU hardware.

Part of #274. Do not merge until #232 lands.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
beriberikix added a commit that referenced this pull request Jul 23, 2026
The serial device wired the server-cert and device-cert channels
unconditionally, but those endpoints are only compiled when SAEAD encryption
is enabled, so a POUCH_ENCRYPTION_NONE build failed to link with undefined
references to pouch_device_endpoint_server_cert/device_cert. Guard the two
cert channels on CONFIG_POUCH_ENCRYPTION_SAEAD (mirroring the BLE GATT
transport, which only exposes its cert characteristics under SAEAD), and
defensively ignore frames that arrive for a channel with no endpoint.

This lets the serial transport build without encryption, e.g. for the
native_sim rpmsg_device example.

Candidate to fold into the serial transport core (#232).

Part of #274. Do not merge until #232 lands.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
beriberikix added a commit that referenced this pull request Jul 23, 2026
Extract the UART adapter's length-delimited framing (SOF + 2-byte length) into
a pure, dependency-free module (uart_framing.[ch]) with an encoder and a
byte-at-a-time streaming decoder, and use it from uart_device.c. This makes the
framing unit-testable and lets a host-side broker mirror the exact wire format.

Add tests/pouch/rpmsg/exchange (native_sim ztest):
- direct framing tests: encode/decode round-trip, resync after garbage,
  rejection of oversized and malformed frames;
- an end-to-end test that runs a full broker <-> device serial exchange (reusing
  the serial exchange stub endpoints, so no gateway/cloud is needed) with every
  frame passed through encode + streaming decode, asserting all channel payloads
  survive the framing round-trip.

Unlike the coap/http example pytests, this runs in the twister CI without any
backend.

Part of #274. Do not merge until #232 lands.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
beriberikix added a commit that referenced this pull request Jul 23, 2026
…ample

The earlier cert-channel SAEAD guard regressed the serial transport unit tests
(pouch.serial.exchange, pouch.serial.device): those tests build the serial
device core without CONFIG_POUCH_ENCRYPTION_SAEAD but still exercise the
certificate channels through stub endpoints, so guarding the channels on SAEAD
removed endpoints the tests rely on.

Revert src/transport/serial/{device,channel}.c to the #232 base so this PR no
longer modifies the serial core, and drop the native_sim rpmsg_device example
that depended on POUCH_ENCRYPTION_NONE. The serial device requires SAEAD in a
real build (its certificate endpoints are only compiled under SAEAD); a proper
SAEAD-based example (which also compiles the UART/rpmsg adapters) is a
follow-up.

The rpmsg framing + end-to-end exchange test (tests/pouch/rpmsg/exchange) is
unaffected: it drives the real broker/device serial cores with the shared stub
endpoints, so it needs no encryption and exercises the framing round-trip.

Part of #274. Do not merge until #232 lands.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
beriberikix added a commit that referenced this pull request Jul 23, 2026
Add a native_sim example that exercises the serial transport's UART device
adapter with SAEAD end-to-end encryption (Pouch's default), so CI compiles the
UART adapter and the serial device path. The device credentials are embedded
self-signed placeholders (P-256), keeping the example self-contained with no
provisioned filesystem; a real deployment provisions a CA-signed device
certificate (see coap_client/ble_gatt for the filesystem pattern).

The example reports telemetry over the serial transport; on native_sim the link
is a host pty (native-tty UART) and on a real MPU+MCU part the same device code
runs over the rpmsg adapter. Built build_only on native_sim in CI.

Part of #274. Do not merge until #232 lands.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
beriberikix added a commit that referenced this pull request Jul 23, 2026
Capture the work-in-progress harness for validating the Pouch rpmsg (OpenAMP)
transport against a real Linux remoteproc + virtio-rpmsg stack in Renode, with
no hardware. This complements the native_sim tests, which cannot exercise the
ipc_service/OpenAMP adapter.

Includes:
- README documenting the validated approach: platform (AMD Kria KV260 / ZynqMP
  Cortex-R5, Zephyr board kv260_r5), the OpenAMP DDR carve-out addresses taken
  from the demo's Linux DTB (vrings @0x3ed40000/0x3ed44000, buffers
  @0x3ed48000), the IPI mailbox wiring, and how to build/run.
- kv260_r5 openamp overlay + conf (zephyr,ipc_shm @0x3ed40000, zephyr,ipc =
  &rpu0_ipi) that lets zephyr/samples/subsys/ipc/openamp_rsc_table build for
  kv260_r5.
- A robot smoke test that boots a kv260_r5 Zephyr image on the emulated R5.

Validated so far: Renode's shipped ZynqMP OpenAMP echo test passes (real
remoteproc + virtio-rpmsg round-trip); a kv260_r5 hello_world boots on the
emulated R5; the openamp_rsc_table firmware builds for kv260_r5. Remaining:
load the firmware via Linux remoteproc, swap in the Pouch device + mock broker,
Ubuntu rootfs, and an end-to-end Robot test.

Part of #274. Do not merge until #232 lands.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
beriberikix added a commit that referenced this pull request Jul 23, 2026
The Zephyr OpenAMP firmware built for kv260_r5 now loads and starts on the R5
via real Linux remoteproc in the Renode ZynqMP demo (remoteproc state ->
running). Add the robot test that boots the demo Linux, injects the firmware
into a copy of the rootfs (debugfs, no root), and drives the remoteproc
load/start; document the debugfs injection flow.

Fix firmware placement: device-address 0x0 maps to the 64K R5 TCM, which the
~155K openamp image overflows (remoteproc "bad phdr da 0x0"). Relocate &sram0
to the DDR rproc carve-out at 0x3ed00000 (256K), which the Xilinx R5
remoteproc driver maps.

Remaining: confirm the rpmsg channel end to end, swap in the Pouch device +
adapter firmware, an Ubuntu rootfs, and a mock broker.

Part of #274. Do not merge until #232 lands.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
beriberikix added a commit that referenced this pull request Jul 24, 2026
Root-cause the R5-side bring-up: the ZynqMP R5 boots/executes from TCM (reset
at 0x0), not DDR. A DDR-relinked firmware loads via remoteproc (state=running)
but never runs. Switch the overlay back to the default TCM link address and
keep only the vrings/buffers in DDR (zephyr,ipc_shm @ 0x3ed40000); route the
R5 console to uart0 (Linux owns uart1). Trim the OpenAMP sample (no shell/log)
to ~45 KiB so it fits ATCM.

With that, the Linux side is fully up (virtio_rpmsg_bus online, /dev/rpmsg_ctrl0
+ rpmsg_ctrl/rpmsg_ns devices) and the R5 now executes the firmware (prints on
uart0). Remaining open blocker: the firmware faults in its IPM/IPI setup
(ipm_set_enabled -> prefetch abort) before completing the rpmsg handshake;
documented as the next step.

Part of #274. Do not merge until #232 lands.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
beriberikix added a commit that referenced this pull request Jul 24, 2026
Document and capture the R5-side bring-up progress. Three real, sequential
ZynqMP-R5-OpenAMP requirements the board lacked out of the box:

1. TCM-linked firmware (R5 boots from TCM, not DDR).
2. zephyr,ipc must point at the child mailbox (&rpu0_apu_mailbox); the
   ipm_xlnx_ipi driver leaves the parent rpu0_ipi device with a NULL api, so
   pointing at the parent faults in ipm_set_enabled.
3. An MPU region for the DDR OpenAMP carve-out: the ZynqMP SoC uses a static
   MPU table that doesn't map it, so the R5 data-aborts in virtqueue init.
   Captured as zynqmp-r5-openamp-mpu.patch.

With all three the R5 runs the OpenAMP firmware without faulting. Remaining
open blocker: after remoteproc start the emulation nearly stalls and no
R5-announced rpmsg channel is observed yet (likely an IPI notify /
interrupt-storm issue); documented for follow-up.

Part of #274. Do not merge until #232 lands.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
beriberikix added a commit that referenced this pull request Jul 24, 2026
…rdware

Frame the Renode Tier-2 outcome: the Linux remoteproc + virtio-rpmsg mechanism
(the RFC's /dev/rpmsg_ctrl0 interface) and the full Zephyr R5 firmware bring-up
(TCM link, child-mailbox IPM, MPU shared-memory region) are validated and
documented. The final fault-free R5 rpmsg channel announcement is blocked by an
emulator-model IPI interrupt-storm/stall after remoteproc start, so the
end-to-end Pouch-over-rpmsg round-trip is deferred to real MPU+MCU hardware.

Part of #274. Do not merge until #232 lands.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

size_t len = maxlen - POUCH_SERIAL_HEADER_LEN;
enum pouch_result result = ch->endpoint->send(&ch->bearer, &buf[POUCH_SERIAL_HEADER_LEN], &len);
if (len == 0 && result == POUCH_MORE_DATA)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found while bringing up the rpmsg transport in #304, an adapter on this serial core (RFC #274), running against the real device endpoints instead of this PR's mocks.

This early return is right in spirit, but CH_FLAG_PENDING was consumed at the top of the function (line 136, test_and_clear_bit) and isn't restored here — so "wait for the next call" never happens. The channel is permanently un-pending and every later frame_get() bails at line 137.

Worth contrasting with sar/sender.c:94, which has the same "no data at this time, will come back later" early return — but the SAR sender doesn't disarm itself on entry, so a transport can just re-drive it. This path both disarms itself and depends on bearer_ready() to re-arm, which the device endpoints never call: src/transport/endpoints/device/uplink.c implements only start/send/end.

It only bites when an endpoint produces data asynchronously — which is exactly the uplink, since pouch encrypts on its work queue a few hundred ms after the broker prompts the channel. Info and device-cert have their data buffered already and never reach this branch, so a session reliably completes the first three verbs and then hangs. From the Linux-side broker in #304:

{"level":"ERROR","msg":"session handler failed",
 "error":"read uplink: channel 4: timed out waiting for transfer"}

Minimal fix is to re-arm before returning:

Suggested change
if (len == 0 && result == POUCH_MORE_DATA)
if (len == 0 && result == POUCH_MORE_DATA)
{
// Endpoint has no data ready to send, but expects to be called again later.
// Re-arm so the transport polls us again; nothing else will.
pouch_serial_ch_ready(ch);
return 0;
}

That's necessary but not sufficient by itself — it only helps if the transport re-drives the channel (the rpmsg adapter uses a short timed wait rather than blocking indefinitely on the ready signal). The alternative, and arguably the better design, is for the uplink endpoint to call bearer->ready() when the work queue produces a block: fully event-driven, no polling, and it fixes every transport at once. That touches src/transport/endpoints/, which this PR doesn't own, so I'd defer to you on the direction.

Possibly related: your note on the clear_bit at line 165 being "a bit of a mix up" — same function, same class of flag-lifecycle issue, so there may be a common cleanup here rather than two isolated patches.

@beriberikix

Copy link
Copy Markdown

Context: RFC #274 proposes running Pouch over rpmsg on heterogeneous MPU+MCU SoCs, where an MCU core runs the Pouch device stack and syncs through a gateway on the Linux side. #304 implements that as an adapter on this serial core rather than a new transport — one Pouch Serial frame per rpmsg message, with no changes to the code in this PR.

This PR notes it "does not include a real transport adapter… each using mock serial adapters and endpoints." #304 is that adapter, and bringing it up on hardware (NXP FRDM-IMX95: Zephyr on the Cortex-M7, Linux plus a Go broker on the Cortex-A55s) exercised the core against the real endpoints for the first time. That turned up one defect not yet raised — commented inline at channel.c:185 — and independently confirmed the one @mniestroj already flagged at channel.c:234.

Both are invisible to CI for the same structural reason, which seems worth fixing alongside them. Across all three suites the mocks in tests/pouch/serial/*/src/stub_endpoints.c differ from the real endpoints in exactly the two ways that matter:

  1. All 20 mock vtables define .end. The real pouch_device_endpoint_info and device_cert don't, so the unguarded call at channel.c:234 is unreachable in test but fatal on hardware — the first ACK the broker sends closes the info channel and faults the device.
  2. All three mocks share one synchronous sendmemcpy from a pre-filled buffer, then return (s->tx_offset >= s->tx_len) ? POUCH_NO_MORE_DATA : POUCH_MORE_DATA;. It never returns POUCH_MORE_DATA with *dst_len == 0. The real uplink endpoint hits that combination on every session, because pouch encrypts on its work queue a few hundred ms after the broker prompts the channel — and the branch at channel.c:185 then deadlocks the transfer.

Two mock variants would close the gap without needing hardware: one with .end = NULL, and one that returns POUCH_MORE_DATA with zero bytes for the first N calls before producing data.

With both fixes applied, a full session completes end-to-end against gw.golioth.io — info, server certificate, device certificate, uplink, downlink — and the MCU registers in the Golioth console as its own device alongside the gateway. Device-side trace of the uplink path that previously hung:

[00:00:05.760] <dbg> pouch_serial_ch: handle_ack: ch4: sender start -> 0
[00:00:05.760] <dbg> pouch_serial_ch: ch4: no data yet, re-arming
[00:00:05.760] <dbg> saead_session: session_encrypt_block: Session key: …
[00:00:08.098] <dbg> downlink: pouch_downlink_start: Pouch downlink start

Ten consecutive sessions, zero errors.

beriberikix added a commit that referenced this pull request Aug 10, 2026
… on hardware

Completes this PR's deferred item: the end-to-end Pouch-over-rpmsg round-trip
now runs on real MPU+MCU hardware. Validated on an FRDM-IMX95 (Cortex-M7 under
Linux remoteproc) against production Golioth, with connect-agent as the broker
on the Linux side.

Resource-table transport
------------------------

port/zephyr/transport/serial/rpmsg_rsc_device.c is a second device adapter
alongside the ipc_service one. Zephyr's DT-instantiable ipc_service backends
use a static vring layout that the Linux kernel's virtio_rpmsg_bus does not
understand, so against Linux the vdev has to come from the resource table, as
in the openamp_rsc_table sample. Naming the endpoint "rpmsg-raw" gets a
/dev/rpmsgN for free, with no RPMSG_CREATE_EPT_IOCTL.

Two bugs that only a fast host exposes:

- rpmsg_send() blocks until a TX buffer frees, and it is the thread servicing
  the vrings that would free one. Use rpmsg_trysend() with a bounded retry.
- The management thread ran at the same priority as the Pouch work queue, so
  it drained the entire RX vring without yielding while the decrypt queue
  never ran. A host pushed 137 KB in 6 ms and pouch_downlink_push() kept
  heap-allocating encrypted blocks until malloc failed, which left the
  downlink channel permanently erroring. The management thread now runs below
  the work queue, and the RX path stalls while the firmware relay is backed
  up so the backpressure reaches the broker instead of piling up on-device.

MCU-mediated OTA
----------------

A core loaded by remoteproc has no flash to write to, but it is still the
authenticated Pouch endpoint. It downloads its own image through Pouch OTA and
relays the plaintext to the host over two new serial channels, and the host
verifies the SHA-256 from the manifest, installs alongside the previous image
and restarts the core.

The relay never holds the image. Golioth OTA reports only a state upstream,
never a received offset, so the cloud streams a whole component within one
session and expects the device to consume it - which a flashless MCU with
32 KB of buffer cannot do if the firmware channel is only collected after the
downlink verb completes. The broker therefore collects the firmware channel
concurrently with the downlink that feeds it, and the channel holds its
transfer open when the relay is momentarily empty rather than reporting
"nothing to collect".

Results
-------

Three complete OTA runs (install, self-hosted upgrade, downgrade), each
installed image byte-identical to the cloud artifact by SHA-256, plus a
negative test: a corrupted image was rejected on hash, nothing was installed,
the core was not restarted and the running firmware was untouched.

Notes
-----

- This branch adds two channels to the serial common header and registers
  them in device.c, so unlike the rest of the PR it does touch the serial
  core. It also depends at runtime on two serial-core fixes sent separately
  against #232's branch; without them the first ACK crashes the device and
  the uplink stalls every session.
- The example's device credentials remain placeholders.
- Known gap: a rejected image leaves the component marked "updating", and the
  device needs a restart to retry, because the OTA manifest is delivered once
  per connection. A fix is drafted but not yet hardware-verified.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@trond-snekvik
trond-snekvik force-pushed the transport/serial branch 2 times, most recently from 06f50f7 to b39bbd2 Compare August 10, 2026 15:29

@mniestroj mniestroj left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 things from my side, otherwise looks good.

Comment thread src/transport/serial/channel.c Outdated
Comment thread src/transport/serial/channel.c
Adds the serial transport layer, as outlined in golioth/architecture#85.
This does not include a real transport adapter, but includes three test
suites that test the serial device, serial broker and a test that covers
both together, each using mock serial adapters and endpoints.

Signed-off-by: Trond Snekvik <trond.snekvik@canonical.com>

@mniestroj mniestroj left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome work on the design, implementation and refinement!

@trond-snekvik
trond-snekvik merged commit 7b0a718 into main Sep 2, 2026
50 of 51 checks passed
@trond-snekvik
trond-snekvik deleted the transport/serial branch September 2, 2026 09:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants