hiroz typed and dynamic subscribers receive nothing from buffer-aware rmw_zenoh publishers #371

Part of #370. Reported in #369 for a raw subscriber. The raw subscriber is #372.

Problem

A hiroz typed or dynamic subscriber on a buffer-typed topic declares a liveliness token without a backends: field. The rmw_zenoh publisher routes such a subscriber to <base>/_buf/<gid>. Hiroz never reads that key, so the subscriber receives nothing. It stays matched, and nothing logs an error. Messages without buffer fields are not affected.

Fixed by

The issue is fixed when both pull requests merge.

Mechanism

sequenceDiagram
    autonumber
    participant Sub as hiroz subscriber
    participant Z as Zenoh
    participant Pub as rmw_zenoh publisher
    Sub->>Z: declare token (no backends: field)
    Sub->>Z: subscribe to base key
    Z->>Pub: token appears
    Note over Pub: not a CPU-group member, so it counts as an accelerated subscriber
    Pub->>Z: put on base/_buf/gid
    Note over Sub: never subscribed to base/_buf/gid, receives nothing
    rect rgb(235, 245, 235)
    Note over Sub,Pub: with a joined subscriber
    Sub->>Z: declare token ending in backends:cpu:
    Sub->>Z: subscribe to base/**, keep the base key and base/_buf_cpu
    Pub->>Z: put on base/_buf_cpu
    Z->>Sub: every message
    end

Reproducer

Use a container with ros-lyrical-rmw-zenoh-cpp 0.10.6.

1. Start the router: ros2 run rmw_zenoh_cpp rmw_zenohd.
2. Start a publisher: ros2 topic pub -r 5 /cloud sensor_msgs/msg/PointCloud2 "{}".
3. Start a hiroz typed subscriber on /cloud, built from main before the fix.
4. Count the messages for 10 s.

Before and after

Behaviour

• The default in feat(hiroz): join the rmw_zenoh cpu buffer group #384 is Off. The default in feat(hiroz): join the buffer group by default #387 is Auto.
• A joined subscriber declares <base>/**. A router rule that allows only the exact topic key starves it, and build still returns Ok. A call to with_buffer_group(BufferGroup::Off) avoids this.
• A subscriber without a backends: field receives nothing from a buffer-aware publisher until Publish on the base key to subscribers without a backends: field ros2/rmw_zenoh#1078 ships. This covers Off, transient-local subscribers and other key formats.
• The Python binding has a buffer_group argument in feat(hiroz-py): add a buffer_group argument #388, and the action client status subscriber stays outside the group there. The C and Go bindings are in feat(ffi): join the cpu group from c and go #389.

Evidence on main

The runs use the released ros-lyrical-rmw-zenoh-cpp 0.10.6 with three runs in each cell. The older releases are ros-humble-rmw-zenoh-cpp 0.1.11, ros-kilted-rmw-zenoh-cpp 0.6.8 and ros-jazzy-rmw-zenoh-cpp 0.2.11, with one run in each cell. The publisher is ros2 topic pub -r 5 with CPU buffers, so complete delivery in a 10 s window is 50 or 51. One process holds a typed subscriber on the topic. The probe was built from hiroz main before the fix. The Humble probe uses the humble feature. The Kilted and Jazzy probes use the default feature set, which is the Jazzy one. The probes count messages and do not compare payload contents.

Lyrical 0.10.6, three runs per cell, messages received on main:

The 0 cells have two controls. In the same cell, a native ros2 topic hz subscriber read 5.000 Hz, so the publisher was producing. The same probe binary received 50 to 51 messages on Float64MultiArray, so the probe and the router work.

Older releases, one run per cell, typed subscriber, messages received on main:

On these releases main already delivers every message, so they show no loss. The loss follows the rmw_zenoh release and the message definition.

The rule that decides which messages carry buffers follows the message definition. In Lyrical an unbounded uint8[] is a rosidl::Buffer, directly or in a nested message. The reference is the compiled typesupport of one Lyrical install: its has_buffer_fields function, called for every message of the 223 packages that ship .msg files.

118 of the 2227 messages have buffer fields. They include visualization_msgs/Marker, MarkerArray, the four interactive-marker messages, stereo_msgs/DisparityImage, map_msgs/PointCloud2Update and nav2_msgs/Costmap.

The comparison covers the exact rule, not the hiroz code. Hiroz applies the rule to the message schema, and a schema cannot tell byte from uint8. Hiroz therefore also counts an unbounded byte[]. In the bundled Jazzy .msg files, five messages count by the schema rule and not by the exact rule: std_msgs/ByteMultiArray, example_interfaces/ByteMultiArray, rcl_interfaces/ParameterValue, rcl_interfaces/Parameter and rcl_interfaces/ParameterEvent. A subscriber on one of them joins without need, and receives each message once.