Repository navigation
feat(hiroz): join the rmw_zenoh cpu buffer group - #384
Merged
Merged
Conversation
The three mutant tests failed through a plain assert. They now fail through r1_assert with their own name, so a failure carries the R1ASSERT marker.
A joined typed subscriber that is dropped must stop its callback on the group key. No test owned the mutant that forgets the group subscription on drop. The callback runs on the group key before the drop, then ten puts after it add none.
Add the exact-join tests: delivery under a rule that allows only the base and group keys, the sibling sample key that no subscription matches, subscription and token counts, a typed topic with no buffer field, a transient-local subscriber, a typed subscriber with no schema, and the join decision per field kind. Each test pairs its negative check with a positive control. The field kind table pins that the typed path counts byte[]. ros-msgs now enables dynamic-schema-loader so the table can read load_schema.
Cite the rmw_zenoh_cpp code the join copies, state the measured buffer field rule, drop the claim that transient-local history uses the base key only, and label the guide sentences that were read from source and not measured.
The plan held the topic key that both builders already hold, and it forced a call order that no text stated. Two functions, group_key and declare_token, give the same behaviour with less code.
The regrouping was formatter output and has nothing to do with the join. The file keeps the layout of main, and the token field uses the LivelinessToken import again.
Two tests repeat rows float_auto and tl_default of typed_subscriber_join_policy. A third repeated the matching-status probe of the sibling key test; its three extra keys now sit in that test. The test of the key that a real publisher writes stays: it is the only one that checks decoded data on the group key.
The rule sits in BufferGroup. The method doc restated it and could drift from it, so it now gives one sentence and a link.
A tuple alias hid what the pair owns. The struct states that it holds the topic-key subscription and the optional group subscription, and that dropping it undeclares both. Field order keeps the drop order of the tuple.
The join declares two exact subscriptions. Name the probes on the topic key and everything under it as controls, and drop the comparison with the wide shape from a test doc and the guide.
The field-kind test loads std_msgs definitions through load_schema. The humble build embeds no std_msgs text, so the test panicked there. The loaded rows now exist only without the humble feature. The typed rows run on every distro.
A transient-local subscriber stays out of the group because the released package serves its history on the base key only. The guide now says what was measured and what was not. A joined subscriber can report lost-message events when other buffer-aware subscribers share the topic. The guide lists this as a limit and notes the interop tests do not cover it.
The policy predicate is pure now. The builder logs one debug line per join and reuses the group key it already built. ZSub and ZCache keep the field order and names of main, because nothing observes drop order.
Private comments no longer carry run counts, setups or revision history. The user guide holds the evidence.
Tables hold the field rule and the join setting. The lost-message limit appears once. The cases that never join form a list.
The depth, occupancy grid, volatile action and loaded-schema rows pinned code that another row pins. The schema-loader feature of hiroz-tests has no user left.
Every joined table row now proves the base key and decodes a nonempty payload, so the cache two-key test, the real-publisher test and the schemaless Num test have no unique check left.
One table now holds the base-only rule and the exact-pair rule. The default-allow deny rule and the wildcard probes of the old design are gone. A wider subscription still turns the pair rows red.
The per-gid key stays as the one negative, with the base and group keys as positives and one broad observer as the control. Three keys that no design proposes are gone.
The join event carries the base key and the group key as fields. The test checks the level, the keys and one event per join, and no longer freezes the English text.
The schema matrix and the field-kind matrix need no router. They read the token of a subscriber on one session. A codegen test pins that char[], byte[] and uint8[] share one schema kind. The router tests keep two dynamic rows with delivery checks.
raw_session had the body of buffer_rmw::client_session.
The plain-topic test repeats the action feedback test, which stays. The group-key counter of that test had no positive control.
The mixed-set test required zero base-key samples and zero delivery to the Off subscriber. A publisher with the fix for legacy subscribers can break both and still deliver every joined message. The guide states the starvation as a limit.
The module comment no longer states the starvation of legacy subscribers as a fact of every publisher.
A rule on the group key stops a joined subscriber from receiving buffer messages only. A message without a buffer arrives on the base key. Off avoids the join but does not restore buffer messages from a released publisher.
YuanYuYuan
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A buffer-aware
rmw_zenohpublisher changes its keys for a message with an unboundeduint8[]orchar[]field, such asImage. It stops writing the plain topic key. It writes<topic key>/_buf_cpufor each subscriber whose token ends withbackends:cpu:. See ros2/rmw_zenoh#930 and ros2/rmw_zenoh#987. A hiroz subscriber announces no backends and reads only the base key. It receives nothing from such a publisher, with no error and no log.Fixes #371. Part of #370.
What changes
#[non_exhaustive]enumBufferGroup { Auto, Cpu, Off }. The subscriber builder and the cache builder getwith_buffer_group. The default isAuto.backends:cpu:. It declares two exact subscriptions, the base key and the group key (<topic key>/_buf_cpu), before the token.Offkeeps the token and the single base subscription ofmain.rmw-zenoh-rspassesOff.joinsanddeclare_tokeninbuffer_group.rsreplace the earlierPlantype. The builder wrapper logs one debug event per join.Auto(default)uint8[]or char arrayAuto, no definitionCpuOffA nested field counts. The typed code generator also counts
byte[]: this costs one extra join and loses no data. Bounded arrays, fixed arrays andint8[]do not count. Transient-local subscribers never join. Subscribers on other key formats thanrmw_zenohnever join.Before and after
mainbackends:cpu:Compatibility
Offstops a subscriber from joining, so it needs no rule for the group key. It does not bring back buffer messages: a subscriber with nobackends:field receives none from a buffer-aware publisher until a release carries ros2/rmw_zenoh#1078.A rule on
<topic key>/**is not needed. With a default-deny router,buildreturnsOkand the subscriber receives nothing from a buffer-aware publisher.Design
rmw_zenoh_cpp; narrow router rulerouter_access_control_starves_a_joined_subscriber_silentlypins the router rule on the two keys.sibling_sample_key_has_no_subscription_and_its_samples_never_reach_a_joined_subscribershows that the process subscribes to no wide key.joined_subscriber_reads_the_group_key_with_its_localityand the rows ofcache_join_policypin delivery on both keys.group_subscription_is_declared_before_the_token_is_visiblepins the declaration order on the typed path.Transient-local measurement on the released 0.10.6 package: n=5 per cell, five kinds of late joiner, 75 Image cells. History arrived on the base key in every cell and never on the group key. An unreleased revision serves it on the group key. No test covers history after ros2/rmw_zenoh#1078.
Declaration cost, from an earlier experimental build on one machine, n=5, N=1000 subscribers: two exact subscriptions took 3.2 times the declaration time of
main, and one wide key took 1.2 times. This head has not repeated it.Tests
The head has 12 router tests, 4 interop tests, 2 router-free join tests and 1 codegen test. The router tests cover join policy, token, delivery, teardown, the debug log and router access control. The router-free tests and the codegen test cover which field kinds join.
typed_subscriber_join_policy,dynamic_subscriber_join_policy,cache_join_policyandaction_client_join_policy.dropping_a_joined_typed_subscriber_removes_its_token_and_stops_callbacksanddropping_a_joined_cache_stops_group_key_updates.dynamic_schema_shapes_decide_the_joinandtyped_message_fields_decide_the_joininbuffer_group_join.rs, andbyte_char_and_uint8_sequences_share_one_schema_kindin the codegen crate.Evidence
cargo testcargo doccargo check --testsThe 2 skipped jobs are
Publish to GitHub Pages. The interop tests ran in this CI job withHIROZ_REQUIRE_BUFFER_RMW=1, which forbids a skip: https://github.com/ZettaScaleLabs/hiroz/actions/runs/39590732929/job/119190731664. The token-before-subscriptions mutant covers the typed path only.Limits
This PR builds on
main. Merge order: #384 (this PR), #385, #388, #389, then #386.