Repository navigation
Conversation
capture.py records raw device traffic as JSON lines (rx/tx bytes with the original socket read boundaries, plus device info) and can load, sanitize and organize those files. Sanitizing is on by default (CYNC_CAPTURE_SANITIZE) and happens in memory before anything is written, so identifiers never reach the disk. Replacements are same-length, use obviously fake markers (DE AD BE EF / SANITIZED) and keep 0x73/0x83 inner checksums valid, so a replay still frames and parses: - by packet layout: the queue id (both directions: our acks and control packets carry it too) and the whole body of the 0x23 hello (the account-derived auth code) - by value, from the user's own setup: device MACs (binary in both byte orders, and text), home ids (decimal text and 4-byte big/little endian), account username/password, CYNC_SECRET_KEY and the device's own IP - the IP in the file header and file name Each file ends with a sanitize_report; bytes outside any recognised packet are kept and counted. Also: per-file and per-directory size caps, 0600 files, a tolerant loader (truncated last line), an offline sanitizer (idempotent, optional --mesh-config to search for the values in cync_mesh.yaml) and 'python -m cync_lan.capture info|organize' which sorts captures into type_<n>/fw_<version>/. Not hooked into the sessions yet. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
CyncTCPSession opens a CaptureWriter per socket when CYNC_CAPTURE_DIR is set (off otherwise): every read() chunk and every write we make is recorded, and what we learn (device type, firmware, protocol version, configured node ids/types/endpoint ids, never names or MACs) is written when the node is identified and again on close. A same-IP reconnect starts a new file. Mobile-app sessions are discarded as soon as they are recognised, because they carry account tokens. The writer is built with the PII patterns from the configured devices and account. Live tests over real loopback sockets cover rx/tx/end records with preserved chunking, default sanitizing (no IP in the file or its name) and app-session discarding. The first version created the writer before ip_address was set, which the live test caught. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
tests/replay_harness.py rebuilds the configured nodes from a capture's info, feeds every rx chunk (original boundaries) to a real CyncTCPSession with a fake writer and MQTT client, and returns what the session sent back plus the framer/parser counters; it restores the global state it touches. tests/test_replay.py replays every tests/captures/type_*/fw_*/*.jsonl and asserts the invariants that must always hold: only sanitized files, no packet failed to parse, the framer dropped no bytes, the capture ends on a packet boundary. One synthetic sample (handshake and pings, read split inside the header) is included so the path always runs and shows the layout; drop real sanitized captures into the same tree and they are picked up automatically. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
docs/capture.md: how to record (env vars, size caps), what sanitizing replaces and what it cannot, the offline organize/info tools, how the replay tests work and how to add a real capture. README env table, a commented CYNC_CAPTURE_DIR in the example compose (inside the mounted config volume), CLAUDE.md module map, PLAN.md status, and /captures/ in .gitignore so a raw capture directory in the repo root can never be committed by accident. Raw captures must stay private: only sanitized files go under tests/captures/. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Record real device traffic to disk and replay it through a real session in tests, so protocol/handler behaviour can be pinned with real data per device type and firmware. Base is
claude-refactor(this adds on top of it, notpython).How it works
CYNC_CAPTURE_DIR(off by default) writes one JSONL file per device TCP connection: everyread()chunk (original boundaries), every write we make, and what we learn (device type, firmware, protocol version, configured node ids/types/endpoints; never names or MACs).CYNC_CAPTURE_SANITIZE) and happens in memory before anything is written: queue ids in both directions, the 0x23 hello body (auth code), device MACs, home ids, account name/password, secret key, the device IP, and the IP in the header/file name. Replacements are same-length and use obvious markers (DE AD BE EF/SANITIZED), inner checksums are recomputed, so replays still frame and parse. Each file ends with asanitize_report.python -m cync_lan.capture info|organizeinspects captures and sorts sanitized copies intotests/captures/type_<n>/fw_<v>/(idempotent;--mesh-configalso searches for the values incync_mesh.yaml).tests/replay_harness.py+tests/test_replay.pyreplay every committed capture and assert: only sanitized files, no packet failed to parse, no bytes dropped by the framer, ends on a packet boundary. One synthetic sample is included.Commits
Known limits
Sanitizing removes what it can recognise or enumerate; unknown fields that hold something personal are not caught, and bytes outside recognised packets are kept (and counted). Captures must be reviewed before they are committed. See
docs/capture.md.Testing
199 tests pass on Python 3.12, 3.13 and 3.14; ruff clean. Mutation-checked the redaction paths (disabling pattern or queue-id redaction fails the tests). Not run against real devices yet: the first real captures are the next step. The HA add-on does not expose
CYNC_CAPTURE_DIRyet (Docker/CLI only).🤖 Generated with Claude Code