This repository was archived by the owner on Nov 11, 2026. It is now read-only.
Repository navigation
fix(py): tolerate partial transcript lines in Claude Agent SDK tracing - #3654
Closed
Tony Cassara (tonycassara) wants to merge 1 commit into
Closed
Tony Cassara (tonycassara) wants to merge 1 commit into
Tony Cassara (tonycassara) wants to merge 1 commit into
Conversation
Tony Cassara (tonycassara)
force-pushed
the
fix/claude-agent-sdk-transcript-decode
branch
6 times, most recently
from
October 13, 2026 17:26
457235c to
18a5631
Compare
The Claude Agent SDK appends to its JSONL session transcript while the
tracer reads it back to reconcile token usage and stop reasons. A read
that lands mid-write can end partway through a multibyte UTF-8
character, so iterating the file raised UnicodeDecodeError ("unexpected
end of data"). That is a ValueError, not an OSError, so it escaped the
readers' error handling.
The readers are called from the finally block of the traced
receive_response generator, outside its except guard, so the error
propagated to the SDK user and aborted their turn after the response
had already streamed successfully.
Open transcripts as UTF-8 with errors="replace" so a truncated trailing
line decodes to replacement characters and is skipped by the existing
JSON parse handling, and widen the readers' except to include
UnicodeDecodeError. Also guard the reconcile_from_transcripts call in
the finally block: reconciliation is observability only and must never
fail the caller's response.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tony Cassara (tonycassara)
force-pushed
the
fix/claude-agent-sdk-transcript-decode
branch
from
October 14, 2026 18:09
18a5631 to
b40e9d8
Compare
Contributor
|
Hey Tony Cassara (@tonycassara), thanks for this PR 🙏 This repo is archived, so it can’t be merged here anymore, the SDKs moved to: If it's still relevant, please reopen it on the right repo ! More context in #3691 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The Claude Agent SDK integration reads the SDK's JSONL session transcript after each response to reconcile token usage and stop reasons (
read_usage_and_stop_reasons_from_transcriptandread_llm_turns_from_transcriptin_usage.py). The SDK may still be appending to that file during the read. If the read ends partway through a multibyte UTF-8 character, iterating the file raises:UnicodeDecodeErroris aValueError, not anOSError, so the readers'except OSErrordoes not catch it. The readers run fromreconcile_from_transcripts(...), which is called in thefinally:block of the tracedreceive_responsegenerator. That call sits outside theexcept Exceptionguard, so the error reaches the SDK user and aborts their turn after the response has already streamed. Usage reconciliation only affects observability, so it should never break the caller.Changes
_usage.py: open transcripts withencoding="utf-8", errors="replace". A truncated trailing line then decodes to replacement characters, failsjson.loads, and is skipped by the existing handling. The readers also catch(OSError, UnicodeDecodeError)as a safety net. Setting the encoding explicitly also stops the read from depending on the platform's default locale encoding._client.py: wrapreconcile_from_transcripts(...)in thefinallyblock withtry/except Exceptionandlogger.exception(...).tracker.flush()and the session cleanup still run, and a reconciliation failure can no longer reach the caller.Tests
Added to
python/tests/unit_tests/wrappers/test_claude_agent_sdk_hooks.py:TestTranscriptTruncatedMidCharacter: writes a transcript with one complete assistant entry followed by a second entry cut off after\xe2\x80with no newline. The test asserts that both readers return the first message's usage and stop reason without raising.TestReceiveMessagesInstrumented::test_reconcile_failure_does_not_reach_caller: makesreconcile_from_transcriptsraise. The test asserts thatreceive_response()still yields theResultMessageand that the root run ends without an error.All three new tests fail on
mainwithUnicodeDecodeErrorand pass with this change.