Skip to content
This repository was archived by the owner on Nov 11, 2026. It is now read-only.

fix(py): tolerate partial transcript lines in Claude Agent SDK tracing - #3654

Closed
Tony Cassara (tonycassara) wants to merge 1 commit into
langchain-ai:mainfrom
tonycassara:fix/claude-agent-sdk-transcript-decode
Closed

Tony Cassara (tonycassara) wants to merge 1 commit into
langchain-ai:mainfrom
tonycassara:fix/claude-agent-sdk-transcript-decode

Conversation

@tonycassara

Copy link
Copy Markdown

Summary

The Claude Agent SDK integration reads the SDK's JSONL session transcript after each response to reconcile token usage and stop reasons (read_usage_and_stop_reasons_from_transcript and read_llm_turns_from_transcript in _usage.py). The SDK may still be appending to that file during the read. If the read ends partway through a multibyte UTF-8 character, iterating the file raises:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe2 in position N: unexpected end of data

UnicodeDecodeError is a ValueError, not an OSError, so the readers' except OSError does not catch it. The readers run from reconcile_from_transcripts(...), which is called in the finally: block of the traced receive_response generator. That call sits outside the except Exception guard, so the error reaches the SDK user and aborts their turn after the response has already streamed. Usage reconciliation only affects observability, so it should never break the caller.

Changes

  • _usage.py: open transcripts with encoding="utf-8", errors="replace". A truncated trailing line then decodes to replacement characters, fails json.loads, and is skipped by the existing handling. The readers also catch (OSError, UnicodeDecodeError) as a safety net. Setting the encoding explicitly also stops the read from depending on the platform's default locale encoding.
  • _client.py: wrap reconcile_from_transcripts(...) in the finally block with try/except Exception and logger.exception(...). tracker.flush() and the session cleanup still run, and a reconciliation failure can no longer reach the caller.

Tests

Added to python/tests/unit_tests/wrappers/test_claude_agent_sdk_hooks.py:

  • TestTranscriptTruncatedMidCharacter: writes a transcript with one complete assistant entry followed by a second entry cut off after \xe2\x80 with no newline. The test asserts that both readers return the first message's usage and stop reason without raising.
  • TestReceiveMessagesInstrumented::test_reconcile_failure_does_not_reach_caller: makes reconcile_from_transcripts raise. The test asserts that receive_response() still yields the ResultMessage and that the root run ends without an error.

All three new tests fail on main with UnicodeDecodeError and pass with this change.

cd python
TEST=tests/unit_tests/wrappers/test_claude_agent_sdk_hooks.py make tests   # 47 passed
uv run ruff format --check / ruff check / mypy on touched files            # clean

@tonycassara
Tony Cassara (tonycassara) force-pushed the fix/claude-agent-sdk-transcript-decode branch 6 times, most recently from 457235c to 18a5631 Compare October 13, 2026 17:26
The Claude Agent SDK appends to its JSONL session transcript while the
tracer reads it back to reconcile token usage and stop reasons. A read
that lands mid-write can end partway through a multibyte UTF-8
character, so iterating the file raised UnicodeDecodeError ("unexpected
end of data"). That is a ValueError, not an OSError, so it escaped the
readers' error handling.

The readers are called from the finally block of the traced
receive_response generator, outside its except guard, so the error
propagated to the SDK user and aborted their turn after the response
had already streamed successfully.

Open transcripts as UTF-8 with errors="replace" so a truncated trailing
line decodes to replacement characters and is skipped by the existing
JSON parse handling, and widen the readers' except to include
UnicodeDecodeError. Also guard the reconcile_from_transcripts call in
the finally block: reconciliation is observability only and must never
fail the caller's response.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tonycassara
Tony Cassara (tonycassara) force-pushed the fix/claude-agent-sdk-transcript-decode branch from 18a5631 to b40e9d8 Compare October 14, 2026 18:09
@thibautlehmann

Copy link
Copy Markdown
Contributor

Hey Tony Cassara (@tonycassara), thanks for this PR 🙏

This repo is archived, so it can’t be merged here anymore, the SDKs moved to:

If it's still relevant, please reopen it on the right repo !

More context in #3691

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants