Skip to content

fix(workspace): write and read the vault as UTF-8, not the host locale - #418

Open
MohammedAlkindi wants to merge 1 commit into
mindsdb:mainfrom
MohammedAlkindi:fix/workspace-utf8-vault
Open

fix(workspace): write and read the vault as UTF-8, not the host locale#418
MohammedAlkindi wants to merge 1 commit into
mindsdb:mainfrom
MohammedAlkindi:fix/workspace-utf8-vault

Conversation

@MohammedAlkindi

Copy link
Copy Markdown
Contributor

AntonSettings loads .anton/.env with env_file_encoding="utf-8" (anton/config/settings.py:32), but Workspace wrote and read that file, and anton.md, with bare read_text / write_text. Those use locale.getpreferredencoding(), which is cp1252 on a Western Windows install, so the writer and the loader disagree.

Two consequences, both silent until they are not:

  • a non-ASCII secret is stored as bytes the loader cannot decode, so every later AntonSettings() raises UnicodeDecodeError;
  • a value outside the code page raises UnicodeEncodeError on write and is never stored at all.

Five tests cover it. Four are behavioural; the fifth runs the workspace under -X warn_default_encoding (PEP 597) and asserts no EncodingWarning names workspace.py, which keeps the regression visible on UTF-8 CI where the behavioural four pass either way.

All five fail on the unpatched tree and pass with the change.

Scoped to the nine test modules that reach Workspace or cli: 403 passed / 9 failed before, 408 passed / 9 failed after, with a byte-identical set of failing names. The nine are pre-existing clipboard, network and POSIX-permission tests unrelated to this change.

The same two lines applied to the consent write in cli._ensure_terms_consent.

AntonSettings loads .anton/.env with env_file_encoding="utf-8" (config/settings.py:32), but Workspace wrote and read that file, and anton.md, with bare read_text/write_text. Those use locale.getpreferredencoding(), which is cp1252 on a Western Windows install, so a non-ASCII secret was stored as bytes the settings loader cannot decode and every later AntonSettings() raised UnicodeDecodeError. A value outside the code page, such as a CJK string, raised UnicodeEncodeError on write and was never stored at all.

The same two-line issue applied to the consent write in cli._ensure_terms_consent.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant