Skip to content

Agent test an_app_that_misbehaves_is_refused_or_dropped_and_the_rest_go_on needs more than 16 GiB #15

Open

Description

@Skyfuller1303

On unmodified main (98bbab3), one agent test uses more than 16 GiB of memory on Windows. That's enough to make cargo xtask ci fail on a machine with a 16 GiB limit, and it might point at an unbounded queue in the agent.

What I see

All runs were on Windows 11 with Rust 1.99.0, inside a job object that caps the whole process tree at 16 GiB (commit memory).

Command Result Peak
cargo test -p craft-cloud-agent --test live_documents --locked a_craft_save_goes_up_with_its_record -- --exact --test-threads=1 passes in 22 s 1.95 GiB
cargo test -p craft-cloud-agent --test live_documents --locked an_app_that_misbehaves_is_refused_or_dropped_and_the_rest_go_on -- --exact --test-threads=1 aborts: memory allocation of 786432 bytes failed, exit 0xc0000409 16.0 GiB
cargo test -p craft-cloud-agent --test live_documents --locked -- --test-threads=1 aborts in the same test 16.0 GiB

0xc0000409 is how Rust's allocation-failure abort shows up on Windows; it isn't a stack overflow. The first run also compiled the test binary, so its peak includes that; the other two ran on the finished build.

So the test alone, with one test thread, keeps allocating until the cap. I mentioned a_craft_save_goes_up_with_its_record in #13 and #14; that was wrong, since that test only happened to be the one still running. Without a cap the test may simply pass on a machine with more memory, and I haven't measured how far it goes.

Where it might come from

From reading the code, not proven:

  • The test sends 60 updates of 200,000 characters each (apps/craft-cloud-agent/tests/live_documents.rs:653). That's 12 million characters retained in the Loro history, plus replicas and exports.
  • The agent's server-output channel is unbounded (apps/craft-cloud-agent/src/live/hub.rs:1373). The 10-second send timeout limits how long a stalled send waits, not how many bytes queue up.
  • The app's event channel is an unbounded std::sync::mpsc::channel (crates/app/src/live.rs:365).

If it's one of the queues, a local app that misbehaves could drive the agent's memory without limit, which seems worth a bound outside the test too. If it's only the Loro history of 12 M characters, a smaller flood in the test would do.

Repro

Run the second command above on a machine or VM with about 16 GiB, or under a memory cap. I'm happy to dig further, with a heap profile or a bounded-queue experiment, if that's useful to you.

🤖 Generated with Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions