latest version fails to compact context and endlessly throws a malformed ContextWindowExceededError exception

[david-thrower]

Latest version fails to handle ContextWindowExceededError and compact the context.

Thank you in advance for looking into this.

I need to report a bug in error handling that apparently affects the version last merged into main, but not the version I previously installed :

I updated ml-intern a few hours ago.

cd ml-intern
git pull
uv sync
uv tool install -e .


... and started ml intern ...
cd [my project's directory]
export GITHUB_TOKEN=[...]
export HF_TOKEN=[...]
export OPENAI_API_KEY="$HF_TOKEN"
export OPENAI_BASE_URL="https://router.huggingface.co/v1"
ml-intern --model moonshotai/Kimi-K2.6:fireworks-ai --sandbox-tools

Afterwards,

I ask it to pull a branch in my repo and fire off a series of ablations from it in a sandbox, and not long after it starts traversing the log stream from the ablations it started, it starts throwing this error when in the same situation I would normally expect it to print out a warning like "compacting context 197,000 tokens -> 168,000" and continue working. I tried starting a new conversation a few times, and appears to be a chronic issue ...

  Error: ContextWindowExceededError.__init__() missing 2 required positional 
arguments: 'model' and 'llm_provider'
Traceback (most recent call last):
  File 
"[redacted]/uv/tools/ml-intern/lib/python3.12/site-packages/litell
m/llms/openai/openai.py", line 1113, in async_streaming
    headers, response = await self.make_openai_chat_completion_request(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File 
"[redacted]/uv/tools/ml-intern/lib/python3.12/site-packages/litell
m/litellm_core_utils/logging_utils.py", line 297, in async_wrapper
    result = await func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File 
"[redacted]/uv/tools/ml-intern/lib/python3.12/site-packages/litell
m/llms/openai/openai.py", line 461, in make_openai_chat_completion_request
    raise e
  File 
"[redacted]/uv/tools/ml-intern/lib/python3.12/site-packages/litell
m/llms/openai/openai.py", line 438, in make_openai_chat_completion_request
    await openai_aclient.chat.completions.with_raw_response.create(
  File 
"[redacted]/uv/tools/ml-intern/lib/python3.12/site-packages/openai
/_legacy_response.py", line 384, in wrapped
    return cast(LegacyAPIResponse[R], await func(*args, **kwargs))
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File 
"[redacted]/uv/tools/ml-intern/lib/python3.12/site-packages/openai
/resources/chat/completions/completions.py", line 2700, in create
    return await self._post(
           ^^^^^^^^^^^^^^^^^
  File 
"[redacted]/uv/tools/ml-intern/lib/python3.12/site-packages/openai
/_base_client.py", line 1884, in post
    return await self.request(cast_to, opts, stream=stream, 
stream_cls=stream_cls)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^
  File 
"[redacted]/uv/tools/ml-intern/lib/python3.12/site-packages/openai
/_base_client.py", line 1669, in request
    raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'error': {'object': 'error', 'type': 
'invalid_request_error', 'code': 'invalid_request_error', 'message': 'The prompt
is too long: 2276991, model maximum context length: 262143'}}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File 
"[redacted]/uv/tools/ml-intern/lib/python3.12/site-packages/litell
m/main.py", line 622, in acompletion
    response = await init_response
               ^^^^^^^^^^^^^^^^^^^
  File 
"[redacted]/uv/tools/ml-intern/lib/python3.12/site-packages/litell
m/llms/openai/openai.py", line 1163, in async_streaming
    raise OpenAIError(
litellm.llms.openai.common_utils.OpenAIError: Error code: 400 - {'error': 
{'object': 'error', 'type': 'invalid_request_error', 'code': 
'invalid_request_error', 'message': 'The prompt is too long: 2276991, model 
maximum context length: 262143'}}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/[redacted]/ml-intern/agent/core/agent_loop.py", line 862, in 
_call_llm_streaming
    response = await acompletion(
               ^^^^^^^^^^^^^^^^^^
  File 
"[redacted]/uv/tools/ml-intern/lib/python3.12/site-packages/litell
m/utils.py", line 2097, in wrapper_async
    raise e
  File 
"[redacted]/uv/tools/ml-intern/lib/python3.12/site-packages/litell
m/utils.py", line 1896, in wrapper_async
    result = await original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File 
"[redacted]/uv/tools/ml-intern/lib/python3.12/site-packages/litell
m/main.py", line 641, in acompletion
    raise exception_type(
          ^^^^^^^^^^^^^^^
  File 
"[redacted]/uv/tools/ml-intern/lib/python3.12/site-packages/litell
m/litellm_core_utils/exception_mapping_utils.py", line 2456, in exception_type
    raise e
  File 
"[redacted]/uv/tools/ml-intern/lib/python3.12/site-packages/litell
m/litellm_core_utils/exception_mapping_utils.py", line 478, in exception_type
    raise BadRequestError(
litellm.exceptions.BadRequestError: litellm.BadRequestError: OpenAIException - 
The prompt is too long: 2276991, model maximum context length: 262143

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/[redacted]/ml-intern/agent/core/agent_loop.py", line 1261, in run_agent
    llm_result = await _call_llm_streaming(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/[redacted]/ml-intern/agent/core/agent_loop.py", line 876, in 
_call_llm_streaming
    raise ContextWindowExceededError(str(e)) from e
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: ContextWindowExceededError.__init__() missing 2 required positional 
arguments: 'model' and 'llm_provider'

[lewtun]

Thanks for contributing to ML Intern. The project, including its CLI, has been retired and this repository is being archived. We will therefore not be able to review or merge this work. For continued AI assistance, please use HuggingChat.