LoopLab

An autonomous algorithm optimization framework that generates, implements, tests, and improves methods against a real baseline.

What is LoopLab?

LoopLab is an autonomous experimentation framework for machine learning research. It goes beyond generating ideas: each selected hypothesis is developed into an implementable method, coded by an external coding agent, executed against a real baseline, and evaluated using task-defined metrics.

The result of each experiment becomes context for the next research round, allowing LoopLab to improve both the implementation and the question it explores.

You provide	LoopLab returns
A research goal and constraints	Ranked, evidence-backed research hypotheses
Runnable baseline code	Runnable implementations of selected methods
Baseline evaluation metrics	Metric comparisons and experiment reports
A fixed evaluation protocol	A reusable experience library and an evolved research direction

How it works

1. Generate hypotheses - Multiple agents propose, critique, research, and revise candidate ideas.
2. Develop methods - Candidates are ranked for novelty, plausibility, testability, and task alignment, then expanded into implementation-ready specifications.
3. Run experiments - An external claude CLI edits the baseline code. LoopLab executes the result, captures failures, and drives repair attempts.
4. Evaluate results - Each experiment is compared with the baseline using the task's primary metric and optimization direction.
5. Learn and repeat - Task memory records outcomes, semantic memory distills reusable knowledge, and the next research direction is updated from accumulated evidence.

LoopLab supports two experiment search strategies:

• Linear search iteratively improves the current implementation.
• MCTS search explores multiple implementation branches using Monte Carlo Tree Search.

Key capabilities

• Multi-agent hypothesis generation, reflection, literature search, ranking, and method refinement.
• Real code execution with traceback-guided repair attempts.
• Parallel idea processing and experiment execution with GPU allocation.
• Incremental research rounds that can promote the previous winner to the next baseline.
• Task Experience Memory (TEM) for retrieving related successes and failures.
• Semantic Knowledge Memory (SKM) for experience distillation, idea graphs, and research-direction evolution.
• Compatible task and result layouts with InternAgent 1.5.

Quick start

1. Install dependencies

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env

2. Configure the research model

Edit .env and provide an OpenAI-compatible API endpoint:

OPENAI_API_KEY=your_api_key
OPENAI_API_BASE_URL=https://api.openai.com/v1

Set the model name in models.openai.model_name inside the selected YAML configuration.

Experiment code is written by the Claude Code CLI. Install and authenticate the claude command separately. The coding model is configured through experiment.model.

3. Run the smoke task

python launch.py \
  --config config/debug.yaml \
  --task AutoDebug \
  --exp_backend claudecode

AutoDebug uses a synthetic regression dataset and a small MLP. It is intended to verify the complete idea-to-experiment pipeline at minimal cost.

Use config/default.yaml for a full multi-round run or config/mcts_debug.yaml to exercise the MCTS path.

The default configuration expects a local embedding model at ../InternAgent/models/bge-base-en-v1.5. Change memory.task_memory.embedding.model_name if the model is stored elsewhere.

Output layout

Each launch receives an isolated directory. Knowledge shared across launches is stored at the task level.

results/<Task>/
|-- experience_library.json          # Cross-round semantic experience
|-- <Task>_graph.pkl                 # Historical idea graph
|-- chroma_db/                       # Idea vector index
`-- <timestamp>_launch/
    |-- prompt.json                  # Active and potentially evolved research prompt
    |-- discovery_summary.json       # Round results and final winner
    `-- session_<id>/
        |-- ideas.json               # Selected methods and specifications
        |-- traj.json                # Multi-agent reasoning trajectory
        `-- <timestamp>_<idea>/
            |-- run_0/               # Baseline snapshot
            |-- run_1...N/           # Experiment code and metrics
            |-- experiment_report.txt
            `-- tree.json            # MCTS mode only

Add a task

A task directory is the contract between LoopLab and experiment code:

tasks/<Task>/
|-- prompt.json
|-- code/
|   `-- experiment.py
|-- run_0/
|   `-- final_info.json
`-- launcher.sh                      # Optional

prompt.json should define the task, goal, dataset, baseline, metrics, and constraints. For example:

{
  "task_name": "MyTask",
  "domain": "machine learning",
  "goal": "Improve the baseline under a fixed evaluation protocol.",
  "task_description": "Explore methods that improve generalization.",
  "metrics": {
    "primary": "mse",
    "secondary": ["r2", "mae"],
    "optimization_direction": "minimize"
  },
  "constraints": [
    "Do not change the dataset split",
    "Keep the evaluation protocol unchanged"
  ]
}

run_0/final_info.json contains baseline metrics. Every later run_N/ must write a file with the same metric schema so LoopLab can calculate improvement and select the next baseline.

When a task does not provide launcher.sh, LoopLab runs:

python code/experiment.py

A task directory can also be passed directly:

python launch.py --task /path/to/MyTask --config config/debug.yaml

Configuration

Setting	Purpose
workflow.loop_rounds	Number of autonomous research rounds
workflow.loop_mode	incremental promotes the previous winner; fresh always uses the original baseline
workflow.top_ideas_count	Number of methods sent to experiments per round
workflow.max_concurrent_tasks	Maximum concurrent idea-processing tasks
experiment.max_runs	Maximum experiment attempts per method
experiment.use_mcts	Enables MCTS experiment search
experiment.max_parallel_experiments	Maximum concurrent experiments
experiment.gpu_per_experiment	GPUs assigned to each experiment; fractional sharing is supported
memory.task_memory.enabled	Enables Task Experience Memory
memory.long_memory.enabled	Enables semantic memory, idea graphs, and prompt evolution

Included configurations:

• config/debug.yaml - One round, one idea, serial execution.
• config/default.yaml - Full multi-round incremental research.
• config/mcts_debug.yaml - Minimal MCTS search configuration.

Project structure

core/
|-- loop.py                  # Research rounds and baseline promotion
|-- idea_stage.py            # Hypothesis generation and method development
|-- experiment_stage.py      # Experiment scheduling, GPUs, and evaluation
|-- agents/                  # Generation, reflection, ranking, and memory agents
|-- experiments/             # Linear and MCTS experiment engines
|-- memory/                  # TEM, SKM, retrieval, and the idea graph
|-- workflow/                # Multi-agent state machine
`-- tools/                   # Literature search and repository inspection

config/                      # Models, workflow, experiment, and memory settings
prompts/                     # Agent prompts
launch.py                    # Command-line entry point

Evaluation contract

Autonomous experiment results are meaningful only when the evaluation protocol remains fixed. A task should lock the dataset, random seeds, train/test split, and metric calculation. These rules should also be stated explicitly in constraints so the coding agent does not modify them.

Without a stable evaluation protocol, results from different runs may not be comparable and LoopLab cannot reliably select the best method.

Origin

LoopLab is a focused rewrite of the core ideas in InternAgent 1.5. It retains the multi-agent state machine, task experience memory, semantic knowledge memory, and MCTS search while keeping task directories, final_info.json, and historical result layouts compatible.

Citation

@article{feng2026internagent,
  title={InternAgent-1.5: A Unified Agentic Framework for Long-Horizon Autonomous Scientific Discovery},
  author={Shiyang Feng and Runmin Ma and Xiangchao Yan and Yue Fan and Yusong Hu and Songtao Huang and Shuaiyu Zhang and Zongsheng Cao and Tianshuo Peng and Jiakang Yuan and Zijie Guo and Zhijie Zhong and Shangheng Du and Weida Wang and Jinxin Shi and Yuhao Zhou and Xiaohan He and Zhiyin Yu and Fangchen Yu and Bihao Zhan and Qihao Zheng and Jiamin Wu and Mianxin Liu and Chi Zhang and Shaowei Hou and Shuya Li and Yankai Jiang and Wenjie Lou and Lilong Wang and Zifu Wang and Jiong Wang and Wanghan Xu and Yue Deng and Dongrui Liu and Yiheng Wang and Wenlong Zhang and Fenghua Ling and Shufei Zhang and Xiaosong Wang and Shuangjia Zheng and Xun Huang and Siqi Sun and Shuyue Hu and Peng Ye and Chunfeng Song and Bin Wang and Conghui He and Yihao Liu and Xin Li and Qibin Hou and Tao Chen and Xiangyu Yue and Bin Wang and Liang He and Dahua Lin and Bowen Zhou and Bo Zhang and Lei Bai},
  journal={arXiv preprint arXiv:2602.08990},
  year={2026}
}

@article{team2025internagent,
  title={InternAgent: When Agent Becomes the Scientist--Building Closed-Loop System from Hypothesis to Verification},
  author={Team, InternAgent and Zhang, Bo and Feng, Shiyang and Yan, Xiangchao and Yuan, Jiakang and Ma, Runmin and Hu, Yusong and Yu, Zhiyin and He, Xiaohan and Huang, Songtao and others},
  journal={arXiv e-prints},
  pages={arXiv--2505},
  year={2025}
}