Skip to content

Commit 7692091

Browse files
committed
feat(leaderboard): upload leaderboard data from spectrumbench v1.0
1 parent c1ef86e commit 7692091

6 files changed

Lines changed: 2377 additions & 378 deletions

File tree

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ uv pip install -e .
3535

3636
Benchmark data is hosted on Hugging Face. Please download it from the following link:
3737

38-
[https://huggingface.co/datasets/SpectrumWorld/spectrumbench_v_1.0](https://huggingface.co/datasets/SpectrumWorld/spectrumbench_v_1.0)
38+
[https://huggingface.co/datasets/SpectrumWorld/spectrumbench_v_1.0](https://huggingface.co/SpectrumWorld/spectrumbench_v_1.0/tree/main)
3939

4040
After downloading, extract the data to the `data` directory in the project root.
4141

leaderboard/batch_import_models.py

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
#!/usr/bin/env python3
2+
"""
3+
Batch import models to leaderboard
4+
Usage: python batch_import_models.py models_data.json
5+
"""
6+
7+
import json
8+
import sys
9+
from pathlib import Path
10+
11+
# Add the leaderboard directory to Python path
12+
sys.path.insert(0, str(Path(__file__).parent))
13+
14+
from manage_leaderboard import LeaderboardManager
15+
16+
17+
def batch_import(models_file: str):
18+
"""Batch import models from JSON file"""
19+
20+
with open(models_file, "r", encoding="utf-8") as f:
21+
models_data = json.load(f)
22+
23+
manager = LeaderboardManager()
24+
25+
# Clear existing models if specified
26+
if models_data.get("clear_existing", False):
27+
print("🗑️ Clearing existing models...")
28+
manager.data["models"] = []
29+
30+
# Import models
31+
imported_count = 0
32+
for model_data in models_data["models"]:
33+
model_info = model_data["model_info"]
34+
subcategory_scores = model_data["scores"]
35+
36+
success = manager.add_model(model_info, subcategory_scores)
37+
if success:
38+
imported_count += 1
39+
40+
print(
41+
f"\n✅ Successfully imported {imported_count}/{len(models_data['models'])} models"
42+
)
43+
44+
45+
if __name__ == "__main__":
46+
if len(sys.argv) != 2:
47+
print("Usage: python batch_import_models.py models_data.json")
48+
sys.exit(1)
49+
50+
batch_import(sys.argv[1])

0 commit comments

Comments
 (0)