title	GitHub Learner Env (OpenEnv RL Environment)
emoji	🧠
colorFrom	blue
colorTo	gray
sdk	docker
app_port	8000
tags	openenv reinforcement-learning github

🧠 GitHub Learner Environment

A Reinforcement Learning environment built with OpenEnv where an agent learns GitHub skills by selecting repositories in the correct order.

This is NOT a toy echo environment — it simulates:

• Skill progression 📈
• Dependency constraints 🔗
• Reward optimization 🎯

---

🚀 What This Project Does

The agent must:

• Choose repositories
• Learn skills
• Respect prerequisites
• Maximize reward

---

🧠 RL Loop

Agent → selects repo → Environment updates state → Reward → Repeat

---

🎮 Environment Design

🔹 Action

Action(repo_id: int)

👉 Agent selects a repository

---

🔹 Observation

Observation:
  available_repos: List[RepoInfo]
  current_state: State
  reward: float
  done: bool

---

🔹 State

Tracks:

State:
  mastered_skills: List[str]
  step_count: int
  progress_score: float

---

🧩 Repository Structure

Each repo contains:

RepoInfo:
  id: int
  skill: str
  difficulty: int
  prereqs: List[str]

---

🎯 Reward Function (IMPORTANT)

Reward is calculated using:

reward =
    +2   (new skill)
    -1   (duplicate skill)
    +1   (prereqs satisfied)
    -1   (wrong order)
    +0.5 * difficulty
    +0.1 * unique_skills

---

💡 Example

Scenario	Reward
Learn new skill correctly	High
Repeat same skill	Negative
Skip prerequisites	Penalty

---

🏁 Episode Termination

done = step_count >= 5

---

🧪 Grading System (Hackathon Critical 🚨)

✔ Task 1 — Exploration

unique_skills >= 3

---

✔ Task 2 — Dependency Correctness

All skills must follow prerequisites

---

✔ Task 3 — Efficiency

progress_score >= 5

---

🎯 Final Evaluation

env.evaluate()

Returns:

{
  "exploration": 0.0–1.0,
  "dependencies": 0.0–1.0,
  "efficiency": 0.0–1.0
}

---

🤖 Inference Script

File: inference.py

This script:

• Runs the environment
• Chooses actions (LLM or heuristic)
• Logs output in required format

---

📊 Required Log Format

[START] ...
[STEP] ...
[END] ...

---

🧠 Action Strategy

1. LLM-based (if API key present)

Uses:

MODEL_NAME = Qwen/Qwen2.5-72B-Instruct

---

2. Fallback Strategy

• Picks repo with:

  • unmet skill
  • satisfied prerequisites
  • highest difficulty

---

⚙️ Setup

1. Install dependencies

pip install -e .

---

2. Run environment locally

uvicorn server.app:app --reload

---

3. Test environment

python server/test.py

---

4. Run inference

python inference.py

---

🐳 Docker

docker build -t github_learner_env-env -f server/Dockerfile .

---

🌍 Deploy (Hugging Face)

openenv push

---

📁 Project Structure

github_learner_env/
│
├── client.py
├── models.py
├── inference.py   ← agent logic
├── openenv.yaml
│
└── server/
    ├── app.py
    ├── github_learner_env_environment.py  ← CORE LOGIC
    ├── repos.json
    └── Dockerfile

---

🧠 Core Learning Concept

This environment teaches:

• Sequential decision making
• Dependency planning
• Reward optimization

---

🚀 Why This Matters

Unlike simple environments, this project includes:

✅ Multi-step reasoning ✅ Dependency constraints ✅ Custom reward shaping ✅ Automated grading ✅ LLM + heuristic agent

---

⚠️ Notes

• Max steps: 5
• Max reward normalized to 0–1
• Works with or without LLM

---

🔥 Future Improvements

• Add dynamic repo generation
• Introduce time penalties
• Multi-agent competition
• Curriculum learning

---

🎯 Summary

This is a skill-based RL environment where agents must:

• Learn efficiently
• Follow correct order
• Maximize reward

---

💡 Built for Meta PyTorch OpenEnv Hackathon