A sophisticated CI/CD failure monitoring agent that uses LLM-powered analysis to diagnose build failures and apply safe, autonomous remediation actions. Integrated with Featherless AI and Bright Data for enhanced intelligence.
- 🤖 LLM-Powered Error Classification: Uses Featherless AI with chain-of-thought reasoning to analyze CI/CD failures
- 🔧 Autonomous Remediation: Detects root cause and applies appropriate fixes from a safe playbook
- 🛡️ Safety Guardrails: Confidence thresholds, retry limits, and validation to prevent unsafe actions
- 📊 Bright Data Enrichment: Cross-references errors with external signals and known issue database
- 📌 Human Feedback Loop: Notifies humans AFTER resolution, not before
- 📈 Learning & Monitoring: Tracks success rates and patterns to improve over time
- 🎨 Rich Slack Integration: Explainable summaries with reasoning steps and metrics
CI Pipeline Failure
↓
GitLab Webhook
↓
Error Log Extraction
↓
Bright Data Enrichment (external signals)
↓
LLM Analysis (chain-of-thought reasoning)
↓
Confidence & Safety Checks
↓
Playbook Action Selection
↓
Autonomous Execution
↓
Slack Notification (AFTER fixing)
↓
Monitoring & Analytics
The system identifies and handles:
- Flaky Tests - Transient test failures → Retry
- Dependency Issues - Package conflicts or missing deps → Reinstall/Clear Cache
- Environment Issues - Network/connectivity problems → Retry
- Timeout - Operations exceeding time limits → Retry with backoff
- Rate Limit - External service quota exceeded → Backoff & Retry
- Permission Error - Auth/credential issues → Manual review
- Code Error - Application logic errors → Manual review
- Node.js 18+
- npm dependencies
- GitLab Personal Access Token
- Slack Webhook URL
- Featherless AI API Key
- Bright Data API Key (optional)
# Clone & install
npm install
# Setup environment
cp .env.example .env
# Edit .env with your credentialsCreate a .env file:
# Server
PORT=3000
NODE_ENV=development
# GitLab
GITLAB_TOKEN=glpat_xxxxxxxxxxxxx
GITLAB_API_BASE=https://gitlab.com/api/v4
# Slack
SLACK_WEBHOOK=https://hooks.slack.com/services/...
# AI
FEATHERLESS_API_KEY=ftr_xxxxxxxxxxxxx
# Enrichment
BRIGHTDATA_API_KEY=xxxxxxxxxxxxx# Development
node server.js
# With logging
DEBUG=* node server.js
# Production
NODE_ENV=production node server.js-
Create a Personal Access Token in GitLab (Settings → Access Tokens)
- Scopes:
api,read_api
- Scopes:
-
Add webhook to your project:
- URL:
https://your-domain.com/webhook/gitlab - Events: Pipeline events
- Trigger: Failed pipelines
- URL:
-
Set
GITLAB_TOKENin.env
NEW: Test CI Agent on actual GitLab repositories with real CI/CD pipelines!
# 1. Start CI Agent
npm start
# 2. Follow the GitLab integration guide
# Follow: GITLAB_INTEGRATION.md
# 3. Create test repository with failing pipeline
# Create repository: ci-agent-test-repo
# 4. Trigger errors and watch CI Agent respond
# Push code that fails the pipeline
# Watch in real-time as CI Agent:
# ✅ Receives webhook
# ✅ Analyzes failure with LLM
# ✅ Applies safe remediation
# ✅ Sends Slack notification
# ✅ Tracks metricsFor a comprehensive step-by-step walkthrough, see:
REAL_WORLD_TESTING.md ← Start here!
This 70-minute guide covers:
- ✅ Setting up ngrok tunnel for webhooks
- ✅ Creating test GitLab repository
- ✅ Configuring GitLab webhook integration
- ✅ Testing each error classification type (7 types)
- ✅ Monitoring CI Agent in real-time
- ✅ Capturing evidence for presentation
- ✅ Live demo scenarios for judges
For detailed setup, see:
GITLAB_INTEGRATION.md ← Technical details
Step-by-step configuration of:
- GitLab Personal Access Token
- ngrok tunnel setup
- Environment variable configuration
- Webhook configuration
- Slack integration
- End-to-end testing
- Troubleshooting common issues
Your Test Repository
↓
You push code with error
↓
GitLab pipeline fails
↓
GitLab sends webhook to ngrok
↓
CI Agent receives event
↓
Extracts error logs from GitLab
↓
LLM analyzes with chain-of-thought
↓
Classifies error type (flaky? dependency? env?)
↓
Checks safety guardrails
↓
Applies remediation (retry, reinstall, etc.)
↓
Pipeline re-runs automatically
↓
Success! Pipeline passes
↓
Slack notification sent to team
↓
Metrics updated showing fix success
After following the real-world testing guide, you'll have:
| Metric | Expected Result |
|---|---|
| Test Errors Created | 5-7 different error types |
| Classification Accuracy | 90%+ correct categorization |
| Autonomous Fixes Applied | 12-15 automatic remediation actions |
| Fix Success Rate | 85-90% successful resolutions |
| Response Time | <30 seconds error detection |
| Slack Notifications | 15-20 rich notification messages |
| Pipeline Efficiency Gain | 25-30% faster resolution |
This real-world testing demonstrates to judges:
- ✅ NOT just unit tests - actual GitLab integration working
- ✅ Live failures - real CI/CD errors being detected
- ✅ Autonomous action - system fixing problems automatically
- ✅ Safety enforcement - all actions validated before execution
- ✅ Team communication - Slack notifications with reasoning
- ✅ Measurable impact - success rates proving value
# Terminal 1: CI Agent running
npm start
# Terminal 2: Monitor real-time logs
tail -f logs/combined.log
# Terminal 3: Push test code to trigger pipeline
cd ci-agent-test-repo
echo "test" >> test.txt
git add test.txt
git commit -m "Trigger test pipeline"
git push origin mainThen show judges:
- Pipeline failing on GitLab
- CI Agent terminal showing real-time analysis
- Slack notification with automated fix + reasoning
- Pipeline re-running and succeeding
- Metrics showing success rate improvement
Track project success rates:
const monitor = require('./services/monitor');
// Get success rate for a project
const rate = monitor.getSuccessRate(projectId);
// Get recent execution history
const history = monitor.getRecentHistory(pipelineId, limit);
// Analyze recurring patterns
const patterns = monitor.getPipelinePattern(projectId);- Max 2 auto-retries per pipeline
- Prevents infinite loops
- Escalates to manual review after threshold
- Requires ≥75% confidence for auto-action
- High severity errors require ≥85% confidence
- Low confidence → manual review
- Actions matched to error types
- Prevents inappropriate fixes
- Falls back to manual review if validation fails
- Configurable cooldown between actions
- Per-action retry logic with exponential backoff
Safe remediation actions mapped to error types:
{
flaky_test: {
primary: "retry",
fallback: ["notify"],
maxAttempts: 2
},
dependency_issue: {
primary: "reinstall",
fallback: ["clear_cache", "retry"],
maxAttempts: 2
},
env_issue: {
primary: "retry",
fallback: ["update_env", "notify"],
maxAttempts: 2
}
// ... more error types
}Supported Actions:
retry- Restart the pipelinereinstall- Clear cache and reinstall dependenciesclear_cache- Clear build cacherestart_service- Restart external serviceupdate_env- Update environment variablesnotify- Escalate to manual review
Uses Featherless AI with:
- Chain-of-Thought Reasoning: Step-by-step analysis for better accuracy
- Multi-Context Analysis:
- Error logs
- Stack traces
- External signals (Bright Data)
- Log metadata
- Confidence Scoring: 0-1 scale to indicate analysis reliability
- Fallback Classification: Heuristic-based patterns when LLM fails
Enriches error analysis with external signals:
- Known issue database matching
- External service status
- Pattern correlation
- Suggestion generation
Fallback uses local pattern database:
- npm registry issues
- Python PyPI problems
- Network/SSL issues
- Disk space / memory issues
- Version conflicts
Rich, explainable notifications include:
{
"error_classification": "dependency_issue",
"confidence": "92%",
"severity": "medium",
"root_cause": "npm package version conflict",
"action_taken": "reinstall",
"reasoning": [
"1. Detected npm ERR! pattern",
"2. Version mismatch identified",
"3. Selected safe reinstall action"
],
"success_rate": "87%",
"new_pipeline_link": "https://gitlab.com/-/pipelines/12345"
}.
├── server.js # Express app
├── package.json # Dependencies
├── .env.example # Environment template
├── routes/
│ └── webhook.js # GitLab webhook handler
├── services/
│ ├── ai.js # LLM classification
│ ├── brightdata.js # External enrichment
│ ├── executor.js # Action execution
│ ├── gitlab.js # GitLab API
│ ├── monitor.js # Analytics
│ ├── playbook.js # Action selection
│ └── slack.js # Notifications
├── config/
│ └── playbook-guard.js # Safety guardrails
└── utils/
├── logger.js # Logging
├── llm-reasoning.js # Chain-of-thought
└── parser.js # Log parsing
- Webhook Reception → Validate GitLab event
- Log Extraction → Fetch & parse job logs
- Enrichment → Query Bright Data for signals
- Analysis → LLM classification with reasoning
- Validation → Check confidence & safety rules
- Action Selection → Choose from safe playbook
- Execution → Apply fix to pipeline
- Notification → Send Slack summary
- Tracking → Record metrics & patterns
# Health check
curl http://localhost:3000/
# Webhook simulation
curl -X POST http://localhost:3000/webhook/gitlab \
-H "Content-Type: application/json" \
-d @test-payload.json# Development logging
tail -f logs/combined.log
# Error logs only
tail -f logs/error.log- Check
FEATHERLESS_API_KEYin.env - Verify Featherless AI API status
- Check fallback heuristics are working
- Optional - system continues with local patterns
- Verify API key if integration is needed
- Check rate limits
- Verify
SLACK_WEBHOOKURL - Check Slack app permissions
- Review notification size limits
- Check log quality
- Verify error patterns in logs
- Review LLM prompt
Key metrics to track:
{
total_failures: 150,
resolved_auto: 142, // 94.7% auto-resolution
failed_auto: 5,
manual_reviews: 3,
by_type: {
flaky_test: 42,
dependency_issue: 67,
env_issue: 28,
code_error: 13
},
avg_resolution_time: "2.3s"
}- ✅ No deployment or production changes without human approval
- ✅ Actions limited to safe operations (retry, reinstall, cache clear)
- ✅ Confidence requirements prevent over-zealous fixes
- ✅ All actions logged for audit trail
- ✅ Secrets in
.envnot committed to git
- Slack slash commands for manual actions
- Database persistence for long-term analytics
- ML model training on resolution patterns
- GitHub/Gitea support
- Custom action plugins
- Webhook signature verification
- Rate limiting by project
MIT
- Fork the repository
- Create feature branch
- Commit changes
- Push and create PR
Note: This agent is designed to resolve transient CI/CD failures autonomously while maintaining safety guardrails. For errors requiring developer intervention, manual review is requested AFTER attempting automatic fixes, not before.