feat: production API stack with security and observability - #2
Open
devhub-agent-maxim wants to merge 102 commits into
Open
feat: production API stack with security and observability#2devhub-agent-maxim wants to merge 102 commits into
devhub-agent-maxim wants to merge 102 commits into
Conversation
- claude-runner.js: replace hardcoded npm path with findClaudeCli() which scans VS Code extension installs (newest first), Claude desktop app, then falls back to PATH. Resolves 'claude.cmd is not recognized' error since Claude Code ships as a VS Code extension, not a global npm package. - video-intel-agent.js: add vt.tiktok.com pattern to SUPPORTED_PATTERNS so short share links like https://vt.tiktok.com/ZSHJmao5G are detected and processed correctly. Co-Authored-By: claude-flow <ruv@ruv.net>
TikTok blocks yt-dlp without impersonation. Add --impersonate chrome to both the caption fetch and audio download commands. Also add --no-check-formats to the audio extraction to avoid ffprobe codec detection failure when TikTok serves mp4 video that gets converted to mp3. Requires: pip install curl_cffi (installed on host) Co-Authored-By: claude-flow <ruv@ruv.net>
Three issues fixed: - Download raw video instead of --extract-audio to avoid ffprobe codec error on TikTok's proprietary bytevc1 format - Derive Python path from yt-dlp location (findPython()) instead of using system python3 which resolves to Windows Store stub - Use string concatenation instead of template literal for Whisper Python script to avoid backslash escaping issues on Windows paths - Use forward slashes in paths passed to Whisper (ffmpeg accepts them) Tested: vt.tiktok.com/ZSHJuqfjC → 2009 char transcript → 10/10 relevance summary delivered end-to-end via Telegram. Co-Authored-By: claude-flow <ruv@ruv.net>
- Fix TikTok audio: bytevc1/HEVC formats deliver video-only despite claiming aac; force best[vcodec=h264] which reliably includes audio - Add ffmpeg PATH injection into Python Whisper subprocess so it works in Windows CMD environment (not just bash) - Add findFfmpegDir() to locate C:\ffmpeg\...\bin reliably - Add npm global path to claude CLI discovery (AppData/Roaming/npm) - Preserve VSCode extension path as primary fallback for claude.cmd Co-Authored-By: claude-flow <ruv@ruv.net>
Goal 2: Intel pipeline quality improvements Goal 3: Agent self-improvement via log analysis Goal 4: Scaffold new TypeScript project Agent work loop was idle because goals.md had no active goals. Co-Authored-By: claude-flow <ruv@ruv.net>
…alidator AUTONOMOUS DECIDER (decider.js rewrite): - No longer requires goals.md to have content - Gathers full context: git status, projects, intel, daily log, tasks - Asks Claude "given what you see, what's the best thing to work on?" - Agent self-directs based on codebase state — true OpenClaw-style autonomy AUTO-COMMIT (git-ops.js, change-validator.js): - After every worker completes, validator checks git diff - Runs tests if available - AI reviews change quality (scores 1-10) - If score ≥ 6 AND tests pass: auto-commit + push to branch - Sends diff summary + suggestions to Telegram so Maxim can see exactly what changed JIRA INTEGRATION (jira.js): - Creates Jira ticket for every committed change - Configure via: JIRA_URL, JIRA_USER, JIRA_TOKEN, JIRA_PROJECT_KEY in .env CHANGE VALIDATOR (change-validator.js): - Scores each change 1-10 via Sonnet - Generates 3 specific suggestions for next work - Posts formatted report to Telegram: what changed, quality score, next steps Co-Authored-By: claude-flow <ruv@ruv.net>
Registers a new daily job at 07:00 AM that sends to Telegram: - Overnight work summary (bullet points via Sonnet) - All GitHub commits made since yesterday midnight - Open Jira tickets and their status - Reminder that agent continues running all day Schedule is now: 07:00 — Daily brief (GitHub + Jira + summary) 08:00 — Intel scraper (curated digest → Social Monitor) 02:00 — Nightly consolidation Co-Authored-By: claude-flow <ruv@ruv.net>
Worker: AUTO-1774717084032 Score: 6/10 Co-Authored-By: claude-flow <ruv@ruv.net>
Worker: AUTO-1774717230611 Score: 6/10 Co-Authored-By: claude-flow <ruv@ruv.net>
- Archive 2026-03-28.md (981K lines) to gzipped archive - Remove garbage files created by broken pipe errors - Create fresh 2026-03-29.md daily note - Update TASKS.md with completed TASK-004 Co-Authored-By: claude-flow <ruv@ruv.net>
Problem: - Work loop logged "waiting" every 10 min → 144+ entries/day - Caused yesterday's log to balloon to 981K lines - Led to EPIPE errors and file bloat Solution: - Remove memory.log() for routine "wait" states - Only log to daily note when spawning actual work - Keep console logging for debugging Impact: - Daily notes now contain only meaningful work - Prevents log bloat and associated errors - Aligns with Goal 3: Agent Self-Improvement Co-Authored-By: claude-flow <ruv@ruv.net>
Moved Goal 3 from Active to Completed. Agent now in steady-state — autonomous mode will identify future improvements. Co-Authored-By: claude-flow <ruv@ruv.net>
Documented work loop fix and current status. Agent now in steady-state autonomous mode. Co-Authored-By: claude-flow <ruv@ruv.net>
Co-Authored-By: claude-flow <ruv@ruv.net>
Worker: AUTO-1774719011864 Score: 6/10 Co-Authored-By: claude-flow <ruv@ruv.net>
Installed swagger-jsdoc and swagger-ui-express packages to provide interactive API documentation for all TODO endpoints. Changes: - Created src/swagger.ts with complete OpenAPI 3.0 specification - Defined schemas for all models with validation rules - Documented all endpoints: GET/POST /todos, GET/PUT/DELETE /todos/:id - Mounted Swagger UI at /api-docs in src/index.ts - Added test to verify /api-docs endpoint is accessible - All 33 tests passing, build successful Co-Authored-By: claude-flow <ruv@ruv.net>
Worker: AUTO-1774719636829 Score: 6/10 Co-Authored-By: claude-flow <ruv@ruv.net>
- SQLite database with better-sqlite3 - TodoRepository for persistence - Bearer token authentication middleware - 45 passing tests (39 persistence + 6 auth) - Environment-based API key validation Co-Authored-By: claude-flow <ruv@ruv.net>
Worker: AUTO-1774720108512 Score: 6/10 Co-Authored-By: claude-flow <ruv@ruv.net>
Implemented rate limiting using express-rate-limit to prevent API abuse: - Created src/middleware/rate-limiter.ts with 100 req/15min default - Applied rate limiter to all TODO routes in src/routes/todos.ts - Added 3 comprehensive tests in tests/rate-limiter.test.ts - Updated README.md with rate limiting documentation All 48 tests pass. Security enhancement complete. Co-Authored-By: claude-flow <ruv@ruv.net>
Worker: AUTO-1774720673152 Score: 6/10 Co-Authored-By: claude-flow <ruv@ruv.net>
Co-Authored-By: claude-flow <ruv@ruv.net>
Worker: AUTO-1774721262333 Score: 6/10 Co-Authored-By: claude-flow <ruv@ruv.net>
Added winston for structured JSON logging with configurable log levels and formats. Created request-logger middleware that logs all incoming requests with unique IDs, duration tracking, and comprehensive request/response details. Added 25 passing tests covering logger configuration and request logging functionality. Co-Authored-By: claude-flow <ruv@ruv.net>
Worker: AUTO-1774721859102 Score: 6/10 Co-Authored-By: claude-flow <ruv@ruv.net>
- Created src/middleware/validate.ts: Validation middleware factory - Created src/validation/todo-schemas.ts: Joi schemas for TODO CRUD - Added validation to POST /todos and PUT /todos/:id endpoints - Fixed validate middleware to use next(error) instead of throw - Added 24 comprehensive validation tests covering: * Required fields, empty values, type validation * Length limits (title: 200 chars, description: 1000 chars) * Whitespace trimming, unknown field stripping * Boundary conditions and edge cases - Updated error response format for consistency - All 105 tests passing Co-Authored-By: claude-flow <ruv@ruv.net>
- Added detailed validation rules section for title, description, and completed fields - Documented error messages and type constraints - Updated features list to include Joi validation - Updated test count to 105+ Co-Authored-By: claude-flow <ruv@ruv.net>
Co-Authored-By: claude-flow <ruv@ruv.net>
Worker: AUTO-1774723219021 Score: 6/10 Co-Authored-By: claude-flow <ruv@ruv.net>
- Install helmet and @types/helmet packages - Create src/middleware/security-headers.ts with strict CSP, HSTS, frameguard, and comprehensive security headers - Apply security headers middleware first in middleware chain - Add 11 comprehensive tests covering CSP, X-Frame-Options, X-Content-Type-Options, HSTS, DNS prefetch control, download options, referrer policy, cross-domain policies, and X-Powered-By removal - Update README.md with Security Headers section documenting all protections - All 128 tests passing Co-Authored-By: claude-flow <ruv@ruv.net>
Worker: AUTO-1774768974469 Score: 6/10 Co-Authored-By: claude-flow <ruv@ruv.net>
Previously, when Claude timed out generating commit messages, the commit would be titled "Task timed out after 1 minutes." Now extracts the real worker summary from daily log entries. Changes: - Added extractWorkerMessage() to parse "Worker done:" log entries - Updated generateCommitMessage() to use worker message directly - Intelligent commit type detection (feat/fix/refactor/chore/docs/test) - Fallback to Claude only if worker message not found - Better timeout protection Impact: - Future commits have meaningful messages like: "feat: add deployment guide to README" instead of "Task timed out after 1 minutes." - Faster (no Claude call in most cases) - Better commit history readability Co-Authored-By: claude-flow <ruv@ruv.net>
Worker: AUTO-1774769561548 Score: 6/10 Co-Authored-By: claude-flow <ruv@ruv.net>
- Add scripts/lib/sprint-manager.js: - Tracks active sprint per project - onWorkerDone() → immediately picks next backlog item (no 10-min wait) - Sprint complete detection → sends Telegram summary + board link - seedBacklogFromProjectMd() → seeds GitHub Issues from PROJECT.md backlog - Reads PROJECT.md brief to give agent rich context per task - Update scripts/lib/decider.js: - Check GitHub Issues backlog FIRST (step 2, before TASKS.md) - Pulls PROJECT.md sprint goal + constraints into worker prompt - Falls back to autonomous context-driven mode when backlog empty - Update scripts/agent.js onComplete handler: - After validator runs → call sprintMgr.onWorkerDone() - If backlog has more → immediately spawn next worker (3s pause only) - Sprint complete → single Telegram message with board link - Telegram silent during sprint (no per-worker noise) - Fix github-issues.js createIssue to accept status param (backlog/in-progress) - Seed 14 GitHub Issues backlog items from PROJECT.md (agent-tools + agent-dashboard) Result: restart start-agent.bat → agent team picks up GitHub Issues → works continuously until backlog empty → sends sprint complete summary to Telegram. Co-Authored-By: claude-flow <ruv@ruv.net>
Removed 640 lines of unused Jira integration code: - Deleted scripts/lib/jira.js (165 lines) - Deleted scripts/agents/jira-sync-agent.js (475 lines) - Removed all Jira references from scripts/agent.js (7 locations) - Updated memory/areas/integrations.md to remove Jira/Linear entries GitHub Issues now handles all task tracking with full agent integration: - Auto-create issues on worker spawn - Auto-close with completion status - Telegram notifications with issue links - Kanban workflow (backlog → in-progress → done) Impact: Simplified daily brief, removed unused dependencies, cleaner codebase Co-Authored-By: claude-flow <ruv@ruv.net>
…Telegram standups Removed all dead Jira integration code now that GitHub Issues has fully replaced it: Deleted files: - scripts/lib/jira.js (165 lines) - scripts/agents/jira-sync-agent.js (475 lines) Updated files to remove Jira references: - scripts/agent.js: Cleaned dailyBrief function (already done) - scripts/lib/config.js: Removed jira and linear config objects - scripts/lib/task-queue.js: Removed [jira] tag and jiraId extraction - scripts/heartbeat.js: Removed jira agent mapping - memory/areas/integrations.md: Removed Jira/Linear sections (already done) - memory/patterns/scheduling-pattern.md: Updated daily brief description - scripts/lib/schedule-manager.js: Updated daily brief description GitHub Issues integration: - ✅ Fully operational with Kanban workflow (backlog → in-progress → in-review → done) - ✅ Integrated with agent lifecycle (auto-create on worker start, auto-close on completion) - ✅ Telegram standup messages include issue links - ✅ Board URL: https://github.com/devhub-agent-maxim/Agent/issues?q=label%3Aagent-task Commit 1ab0f28 (2026-03-29 15:25) replaced Jira with GitHub Issues. This commit removes 640+ lines of dead code. Co-Authored-By: claude-flow <ruv@ruv.net>
Worker: AUTO-1774770787100 Score: 6/10 Co-Authored-By: claude-flow <ruv@ruv.net>
- Created fly.toml with production-ready configuration - Node 20 runtime via Dockerfile - 256MB RAM with auto-scaling - Health checks at /health endpoint - HTTPS enforced, Singapore region - Created FLY_DEPLOYMENT.md with complete deployment guide - Step-by-step authentication instructions - Environment variable setup - Deployment commands - Troubleshooting guide - Cost estimates and security notes - Installed Fly CLI at C:\Users\maxim\.fly\bin\flyctl.exe BLOCKED: Awaiting Fly.io authentication to complete deployment. Next step: Maxim needs to authenticate via browser, then run deploy commands. Co-Authored-By: claude-flow <ruv@ruv.net>
Rewrote extractWorkerMessage() to correctly parse worker completion entries from daily log. - Changed from regex to robust string split approach - Handles em dash (—) separators correctly - Removes time prefix "- HH:MM am/pm —" properly - Tested with 3 workers - all extracted successfully - Auto-commits now use actual worker summaries instead of "Task timed out" Co-Authored-By: claude-flow <ruv@ruv.net>
…ojects Added validateEnvironmentOrExit() to all three API projects (agent-tools, agent-dashboard, agent-scheduler). Implementation: - agent-tools: Required API_KEYS (16+ chars), optional PORT/CORS/DATABASE_PATH - agent-dashboard: Required AGENT_TOOLS_URL/AGENT_SCHEDULER_URL, optional PORT/CORS - agent-scheduler: All optional (PORT/DATABASE_PATH/CORS with defaults) - Server exits with code 1 and clear error messages if validation fails - Format validation for URLs, ports, file paths, comma-separated lists Testing: - 69 new validation tests across all projects - All 397 tests passing: agent-tools (171), agent-dashboard (131), agent-scheduler (95) - Tests cover: missing required vars, invalid formats, weak keys, boundary conditions Documentation: - Updated README.md for all projects with validation sections - Documented required vs optional variables - Listed validation rules (formats, ranges, minimums) - Example validation error messages Co-Authored-By: claude-flow <ruv@ruv.net>
Worker: AUTO-1774772591245 Score: 6/10 Co-Authored-By: claude-flow <ruv@ruv.net>
Worker: AUTO-1774773194562 Score: 6/10 Co-Authored-By: claude-flow <ruv@ruv.net>
Worker: AUTO-1774773748555 Score: 6/10 Co-Authored-By: claude-flow <ruv@ruv.net>
Worker: AUTO-1774773751176 Score: 6/10 Co-Authored-By: claude-flow <ruv@ruv.net>
Worker: AUTO-1774774364251 Score: 6/10 Co-Authored-By: claude-flow <ruv@ruv.net>
Worker: AUTO-1774774347635 Score: 6/10 Co-Authored-By: claude-flow <ruv@ruv.net>
Worker: AUTO-1774774954401 Score: 6/10 Co-Authored-By: claude-flow <ruv@ruv.net>
Worker: AUTO-1774774957193 Score: 6/10 Co-Authored-By: claude-flow <ruv@ruv.net>
Worker: AUTO-1774775546562 Score: 6/10 Co-Authored-By: claude-flow <ruv@ruv.net>
Every runClaude() call now: - Appends a JSONL entry to memory/usage-log.jsonl (timestamp, model, chars, summary) - Fires a rate-limited Telegram ping: '⚡ Claude call #N [summary] • model • ~Nk chars' - Increments session-level call + char counters New /usage Telegram command shows session totals + last 10 calls. Rate-limited to 1 Telegram ping per 15s to avoid spam. No blocking — Telegram send is always fire-and-forget. Closes: usage transparency request (too many silent Claude calls) Co-Authored-By: claude-flow <ruv@ruv.net>
After each successful auto-commit+push, the agent: 1. Creates (or surfaces existing) a PR from the sprint branch to main 2. Sends a Telegram card: PR title, score, commit SHA, + ✅ Merge / ❌ Skip buttons 3. ✅ Merge → squash-merges via GitHub API, confirms in Telegram 4. ❌ Skip → leaves PR open, dismisses card Also adds createOrGetPR() and mergePR() to github-issues.js. The validate() return value now includes prUrl and prNumber. Co-Authored-By: claude-flow <ruv@ruv.net>
Each project can now have its own Telegram topic (thread). Standup messages from the change validator are routed to the project's thread instead of the main group. New module: scripts/lib/project-threads.js - getThreadId(projectName) → thread ID or null (falls back to main) - registerThread(name, id) → persists to memory/project-threads.json - listAll() → dump the map New Telegram commands: - /threads → list all configured project threads - /threads set <name> <id> → register a new project thread Config lives in memory/project-threads.json (human-editable). TELEGRAM_PROJECT_THREADS env var can override it as a JSON string. Co-Authored-By: claude-flow <ruv@ruv.net>
…nal-improvement - memory/project-threads.json: blank map for agent-tools + agent-dashboard (fill in thread IDs via /threads set <name> <id> in Telegram) - GitHub Issues #5–18: labelled "internal-improvement" Co-Authored-By: claude-flow <ruv@ruv.net>
Worker: ISSUE-22 Score: 7/10 Co-Authored-By: claude-flow <ruv@ruv.net>
Worker: ISSUE-23 Score: 8/10 Co-Authored-By: claude-flow <ruv@ruv.net>
Worker: ISSUE-23 Score: 6/10 Co-Authored-By: claude-flow <ruv@ruv.net>
Worker: ISSUE-23 Score: 7/10 Co-Authored-By: claude-flow <ruv@ruv.net>
Worker: ISSUE-29 Score: 7/10 Co-Authored-By: claude-flow <ruv@ruv.net>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds agent-tools, agent-dashboard, and agent-scheduler with 303 tests