Skip to content

Repository files navigation

☁️ Cloud Platform FinOps

DevOps & Cloud Engineer | Terraform · ECS Fargate · GitHub Actions · Lambda Cost Governance

  • Multi-Account Governance: AWS Organizations with SCPs, OUs, and cross-account OIDC federation
  • Infrastructure as Code: 8 Terraform modules provisioning VPC, ECS Fargate, DynamoDB, monitoring across 3 environments
  • CI/CD Pipeline: GitHub Actions with OIDC auth, automated rollback, security gates, and LocalStack integration testing
  • FinOps Automation: Lambda-driven cost governance — business-hours scaling, orphaned resource scanning, anomaly detection
  • Zero-Cost Model: Full platform runs locally with LocalStack/Docker; on-demand AWS demo costs < £2/session
  • Security-First: tfsec/Trivy scanning, least-privilege IAM, private compute subnets, VPC Flow Logs

Tests Property Tests AWS Cost License: MIT


🎯 What This Project Demonstrates

Skill Area Components
🏗️ Infrastructure as Code Terraform modules, workspaces, remote state, multi-environment
☁️ Cloud Architecture Multi-account AWS Organizations, VPC design, private subnets
🐳 Container Orchestration ECS Fargate, ECR, multi-stage Docker builds, health checks
🔄 CI/CD GitHub Actions, OIDC federation, automated rollback, security gates
📡 Monitoring & Observability CloudWatch, X-Ray, structured logging, Grafana
🔒 Security tfsec/Checkov, Trivy, least-privilege IAM, Secrets Manager, VPC Flow Logs
💰 Cost Management Budgets, anomaly detection, scheduling, orphaned resource scanning
⚡ Serverless Lambda functions for automation (Node.js ESM)
🧪 Testing Unit, property-based (fast-check), integration, LocalStack
📝 Documentation ADRs, runbooks, architecture diagrams

🏗️ System Architecture

Architecture

🔄 Request Flow

Request Flow


🔄 Data Flow

sequenceDiagram
    participant Dev as Developer
    participant GH as GitHub Actions
    participant AWS as AWS (ECS/Lambda)
    participant LS as LocalStack

    Dev->>GH: Push PR
    GH->>GH: Lint → Test → Security Scan
    GH->>GH: Terraform Plan (comment on PR)
    Dev->>GH: Merge to main
    GH->>AWS: OIDC AssumeRole
    GH->>AWS: Docker Build → ECR Push
    GH->>AWS: Terraform Apply (staging)
    AWS->>GH: Health Check ✓
    GH->>AWS: Terraform Apply (production)
    
    Note over AWS: Lambda Cost Governance
    AWS->>AWS: 18:00 Scale Down (non-prod)
    AWS->>AWS: 08:00 Scale Up (business hours)
    AWS->>AWS: Daily Orphaned Resource Scan
    AWS->>AWS: Monthly Cost Report

    Note over LS: Local Development
    Dev->>LS: docker compose up
    LS->>LS: Full AWS emulation (£0)
Loading

📊 Test Coverage

Component Unit Property Integration Coverage
FinOps API 4 properties Full endpoint suite 80%+
Remediation Lambda 1 property LocalStack execution 90%+
Scanner Lambda 1 property LocalStack execution 90%+
Reporter Lambda 1 property LocalStack execution 90%+
Notification formatting 1 property Slack webhook mock 85%+
Terraform modules validate/plan LocalStack provisioning N/A

Property Invariants (fast-check, 100+ iterations each):

  1. Business hours scheduling decisions are always correct
  2. Orphaned resource classification has zero false positives
  3. Notification messages contain all required fields
  4. Cost report aggregation totals are mathematically consistent
  5. Structured JSON logging always produces valid JSON
  6. Cost summary API aggregation matches stored data
  7. DynamoDB data round-trips preserve all fields
  8. Invalid request parameters always return HTTP 400

🚀 Quick Start

Prerequisites

Local Development

# Clone
git clone https://github.com/Steven-Owen-21/cloud-platform-finops.git
cd cloud-platform-finops

# Environment setup
cp .env.example .env.local

# Start full local stack (LocalStack, DynamoDB Local, API, Grafana)
docker compose up -d

# Provision infrastructure locally
./scripts/localstack-deploy.sh

# Seed sample data
./scripts/seed-data.sh

# Verify
curl http://localhost:3000/health

Running Tests

# FinOps API unit & property tests
cd finops-api && npm test

# Lambda unit & property tests
cd lambdas && npm test

# Terraform validation
cd infrastructure && terraform validate

# Full integration suite (requires Docker)
docker compose up -d && npm run test:integration

Demo Deployment (Real AWS)

# One-command deploy via GitHub Actions
gh workflow run demo.yml

# Deploys in ~10-15 mins → outputs ALB URL → auto-teardown after demo

💰 Cost Model

Scenario Cost Notes
Local development £0 Docker + LocalStack
CI pipeline runs £0 GitHub Actions free (public repo)
Single demo (2 hours) < £2 Fargate + ALB + NAT Instance
Extended demo (4 hours) < £4 Alert triggers at 4-hour mark
Monthly ongoing £0 No always-on infrastructure

NAT Instance (t3.micro) saves ~£30/month vs NAT Gateway. Non-prod uses Fargate Spot. DynamoDB within Free Tier.


🌍 Environments

Environment Purpose Compute Cost
Local Development & testing Docker + LocalStack £0
Dev Feature development ECS Fargate Spot Free Tier
Staging Pre-production validation ECS Fargate Spot Free Tier
Production Live demonstration ECS Fargate (on-demand) On-demand only

Each environment: separate Terraform state, isolated VPC/CIDR, environment-specific tfvars, consistent tagging.


📁 Project Structure

cloud-platform-finops/
├── .github/workflows/
│   ├── ci.yml                  # PR: lint → test → security-scan → plan
│   ├── ci-integration.yml      # LocalStack integration tests
│   ├── cd.yml                  # Deploy: build → staging → prod
│   └── demo.yml                # One-command demo deploy & teardown
├── infrastructure/
│   ├── main.tf                 # Root Terraform configuration
│   ├── modules/
│   │   ├── network/            # VPC, subnets, NAT Instance, routing
│   │   ├── compute/            # ECS Fargate, ALB, auto-scaling
│   │   ├── data/               # DynamoDB tables, ECR repository
│   │   ├── monitoring/         # CloudWatch dashboards, alarms, X-Ray
│   │   ├── security/           # IAM roles, Secrets Manager, VPC Flow Logs
│   │   ├── governance/         # Budgets, Cost Anomaly Detection, Lambdas
│   │   ├── notifications/      # SNS topics, subscriptions
│   │   └── organizations/      # OUs, SCPs, cross-account roles
│   └── environments/
│       ├── dev/                 # Dev tfvars & overrides
│       ├── staging/            # Staging tfvars & overrides
│       ├── production/         # Production tfvars & overrides
│       └── local/              # LocalStack-targeted config
├── finops-api/
│   ├── src/                    # Express.js REST API source
│   ├── tests/                  # Unit, integration & property tests
│   ├── Dockerfile              # Multi-stage production build
│   └── package.json
├── lambdas/
│   ├── remediation/            # Business-hours ECS scaling
│   ├── scanner/                # Orphaned resource detection
│   ├── reporter/               # Monthly cost report generator
│   └── shared/                 # Common utilities across Lambdas
├── localstack/
│   └── init-scripts/           # LocalStack bootstrap scripts
├── scripts/
│   ├── localstack-deploy.sh    # Provision against LocalStack
│   ├── localstack-teardown.sh  # Tear down local resources
│   └── seed-data.sh            # Seed DynamoDB with sample data
├── docs/
│   └── adr/                    # Architecture Decision Records
├── docker-compose.yml          # Full stack compose
└── README.md                   # ← You are here

🛡️ Technology Stack

Category Technology Purpose
IaC Terraform Multi-environment provisioning with reusable modules
Cloud AWS (eu-west-2) Multi-account Organizations structure
Compute ECS Fargate Serverless container orchestration (Spot for non-prod)
CI/CD GitHub Actions OIDC auth, automated rollback, security gates
Application Node.js (ESM) FinOps Dashboard REST API
Database DynamoDB Serverless data store (Free Tier)
Monitoring CloudWatch + X-Ray Dashboards, alarms, structured logging, tracing
Security tfsec / Trivy Infrastructure and container scanning
Cost Budgets + Anomaly Detection Automated alerts and remediation
Serverless Lambda (Node.js) Scheduling, scanning, reporting
Notifications SNS + Slack Event-driven ops and cost alerts
Local Dev Docker + LocalStack Full AWS emulation at £0
Testing Vitest + fast-check Unit, property-based, integration
Observability Grafana Local metrics dashboard

📚 Documentation

Document Description
ADR-001 Why Terraform over CDK/CloudFormation
ADR-002 ECS Fargate selection rationale
ADR-003 GitHub Actions pipeline architecture
ADR-004 FinOps automation approach
ADR-005 LocalStack + on-demand deployment strategy
Centralised Logging Cross-account log aggregation design

📄 License

MIT

About

Multi-environment cloud platform: Terraform IaC, ECS Fargate, GitHub Actions CI/CD, Lambda cost governance. 198 tests, 8 property-based invariants. Zero-cost operational model.

Topics

Resources

Contributing

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages