π’ Hobby Project Notice: This is a research and learning project exploring FastAPI best practices. Feel free to use it as a reference, report issues, or suggest improvements! Contributions and feedback are always welcome.
A comprehensive FastAPI template with HTTP REST API and WebSocket real-time communication, featuring role-based access control (RBAC), Keycloak authentication, PostgreSQL database, and comprehensive monitoring.
π View Full Documentation | π Quick Start Guide | π― Development Guide
- Features
- Prerequisites
- Quick Start
- Installation
- Documentation
- Configuration
- Usage
- Architecture Overview
- API Reference
- Development
- Testing
- Troubleshooting
- Contributing
- License
- π Dual Protocol Support: HTTP REST API and WebSocket real-time communication
- π Authentication: Keycloak-based JWT token authentication with token caching
- π‘οΈ Authorization: Fine-grained RBAC with decorator-based permissions
- β±οΈ Rate Limiting: Redis-based sliding window rate limiting for HTTP and WebSocket
- π Audit Logging: Queue-based async audit logging with PostgreSQL storage
- ποΈ Database: PostgreSQL with async SQLModel/SQLAlchemy and Alembic migrations
- πΎ Caching: Redis integration for session management, rate limiting, and token caching
- π¦ Package Router: Custom routing system for WebSocket request handling with JSON schema validation
- β Type Safety: Strict type checking with mypy (--strict mode)
- π¨ Code Quality: Pre-commit hooks with ruff, bandit, interrogate, typos, and security scanning
- π§ͺ Testing: Comprehensive test suite with pytest, 81%+ coverage, load tests, and chaos tests
- π Documentation: Auto-generated docs with MkDocs Material, published to GitHub Pages
- β‘ Code Generation: Template-based WebSocket handler generator with AST validation
- π Monitoring: Prometheus metrics with Grafana dashboards (HTTP, WebSocket, DB, Redis, Keycloak, Circuit Breakers) β powered by
fastapi-telemetry - π Logging: Structured JSON logging with correlation IDs and Grafana Loki integration via Alloy β powered by
fastapi-correlation - π Alerting: Prometheus alerting rules for critical system events (errors, downtime, circuit breakers)
- π Security: Security headers middleware, request size limits, IP spoofing protection
- π Resilience: Circuit breaker pattern for Keycloak and Redis with fail-fast protection
- π³ Containerization: Docker Compose with health checks, Traefik reverse proxy, and BasicAuth
- β‘ Performance: Connection pooling, pagination caching, slow query detection
- π‘ Protocol Buffers: Optional protobuf support for WebSocket (30-50% smaller messages)
- π Python: 3.13+ (specified in pyproject.toml:6)
- π¦ uv: Modern Python package manager (installation guide)
- π³ Docker & Docker Compose: For containerized services
- ποΈ PostgreSQL: 14+ (provided via Docker)
- πΎ Redis: 7+ (provided via Docker)
- π Keycloak: Authentication server (provided via Docker)
# 1. Clone the repository
git clone https://github.com/acikabubo/fastapi-http-websocket.git
cd fastapi-http-websocket
# 2. Set up environment variables
cp docker/.srv_env.example docker/.srv_env
cp docker/.db_env.example docker/.db_env
cp docker/.kc_env.example docker/.kc_env
# Edit the .env files with your configuration
# 3. Start all services (PostgreSQL, Redis, Keycloak, Traefik, monitoring stack)
make start
# 4. Install dependencies and run database migrations
uv sync
make migrate
# 5. Start the development server with hot-reload
make serveπ Access Points:
- π Application:
http://localhost:8000 - π API Docs:
http://localhost:8000/docs - π Grafana:
http://localhost:3000(admin/admin) - π Prometheus:
http://prometheus.localhost(requires BasicAuth: admin/admin) - π Loki:
http://loki.localhost(requires BasicAuth: admin/admin) - π Keycloak:
http://localhost:8080(admin/admin)
πΊοΈ Services Hub: Open docker/hub.html in your browser (or run make hub) to get a quick-launch page with links to all local services.
Some cards show a
β run: make <command>hint when a service needs to be started separately (e.g.make serve,make start-server,make docs-serve).
# Install uv package manager (if not already installed)
curl -LsSf https://astral.sh/uv/install.sh | sh
# Install project dependencies
uv sync
# Install pre-commit hooks
uv run pre-commit install# Build all containers
make build
# Start services in detached mode
make start
# View logs
docker-compose logs -f
# Stop services
make stop
# Enter development shell
make shellThe project uses separate environment files for different services:
Application (docker/.srv_env):
# Environment
ENV=dev # dev, staging, production
# Database
DB_USER=postgres
DB_PASSWORD=postgres
DB_HOST=hw-db
DB_PORT=5432
DB_NAME=hw
# Redis
REDIS_IP=hw-redis
REDIS_PORT=6379
MAIN_REDIS_DB=1
AUTH_REDIS_DB=10
# Keycloak
KEYCLOAK_BASE_URL=http://hw-keycloak:8080
KEYCLOAK_REALM=hw
KEYCLOAK_CLIENT_ID=hw-client
KEYCLOAK_ADMIN_USERNAME=admin
KEYCLOAK_ADMIN_PASSWORD=admin
# Logging
LOG_LEVEL=INFO
LOG_CONSOLE_FORMAT=json # json or human
# Rate Limiting
RATE_LIMIT_ENABLED=true
RATE_LIMIT_PER_MINUTE=60
RATE_LIMIT_FAIL_MODE=open # open or closed
# Audit Logging
AUDIT_QUEUE_MAX_SIZE=10000
AUDIT_BATCH_SIZE=100Database (docker/.db_env):
POSTGRES_USER=postgres
POSTGRES_PASSWORD=postgres
POSTGRES_DB=hwKeycloak (docker/.kc_env):
KEYCLOAK_ADMIN=admin
KEYCLOAK_ADMIN_PASSWORD=admin
KC_DB=postgres
KC_DB_URL=jdbc:postgresql://hw-db:5432/keycloak
KC_DB_USERNAME=postgres
KC_DB_PASSWORD=postgres
KC_METRICS_ENABLED=trueSee .env.*.example files for complete configuration options.
Role-based permissions are defined directly in code using decorators:
WebSocket handlers (app/api/ws/handlers/):
@pkg_router.register(
PkgID.GET_AUTHORS,
json_schema=GetAuthorsModel,
roles=["get-authors"] # Required roles
)
async def get_authors_handler(request: RequestModel) -> ResponseModel:
...HTTP endpoints (app/api/http/):
from app.dependencies.permissions import require_roles
@router.get(
"/authors",
dependencies=[Depends(require_roles("get-authors"))]
)
async def get_authors():
...No external configuration file needed - all permissions are co-located with handler code.
All HTTP requests (except excluded paths) require Bearer token authentication:
# Login to get access token
curl -X POST http://localhost:8000/auth/login \
-H "Content-Type: application/json" \
-d '{"username": "user", "password": "password"}'
# Use token in subsequent requests
curl -X GET http://localhost:8000/api/authors \
-H "Authorization: Bearer <your-token>"GET /health- Health check endpointGET /api/authors- Get authors listPOST /api/authors- Create new author (requires admin role)
See app/api/http/ for all available HTTP endpoints.
// Connect with authentication token (URL-encoded Bearer token in query params)
const token = 'your-jwt-token';
const ws = new WebSocket(
`ws://localhost:8000/web?Authorization=${encodeURIComponent('Bearer ' + token)}`
);
ws.onopen = () => {
console.log('Connected to WebSocket');
};
ws.onmessage = (event) => {
const response = JSON.parse(event.data);
console.log('Received:', response);
};
ws.onerror = (error) => {
console.error('WebSocket error:', error);
};
ws.onclose = (event) => {
console.log('WebSocket closed:', event.code, event.reason);
};π Security Note: WebSocket authentication uses query parameters due to browser WebSocket API limitations. Always use WSS (WebSocket over TLS) in production and implement token refresh for long-lived connections. See CLAUDE.md for security best practices.
{
"pkg_id": 1,
"req_id": "550e8400-e29b-41d4-a716-446655440000",
"data": {}
}Paginated Request:
{
"pkg_id": 2,
"req_id": "550e8400-e29b-41d4-a716-446655440000",
"data": {
"page": 1,
"per_page": 20
}
}Note: If pagination params are omitted, defaults are page=1 and per_page=20
Regular Response:
{
"pkg_id": 1,
"req_id": "550e8400-e29b-41d4-a716-446655440000",
"status_code": 0,
"data": {...}
}Paginated Response:
{
"pkg_id": 2,
"req_id": "550e8400-e29b-41d4-a716-446655440000",
"status_code": 0,
"meta": {
"page": 1,
"per_page": 20,
"total": 45,
"pages": 3
},
"data": [...]
}sequenceDiagram
actor C as Client
participant WA as Web Application
participant S as Backend Server
alt Establishing WebSocket Connection
C->>WA: User action
WA->>S: GET /web?token=<jwt>
S-->>WA: 101 Switching Protocols
S->>WA: Send initial data to client
end
alt Make WebSocket Request/Response
WA->>S: Send request with pkg_id & data
S->>S: Validate permissions & handle request
S-->>WA: Return response with status_code
end
HTTP Requests:
- Request hits FastAPI endpoint
AuthenticationMiddlewareauthenticates user via Keycloak tokenrequire_roles()FastAPI dependency checks RBAC permissions (defined in decorator)- Request reaches endpoint handler in app/api/http/
WebSocket Requests:
- Client connects to
/webwith JWT token in query params PackageAuthWebSocketEndpointauthenticates the connection- Client sends JSON with
pkg_id,req_id, anddata PackageRoutervalidates permissions, validates data schema, and dispatches to handler- Handler returns
ResponseModelsent back to client
Central routing system for WebSocket requests defined in app/routing.py:
# Register a handler
@pkg_router.register(PkgID.GET_AUTHORS, json_schema=AuthorSchema)
async def get_authors_handler(request: RequestModel) -> ResponseModel:
# Handler logic
return ResponseModel.success(request.pkg_id, request.req_id, data=[...])Features:
- Dynamic handler registration using decorators
- Optional JSON schema validation
- Built-in permission checking
- Type-safe request/response models
app/auth.py - Keycloak JWT token validation powered by fastapi-keycloak-rbac (PyPI):
- Decodes JWT tokens from Authorization header (HTTP) or query string (WebSocket)
- Extracts user data into
UserModelwith roles - Configurable excluded paths via
EXCLUDED_PATHSregex
app/managers/rbac_manager.py - Singleton permission manager backed by fastapi-keycloak-rbac (PyPI):
check_ws_permission(pkg_id, user)- Validates WebSocket permissions frompkg_router.permissions_registryrequire_roles(*roles)- FastAPI dependency for HTTP permission checking- Permissions defined in code via decorators, not external config files
app/managers/keycloak_manager.py - Singleton Keycloak client wrapping fastapi-keycloak-rbac (PyPI):
- Adds circuit breaker protection (pybreaker) and Prometheus metrics on top of the package
login_async(username, password)- Returns access tokendecode_token(token)- Decodes and validates a JWT token- Configured via environment variables
app/middlewares/pipeline.py β middleware stack powered by two in-house packages:
fastapi-correlation(PyPI) βCorrelationIDMiddleware(injectsX-Correlation-IDper request),LoggingContextMiddleware(enriches every log record withendpoint,method,status_code,user_id),StructuredJSONFormatter/HumanReadableFormatterfastapi-telemetry(PyPI) βPrometheusMiddleware(HTTP request duration/count/in-progress metrics via injectable callbacks),CircuitBreakerMetricsListener(exports pybreaker state to Prometheus), safeget_or_create_*metric helpers
app/managers/websocket_connection_manager.py:
- Manages active WebSocket connections
broadcast(message)- Sends message to all connected clients- Connection lifecycle tracking with logging
app/storage/db.py - PostgreSQL with async SQLModel:
get_paginated_results(model, page, per_page, filters=...)- Pagination helper- Custom filter functions via
apply_filtersparameter - Automatic retry logic in
wait_and_init_db()
app/
βββ __init__.py # Application factory
βββ api/
β βββ http/ # HTTP endpoint routers
β β βββ author.py
β β βββ health.py
β βββ ws/
β βββ consumers/ # WebSocket endpoint classes
β βββ handlers/ # WebSocket request handlers
β βββ constants.py # PkgID and RSPCode enums
βββ managers/ # Singleton managers
β βββ rbac_manager.py
β βββ keycloak_manager.py
β βββ websocket_connection_manager.py
βββ middlewares/ # Custom middleware
βββ models/ # SQLModel database models
βββ schemas/ # Pydantic validation schemas
βββ tasks/ # Background tasks
βββ storage/ # Database and Redis utilities
Available PkgID values defined in app/api/ws/constants.py:
| PkgID | Value | Handler | Description | Required Role |
|---|---|---|---|---|
GET_AUTHORS |
1 | get_authors_handler |
Retrieve author list with optional filters (id, name, search_term) | get-authors |
GET_PAGINATED_AUTHORS |
2 | get_paginated_authors_handler |
Retrieve paginated authors with metadata (page, per_page, filters) | get-authors |
CREATE_AUTHOR |
3 | create_author_handler |
Create new author (requires name) | create-author |
UNREGISTERED_HANDLER |
999 | (none) | Test-only PkgID for testing unregistered handlers | (test only) |
Handler Location: All WebSocket handlers are in app/api/ws/handlers/author_handlers.py and registered using the @pkg_router.register() decorator.
RSPCode enumeration in app/api/ws/constants.py:4-30:
| Code | Value | Description |
|---|---|---|
OK |
0 | Operation completed successfully |
ERROR |
1 | General error occurred |
INVALID_DATA |
2 | Provided data is invalid or malformed |
PERMISSION_DENIED |
3 | User lacks required permissions |
# Add new PkgID to app/api/ws/constants.py
# Then generate handler from template
make new-ws-handlers
# Or manually create in app/api/ws/handlers/See CLAUDE.md for detailed instructions.
For comprehensive development workflows, commands, and best practices, see CLAUDE.md.
# Start development server with hot-reload
make serve
# Run tests
uv run pytest
# Run single test
uv run pytest tests/test_check.py::test_function_name
# Code quality checks
make ruff-check # Linting
uvx ruff format # Formatting
uvx mypy app/ # Type checking
uvx interrogate app/ # Docstring coverage (β₯80%)
# Security scanning
make bandit-scan # SAST scanning
make skjold-scan # Dependency vulnerabilities
make dead-code-scan # Find unused code
# WebSocket handlers
make ws-handlers # Show PkgID table
make new-ws-handlers # Generate new handlerAll commits must pass:
- ruff: Linting and formatting (79 char line length)
- mypy: Strict type checking
- interrogate: β₯80% docstring coverage
- typos: Spell checking
- bandit: Security scanning (low severity threshold)
- skjold: Dependency vulnerability checks
- Line length: 79 characters
- Type hints: Required on all functions (mypy --strict)
- Docstrings: Required on all public functions, classes, and methods
- Formatting: Double quotes, 4-space indentation
# Run all tests in parallel (default, uses pytest-xdist)
make test
# Or: uv run pytest -n auto tests
# Run tests sequentially (for debugging)
make test-serial
# Or: uv run pytest tests
# Run specific test file
uv run pytest tests/test_routing.py -v
# Run with coverage report
make test-coverage
# Or: uv run pytest --cov=app --cov-report=html tests
# Run load tests (marked with @pytest.mark.load)
uv run pytest -m load tests/load/ -v -s
# Run chaos tests (failure scenarios, marked with @pytest.mark.chaos)
uv run pytest -m chaos tests/chaos/ -v -s
# Skip slow tests
uv run pytest -m "not load and not chaos"β Test Infrastructure:
- Parallel Execution: Tests run in parallel by default (3-5x faster)
- Centralized Fixtures: Common test data in
tests/conftest.py - Mock Factories: Reusable mocks in
tests/mocks/(auth, Redis, WebSocket, repositories) - Coverage: 81%+ test coverage enforced by pre-commit hook
- Load Testing: WebSocket load tests (100-1000 concurrent connections)
- Chaos Engineering: Failure scenario tests (Redis down, DB failures, Keycloak unavailable)
See docs_site/development/testing.md for comprehensive testing guide.
Problem: Connection refused when connecting to services
Solution:
# Verify services are running
docker-compose ps
# Restart services
make stop
make start
# Check logs for errors
docker-compose logs -fProblem: Container exits immediately after upgrading PostgreSQL image (e.g. 13 β 18)
Root cause: PostgreSQL data format is incompatible across major versions. The old data volume cannot be read by the new engine.
π‘ Solution β dump & restore:
# 1. Revert to old version temporarily
# Edit docker/docker-compose.yml β set postgres image back to old version
# 2. Start the old container and dump ALL databases (app + Keycloak)
make start-server
docker exec hw-db pg_dump -U postgres hw > hw_backup.sql
docker exec hw-db pg_dump -U postgres keycloak > keycloak_backup.sql
# 3. Stop containers and delete the old data volume
make stop
docker volume rm fastapi-http-websocket_postgres-hw-data
# 4. Update to new postgres version in docker/docker-compose.yml
# Also update the volume mount path:
# OLD: /var/lib/postgresql/data
# NEW: /var/lib/postgresql (required for PostgreSQL 15+)
# 5. Start fresh container and restore both databases
make start-server
cat hw_backup.sql | docker exec -i hw-db psql -U postgres hw
cat keycloak_backup.sql | docker exec -i hw-db psql -U postgres keycloak
# 6. Run migrations to catch up any schema changes
make migrateNote: Dumping the
keycloakdatabase preserves all realms, users, roles, clients, and signing keys. Skipping this step will cause Keycloak to regenerate its RSA keys on first boot, invalidating all existing tokens (see Keycloak Major Version Upgrade below).
Problem: After upgrading Keycloak image, JWTMissingKey errors appear in the app
Root cause: Keycloak regenerates RSA signing keys on first startup with a new version (or after DB wipe). The app and Redis hold stale cached tokens/JWKS signed by the old keys.
π‘ Solution:
# 1. Clear the Redis token cache (forces fresh JWKS fetch)
docker exec hw-redis redis-cli FLUSHDB
# 2. Restart the app server to reload keys
docker compose -f docker/docker-compose.yml restart hw-server
# 3. Clear browser storage to remove stale tokens
# Chrome/Firefox: DevTools (F12) β Application β Local Storage β Clear All
# Or simply open an Incognito/Private window
# 4. Log out of all clients (browser, Grafana, etc.) and log back in
# Existing JWTs were signed with the old key and cannot be reusedProblem: 401 Unauthorized or invalid token errors
π‘ Solutions:
- Verify Keycloak is running:
curl http://localhost:8080 - Check realm and client configuration in
.env - Ensure token hasn't expired (tokens have limited lifetime)
- Verify user has required roles in Keycloak admin console
Problem: Database schema out of sync
π‘ Solutions:
# Reset database (WARNING: destroys data)
docker-compose down -v
make start
# Or manually run migrations
uv run alembic upgrade headProblem: WebSocket connections disconnect unexpectedly
π‘ Solutions:
- Check token validity in query parameters
- Verify user has required roles defined in handler's
@pkg_router.register(roles=[...])decorator - Check server logs for permission denied errors
- Ensure Redis is running for session management
Problem: ModuleNotFoundError when running the application
π‘ Solutions:
# Reinstall dependencies
uv sync
# Clear Python cache
find . -type d -name "__pycache__" -exec rm -rf {} +
find . -type f -name "*.pyc" -deleteProblem: Commits blocked by pre-commit hooks
π‘ Solutions:
# Run individual hooks to identify issues
uvx ruff check --config=pyproject.toml # Linting
uvx mypy app/ # Type checking
uvx interrogate app/ # Docstrings
# Auto-fix formatting issues
uvx ruff format- Check application logs:
docker-compose logs -f - Enable debug logging: Set
LOG_LEVEL=DEBUGin.env - Review CLAUDE.md for detailed development guidance
- Check Keycloak admin console:
http://localhost:8080/admin
- Create a feature branch from
main - Make your changes following code style requirements
- Ensure all pre-commit hooks pass
- Write tests for new functionality
- Update documentation if needed
- Submit a pull request to
mainbranch
- All code must pass pre-commit hooks (ruff, mypy, interrogate, etc.)
- Minimum 80% docstring coverage required
- Type hints required on all functions
- Follow 79-character line length limit
- Write tests for new features and bug fixes
- Never commit sensitive data (
.envfiles, tokens, passwords) - Run security scans before submitting PR:
make bandit-scan && make skjold-scan - Report security vulnerabilities privately to maintainers
This project is licensed under the MIT License - see the LICENSE file for details.
- β Test Coverage Improvement: Increased from 80% to 81%+ with comprehensive tests for logging and routing modules
- β
Documentation Reorganization: Moved all documentation to
docs_site/for better structure and GitHub Pages integration - β BasicAuth for Observability: Added authentication protection for Prometheus, Loki, and Grafana Alloy endpoints
- β Comprehensive Codebase Analysis: Identified 22 improvement opportunities across security, performance, testing, and documentation
See our GitHub Issues for planned improvements:
π΄ Critical Priority:
π High Priority:
- #127 - Replace Bare Exception Handling
- #128 - Cursor-based Pagination
- #129 - Edge Case Tests
- #130 - Type-Safe Filter Validation
- #131 - Pagination Cache Invalidation Guide
π‘ Medium Priority:
- #132 - Explicit Return Types
- #133 - Consolidate Repository Methods
- #134 - Database Connection Pool Monitoring
- #135 - WebSocket Protocol Documentation
π See all issues: GitHub Issues
β‘ Built with: FastAPI Β· PostgreSQL Β· Redis Β· Keycloak Β· Prometheus Β· Grafana Β· Loki Β· Modern Python Tooling
Full documentation available at: https://acikabubo.github.io/fastapi-http-websocket/
All documentation source files are in docs_site/ and automatically published to GitHub Pages.
- Installation - Environment setup and dependencies
- Quick Start - Get up and running in 5 minutes
- Configuration - Environment variables and settings
- Authentication - Keycloak authentication and RBAC
- HTTP Endpoints - REST API usage
- WebSocket Handlers - Real-time WebSocket communication
- Rate Limiting - Request throttling
- Audit Logging - User action tracking
- Database Operations - Repository pattern and pagination
- Monitoring - Prometheus, Grafana, Loki
- Overview - System architecture
- Design Patterns - Repository, Command, Dependency Injection
- Request Flow - HTTP and WebSocket flow diagrams
- RBAC System - Role-based access control
- Testing - Test suite, load tests, chaos tests
- Database Migrations - Alembic migration workflow
- Code Quality - Linting, formatting, type checking
- Protobuf Testing - Protocol Buffers usage
- Swagger UI Testing - API documentation testing
- Docker Deployment - Container orchestration
- Production Setup - Production configuration
- Security Guide - Security best practices
- BasicAuth for Observability - Protecting metrics endpoints
- Monitoring & Observability - Production monitoring stack
- Troubleshooting - Common issues and solutions
- CLAUDE.md - AI assistant development guidelines and project structure
- Need authentication? β Quick Start Guide
- Building WebSocket handlers? β WebSocket Guide
- Understanding architecture? β Design Patterns
- Want to contribute? β CLAUDE.md