Add Application Performance Monitoring (APM) with Prometheus metrics

- Backend: Implement Prometheus metrics collection
  - Add prometheus-client dependency
  - Create metrics utility module with HTTP request tracking counters, histograms, gauges
  - Implement MetricsMiddleware to track request latency, count, and active requests
  - Add /metrics endpoint to expose metrics in Prometheus text format
  - Normalize paths to prevent cardinality explosion (e.g., /api/{id} for UUIDs)
  - Exclude /metrics and /health from detailed tracking

- Frontend: Add web vitals and API metrics collection
  - Install web-vitals library (v4.0.0) for Core Web Vitals tracking
  - Create metrics utility module for FCP, LCP, CLS, INP, TTFB collection
  - Implement useTrackedFetch hook for automatic API call metrics (method, endpoint, status, duration)
  - Initialize web vitals tracking in App component on mount
  - Provide exportMetrics() for sending metrics to backend

- Testing:
  - Add comprehensive backend metrics tests (9 tests, 100% coverage)
  - Add comprehensive frontend metrics tests (10 tests)
  - All tests passing

- Documentation:
  - Expand Docs/Observability.md with complete APM section
  - Include metrics reference, integration examples (Prometheus, Datadog, NewRelic)
  - Add troubleshooting guide and best practices for cardinality management
  - Update Tasks.md to mark APM task as complete

Metrics exposed:
- bangui_http_requests_total: HTTP request count by method, endpoint, status
- bangui_http_request_duration_seconds: Request latency histogram
- bangui_http_active_requests: Active request gauge
- Web Vitals: CLS, FCP, INP, LCP, TTFB with ratings
- API metrics: endpoint, method, status, duration, timestamp

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
2026-05-01 18:33:14 +02:00
parent 37078b742b
commit 1af67eb0ce
14 changed files with 969 additions and 74 deletions

View File

@@ -0,0 +1,95 @@
"""Metrics collection middleware for BanGUI.
Tracks HTTP request count, latency, and active requests.
Excludes the /metrics endpoint to prevent recursive metrics collection.
"""
from __future__ import annotations
import re
import time
from typing import TYPE_CHECKING
import structlog
from starlette.middleware.base import BaseHTTPMiddleware
from app.utils.metrics import http_active_requests, http_request_count, http_request_latency
if TYPE_CHECKING:
from collections.abc import Awaitable, Callable
from starlette.requests import Request
from starlette.responses import Response
log = structlog.get_logger()
# Paths excluded from detailed metrics (to avoid cardinality explosion)
EXCLUDED_PATHS = {"/metrics", "/health", "/api/health"}
# Pattern to normalize endpoint paths (convert IDs to placeholders)
PATH_PATTERN = re.compile(r"/api/[^/]+/[a-f0-9\-]{36}|/api/[^/]+/\d+")
def _normalize_path(path: str) -> str:
"""Normalize path by replacing IDs with placeholders.
Converts paths like /api/resource/123 to /api/resource/{id}
to prevent cardinality explosion from dynamic IDs.
Args:
path: The request path.
Returns:
Normalized path with IDs replaced by {id}.
"""
return PATH_PATTERN.sub(r"/api/{id}", path)
class MetricsMiddleware(BaseHTTPMiddleware):
"""Middleware to collect Prometheus metrics for HTTP requests."""
async def dispatch(
self,
request: Request,
call_next: Callable[[Request], Awaitable[Response]],
) -> Response:
"""Collect metrics for the request and response.
Args:
request: The incoming request.
call_next: The next middleware/route handler.
Returns:
The response.
"""
# Skip metrics for excluded paths
if request.url.path in EXCLUDED_PATHS:
return await call_next(request)
method: str = request.method
endpoint: str = _normalize_path(request.url.path)
# Track active requests
http_active_requests.labels(method=method, endpoint=endpoint).inc()
start_time = time.perf_counter()
status_code = 500
try:
response: Response = await call_next(request)
status_code = response.status_code
return response
finally:
# Record metrics
duration: float = time.perf_counter() - start_time
http_request_latency.labels(method=method, endpoint=endpoint).observe(duration)
http_request_count.labels(method=method, endpoint=endpoint, status_code=status_code).inc()
http_active_requests.labels(method=method, endpoint=endpoint).dec()
log.debug(
"http_request_recorded",
method=method,
endpoint=endpoint,
status_code=status_code,
duration_ms=duration * 1000,
)