Implement global rate limiter and refactor auth middleware

- Add global rate limiter utility with configurable limits and cleanup
- Move rate limiting logic to middleware for consistent application
- Update auth routes to use new rate limiter
- Add comprehensive tests for rate limiter functionality
- Update documentation with backend development guidelines and tasks

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
2026-04-30 21:26:31 +02:00
parent d1316ca66e
commit 3bd9848a08
9 changed files with 511 additions and 61 deletions

View File

@@ -2224,6 +2224,43 @@ The login endpoint (`POST /api/auth/login`) is protected against brute-force att
- Dependency: `LoginRateLimiterDep` in `app.dependencies`
### Global Rate Limiting
In addition to login-specific rate limiting, all API endpoints are protected by global per-IP rate limiting to prevent resource exhaustion, CPU spikes, and network bandwidth attacks from malicious or misconfigured clients.
**Design:**
- Uses a `dict[str, deque[float]]` keyed by client IP, storing request timestamps within a time window.
- Implements a sliding-window algorithm: when an IP exceeds the limit, subsequent requests are blocked until the oldest request timestamp in the window expires.
- Applied globally via middleware that runs on every request.
- Respects the same IP extraction logic (trusted proxies) as login rate limiting.
**Rate Limit Rules:**
- **Default limit:** 200 requests per 60 seconds per IP.
- Blocked requests return **HTTP 429 Too Many Requests** with a `Retry-After` header indicating the estimated seconds until the IP can retry.
- The `Retry-After` value is dynamically calculated based on when the oldest request in the window will expire.
- Different endpoints can be configured with different limits by adjusting the global rate limiter settings or using per-endpoint decorators (future enhancement).
**IP Extraction (Proxy Safety):**
- Same as login rate limiting: reads real client IP from `X-Forwarded-For` or `X-Real-IP` headers when the immediate connection is from a trusted proxy.
- Falls back to direct connection IP when headers cannot be trusted.
**Process-Local Limitation:**
- The global rate limiter is process-local (in-memory), like the login rate limiter.
- In single-worker deployments (enforced elsewhere), this is not a constraint.
- Each worker in a multi-worker setup maintains independent counters, which is acceptable under the single-worker enforcement model.
**Memory Management:**
- Old request timestamps outside the rate-limit window are automatically pruned during validation checks.
- A scheduled background task (`rate_limiter_cleanup` in `app.tasks.rate_limiter_cleanup`) runs every 30 minutes to remove dormant IPs from memory, preventing unbounded growth.
**Implementation:**
- Rate limiter: `app.utils.rate_limiter.GlobalRateLimiter`
- Middleware: `app.middleware.rate_limit.RateLimitMiddleware`
- IP extraction: `app.utils.client_ip.get_client_ip()`
- Cleanup task: `app.tasks.rate_limiter_cleanup` (registered in `app.startup`)
- Initialized in: `app.main.create_app()` and the lifespan handler
---
## 12. Authentication Endpoints

View File

@@ -1,53 +1,3 @@
## [CRITICAL] Docker containers lack resource limits
**Where found**
- `Docker/docker-compose.yml` — no `deploy.limits` or `deploy.reservations` sections
**Why this is needed**
Without resource limits, single container can consume all host CPU, memory, disk. "Noisy neighbor" scenario where backend memory leak → uses 100% RAM → OOM kill → host unresponsive.
**Goal**
Set hard and soft resource limits for all containers.
**What to do**
1. Add resource limits to `docker-compose.yml`:
```yaml
backend:
deploy:
limits:
cpus: '2'
memory: 512M
reservations:
cpus: '1'
memory: 256M
```
2. Document these limits in `Docs/Deployment.md`
3. For Kubernetes, add equivalent `resources.limits` and `resources.requests`
**Possible traps and issues**
- Limits set too low → OOM kill or throttling
- Backend may need more memory for large blocklists
- Test under expected load before finalizing
- Different environments may need different limits
**Docs changes needed**
- Update `Docker/docker-compose.yml` with `deploy` sections
- Add section in `Docs/Deployment.md` § Resource Allocation
**Doc references**
- `Docker/docker-compose.yml`
- `Docs/Deployment.md` (resource allocation)
---
## [CRITICAL] Global rate limiting missing
**Where found**