feat: implement API versioning /api/v1/
- All backend routers moved to /api/v1/ prefix
- Frontend BASE_URL updated to /api/v1
- Setup redirect middleware updated to redirect to /api/v1/setup
- Health router path fixed: prefix=/api/v1/health, @router.get('')
- conftest.py: set server_status=online for test fixture
- Created Docs/API_VERSIONING.md with deprecation policy
- Updated Docs/Backend-Development.md with versioning section
- Updated Instructions.md curl examples
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
@@ -1,97 +1,3 @@
|
||||
## HIGH PRIORITY ISSUES
|
||||
|
||||
---
|
||||
|
||||
### Issue #3: HIGH - Unbounded Query Results Causing OOM (Out of Memory)
|
||||
|
||||
**Where found**:
|
||||
- `backend/app/repositories/history_archive_repo.py` - `get_all_archived_history()`
|
||||
- `backend/app/services/ban_service.py` (lines 589-600) - `bans_by_country()` loads all unique IPs into memory
|
||||
- `backend/app/services/ban_service.py` (lines 650-680) - N+1 geo lookup pattern
|
||||
|
||||
**Why this is needed**:
|
||||
With large deployments having millions of ban records, queries that load entire tables into memory cause:
|
||||
- Memory spikes that crash the container
|
||||
- Slow dashboard performance
|
||||
- Database file growth without bounds
|
||||
|
||||
**Goal**:
|
||||
Implement pagination, streaming, and batch processing for all large queries to ensure bounded memory usage and consistent performance.
|
||||
|
||||
**What to do**:
|
||||
1. Refactor `get_all_archived_history()` to only be called with pagination parameters
|
||||
2. Refactor `bans_by_country()` to:
|
||||
- Process countries in batches
|
||||
- Stream results instead of collecting all in memory
|
||||
- Implement server-side aggregation in SQL instead of Python loops
|
||||
3. Add `LIMIT` + `OFFSET` or cursor-based pagination to all list endpoints
|
||||
4. Implement batch geo lookups instead of per-IP loops
|
||||
5. Add tests with large datasets (1M+ records) to catch performance regressions
|
||||
|
||||
**Possible traps and issues**:
|
||||
- Changing query patterns might break sorting/filtering logic
|
||||
- Pagination cursor format must be consistent across endpoints
|
||||
- Memory usage must be monitored in production
|
||||
- Aggregation queries might need new database indexes
|
||||
- Frontend pagination UI assumes cursor format - changes will break old clients
|
||||
|
||||
**Docs changes needed**:
|
||||
- Add performance guidelines to `Docs/Backend-Development.md` - "Never load unbounded result sets"
|
||||
- Create `Docs/PERFORMANCE.md` with query optimization patterns
|
||||
- Document pagination standards in API docs
|
||||
|
||||
**Doc references**:
|
||||
- DETAILED_FINDINGS.md - Issues #2, #3, #4 (Unbounded queries, N+1, Large structures)
|
||||
- DATABASE_API_DEPLOYMENT_ISSUES.md - Section "Database Design Issues"
|
||||
|
||||
---
|
||||
|
||||
### Issue #4: HIGH - Missing Rate Limiting on Write Operations
|
||||
|
||||
**Where found**:
|
||||
- `backend/app/middleware/rate_limit.py` - Only applied to login endpoint
|
||||
- `backend/app/routers/bans.py` - POST /api/bans/ban, POST /api/bans/unban (NO rate limit)
|
||||
- `backend/app/routers/blocklist.py` - POST /api/blocklists/:id/import (NO rate limit)
|
||||
- `backend/app/routers/config.py` - PUT endpoints (NO rate limit)
|
||||
|
||||
**Why this is needed**:
|
||||
Without rate limiting on state-mutating endpoints, an attacker can:
|
||||
- Spam ban requests to exhaust fail2ban resources
|
||||
- Trigger repeated blocklist imports consuming bandwidth/CPU
|
||||
- Cause DoS by hammering config updates
|
||||
|
||||
**Goal**:
|
||||
Extend rate limiting to all write operations (POST, PUT, DELETE) with appropriate rate limits per operation type.
|
||||
|
||||
**What to do**:
|
||||
1. Create rate limit buckets for different operations:
|
||||
- `bans:ban` - 100/minute per IP
|
||||
- `bans:unban` - 100/minute per IP
|
||||
- `blocklist:import` - 10/hour per IP
|
||||
- `config:update` - 50/minute per IP
|
||||
2. Apply rate limiting middleware to all write endpoints
|
||||
3. Return 429 with `Retry-After` header when limit exceeded
|
||||
4. Add metrics/monitoring for rate limit hits
|
||||
5. Make rate limits configurable via environment variables
|
||||
|
||||
**Possible traps and issues**:
|
||||
- Rate limiting at IP level doesn't work behind proxies (need proper X-Forwarded-For handling)
|
||||
- Different operations need different rate limits (can't use global limit)
|
||||
- Legitimate bulk operations might hit limits unexpectedly
|
||||
- Rate limit state must be persistent across process restarts (use database or Redis)
|
||||
- False positives from internal monitoring scripts hammering endpoints
|
||||
|
||||
**Docs changes needed**:
|
||||
- Add rate limit table to API documentation
|
||||
- Document in `Docs/CONFIGURATION.md` how to adjust rate limits
|
||||
- Add to `Docs/TROUBLESHOOTING.md` - "Getting 429 Too Many Requests"
|
||||
|
||||
**Doc references**:
|
||||
- DETAILED_FINDINGS.md - Issue #5 "Missing Rate Limiting"
|
||||
- `backend/app/middleware/rate_limit.py` - Current implementation
|
||||
|
||||
---
|
||||
|
||||
### Issue #5: HIGH - API Has No Versioning Strategy
|
||||
|
||||
**Where found**:
|
||||
|
||||
Reference in New Issue
Block a user