Add explicit database transaction isolation to multi-step operations
This commit addresses race conditions in multi-step database operations by: 1. Wrap write operations in BEGIN IMMEDIATE ... COMMIT transactions: - import_run_repo: create_pending, mark_completed, mark_failed - geo_cache_repo: all upsert_*_and_commit functions - geo_cache_repo: bulk_upsert_entries_and_neg_entries_and_commit 2. Handle concurrent write collisions gracefully: - import_run_repo.create_pending can now raise IntegrityError - blocklist_import_workflow catches IntegrityError and retries lookup - Logs 'blocklist_import_lost_race' event when another request wins the race 3. Add comprehensive documentation: - Backend-Development.md § 6.3 Database Transactions - Explains when to use BEGIN IMMEDIATE - Shows transaction pattern with try-except-rollback - Documents race condition error handling pattern The solution leverages SQLite's UNIQUE constraint for data integrity while handling the concurrent case gracefully in application logic. This is more efficient than using BEGIN EXCLUSIVE which would serialize all writers. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
@@ -1,48 +1,3 @@
|
||||
## [CRITICAL] Health check endpoint returns wrong status code
|
||||
|
||||
**Where found**
|
||||
|
||||
- `backend/app/routers/health.py` — always returns 200, even when fail2ban offline
|
||||
|
||||
**Why this is needed**
|
||||
|
||||
Docker health checks interpret 200 as "healthy". If fail2ban offline but backend returns 200, Docker thinks container healthy and doesn't restart it.
|
||||
|
||||
**Goal**
|
||||
|
||||
Return 503 Service Unavailable when fail2ban is offline.
|
||||
|
||||
**What to do**
|
||||
|
||||
1. Change health endpoint to return 503 when offline:
|
||||
```python
|
||||
if not server_status.online:
|
||||
return JSONResponse(
|
||||
status_code=503,
|
||||
content={"status": "unavailable", "fail2ban": "offline"}
|
||||
)
|
||||
```
|
||||
|
||||
2. Update Docker health check to expect 503 as "unhealthy"
|
||||
|
||||
**Possible traps and issues**
|
||||
|
||||
- Returning 503 causes orchestration tools to restart container
|
||||
- If fail2ban restarts frequently, health check becomes flaky
|
||||
- Consider gradual degradation
|
||||
|
||||
**Docs changes needed**
|
||||
|
||||
- Update `Docker/Dockerfile.backend` health check documentation
|
||||
- Update `Docs/Deployment.md` § Health Checks
|
||||
|
||||
**Doc references**
|
||||
|
||||
- `backend/app/routers/health.py`
|
||||
- `Docker/Dockerfile.backend`
|
||||
|
||||
---
|
||||
|
||||
## [IMPORTANT] Database transactions lack explicit isolation
|
||||
|
||||
**Where found**
|
||||
|
||||
Reference in New Issue
Block a user