Add explicit database transaction isolation to multi-step operations

This commit addresses race conditions in multi-step database operations by:

1. Wrap write operations in BEGIN IMMEDIATE ... COMMIT transactions:
   - import_run_repo: create_pending, mark_completed, mark_failed
   - geo_cache_repo: all upsert_*_and_commit functions
   - geo_cache_repo: bulk_upsert_entries_and_neg_entries_and_commit

2. Handle concurrent write collisions gracefully:
   - import_run_repo.create_pending can now raise IntegrityError
   - blocklist_import_workflow catches IntegrityError and retries lookup
   - Logs 'blocklist_import_lost_race' event when another request wins the race

3. Add comprehensive documentation:
   - Backend-Development.md § 6.3 Database Transactions
   - Explains when to use BEGIN IMMEDIATE
   - Shows transaction pattern with try-except-rollback
   - Documents race condition error handling pattern

The solution leverages SQLite's UNIQUE constraint for data integrity while
handling the concurrent case gracefully in application logic. This is more
efficient than using BEGIN EXCLUSIVE which would serialize all writers.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
2026-04-30 22:04:15 +02:00
parent 94d6352d1d
commit f9e283541b
5 changed files with 259 additions and 112 deletions

View File

@@ -1,48 +1,3 @@
## [CRITICAL] Health check endpoint returns wrong status code
**Where found**
- `backend/app/routers/health.py` — always returns 200, even when fail2ban offline
**Why this is needed**
Docker health checks interpret 200 as "healthy". If fail2ban offline but backend returns 200, Docker thinks container healthy and doesn't restart it.
**Goal**
Return 503 Service Unavailable when fail2ban is offline.
**What to do**
1. Change health endpoint to return 503 when offline:
```python
if not server_status.online:
return JSONResponse(
status_code=503,
content={"status": "unavailable", "fail2ban": "offline"}
)
```
2. Update Docker health check to expect 503 as "unhealthy"
**Possible traps and issues**
- Returning 503 causes orchestration tools to restart container
- If fail2ban restarts frequently, health check becomes flaky
- Consider gradual degradation
**Docs changes needed**
- Update `Docker/Dockerfile.backend` health check documentation
- Update `Docs/Deployment.md` § Health Checks
**Doc references**
- `backend/app/routers/health.py`
- `Docker/Dockerfile.backend`
---
## [IMPORTANT] Database transactions lack explicit isolation
**Where found**