Add fail2ban DB index management and socket-based path resolution
- New get_fail2ban_db_path() in setup_service resolves DB path from configured socket path - New ensure_fail2ban_indexes() creates missing performance indexes on bans table - Call ensure_fail2ban_indexes on every startup before first ban query - Remove completed tasks from Docs/Tasks.md - Update Docs/PERFORMANCE.md with index findings
This commit is contained in:
@@ -95,4 +95,52 @@ Use `EXPLAIN QUERY PLAN` to verify index usage:
|
||||
EXPLAIN QUERY PLAN SELECT ip, COUNT(*) FROM history_archive WHERE timeofban >= ? GROUP BY ip;
|
||||
```
|
||||
|
||||
Expected: `USING INDEX idx_history_archive_timeofban` in the output.
|
||||
Expected: `USING INDEX idx_history_archive_timeofban` in the output.
|
||||
|
||||
---
|
||||
|
||||
## Fail2ban Database Indexes
|
||||
|
||||
BanGUI reads from fail2ban's SQLite database (`/var/run/fail2ban/fail2ban.db`). Query performance degrades without appropriate indexes.
|
||||
|
||||
### Current fail2ban bans Indexes
|
||||
|
||||
Fail2ban creates these indexes on the `bans` table:
|
||||
- `bans_jail_timeofban_ip` — composite (jail, timeofban, ip)
|
||||
- `bans_jail_ip` — composite (jail, ip)
|
||||
- `bans_ip` — single (ip)
|
||||
|
||||
**Missing**: standalone index on `timeofban` alone.
|
||||
|
||||
### BanGUI Automatic Index Creation
|
||||
|
||||
On startup, BanGUI calls `ensure_fail2ban_indexes()` to add missing indexes idempotently:
|
||||
|
||||
```python
|
||||
# From fail2ban_db_utils.py
|
||||
CREATE INDEX IF NOT EXISTS idx_bans_timeofban_desc ON bans(timeofban DESC);
|
||||
```
|
||||
|
||||
This improves queries like:
|
||||
```sql
|
||||
SELECT * FROM bans WHERE timeofban >= ? ORDER BY timeofban DESC;
|
||||
```
|
||||
|
||||
### Verifying Index Usage
|
||||
|
||||
Check if a query uses the index:
|
||||
```sql
|
||||
EXPLAIN QUERY PLAN SELECT * FROM bans WHERE timeofban >= 1700000000 ORDER BY timeofban DESC;
|
||||
-- With index: SEARCH USING INDEX idx_bans_timeofban_desc
|
||||
-- Without: SCAN TABLE bans
|
||||
```
|
||||
|
||||
### Adding Indexes to Migrations
|
||||
|
||||
For BanGUI's own `history_archive` table, indexes go in migrations via `_ Migration.add_table_indexes()`:
|
||||
|
||||
```python
|
||||
def _add_history_archive_indexes(m: Migration) -> None:
|
||||
m.add_index("history_archive", ["timeofban"], unique=False, if_not_exists=True)
|
||||
m.add_index("history_archive", ["jail", "timeofban"], unique=False, if_not_exists=True)
|
||||
```
|
||||
@@ -1,51 +1,3 @@
|
||||
### Issue #20: MEDIUM - No Correlation ID in Background Tasks
|
||||
|
||||
**Where found**:
|
||||
- All task files in `backend/app/tasks/`
|
||||
- Background tasks don't propagate correlation ID
|
||||
- Can't correlate task logs with triggering request
|
||||
|
||||
**Why this is needed**:
|
||||
Troubleshooting becomes hard:
|
||||
- Task fails
|
||||
- Logs show task name
|
||||
- Can't find what triggered it
|
||||
|
||||
**Goal**:
|
||||
Track correlation ID through entire request lifecycle including background tasks.
|
||||
|
||||
**What to do**:
|
||||
1. Use contextvars for correlation ID:
|
||||
```python
|
||||
from contextvars import ContextVar
|
||||
|
||||
correlation_id_var: ContextVar[str] = ContextVar('correlation_id', default='bg-task')
|
||||
|
||||
async def blocklist_import_task(source_id: str, correlation_id: str):
|
||||
token = correlation_id_var.set(correlation_id)
|
||||
try:
|
||||
logger.info(f"Starting import for source {source_id}")
|
||||
finally:
|
||||
correlation_id_var.reset(token)
|
||||
```
|
||||
2. Pass correlation ID to background tasks
|
||||
3. Include in structured logs
|
||||
4. Create task tracking UI showing correlation ID
|
||||
|
||||
**Possible traps and issues**:
|
||||
- Correlation ID must flow through all async contexts
|
||||
- Need to pass ID when scheduling tasks
|
||||
- Multiple nested tasks might have parent/child correlation IDs
|
||||
|
||||
**Docs changes needed**:
|
||||
- Add observability guide for background tasks
|
||||
- Document correlation ID format
|
||||
|
||||
**Doc references**:
|
||||
- DETAILED_FINDINGS.md - Issue #26 "Missing Correlation ID"
|
||||
|
||||
---
|
||||
|
||||
### Issue #21: MEDIUM - Missing Database Indexes for Performance
|
||||
|
||||
**Where found**:
|
||||
|
||||
Reference in New Issue
Block a user