Commit Graph

7 Commits

Author SHA1 Message Date
28f7b1cfcd Make geo lookups non-blocking with bulk DB writes and background tasks 2026-03-12 18:10:00 +01:00
a61c9dc969 Fix empty error field in geo_lookup_request_failed log events
- Replace str(exc) with repr(exc) in lookup() and _batch_api_call()
  so exception class name is always present even for no-message errors
  (e.g. aiohttp.ServerDisconnectedError() whose str() is empty)
- Add exc_type=type(exc).__name__ field to network-error log events
  for easy structured-log filtering
- Move import aiohttp to runtime import; use aiohttp.ClientTimeout()
  instead of raw float, removing # type: ignore[arg-type] workarounds
- Add TestErrorLogging with 3 tests covering empty-message exceptions
2026-03-12 17:50:58 +01:00
44a5a3d70e Fix geo cache write performance: batch commits, read-only GETs, dirty flush
- Remove per-IP db.commit() from _persist_entry() and _persist_neg_entry();
  add a single commit after the full lookup_batch() chunk loop instead.
  Reduces commits from ~5,200 to 1 per bans/by-country request.

- Remove db dependency from GET /api/dashboard/bans and
  GET /api/dashboard/bans/by-country; pass app_db=None so no SQLite
  writes occur during read-only requests.

- Add _dirty set to geo_service; _store() marks resolved IPs dirty.
  New flush_dirty(db) batch-upserts all dirty entries in one transaction.
  New geo_cache_flush APScheduler task flushes every 60 s so geo data
  is persisted without blocking requests.
2026-03-10 18:45:58 +01:00
6877637507 Fix startup crash caused by top-level geoip2 import
geoip2 is an optional dependency used only when a MaxMind mmdb path is
configured. Importing it unconditionally at module level caused the server
to crash on startup with ModuleNotFoundError when the package was absent
from the environment.

Move the imports under TYPE_CHECKING (for static analysis) and add lazy
local imports inside init_geoip() and _geoip_lookup() where geoip2 is
actually needed at runtime. The server now starts normally without a
MaxMind database, and geoip2 is loaded on demand if the feature is used.
2026-03-10 15:48:00 +01:00
12a859061c Fix missing country: neg cache, geoip2 fallback, re-resolve endpoint
- Add 5-min negative cache (_neg_cache) so failing IPs are throttled
  rather than hammering the API on every request
- Add MaxMind GeoLite2 fallback (init_geoip / _geoip_lookup) that fires
  when ip-api fails; controlled by BANGUI_GEOIP_DB_PATH env var
- Fix lookup_batch bug: failed API results were stored in positive cache
- Add _persist_neg_entry: INSERT OR IGNORE into geo_cache with NULL
  country_code so re-resolve can find historically failed IPs
- Add POST /api/geo/re-resolve: clears neg cache, batch-retries all
  geo_cache rows with country_code IS NULL, returns resolved/total count
- BanTable + MapPage: wrap the country — placeholder in a Fluent UI
  Tooltip explaining the retry behaviour
- Add geoip2>=4.8.0 dependency; geoip_db_path config setting
- Tests: add TestNegativeCache (4), TestGeoipFallback (4), TestReResolve (4)
2026-03-07 20:42:34 +01:00
ddfc8a0b02 Optimise geo lookup and aggregation for 10k+ IPs
- Add persistent geo_cache SQLite table (db.py)
- Rewrite geo_service: batch API (100 IPs/call), two-tier cache,
  no caching of failed lookups so they are retried
- Pre-warm geo cache from DB on startup (main.py lifespan)
- Rewrite bans_by_country: SQL GROUP BY ip aggregation + lookup_batch
  instead of 2000-row fetch + asyncio.gather individual calls
- Pre-warm geo cache after blocklist import (blocklist_service)
- Add 300ms debounce to useMapData hook to cancel stale requests
- Add perf benchmark asserting <2s for 10k bans
- Add seed_10k_bans.py script for manual perf testing
2026-03-07 20:28:51 +01:00
9ac7f8d22d feat: implement dashboard ban overview (Stage 5)
- Add ban_service reading fail2ban SQLite DB via read-only aiosqlite
- Add geo_service resolving IPs via ip-api.com with 10k in-memory cache
- Add GET /api/dashboard/bans and GET /api/dashboard/accesses endpoints
- Add TimeRange, DashboardBanItem, DashboardBanListResponse, AccessListItem,
  AccessListResponse models in models/ban.py
- Build BanTable component (Fluent UI DataGrid) with bans/accesses modes,
  pagination, loading/error/empty states, and ban-count badges
- Build useBans hook managing time-range and pagination state
- Update DashboardPage: status bar + time-range toolbar + tab switcher
- Add 37 new backend tests (ban service, geo service, dashboard router)
- All 141 tests pass; ruff/mypy --strict/tsc --noEmit clean
2026-03-01 12:57:19 +01:00