Commit Graph

18 Commits

Author SHA1 Message Date
654dbdb000 T-04: Encapsulate geo_service module-level mutable state in GeoCache class
Create GeoCache class with all mutable state as instance attributes:
- _cache, _neg_cache, _dirty, _geoip_reader, _geoip_initialized, _cache_lock
- All public methods: lookup(), lookup_batch(), lookup_cached_only(), flush_dirty(), load_from_db(), clear(), etc.

Initialization & Dependency Injection:
- Instantiate GeoCache in startup.py and store on app.state.geo_cache
- Add get_geo_cache() dependency function in dependencies.py
- Inject into routes and tasks via FastAPI's dependency system

Backward Compatibility:
- Maintain module-level functions in geo_service.py as deprecated wrappers
- All old callers continue to work through _default_geo_cache instance
- Remove test-escape-hatch functions (clear_cache, clear_neg_cache moved to methods)

Background Tasks:
- Update geo_cache_flush.py and geo_re_resolve.py to receive GeoCache instance
- Tasks now operate on injected instance rather than module globals

Tests:
- Refactor test_geo_service.py with geo_cache fixture providing fresh instances
- Update patch paths to target GeoCache methods correctly
- Fix internal state assertions to access instance attributes

Documentation:
- Update Architekture.md to document GeoCache as managed stateful service
- Describe cache lifecycle (load on startup, flush periodically, re-resolve stale)
- Note process-local limitations for multi-worker deployments

Fixes violation of Single Responsibility Principle: module no longer owns both
lookup logic and cache lifecycle management. Cache is now a first-class
injectable service with transparent lifecycle.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-23 16:18:09 +02:00
c1f188643c Move geo cache commit handling into repository layer 2026-04-18 20:10:05 +02:00
52e08e17a4 Guard geo_service.init_geoip against repeated initialization 2026-04-18 19:54:05 +02:00
1e2850a34e Add async lock protection to geo service cache and mark Task 16 done 2026-04-17 16:51:05 +02:00
a8f2d2d7b9 Refactor geo re-resolve endpoint into geo_service and add typed response 2026-04-15 08:56:37 +02:00
bf2abda595 chore: commit local changes 2026-03-22 14:24:32 +01:00
a442836c5c refactor: complete Task 2/3 geo decouple + exceptions centralization; mark as done 2026-03-22 14:24:25 +01:00
bdcdd5d672 Fix geo_re_resolve async mocks and mark tasks complete 2026-03-22 14:24:24 +01:00
ce59a66973 Move conffile_parser from services to utils 2026-03-22 14:24:24 +01:00
c9e688cc52 Refactor geo cache persistence into repository + remove raw SQL from tasks/main, update task list 2026-03-22 14:24:24 +01:00
93f0feabde Refactor geo re-resolve to use geo_cache repo and move data-access out of router 2026-03-22 14:24:24 +01:00
28f7b1cfcd Make geo lookups non-blocking with bulk DB writes and background tasks 2026-03-12 18:10:00 +01:00
a61c9dc969 Fix empty error field in geo_lookup_request_failed log events
- Replace str(exc) with repr(exc) in lookup() and _batch_api_call()
  so exception class name is always present even for no-message errors
  (e.g. aiohttp.ServerDisconnectedError() whose str() is empty)
- Add exc_type=type(exc).__name__ field to network-error log events
  for easy structured-log filtering
- Move import aiohttp to runtime import; use aiohttp.ClientTimeout()
  instead of raw float, removing # type: ignore[arg-type] workarounds
- Add TestErrorLogging with 3 tests covering empty-message exceptions
2026-03-12 17:50:58 +01:00
44a5a3d70e Fix geo cache write performance: batch commits, read-only GETs, dirty flush
- Remove per-IP db.commit() from _persist_entry() and _persist_neg_entry();
  add a single commit after the full lookup_batch() chunk loop instead.
  Reduces commits from ~5,200 to 1 per bans/by-country request.

- Remove db dependency from GET /api/dashboard/bans and
  GET /api/dashboard/bans/by-country; pass app_db=None so no SQLite
  writes occur during read-only requests.

- Add _dirty set to geo_service; _store() marks resolved IPs dirty.
  New flush_dirty(db) batch-upserts all dirty entries in one transaction.
  New geo_cache_flush APScheduler task flushes every 60 s so geo data
  is persisted without blocking requests.
2026-03-10 18:45:58 +01:00
6877637507 Fix startup crash caused by top-level geoip2 import
geoip2 is an optional dependency used only when a MaxMind mmdb path is
configured. Importing it unconditionally at module level caused the server
to crash on startup with ModuleNotFoundError when the package was absent
from the environment.

Move the imports under TYPE_CHECKING (for static analysis) and add lazy
local imports inside init_geoip() and _geoip_lookup() where geoip2 is
actually needed at runtime. The server now starts normally without a
MaxMind database, and geoip2 is loaded on demand if the feature is used.
2026-03-10 15:48:00 +01:00
12a859061c Fix missing country: neg cache, geoip2 fallback, re-resolve endpoint
- Add 5-min negative cache (_neg_cache) so failing IPs are throttled
  rather than hammering the API on every request
- Add MaxMind GeoLite2 fallback (init_geoip / _geoip_lookup) that fires
  when ip-api fails; controlled by BANGUI_GEOIP_DB_PATH env var
- Fix lookup_batch bug: failed API results were stored in positive cache
- Add _persist_neg_entry: INSERT OR IGNORE into geo_cache with NULL
  country_code so re-resolve can find historically failed IPs
- Add POST /api/geo/re-resolve: clears neg cache, batch-retries all
  geo_cache rows with country_code IS NULL, returns resolved/total count
- BanTable + MapPage: wrap the country — placeholder in a Fluent UI
  Tooltip explaining the retry behaviour
- Add geoip2>=4.8.0 dependency; geoip_db_path config setting
- Tests: add TestNegativeCache (4), TestGeoipFallback (4), TestReResolve (4)
2026-03-07 20:42:34 +01:00
ddfc8a0b02 Optimise geo lookup and aggregation for 10k+ IPs
- Add persistent geo_cache SQLite table (db.py)
- Rewrite geo_service: batch API (100 IPs/call), two-tier cache,
  no caching of failed lookups so they are retried
- Pre-warm geo cache from DB on startup (main.py lifespan)
- Rewrite bans_by_country: SQL GROUP BY ip aggregation + lookup_batch
  instead of 2000-row fetch + asyncio.gather individual calls
- Pre-warm geo cache after blocklist import (blocklist_service)
- Add 300ms debounce to useMapData hook to cancel stale requests
- Add perf benchmark asserting <2s for 10k bans
- Add seed_10k_bans.py script for manual perf testing
2026-03-07 20:28:51 +01:00
9ac7f8d22d feat: implement dashboard ban overview (Stage 5)
- Add ban_service reading fail2ban SQLite DB via read-only aiosqlite
- Add geo_service resolving IPs via ip-api.com with 10k in-memory cache
- Add GET /api/dashboard/bans and GET /api/dashboard/accesses endpoints
- Add TimeRange, DashboardBanItem, DashboardBanListResponse, AccessListItem,
  AccessListResponse models in models/ban.py
- Build BanTable component (Fluent UI DataGrid) with bans/accesses modes,
  pagination, loading/error/empty states, and ban-count badges
- Build useBans hook managing time-range and pagination state
- Update DashboardPage: status bar + time-range toolbar + tab switcher
- Add 37 new backend tests (ban service, geo service, dashboard router)
- All 141 tests pass; ruff/mypy --strict/tsc --noEmit clean
2026-03-01 12:57:19 +01:00