Optimise geo lookup and aggregation for 10k+ IPs
- Add persistent geo_cache SQLite table (db.py) - Rewrite geo_service: batch API (100 IPs/call), two-tier cache, no caching of failed lookups so they are retried - Pre-warm geo cache from DB on startup (main.py lifespan) - Rewrite bans_by_country: SQL GROUP BY ip aggregation + lookup_batch instead of 2000-row fetch + asyncio.gather individual calls - Pre-warm geo cache after blocklist import (blocklist_service) - Add 300ms debounce to useMapData hook to cancel stale requests - Add perf benchmark asserting <2s for 10k bans - Add seed_10k_bans.py script for manual perf testing
This commit is contained in:
@@ -134,6 +134,11 @@ async def _lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
|
||||
http_session: aiohttp.ClientSession = aiohttp.ClientSession()
|
||||
app.state.http_session = http_session
|
||||
|
||||
# --- Pre-warm geo cache from the persistent store ---
|
||||
from app.services import geo_service # noqa: PLC0415
|
||||
|
||||
await geo_service.load_cache_from_db(db)
|
||||
|
||||
# --- Background task scheduler ---
|
||||
scheduler: AsyncIOScheduler = AsyncIOScheduler(timezone="UTC")
|
||||
scheduler.start()
|
||||
|
||||
Reference in New Issue
Block a user