diff --git a/Docs/Tasks.md b/Docs/Tasks.md index e15f7b3..a116ff0 100644 --- a/Docs/Tasks.md +++ b/Docs/Tasks.md @@ -4,224 +4,479 @@ This document breaks the entire BanGUI project into development stages, ordered --- -## Issue: World Map Loading Time — Architecture Fix +## Stage 1 — Dashboard Charts Foundation -### Problem Summary +### Task 1.1 — Install and configure a charting library -The `GET /api/dashboard/bans/by-country` endpoint is extremely slow on first load. A single request with ~5,200 unique IPs produces **10,400 SQLite commits** and **6,000 INSERT statements** against the app database — all during a read-only GET request. The log shows 21,000+ lines of SQL trace for just 18 HTTP requests. +**Status:** `done` -Root causes (ordered by impact): +The frontend currently has no charting library. Install **Recharts** (`recharts`) as the project charting library. Recharts is React-native, composable, and integrates cleanly with Fluent UI v9 theming. -1. **Per-IP commit during geo cache writes** — `geo_service._persist_entry()` and `_persist_neg_entry()` each call `await db.commit()` after every single INSERT. With 5,200 uncached IPs this means 5,200+ individual commits, each forcing an `fsync`. This is the dominant bottleneck. -2. **DB writes on a GET request** — The bans/by-country endpoint passes `app_db` to `geo_service.lookup_batch()`, which triggers INSERT+COMMIT for every resolved IP. A GET request should never produce database writes/commits. Users do not expect loading a map page to mutate the database. -3. **Same pattern exists in other endpoints** — The following GET endpoints also trigger geo cache commits: `/api/dashboard/bans`, `/api/bans/active`, `/api/history`, `/api/history/{ip}`, `/api/geo/lookup/{ip}`. +**Steps:** -### Evidence from `log.log` +1. Run `npm install recharts` in the `frontend/` directory. +2. Verify the dependency appears in `package.json` under `dependencies`. +3. Confirm the build still succeeds with `npm run build` (no type errors, no warnings). -- Log line count: **21,117 lines** for 18 HTTP requests -- `INSERT INTO geo_cache`: **6,000** executions -- `db.commit()`: **10,400** calls (each INSERT + its commit = 2 ops per IP) -- `geo_batch_lookup_start`: reports `total=5200` uncached IPs -- The bans/by-country response is at line **21,086** out of 21,117 — the entire log is essentially one request's geo persist work -- Other requests (`/api/dashboard/status`, `/api/blocklists/schedule`, `/api/config/map-color-thresholds`) interleave with the geo persist loop because they share the same single async DB connection +No wrapper or configuration file is needed — Recharts components are imported directly where used. + +**Acceptance criteria:** + +- `recharts` is listed in `frontend/package.json`. +- `npm run build` succeeds with zero errors or warnings. --- -### Task 1: Batch geo cache writes — eliminate per-IP commits ✅ DONE +### Task 1.2 — Create a shared chart theme utility -**File:** `backend/app/services/geo_service.py` +**Status:** `done` -**What to change:** +Create a small utility at `frontend/src/utils/chartTheme.ts` that exports a function (or constant object) mapping Fluent UI v9 design tokens to Recharts-compatible colour values. The charts must respect the current Fluent theme (light and dark mode). At minimum export: -The functions `_persist_entry()` and `_persist_neg_entry()` each call `await db.commit()` after every INSERT. Instead, the commit should happen once after the entire batch is processed. +- A palette of 5+ distinct categorical colours for pie/bar slices, derived from Fluent token aliases (e.g. `colorPaletteBlueBorderActive`, `colorPaletteRedBorderActive`, etc.). +- Axis/grid/tooltip colours derived from `colorNeutralForeground2`, `colorNeutralStroke2`, `colorNeutralBackground1`, etc. +- A helper that returns the CSS value of a Fluent token at runtime (since Recharts needs literal CSS colour strings, not CSS custom properties). -1. Remove `await db.commit()` from both `_persist_entry()` and `_persist_neg_entry()`. -2. In `lookup_batch()`, after the loop over all chunks is complete and all `_persist_entry()` / `_persist_neg_entry()` calls have been made, issue a single `await db.commit()` if `db is not None`. -3. Wrap the single commit in a try/except to handle any errors gracefully. +Keep the file under 60 lines. No React components here — pure utility. -**Expected impact:** Reduces commits from ~5,200 to **1** per request. This alone should cut the endpoint response time dramatically. +**References:** [Web-Design.md](Web-Design.md) § colour tokens. -**Testing:** Existing tests in `test_services/test_ban_service.py` and `test_services/test_geo_service.py` should continue to pass. Verify the geo_cache table still gets populated after a batch lookup by checking the DB contents in an integration test. +**Acceptance criteria:** + +- The exported palette contains at least 5 distinct colours. +- Colours change correctly between light and dark mode. +- `tsc --noEmit` and `eslint` pass with zero warnings. --- -### Task 2: Do not write geo cache during GET requests ✅ DONE +## Stage 2 — Country Pie Chart (Top 4 + Other) -**Files:** `backend/app/routers/dashboard.py`, `backend/app/routers/bans.py`, `backend/app/routers/history.py`, `backend/app/routers/geo.py` +### Task 2.1 — Create the `TopCountriesPieChart` component -**What to change:** +**Status:** `done` -GET endpoints should not pass `app_db` (or equivalent) to geo_service functions. The geo resolution should still populate the in-memory cache (which is fast, free, and ephemeral), but should NOT write to SQLite during a read request. +Create `frontend/src/components/TopCountriesPieChart.tsx`. This component renders a **pie chart (Kuchendiagramm)** showing the **top 4 countries by ban count** plus an **"Other"** slice that aggregates every remaining country. -For each of these GET endpoints: -- `GET /api/dashboard/bans/by-country` in `dashboard.py` — stop passing `app_db=db` to `bans_by_country()`; pass `app_db=None` instead. -- `GET /api/dashboard/bans` in `dashboard.py` — stop passing `app_db=db` to `list_bans()`; pass `app_db=None` instead. -- `GET /api/bans/active` in `bans.py` — the enricher callback should not pass `db` to `geo_service.lookup()`. -- `GET /api/history` and `GET /api/history/{ip}` in `history.py` — same: enricher should not pass `db`. -- `GET /api/geo/lookup/{ip}` in `geo.py` — do not pass `db` to `geo_service.lookup()`. +**Data source:** The component receives the `countries` map (`Record`) and `country_names` map (`Record`) from the existing `/api/dashboard/bans/by-country` endpoint response (`BansByCountryResponse`). No new API endpoint is needed. -The persistent geo cache should only be written during explicit write operations: -- `POST /api/geo/re-resolve` (already a POST — this is correct) -- Blocklist import tasks (`blocklist_service.py`) -- Application startup via `load_cache_from_db()` +**Aggregation logic (frontend):** -**Expected impact:** GET requests become truly read-only. No commits, no `fsync`, no write contention on the app DB during map loads. +1. Sort the `countries` entries descending by ban count. +2. Take the top 4 entries. +3. Sum all remaining entries into a single `"Other"` bucket. +4. The result is exactly 5 slices (or fewer if fewer than 5 countries exist). -**Testing:** Run the full test suite. Verify that: -1. The bans/by-country endpoint still returns correct country data (from in-memory cache). -2. The `geo_cache` table is still populated when `POST /api/geo/re-resolve` is called or after blocklist import. -3. After a server restart, geo data is still available (because `load_cache_from_db()` warms memory from previously persisted data). +**Visual requirements:** ---- +- Use `` and `` from Recharts with `` for per-slice colours from the chart theme palette (Task 1.2). +- Display a `` on hover showing the country name and ban count. +- Display a `` listing each slice with its country name (full name from `country_names`, not just the code) and percentage. +- Label each slice with the percentage (use Recharts `label` prop or `