backup

Update fail2ban dbpurgeage to 648000 and history sync backfill/pagination for archive-based 7.5 day history.

Docker/VERSION

@@ -1 +1 @@
-v0.9.16
+v0.9.18

Docker/fail2ban-dev-config/README.md

@@ -78,6 +78,11 @@ Chains steps 1–3 automatically with appropriate sleep intervals.
 Inside the container the log file is mounted at `/remotelogs/bangui/auth.log`
 (see `fail2ban/paths-lsio.conf` — `remote_logs_path = /remotelogs`).
 
+BanGUI also extends fail2ban history retention for archive backfill. In
+the development config `fail2ban/fail2ban.conf` the database purge age is
+set to `648000` seconds (7.5 days) so the first archive sync can recover a
+full 7-day window before fail2ban purges old rows.
+
 To change sensitivity, edit `fail2ban/jail.d/manual-Jail.conf`:
 
 ```ini

Docs/Features.md

@@ -52,6 +52,8 @@ The main landing page after login. Shows recent ban activity at a glance.
   - Last 7 days (week)
   - Last 30 days (month)
   - Last 365 days (year)
+- **Data source selection:** The "Last 24 hours" preset queries fail2ban's live database directly for real-time accuracy. All longer presets (7 days, 30 days, 365 days) query the BanGUI long-term archive, because fail2ban's own database only retains the last 24 hours by default.
+- A **data-source badge** next to the time-range selector indicates whether the current view is showing **Live (fail2ban DB)** or **Archive (BanGUI DB)** data.
 
 ---
 
@@ -70,14 +72,16 @@ A geographical overview of ban activity.
   - Colors are smoothly interpolated between the thresholds (e.g., 35 bans shows a yellow-green blend)
   - The color threshold values are configurable through the application settings
 - **Interactive zoom and pan:** Users can zoom in/out using mouse wheel or touch gestures, and pan by clicking and dragging. This allows detailed inspection of densely-affected regions. Zoom controls (zoom in, zoom out, reset view) are provided as overlay buttons in the top-right corner.
-- For every country that has bans, the total count is displayed centred inside that country's borders in the selected time range.
-- Countries with zero banned IPs show no number and no label — they remain blank and transparent.
+- For every country that has bans, the total count is shown only in the country tooltip, not rendered on the map itself.
+- Countries with zero banned IPs show no tooltip and remain blank and transparent.
 - Clicking a country filters the companion table below to show only bans from that country.
 - Time-range selector with the same quick presets:
   - Last 24 hours
   - Last 7 days
   - Last 30 days
   - Last 365 days
+- **Data source selection:** Same rule as the Dashboard — "Last 24 hours" uses the live fail2ban database; all other ranges use the BanGUI archive.
+- A **data-source badge** is displayed alongside the time-range selector indicating **Live (fail2ban DB)** or **Archive (BanGUI DB)**.
 
 ---
 
@@ -245,13 +249,15 @@ A page to inspect and modify the fail2ban configuration without leaving the web
 
 ## 7. Ban History
 
-A view for exploring historical ban data stored in the fail2ban database.
+A view for exploring historical ban data stored in the BanGUI long-term archive.
 
 ### History Table
 
 - Browse all past bans across all jails, not just the currently active ones.
 - **Columns:** Time of ban, IP address, jail, ban duration, ban count (how many times this IP was banned), country.
 - Filter by jail, by IP address, or by time range.
+- The default time range on first load is **Last 7 days** and the data source is always the **BanGUI archive**, ensuring the full retention window is visible regardless of fail2ban's `dbpurgeage` setting.
+- A **data-source badge** is displayed indicating **Archive (BanGUI DB)**.
 - See at a glance which IPs are repeat offenders (high ban count).
 
 ### Per-IP History
@@ -265,7 +271,7 @@ A view for exploring historical ban data stored in the fail2ban database.
 - On each configured sync cycle (default every 5 minutes), BanGUI reads latest entries from fail2ban `bans` table and appends any new events to BanGUI history storage.
 - Supports both `ban` and `unban` events; audit record includes: `timestamp`, `ip`, `jail`, `action`, `duration`, `origin` (manual, auto, blocklist, etc.), `failures`, `matches`, and optional `country` / `ASN` enrichment.
 - Includes incremental import logic with dedupe: using unique constraint on (ip, jail, action, timeofban) to prevent duplication across sync cycles.
-- Provides backfill mode for initial startup: import last N days (configurable, default 7 days) of existing fail2ban history into BanGUI to avoid dark gaps after restart.
+- Provides backfill mode for initial startup: import the last 7.5 days of existing fail2ban history into BanGUI to avoid dark gaps after restart. Requires fail2ban's `dbpurgeage` to be set to at least `648000` (7.5 days) — BanGUI ships with this value pre-configured in its Docker setup.
 - Includes configurable archive purge policy in BanGUI (default 365 days), separate from fail2ban `dbpurgeage`, to keep app storage bounded while preserving audit data.
 - Expose API endpoints for querying persistent history, with filters for timeframe, jail, origin, IP, and current ban status.
 - On fail2ban connectivity failure, BanGUI continues serving historical data; next successful sync resumes ingestion without data loss.

Docs/Tasks.md

@@ -8,128 +8,71 @@ Reference: `Docs/Refactoring.md` for full analysis of each issue.
 
 ## Open Issues
 
-### Replace `react-simple-maps` with `d3-geo` in WorldMap
+### Backend Architecture
 
-The current `WorldMap` component (`frontend/src/components/WorldMap.tsx`) uses the `react-simple-maps` library (`ComposableMap`, `ZoomableGroup`, `Geography`, `useGeographies`). This library wraps d3-geo but adds a heavy abstraction layer and fetches the TopoJSON geography file from a remote CDN at runtime. Replace it with direct d3-geo rendering, following the pattern demonstrated in the reference project at `/media/lukas/Volume/repo/worldmaptest/`.
+- **Replace the single shared SQLite connection.**
+  - Current startup code opens one `aiosqlite.Connection` and reuses it for every request.
+  - This should be replaced with either a connection pool or request-scoped connections to avoid concurrency and locking issues.
+  - Update request dependencies, application lifecycle, and tests to use the new pattern.
 
-Reference: `Docs/Features.md` §4 (World Map View) for the full feature specification.
+- **Refactor dependency wiring and shared resource management.**
+  - Remove hidden module-level import coupling between routers, services, and shared utilities.
+  - Introduce explicit factories or providers for shared resources such as DB, HTTP client session, scheduler, and settings.
+  - Ensure routers depend on injected providers rather than global state or dynamic imports.
 
-**All existing features must be preserved.** The component's public API (`WorldMapProps`) and behaviour must remain identical so that `MapPage.tsx`, `HistoryPage.tsx`, and the existing unit test continue to work after the migration.
+- **Harden fail2ban integration.**
+  - Remove the `sys.path` hack that locates `fail2ban-master` at runtime.
+  - Replace it with a deterministic packaging or configuration model so the backend does not depend on repository layout.
+  - Refactor `Fail2BanClient` so concurrency control is instance-based and not backed by hidden module globals.
 
----
+- **Improve startup / setup guard behavior.**
+  - Convert `SetupRedirectMiddleware` from an on-demand DB check into a startup/initialisation guard where possible.
+  - Cache setup completion in a safe way and provide an explicit invalidation path if the application state changes.
+  - Reduce middleware responsibility and avoid DB access during normal request dispatch.
 
-#### Task 1 — Swap npm dependencies [DONE]
+- **Make deployment configuration explicit.**
+  - Move hard-coded environment assumptions such as CORS origins into settings.
+  - Ensure `fail2ban_socket`, `fail2ban_config_dir`, and startup commands are fully configurable via `Settings`.
+  - Document production-ready defaults separately from development defaults.
 
-Remove `react-simple-maps` and `@types/react-simple-maps` from `frontend/package.json`. Add the following packages that the new implementation requires:
+### Reliability and Resilience
 
-- `d3-geo` — geographic projection and SVG path generation.
-- `@types/d3-geo` — TypeScript definitions for d3-geo.
-- `topojson-client` — converts TopoJSON to GeoJSON `FeatureCollection`.
-- `@types/topojson-client` — TypeScript definitions for topojson-client.
-- `world-atlas` — provides the `countries-110m.json` TopoJSON file as a local npm asset (no more CDN fetch at runtime).
+- **Add backend lifecycle tests for resource cleanup.**
+  - Verify startup opens and initialises DB, HTTP session, scheduler, and geo cache correctly.
+  - Verify shutdown closes those resources cleanly.
 
-Run `npm install` and verify the lock file updates cleanly.
+- **Add concurrency/regression coverage for DB and fail2ban socket use.**
+  - Add tests that simulate multiple concurrent requests using the same DB dependency.
+  - Add tests around fail2ban socket retries, protocol errors, and rate limiting.
 
----
+- **Improve state caching and invalidation.**
+  - Add tests for session cache invalidation on logout.
+  - Add tests for setup completion caching so stale state is never served.
 
-#### Task 2 — Rewrite `WorldMap.tsx` to use d3-geo directly [DONE]
+### Backend Feature Work
 
-Rewrite the component so that it renders a plain `<svg>` with `<path>` elements generated by d3-geo instead of the react-simple-maps wrappers. The implementation should follow this approach (as seen in the reference project):
+- **Document and implement backend-safe environment-driven CORS.**
+  - Add support for production and local development origins through configuration.
+  - Avoid a hardcoded Vite origin in the core app factory.
 
-1. **Import the TopoJSON locally** — `import worldData from "world-atlas/countries-110m.json"` instead of fetching from a CDN URL. Use `topojson-client`'s `feature()` to extract the GeoJSON `FeatureCollection` once (memoised).
+- **Centralise scheduler job registration.**
+  - Refactor APScheduler registration so background tasks are registered through a common lifecycle helper.
+  - Ensure jobs can be discovered, replaced, and tested without requiring implicit `app.state` side effects.
 
-2. **Create a projection** — Use `geoMercator()` from d3-geo (matching the current Mercator projection) with `.fitSize([width, height], featureCollection)` to auto-scale. Memoise the projection so it is only recomputed when the geometry changes.
+- **Strengthen fail2ban error handling and reporting.**
+  - Standardise `502` responses for connection/protocol failures across all endpoints.
+  - Add structured logging for retries and fatal socket failures.
+  - Ensure the UI can distinguish offline fail2ban from internal backend failures.
 
-3. **Create a path generator** — `geoPath().projection(projection)`. Memoise.
+- **Improve documentation of backend responsibilities.**
+  - Keep `Docs/Tasks.md` aligned with the backend architecture review.
+  - Add references to the backend modules, resource lifecycle, and dependency model in the documentation.
 
-4. **Render countries** — Map over the GeoJSON features and render a `<path>` element for each country. Use the `ISO_NUMERIC_TO_ALPHA2` lookup (already exists in `frontend/src/data/isoNumericToAlpha2.ts`) to translate the numeric feature id to the alpha-2 code expected by the `countries` prop.
+### Priority Execution Plan
 
-5. **Preserve colour coding** — Continue using `getBanCountColor()` from `frontend/src/utils/mapColors.ts` to compute each country's fill colour based on its ban count and the three threshold props.
-
-6. **Preserve ban-count labels** — For every country with `count > 0`, compute the centroid with `pathGenerator.centroid(feature)` and render a `<text>` element at that position showing the count. Countries with zero bans must remain blank and transparent (no fill, no label).
-
-7. **Preserve country selection** — Clicking a country calls `onSelectCountry` with the alpha-2 code (or `null` to deselect). The selected country must receive a distinct brand fill colour, matching the current behaviour.
-
-8. **Preserve hover tooltip** — On `mouseenter` / `mousemove` / `mouseleave`, show/hide a tooltip portal (`createPortal` into `document.body`) displaying the country name and ban count. Use the same Fluent UI styled tooltip div that the current implementation uses.
-
-9. **Preserve keyboard accessibility** — Each country with a known alpha-2 code must have `role="button"`, `tabIndex={0}`, an `aria-label` (`"CC: N ban(s)"`), and `aria-pressed` when selected. `Enter` and `Space` must trigger selection/deselection.
-
-10. **Use a `viewBox`-based responsive SVG** — Set `viewBox="0 0 {width} {height}"` and `style={{ width: "100%", height: "auto" }}` so the map scales with its container, matching the reference project's approach.
-
----
-
-#### Task 3 — Implement zoom and pan without `react-simple-maps` [DONE]
-
-The current implementation relies on `ZoomableGroup` from react-simple-maps for zoom/pan. Reimplement this using a `<g>` wrapper with an SVG `transform` attribute driven by React state:
-
-1. **State:** Track `zoom` (number, 1–8) and `center` (translate offset `[x, y]`).
-
-2. **Zoom controls:** Keep the three overlay buttons (Zoom In `+`, Zoom Out `−`, Reset `⟲`) in the top-right corner. Each button adjusts the `zoom` state by ±0.5, clamped to `[1, 8]`. Reset sets zoom to 1 and center to `[0, 0]`.
-
-3. **Mouse-wheel zoom:** Attach a `wheel` event handler to the SVG that increments/decrements zoom on scroll, zooming toward the cursor position.
-
-4. **Click-and-drag pan:** Track `mousedown` → `mousemove` → `mouseup` on the SVG to translate the `center` offset. Only pan when the drag exceeds a small threshold (e.g. 3 px) to avoid conflicting with country click events.
-
-5. **Touch support (stretch goal):** Optionally support pinch-to-zoom and touch-drag for tablet users.
-
-6. **Apply transform:** Wrap all `<path>` and `<text>` elements in a `<g transform="translate(tx, ty) scale(zoom)">` group. Alternatively, use `d3-zoom` if a more robust implementation is preferred, but keep React as the rendering layer (no d3 DOM manipulation).
-
----
-
-#### Task 4 — Update hover and selection styles to use CSS transitions [DONE]
-
-The reference project applies hover highlights via CSS classes (`.country`, `.country.hovered`) with CSS `transition` instead of the react-simple-maps `style={{ default, hover, pressed }}` object. Adopt the same approach:
-
-- Define CSS classes (or Fluent UI `makeStyles` rules) for default, hovered, and selected states.
-- Apply the correct class based on component state (`isSelected`, `isHovered`).
-- Use a CSS `transition` on `fill` and `stroke` for a smooth 150 ms highlight effect.
-- This avoids the react-simple-maps per-geography style object entirely.
-
-Ensure the selected state still uses `tokens.colorBrandBackground` / `tokens.colorBrandBackgroundHover` / `tokens.colorBrandBackgroundPressed` from Fluent UI so the map integrates visually with the rest of the application.
-
----
-
-#### Task 5 — Update the WorldMap unit test [DONE]
-
-The existing test at `frontend/src/components/__tests__/WorldMap.test.tsx` mocks `react-simple-maps`. After the migration those mocks are invalid. Update the test:
-
-1. **Remove the `vi.mock("react-simple-maps", ...)` block.**
-
-2. **Mock the TopoJSON data instead.** Since the new implementation imports `world-atlas/countries-110m.json` directly, mock that module to return a minimal TopoJSON object containing a single country feature (e.g. id `"840"` for the US). Use `topojson-client`'s `feature()` to verify the mock produces a valid GeoJSON feature.
-
-3. **Keep the same assertions:** tooltip appears on hover with country name and ban count, tooltip disappears on mouse leave, country element has correct ARIA attributes (`role="button"`, `aria-label`, `aria-pressed`).
-
-4. **Verify zoom controls render:** assert that the three zoom buttons (Zoom In, Zoom Out, Reset) are present and have the correct `aria-label` values.
-
-5. Also verify that tests in `MapPage.test.tsx` and `HistoryPage.test.tsx` still pass (they mock `WorldMap` at the component level so they should be unaffected, but confirm).
-
----
-
-#### Task 6 — Remove CDN dependency and verify offline capability [DONE]
-
-The old implementation fetched geography data from `https://cdn.jsdelivr.net/npm/world-atlas@2/countries-110m.json` at runtime. The new implementation bundles the data via the `world-atlas` npm package, so:
-
-1. Delete the `GEO_URL` constant.
-2. Confirm the TopoJSON file is included in the Vite bundle (imported as a JSON module).
-3. Verify the map renders correctly without any network request for geography data (check the browser network tab or write a test that asserts no fetch calls are made for the old CDN URL).
-
----
-
-#### Task 7 — Final integration smoke test [DONE]
-
-After all changes, manually verify the following against the feature specification in `Docs/Features.md` §4:
-
-- Countries are colour-coded by ban count (transparent → green → yellow → red) using smooth interpolation.
-- Ban count numbers are displayed centred inside each country that has bans.
-- Countries with zero bans are transparent with no label.
-- Clicking a country filters the companion ban table below.
-- Clicking the same country again deselects it.
-- Zoom in / zoom out / reset buttons work correctly (range 1×–8×).
-- Mouse-wheel zoom and click-drag pan work.
-- Tooltip appears on hover showing country name and localised ban count.
-- Keyboard navigation works (Tab to focus, Enter/Space to toggle selection).
-- The map is responsive and scales with the container width.
-- Time-range selector on `MapPage` still updates the map data correctly.
-- Colour thresholds from settings are applied (thresholdLow, thresholdMedium, thresholdHigh props).
-- Run `npm run test` — all existing tests pass.
-- Run `npm run build` — production build succeeds with no errors or warnings.
+1. Fix the global SQLite connection pattern and tests.
+2. Refactor dependency injection / explicit shared resources.
+3. Harden fail2ban client concurrency and packaging.
+4. Convert setup guard to a safer startup-driven model.
+5. Add deployment-safe configuration and production-ready CORS.
+6. Add lifecycle and concurrency regression tests.

backend/app/routers/dashboard.py

@@ -12,7 +12,7 @@ Also provides ``GET /api/dashboard/bans`` for the dashboard ban-list table,
 
 from __future__ import annotations
 
-from typing import TYPE_CHECKING
+from typing import TYPE_CHECKING, Literal
 
 if TYPE_CHECKING:
     import aiohttp
@@ -83,7 +83,7 @@ async def get_dashboard_bans(
     request: Request,
     _auth: AuthDep,
     range: TimeRange = Query(default=_DEFAULT_RANGE, description="Time-range preset."),
-    source: str = Query(default="fail2ban", description="Data source: 'fail2ban' or 'archive'."),
+    source: Literal["fail2ban", "archive"] = Query(default="fail2ban", description="Data source: 'fail2ban' or 'archive'."),
     page: int = Query(default=1, ge=1, description="1-based page number."),
     page_size: int = Query(default=_DEFAULT_PAGE_SIZE, ge=1, le=500, description="Items per page."),
     origin: BanOrigin | None = Query(
@@ -137,7 +137,7 @@ async def get_bans_by_country(
     request: Request,
     _auth: AuthDep,
     range: TimeRange = Query(default=_DEFAULT_RANGE, description="Time-range preset."),
-    source: str = Query(default="fail2ban", description="Data source: 'fail2ban' or 'archive'."),
+    source: Literal["fail2ban", "archive"] = Query(default="fail2ban", description="Data source: 'fail2ban' or 'archive'."),
     origin: BanOrigin | None = Query(
         default=None,
         description="Filter by ban origin: 'blocklist' or 'selfblock'. Omit for all.",
@@ -185,7 +185,7 @@ async def get_ban_trend(
     request: Request,
     _auth: AuthDep,
     range: TimeRange = Query(default=_DEFAULT_RANGE, description="Time-range preset."),
-    source: str = Query(default="fail2ban", description="Data source: 'fail2ban' or 'archive'."),
+    source: Literal["fail2ban", "archive"] = Query(default="fail2ban", description="Data source: 'fail2ban' or 'archive'."),
     origin: BanOrigin | None = Query(
         default=None,
         description="Filter by ban origin: 'blocklist' or 'selfblock'. Omit for all.",
@@ -235,7 +235,7 @@ async def get_bans_by_jail(
     request: Request,
     _auth: AuthDep,
     range: TimeRange = Query(default=_DEFAULT_RANGE, description="Time-range preset."),
-    source: str = Query(default="fail2ban", description="Data source: 'fail2ban' or 'archive'."),
+    source: Literal["fail2ban", "archive"] = Query(default="fail2ban", description="Data source: 'fail2ban' or 'archive'."),
     origin: BanOrigin | None = Query(
         default=None,
         description="Filter by ban origin: 'blocklist' or 'selfblock'. Omit for all.",

backend/app/routers/history.py

@@ -15,7 +15,7 @@ Routes
 
 from __future__ import annotations
 
-from typing import TYPE_CHECKING
+from typing import TYPE_CHECKING, Literal
 
 if TYPE_CHECKING:
     import aiohttp
@@ -56,7 +56,7 @@ async def get_history(
         default=None,
         description="Filter by ban origin: 'blocklist' or 'selfblock'. Omit for all.",
     ),
-    source: str = Query(
+    source: Literal["fail2ban", "archive"] = Query(
         default="fail2ban",
         description="Data source: 'fail2ban' or 'archive'.",
     ),

backend/app/tasks/history_sync.py

@@ -26,7 +26,7 @@ JOB_ID: str = "history_sync"
 HISTORY_SYNC_INTERVAL: int = 300
 
 #: Backfill window when archive is empty (seconds).
-BACKFILL_WINDOW: int = 7 * 86400
+BACKFILL_WINDOW: int = 648000
 
 
 async def _get_last_archive_ts(db) -> int | None:
@@ -50,7 +50,7 @@ async def _run_sync(app: FastAPI) -> None:
             log.info("history_sync_backfill", window_seconds=BACKFILL_WINDOW)
 
         per_page = 500
-        next_since = last_ts
+        next_since = last_ts + 1
         total_synced = 0
 
         while True:

backend/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "bangui-backend"
-version = "0.9.15"
+version = "0.9.18"
 description = "BanGUI backend — fail2ban web management interface"
 requires-python = ">=3.12"
 dependencies = [

backend/tests/test_routers/test_dashboard.py

@@ -428,6 +428,15 @@ class TestBansByCountry:
         called_range = mock_fn.call_args[0][1]
         assert called_range == "7d"
 
+    async def test_invalid_source_returns_422(
+        self, dashboard_client: AsyncClient
+    ) -> None:
+        """An invalid source value returns HTTP 422."""
+        response = await dashboard_client.get(
+            "/api/dashboard/bans/by-country?source=invalid"
+        )
+        assert response.status_code == 422
+
     async def test_empty_window_returns_empty_response(
         self, dashboard_client: AsyncClient
     ) -> None:
@@ -722,6 +731,15 @@ class TestBanTrend:
         )
         assert response.status_code == 422
 
+    async def test_invalid_source_returns_422(
+        self, dashboard_client: AsyncClient
+    ) -> None:
+        """An invalid source value returns HTTP 422."""
+        response = await dashboard_client.get(
+            "/api/dashboard/bans/trend?source=invalid"
+        )
+        assert response.status_code == 422
+
     async def test_empty_buckets_response(self, dashboard_client: AsyncClient) -> None:
         """Empty bucket list is serialised correctly."""
         from app.models.ban import BanTrendResponse
@@ -857,6 +875,15 @@ class TestBansByJail:
         )
         assert response.status_code == 422
 
+    async def test_invalid_source_returns_422(
+        self, dashboard_client: AsyncClient
+    ) -> None:
+        """An invalid source value returns HTTP 422."""
+        response = await dashboard_client.get(
+            "/api/dashboard/bans/by-jail?source=invalid"
+        )
+        assert response.status_code == 422
+
     async def test_empty_jails_response(self, dashboard_client: AsyncClient) -> None:
         """Empty jails list is serialised correctly."""
         from app.models.ban import BansByJailResponse

backend/tests/test_tasks/test_history_sync.py

@@ -2,7 +2,7 @@
 
 from __future__ import annotations
 
-from unittest.mock import MagicMock
+from unittest.mock import AsyncMock, MagicMock, patch
 
 from app.tasks import history_sync
 
@@ -27,3 +27,33 @@ class TestHistorySyncTask:
         called_args, called_kwargs = fake_scheduler.add_job.call_args
         assert called_kwargs["id"] == history_sync.JOB_ID
         assert called_kwargs["kwargs"]["app"] == app
+
+    async def test_backfill_window_is_7_5_days(self) -> None:
+        assert history_sync.BACKFILL_WINDOW == 648000
+
+    async def test_sync_uses_strict_since_after_restart(self) -> None:
+        fake_app = type("FakeApp", (), {})()
+        fake_app.state = type("FakeState", (), {})()
+        fake_app.state.settings = type("FakeSettings", (), {})()
+        fake_app.state.settings.fail2ban_socket = "/tmp/fake.sock"
+
+        fake_app.state.db = MagicMock()
+
+        async def fake_get_history_page(*, db_path: str, since: int, page: int, page_size: int, **kwargs):
+            assert since == 1001
+            return [], 0
+
+        async def fake_get_fail2ban_db_path(socket_path: str) -> str:
+            return "/tmp/fake.sqlite3"
+
+        with patch(
+            "app.tasks.history_sync._get_last_archive_ts",
+            new=AsyncMock(return_value=1000),
+        ), patch(
+            "app.tasks.history_sync.get_fail2ban_db_path",
+            new=fake_get_fail2ban_db_path,
+        ), patch(
+            "app.tasks.history_sync.fail2ban_db_repo.get_history_page",
+            new=fake_get_history_page,
+        ):
+            await history_sync._run_sync(fake_app)

fail2ban-master/config/fail2ban.conf

@@ -72,7 +72,7 @@ dbfile = /var/lib/fail2ban/fail2ban.sqlite3
 # Options: dbpurgeage
 # Notes.: Sets age at which bans should be purged from the database
 # Values: [ SECONDS ] Default: 86400 (24hours)
-dbpurgeage = 1d
+dbpurgeage = 648000
 
 # Options: dbmaxmatches
 # Notes.: Number of matches stored in database per ticket (resolvable via 

frontend/package-lock.json

@@ -1,12 +1,12 @@
 {
   "name": "bangui-frontend",
-  "version": "0.9.15",
+  "version": "0.9.17",
   "lockfileVersion": 3,
   "requires": true,
   "packages": {
     "": {
       "name": "bangui-frontend",
-      "version": "0.9.15",
+      "version": "0.9.17",
       "dependencies": {
         "@fluentui/react-components": "^9.55.0",
         "@fluentui/react-icons": "^2.0.257",

frontend/package.json

@@ -1,7 +1,7 @@
 {
   "name": "bangui-frontend",
   "private": true,
-  "version": "0.9.16",
+  "version": "0.9.18",
   "description": "BanGUI frontend — fail2ban web management interface",
   "type": "module",
   "scripts": {

frontend/src/api/dashboard.ts

@@ -42,6 +42,7 @@ export async function fetchBans(
   page = 1,
   pageSize = 100,
   origin: BanOriginFilter = "all",
+  source: "fail2ban" | "archive" = "fail2ban",
 ): Promise<DashboardBanListResponse> {
   const params = new URLSearchParams({
     range,
@@ -51,6 +52,9 @@ export async function fetchBans(
   if (origin !== "all") {
     params.set("origin", origin);
   }
+  if (source !== "fail2ban") {
+    params.set("source", source);
+  }
   return get<DashboardBanListResponse>(`${ENDPOINTS.dashboardBans}?${params.toString()}`);
 }
 
@@ -66,11 +70,15 @@ export async function fetchBans(
 export async function fetchBanTrend(
   range: TimeRange,
   origin: BanOriginFilter = "all",
+  source: "fail2ban" | "archive" = "fail2ban",
 ): Promise<BanTrendResponse> {
   const params = new URLSearchParams({ range });
   if (origin !== "all") {
     params.set("origin", origin);
   }
+  if (source !== "fail2ban") {
+    params.set("source", source);
+  }
   return get<BanTrendResponse>(`${ENDPOINTS.dashboardBansTrend}?${params.toString()}`);
 }
 
@@ -86,10 +94,14 @@ export async function fetchBanTrend(
 export async function fetchBansByJail(
   range: TimeRange,
   origin: BanOriginFilter = "all",
+  source: "fail2ban" | "archive" = "fail2ban",
 ): Promise<BansByJailResponse> {
   const params = new URLSearchParams({ range });
   if (origin !== "all") {
     params.set("origin", origin);
   }
+  if (source !== "fail2ban") {
+    params.set("source", source);
+  }
   return get<BansByJailResponse>(`${ENDPOINTS.dashboardBansByJail}?${params.toString()}`);
 }

frontend/src/api/history.ts

@@ -21,6 +21,7 @@ export async function fetchHistory(
   if (query.origin) params.set("origin", query.origin);
   if (query.jail) params.set("jail", query.jail);
   if (query.ip) params.set("ip", query.ip);
+  if (query.source) params.set("source", query.source);
   if (query.page !== undefined) params.set("page", String(query.page));
   if (query.page_size !== undefined)
     params.set("page_size", String(query.page_size));

frontend/src/api/map.ts

@@ -17,10 +17,14 @@ import type { BanOriginFilter } from "../types/ban";
 export async function fetchBansByCountry(
   range: TimeRange = "24h",
   origin: BanOriginFilter = "all",
+  source: "fail2ban" | "archive" = "fail2ban",
 ): Promise<BansByCountryResponse> {
   const params = new URLSearchParams({ range });
   if (origin !== "all") {
     params.set("origin", origin);
   }
+  if (source !== "fail2ban") {
+    params.set("source", source);
+  }
   return get<BansByCountryResponse>(`${ENDPOINTS.dashboardBansByCountry}?${params.toString()}`);
 }

frontend/src/components/BanTable.tsx

@@ -46,6 +46,10 @@ interface BanTableProps {
    * Changing this value triggers a re-fetch and resets to page 1.
    */
   origin?: BanOriginFilter;
+  /**
+   * Data source used for the table query.
+   */
+  source?: "fail2ban" | "archive";
 }
 
 // ---------------------------------------------------------------------------
@@ -186,9 +190,9 @@ function buildBanColumns(styles: ReturnType<typeof useStyles>): TableColumnDefin
  * @param props.timeRange - Active time-range preset from the parent page.
  * @param props.origin    - Active origin filter from the parent page.
  */
-export function BanTable({ timeRange, origin = "all" }: BanTableProps): React.JSX.Element {
+export function BanTable({ timeRange, origin = "all", source = "fail2ban" }: BanTableProps): React.JSX.Element {
   const styles = useStyles();
-  const { banItems, total, page, setPage, loading, error, refresh } = useBans(timeRange, origin);
+  const { banItems, total, page, setPage, loading, error, refresh } = useBans(timeRange, origin, source);
 
   const banColumns = buildBanColumns(styles);
 

frontend/src/components/BanTrendChart.tsx

@@ -53,6 +53,8 @@ interface BanTrendChartProps {
   timeRange: TimeRange;
   /** Origin filter controlling which bans are included. */
   origin: BanOriginFilter;
+  /** Data source used for the chart. */
+  source?: "fail2ban" | "archive";
 }
 
 /** Internal chart data point shape. */
@@ -188,9 +190,10 @@ function TrendTooltip(props: TooltipContentProps): React.JSX.Element | null {
 export function BanTrendChart({
   timeRange,
   origin,
+  source = "fail2ban",
 }: BanTrendChartProps): React.JSX.Element {
   const styles = useStyles();
-  const { buckets, isLoading, error, reload } = useBanTrend(timeRange, origin);
+  const { buckets, isLoading, error, reload } = useBanTrend(timeRange, origin, source);
 
   const isEmpty = buckets.every((b) => b.count === 0);
   const entries = buildEntries(buckets, timeRange);

frontend/src/components/WorldMap.tsx

@@ -346,10 +346,6 @@ export function WorldMap({
               return null;
             }
 
-            const centroid = pathGenerator.centroid(featureItem);
-            const [cx, cy] = centroid;
-            const isCentroidValid = Number.isFinite(cx) && Number.isFinite(cy);
-
             return (
               <g key={String(rawId)}>
                 <path
@@ -410,17 +406,6 @@ export function WorldMap({
                     setTooltip(null);
                   }}
                 />
-                {count > 0 && isCentroidValid && (
-                  <text
-                    x={cx}
-                    y={cy}
-                    textAnchor="middle"
-                    dominantBaseline="central"
-                    className={styles.countLabel}
-                  >
-                    {count}
-                  </text>
-                )}
               </g>
             );
           })}

frontend/src/hooks/useBanTrend.ts

@@ -42,6 +42,7 @@ export interface UseBanTrendResult {
 export function useBanTrend(
   timeRange: TimeRange,
   origin: BanOriginFilter,
+  source: "fail2ban" | "archive" = "fail2ban",
 ): UseBanTrendResult {
   const [buckets, setBuckets] = useState<BanTrendBucket[]>([]);
   const [bucketSize, setBucketSize] = useState<string>("1h");
@@ -58,7 +59,7 @@ export function useBanTrend(
     setIsLoading(true);
     setError(null);
 
-    fetchBanTrend(timeRange, origin)
+    fetchBanTrend(timeRange, origin, source)
       .then((data) => {
         if (controller.signal.aborted) return;
         setBuckets(data.buckets);
@@ -73,7 +74,7 @@ export function useBanTrend(
           setIsLoading(false);
         }
       });
-  }, [timeRange, origin]);
+  }, [timeRange, origin, source]);
 
   useEffect(() => {
     load();

frontend/src/hooks/useBans.ts

@@ -44,6 +44,7 @@ export interface UseBansResult {
 export function useBans(
   timeRange: TimeRange,
   origin: BanOriginFilter = "all",
+  source: "fail2ban" | "archive" = "fail2ban",
 ): UseBansResult {
   const [banItems, setBanItems] = useState<DashboardBanItem[]>([]);
   const [total, setTotal] = useState<number>(0);
@@ -51,16 +52,16 @@ export function useBans(
   const [loading, setLoading] = useState<boolean>(true);
   const [error, setError] = useState<string | null>(null);
 
-  // Reset page when time range or origin filter changes.
+  // Reset page when time range, origin filter, or source changes.
   useEffect(() => {
     setPage(1);
-  }, [timeRange, origin]);
+  }, [timeRange, origin, source]);
 
   const doFetch = useCallback(async (): Promise<void> => {
     setLoading(true);
     setError(null);
     try {
-      const data = await fetchBans(timeRange, page, PAGE_SIZE, origin);
+      const data = await fetchBans(timeRange, page, PAGE_SIZE, origin, source);
       setBanItems(data.items);
       setTotal(data.total);
     } catch (err: unknown) {
@@ -68,7 +69,7 @@ export function useBans(
     } finally {
       setLoading(false);
     }
-  }, [timeRange, page, origin]);
+  }, [timeRange, page, origin, source]);
 
   // Stable ref to the latest doFetch so the refresh callback is always current.
   const doFetchRef = useRef(doFetch);

frontend/src/hooks/useDashboardCountryData.ts

@@ -48,6 +48,7 @@ export interface UseDashboardCountryDataResult {
 export function useDashboardCountryData(
   timeRange: TimeRange,
   origin: BanOriginFilter,
+  source: "fail2ban" | "archive" = "fail2ban",
 ): UseDashboardCountryDataResult {
   const [countries, setCountries] = useState<Record<string, number>>({});
   const [countryNames, setCountryNames] = useState<Record<string, string>>({});
@@ -67,7 +68,7 @@ export function useDashboardCountryData(
     setIsLoading(true);
     setError(null);
 
-    fetchBansByCountry(timeRange, origin)
+    fetchBansByCountry(timeRange, origin, source)
       .then((data) => {
         if (controller.signal.aborted) return;
         setCountries(data.countries);
@@ -85,7 +86,7 @@ export function useDashboardCountryData(
           setIsLoading(false);
         }
       });
-  }, [timeRange, origin]);
+  }, [timeRange, origin, source]);
 
   useEffect(() => {
     load();

frontend/src/hooks/useMapData.ts

@@ -43,6 +43,7 @@ export interface UseMapDataResult {
 export function useMapData(
   range: TimeRange = "24h",
   origin: BanOriginFilter = "all",
+  source: "fail2ban" | "archive" = "fail2ban",
 ): UseMapDataResult {
   const [data, setData] = useState<BansByCountryResponse | null>(null);
   const [loading, setLoading] = useState(true);
@@ -64,7 +65,7 @@ export function useMapData(
       abortRef.current?.abort();
       abortRef.current = new AbortController();
 
-      fetchBansByCountry(range, origin)
+      fetchBansByCountry(range, origin, source)
         .then((resp) => {
           setData(resp);
         })
@@ -75,7 +76,7 @@ export function useMapData(
           setLoading(false);
         });
     }, DEBOUNCE_MS);
-  }, [range, origin]);
+  }, [range, origin, source]);
 
   useEffect((): (() => void) => {
     load();

frontend/src/pages/DashboardPage.tsx

@@ -71,8 +71,10 @@ export function DashboardPage(): React.JSX.Element {
   const [timeRange, setTimeRange] = useState<TimeRange>("24h");
   const [originFilter, setOriginFilter] = useState<BanOriginFilter>("all");
 
+  const source = timeRange === "24h" ? "fail2ban" : "archive";
+
   const { countries, countryNames, isLoading: countryLoading, error: countryError, reload: reloadCountry } =
-    useDashboardCountryData(timeRange, originFilter);
+    useDashboardCountryData(timeRange, originFilter, source);
 
   const sectionStyles = useCommonSectionStyles();
 
@@ -86,12 +88,14 @@ export function DashboardPage(): React.JSX.Element {
       {/* ------------------------------------------------------------------ */}
       {/* Global filter bar                                                  */}
       {/* ------------------------------------------------------------------ */}
+      <div style={{ display: "flex", alignItems: "center", gap: tokens.spacingHorizontalM, flexWrap: "wrap" }}>
         <DashboardFilterBar
           timeRange={timeRange}
           onTimeRangeChange={setTimeRange}
           originFilter={originFilter}
           onOriginFilterChange={setOriginFilter}
         />
+      </div>
 
       {/* ------------------------------------------------------------------ */}
       {/* Ban Trend section                                                   */}
@@ -103,7 +107,7 @@ export function DashboardPage(): React.JSX.Element {
           </Text>
         </div>
         <div className={styles.tabContent}>
-          <BanTrendChart timeRange={timeRange} origin={originFilter} />
+          <BanTrendChart timeRange={timeRange} origin={originFilter} source={source} />
         </div>
       </div>
 
@@ -154,7 +158,7 @@ export function DashboardPage(): React.JSX.Element {
 
         {/* Ban table */}
         <div className={styles.tabContent}>
-          <BanTable timeRange={timeRange} origin={originFilter} />
+          <BanTable timeRange={timeRange} origin={originFilter} source={source} />
         </div>
       </div>
     </div>

frontend/src/pages/HistoryPage.tsx

@@ -143,6 +143,7 @@ function areHistoryQueriesEqual(
     a.origin === b.origin &&
     a.jail === b.jail &&
     a.ip === b.ip &&
+    a.source === b.source &&
     a.page === b.page &&
     a.page_size === b.page_size
   );
@@ -386,11 +387,12 @@ export function HistoryPage(): React.JSX.Element {
   const styles = useStyles();
 
   // Filter state
-  const [range, setRange] = useState<TimeRange>("24h");
+  const [range, setRange] = useState<TimeRange>("7d");
   const [originFilter, setOriginFilter] = useState<BanOriginFilter>("all");
   const [jailFilter, setJailFilter] = useState("");
   const [ipFilter, setIpFilter] = useState("");
   const [appliedQuery, setAppliedQuery] = useState<HistoryQuery>({
+    source: "archive",
     page_size: PAGE_SIZE,
   });
 
@@ -406,6 +408,7 @@ export function HistoryPage(): React.JSX.Element {
       origin: originFilter !== "all" ? originFilter : undefined,
       jail: jailFilter.trim() || undefined,
       ip: ipFilter.trim() || undefined,
+      source: "archive",
       page: 1,
       page_size: PAGE_SIZE,
     };
@@ -487,23 +490,6 @@ export function HistoryPage(): React.JSX.Element {
         />
       </div>
 
-      {/* ---------------------------------------------------------------- */}
-      {/* Error / loading state                                             */}
-      {/* ---------------------------------------------------------------- */}
-      {error && (
-        <MessageBar intent="error">
-          <MessageBarBody>{error}</MessageBarBody>
-        </MessageBar>
-      )}
-
-      {loading && !error && (
-        <div
-          style={{ display: "flex", justifyContent: "center", padding: tokens.spacingVerticalXL }}
-        >
-          <Spinner label="Loading history…" />
-        </div>
-      )}
-
       {/* ---------------------------------------------------------------- */}
       {/* Summary                                                           */}
       {/* ---------------------------------------------------------------- */}

frontend/src/pages/MapPage.tsx

@@ -98,8 +98,10 @@ export function MapPage(): React.JSX.Element {
 
   const PAGE_SIZE_OPTIONS = [25, 50, 100] as const;
 
+  const source = range === "24h" ? "fail2ban" : "archive";
+
   const { countries, countryNames, bans, total, loading, error, refresh } =
-    useMapData(range, originFilter);
+    useMapData(range, originFilter, source);
 
   const {
     thresholds: mapThresholds,

frontend/src/pages/__tests__/HistoryPage.test.tsx

@@ -50,7 +50,8 @@ describe("HistoryPage", () => {
     // Initial load should include the auto-applied default query.
     await waitFor(() => {
       expect(lastQuery).toEqual({
-        range: "24h",
+        range: "7d",
+        source: "archive",
         origin: undefined,
         jail: undefined,
         ip: undefined,

frontend/src/types/history.ts

@@ -57,6 +57,7 @@ export interface HistoryQuery {
   origin?: BanOriginFilter;
   jail?: string;
   ip?: string;
+  source?: "fail2ban" | "archive";
   page?: number;
   page_size?: number;
 }