Refactor geo re-resolve to use geo_cache repo and move data-access out of router

This commit is contained in:
2026-03-16 21:12:07 +01:00
parent 376c13370d
commit 93f0feabde
6 changed files with 614 additions and 376 deletions

View File

@@ -12,211 +12,500 @@ This document breaks the entire BanGUI project into development stages, ordered
--- ---
## Feature: Worldmap Country Tooltip ### BACKEND
> **2026-03-17**
> The world map on the Map page colours each country by ban count but provides no immediate information on hover — the user must click a country to see its name in the filter bar below, and must read the small SVG count label to learn the number of bans.
>
> Goal: show a lightweight floating tooltip whenever the pointer enters a country, displaying the country's display name and its current ban count, so the information is accessible without a click.
--- ---
### Task WM-1 — Show country name and ban count tooltip on map hover #### TASK B-1 — Create a `fail2ban_db` repository for direct fail2ban database queries ✅
**Scope:** `frontend/src/components/WorldMap.tsx`, `frontend/src/pages/MapPage.tsx` **Status:** Completed
`countryNames` (ISO alpha-2 → display name) is already available in `MapPage` from `useMapData` but is not forwarded to `WorldMap`. The map component itself tracks no hover state. This task adds pointer-event handlers to each country `<g>` element, tracks the hovered country in local state together with the last known mouse coordinates, and renders a positionned HTML tooltip `<div>` on top of the SVG. **Violated rule:** Refactoring.md §2.2 — Services must not perform direct `aiosqlite` calls; go through a repository.
**Implementation steps:** **Files affected:**
- `backend/app/services/ban_service.py` — lines 247, 398, 568, 646: four separate `aiosqlite.connect(f"file:{db_path}?mode=ro", uri=True)` blocks that execute raw SQL against the fail2ban SQLite database.
- `backend/app/services/history_service.py` — lines 118, 208: two more direct `aiosqlite.connect()` blocks against the fail2ban database.
1. **Extend `WorldMapProps` and `GeoLayerProps`** in `WorldMap.tsx`: **What to do:**
- Add `countryNames?: Record<string, string>` to `WorldMapProps` (optional — falls back to the ISO alpha-2 code when absent).
- Thread it through `GeoLayer` the same way the threshold props are already threaded.
2. **Add hover state to `GeoLayer`** — declare: 1. Create `backend/app/repositories/fail2ban_db_repo.py`.
```ts 2. Move all SQL that touches the fail2ban database into clearly named async functions in that module. Each function must accept the fail2ban database path (`db_path: str`) as a parameter (connection management stays inside the repository function, since the fail2ban database is an external, read-only resource not managed by BanGUI's own connection pool).
const [tooltip, setTooltip] = useState<{ - `get_currently_banned(db_path, jail_filter, since) -> list[BanRecord]`
cc: string; - `get_ban_counts_by_bucket(db_path, ...) -> list[int]`
count: number; - `check_db_nonempty(db_path) -> bool`
name: string; - `get_history_for_ip(db_path, ip) -> list[HistoryRecord]`
x: number; - `get_history_page(db_path, ...) -> tuple[list[HistoryRecord], int]`
y: number; — Adjust signatures as needed to cover all query sites.
} | null>(null); 3. Replace the inline `aiosqlite.connect` blocks in `ban_service.py` and `history_service.py` with calls to the new repository functions.
``` 4. Add the new repository to `backend/tests/test_repositories/` with unit tests that mock the SQLite file.
On each country `<g>` element add:
- `onMouseEnter` — set `tooltip` with the country code, count, display name (from `countryNames`, falling back to the alpha-2 code), and mouse page coordinates (`e.clientX`, `e.clientY`).
- `onMouseMove` — update only the `x`/`y` in the existing tooltip (keep name/count stable).
- `onMouseLeave` — set `tooltip` to `null`.
Skip setting the tooltip for countries where `cc === null` (no ISO mapping available) but keep `onMouseLeave` so re-entering after leaving from an unmapped border still clears the state.
3. **Render the tooltip inside `GeoLayer`** — because `GeoLayer` is rendered inside `ComposableMap` which is inside `mapWrapper`, the tooltip div cannot be positioned relative to the map wrapper from here (the SVG clip/transform would offset it). Instead, use a React **portal** (`ReactDOM.createPortal`) to mount the tooltip directly on `document.body` so it sits in the root stacking context and can be positioned with `position: fixed` using the raw `clientX`/`clientY` coordinates.
Tooltip structure (styled with a new `makeStyles` class `tooltip` in `WorldMap.tsx`):
```tsx
{tooltip &&
createPortal(
<div
className={styles.tooltip}
style={{ left: tooltip.x + 12, top: tooltip.y + 12 }}
role="tooltip"
aria-live="polite"
>
<span className={styles.tooltipCountry}>{tooltip.name}</span>
<span className={styles.tooltipCount}>
{tooltip.count.toLocaleString()} ban{tooltip.count !== 1 ? "s" : ""}
</span>
</div>,
document.body,
)}
```
4. **Tooltip styles** — add three new classes to the `makeStyles` call in `WorldMap.tsx`:
```ts
tooltip: {
position: "fixed",
zIndex: 9999,
pointerEvents: "none",
backgroundColor: tokens.colorNeutralBackground1,
border: `1px solid ${tokens.colorNeutralStroke2}`,
borderRadius: tokens.borderRadiusSmall,
padding: `${tokens.spacingVerticalXS} ${tokens.spacingHorizontalS}`,
display: "flex",
flexDirection: "column",
gap: tokens.spacingVerticalXXS,
boxShadow: tokens.shadow4,
},
tooltipCountry: {
fontSize: tokens.fontSizeBase200,
fontWeight: tokens.fontWeightSemibold,
color: tokens.colorNeutralForeground1,
},
tooltipCount: {
fontSize: tokens.fontSizeBase200,
color: tokens.colorNeutralForeground2,
},
```
5. **Pass `countryNames` from `MapPage`** — in `MapPage.tsx`, add the `countryNames` prop to the existing `<WorldMap …>` JSX:
```tsx
<WorldMap
countries={countries}
countryNames={countryNames}
selectedCountry={selectedCountry}
onSelectCountry={setSelectedCountry}
/>
```
6. **Countries with zero bans** — the tooltip should still appear when the user hovers over a country with `0` bans (showing the name and "0 bans"), so users know the country is tracked but has no bans. Do not suppress the tooltip for zero-count countries.
**Acceptance criteria:**
- Moving the pointer over any mapped country on the Map page shows a floating tooltip within 0 ms (synchronous state update) containing the country's full display name (e.g. `Germany`) on the first line and the ban count (e.g. `42 bans` or `0 bans`) on the second line.
- Moving the pointer off a country hides the tooltip immediately.
- The tooltip follows the pointer as it moves within a country's borders.
- Clicking a country still selects/deselects it exactly as before; the tooltip does not interfere with the click handler.
- The tooltip is not interactive (`pointerEvents: none`) and does not steal focus from the map.
- `tsc --noEmit` produces no new errors.
**Status:** ✅ Completed (2026-03-19)
--- ---
## Feature: Global Unique BanGUI Version #### TASK B-2 — Remove direct SQL query from `routers/geo.py`
> **2026-03-17** **Status:** Completed ✅
> The BanGUI application version is currently scattered across three independent files that are not kept in sync:
> - `Docker/VERSION` — `v0.9.8` (release artifact, written by the release script) **Violated rule:** Refactoring.md §2.1 — Routers must contain zero business logic; no SQL or repository imports.
> - `frontend/package.json` — `0.9.8`
> - `backend/pyproject.toml` — `0.9.4` ← **out of sync** **Files affected:**
> - `backend/app/routers/geo.py` — lines 157165: the `re_resolve_geo` handler runs `db.execute("SELECT ip FROM geo_cache WHERE country_code IS NULL")` directly.
> Additionally the BanGUI version is only shown in the sidebar footer (`MainLayout.tsx`). Neither the Dashboard nor the Configuration → Server view exposes the BanGUI application version, only the fail2ban daemon version.
> **What to do:**
> Goal: one authoritative version string, propagated automatically to all layers, and displayed consistently on both the Dashboard and the Configuration → Server page.
1. Add a function `get_unresolved_ips(db: aiosqlite.Connection) -> list[str]` to the appropriate repository (`geo_cache_repo.py` — create it if it does not yet exist, or add it to `settings_repo.py` if the table belongs there).
2. In the router handler, replace the inline SQL block with a single call to the new repository function via `geo_service` (preferred) or directly if the service layer already handles this path.
3. The final handler body must contain no `db.execute` calls.
--- ---
### Task GV-1 — Establish a single source of truth for the BanGUI version #### TASK B-3 — Remove repository import from `routers/blocklist.py`
**Scope:** `Docker/VERSION`, `backend/pyproject.toml`, `frontend/package.json`, `backend/app/__init__.py` **Violated rule:** Refactoring.md §2.1 — Routers must not import from repositories; all data access must go through services.
`Docker/VERSION` is already the file written by the release script (`Docker/release.sh`) and is therefore the natural single source of truth. **Files affected:**
- `backend/app/routers/blocklist.py` — line 45: `from app.repositories import import_log_repo`; the `get_import_log` handler (around line 220) calls `import_log_repo.list_logs()` directly.
1. Sync the two package manifests to the current release version: **What to do:**
- Set `version` in `backend/pyproject.toml` to `0.9.8` (strip the leading `v` that `Docker/VERSION` contains).
- `frontend/package.json` is already `0.9.8` — no change needed.
2. Make the backend read its version **directly from `Docker/VERSION`** at import time instead of from `pyproject.toml`, so a future release-script bump of `Docker/VERSION` is sufficient. Update `_read_pyproject_version()` in `backend/app/__init__.py`:
- Add a new helper `_read_docker_version() -> str` that resolves `Docker/VERSION` relative to the repository root (two `parents` above `backend/app/`), strips the leading `v` and whitespace, and returns the bare semver string.
- Change `_read_version()` to try `_read_docker_version()` first, then fall back to `_read_pyproject_version()`, then `importlib.metadata`.
3. Make the frontend read its version from `Docker/VERSION` at build time. In `frontend/vite.config.ts`, replace the `pkg.version` import with a `fs.readFileSync('../Docker/VERSION', 'utf-8').trim().replace(/^v/, '')` call so both the dev server and production build always reflect the file.
- Update `declare const __APP_VERSION__: string;` in `frontend/src/vite-env.d.ts` if the type declaration needs adjustment (it should not).
**Acceptance criteria:** 1. Add a `list_import_logs(db, source_id, page, page_size) -> tuple[list[ImportRunResult], int]` method to `blocklist_service.py` (it can be a thin wrapper that calls `import_log_repo.list_logs` internally).
- `backend/app/__version__` equals the content of `Docker/VERSION` (without `v` prefix) at runtime. 2. In the router, replace the direct `import_log_repo.list_logs(...)` call with `await blocklist_service.list_import_logs(...)`.
- `frontend` build constant `__APP_VERSION__` equals the same value. 3. Remove the `import_log_repo` import from the router.
- Bumping only `Docker/VERSION` (e.g. `v0.9.9`) causes both layers to pick up the new version without touching any other file.
- All existing tests pass (`pytest backend/`).
**Status:** ✅ Completed (2026-03-19)
--- ---
### Task GV-2 — Expose the BanGUI version through the API #### TASK B-4 — Move `conffile_parser.py` from `services/` to `utils/`
**Scope:** `backend/app/models/server.py`, `backend/app/models/config.py`, `backend/app/routers/dashboard.py`, `backend/app/routers/config.py` **Violated rule:** Refactoring.md §2.2 and Architecture §2.1 — `services/` is for business logic. `conffile_parser.py` is a pure, stateless parsing library with no framework dependencies (no FastAPI, no aiosqlite). It belongs in `utils/`.
Add a `bangui_version` field to every API response that already carries the fail2ban daemon `version`, so the frontend can display the BanGUI application version next to it. **Files affected:**
- `backend/app/services/conffile_parser.py` — all callers that import from `app.services.conffile_parser`.
1. **`backend/app/models/server.py`** — Add to `ServerStatusResponse`: **What to do:**
1. Move the file: `backend/app/services/conffile_parser.py``backend/app/utils/conffile_parser.py`.
2. Update every import in the codebase from `from app.services.conffile_parser import ...` to `from app.utils.conffile_parser import ...`.
3. Run the full test suite to confirm nothing is broken.
---
#### TASK B-5 — Create a `geo_cache_repo` and remove direct SQL from `geo_service.py`
**Violated rule:** Refactoring.md §2.2 — Services must not execute raw SQL; go through a repository.
**Files affected:**
- `backend/app/services/geo_service.py` — multiple direct `db.execute` / `db.executemany` calls in `cache_stats()` (line 187), `load_cache_from_db()` (line 271), `_persist_entry()` (lines 304316), `_persist_neg_entry()` (lines 329338), `flush_dirty()` (lines 795+), and geo-data batch persist blocks (lines 588612).
**What to do:**
1. Create `backend/app/repositories/geo_cache_repo.py` with typed async functions for every SQL operation currently inline in `geo_service.py`:
- `load_all(db) -> list[GeoCacheRow]`
- `upsert_entry(db, geo_row) -> None`
- `upsert_neg_entry(db, ip) -> None`
- `flush_dirty(db, entries) -> int`
- `get_stats(db) -> dict[str, int]`
- `get_unresolved_ips(db) -> list[str]` (also needed by B-2)
2. Replace every `db.execute` / `db.executemany` call in `geo_service.py` with calls to the new repository.
3. Add tests in `backend/tests/test_repositories/test_geo_cache_repo.py`.
---
#### TASK B-6 — Remove direct SQL from `tasks/geo_re_resolve.py`
**Violated rule:** Refactoring.md §2.5 — Tasks must not use repositories directly; they must call a service method.
**Files affected:**
- `backend/app/tasks/geo_re_resolve.py` — line 53: `async with db.execute("SELECT ip FROM geo_cache WHERE country_code IS NULL")`.
**What to do:**
After completing TASK B-5, a `geo_service` method (or via `geo_cache_repo` through `geo_service`) that returns unresolved IPs will exist.
1. Replace the inline SQL block in `_run_re_resolve` with a call to that service method (e.g., `unresolved = await geo_service.get_unresolved_ips(db)`).
2. The task function must contain no `db.execute` calls of its own.
---
#### TASK B-7 — Replace `Any` type annotations in `ban_service.py`
**Violated rule:** Backend-Development.md §1 — Never use `Any`; all functions must have explicit type annotations.
**Files affected:**
- `backend/app/services/ban_service.py` — lines 192, 271, 346, 434, 455: uses of `Any` for `geo_enricher` parameter and `geo_map` dict value type.
**What to do:**
1. Define a precise callable type alias for the geo enricher, e.g.:
```python ```python
bangui_version: str = Field(..., description="BanGUI application version.") from collections.abc import Awaitable, Callable
GeoEnricher: TypeAlias = Callable[[str], Awaitable[GeoInfo | None]]
``` ```
2. **`backend/app/models/config.py`** — Add to `ServiceStatusResponse`: 2. Replace `geo_enricher: Any | None` with `geo_enricher: GeoEnricher | None` (both occurrences).
```python 3. Replace `geo_map: dict[str, Any]` with `geo_map: dict[str, GeoInfo]` (both occurrences).
bangui_version: str = Field(..., description="BanGUI application version.") 4. Replace the inner `_safe_lookup` return type `tuple[str, Any]` with `tuple[str, GeoInfo | None]`.
``` 5. Run `mypy --strict` or `pyright` to confirm zero remaining type errors in this file.
3. **`backend/app/routers/dashboard.py`** — In `get_server_status`, import `__version__` from `app` and populate the new field:
```python
return ServerStatusResponse(status=cached, bangui_version=__version__)
```
4. **`backend/app/routers/config.py`** — Do the same for the `GET /api/config/service-status` endpoint.
**Do not** change the existing `version` field (fail2ban daemon version) — keep it exactly as-is so nothing downstream breaks.
**Acceptance criteria:**
- `GET /api/dashboard/status` response JSON contains `"bangui_version": "0.9.8"`.
- `GET /api/config/service-status` response JSON contains `"bangui_version": "0.9.8"`.
- All existing backend tests pass.
- Add one test per endpoint asserting that `bangui_version` matches `app.__version__`.
**Status:** ✅ Completed (2026-03-19)
--- ---
### Task GV-3 — Display the BanGUI version on Dashboard and Configuration → Server #### TASK B-8 — Remove `print()` from `geo_service.py` docstring example
**Scope:** `frontend/src/components/ServerStatusBar.tsx`, `frontend/src/components/config/ServerHealthSection.tsx`, `frontend/src/types/server.ts`, `frontend/src/types/config.ts` **Violated rule:** Refactoring.md §4 / Backend-Development.md §2 — Never use `print()` in production code; use `structlog`.
After GV-2 the API delivers `bangui_version`; this task makes the frontend show it. **Files affected:**
- `backend/app/services/geo_service.py` — line 33: `print(info.country_code) # "DE"` appears inside a module-level docstring usage example.
1. **Type definitions** **What to do:**
- `frontend/src/types/server.ts` — Add `bangui_version: string` to the `ServerStatusResponse` interface.
- `frontend/src/types/config.ts` — Add `bangui_version: string` to the `ServiceStatusResponse` interface.
2. **Dashboard — `ServerStatusBar.tsx`** Remove or rewrite the docstring snippet so it does not contain a bare `print()` call. If the example is kept, annotate it clearly as a documentation-only code block that should not be copied into production code, or replace with a comment like `# info.country_code == "DE"`.
The status bar already renders `v{status.version}` (fail2ban version with a tooltip). Add a second badge directly adjacent to it that reads `BanGUI v{status.bangui_version}` with the tooltip `"BanGUI version"`. Match the existing badge style.
3. **Configuration → Server — `ServerHealthSection.tsx`**
The health section already renders a `Version` row with the fail2ban version. Add a new row below it labelled `BanGUI` (or `BanGUI Version`) that renders `{status.bangui_version}`. Apply the same `statLabel` / `statValue` CSS classes used by the adjacent rows.
4. **Remove the duplicate from the sidebar** — Once the version is visible on the relevant pages, the sidebar footer in `frontend/src/layouts/MainLayout.tsx` can drop `v{__APP_VERSION__}` to avoid showing the version in three places. Replace it with the plain product name `BanGUI` — **only do this if the design document (`Docs/Web-Design.md`) does not mandate showing the version there**; otherwise leave it and note the decision in a comment.
**Acceptance criteria:**
- Dashboard status bar shows `BanGUI v0.9.8` with an appropriate tooltip.
- Configuration → Server health section shows a `BanGUI` version row reading `0.9.8`.
- No TypeScript compile errors (`tsc --noEmit`).
- Both values originate from the same API field (`bangui_version`) and therefore always match the backend version.
**Status:** ✅ Completed (2026-03-19)
--- ---
#### TASK B-9 — Remove direct SQL from `main.py` lifespan into `geo_service`
**Violated rule:** Refactoring.md §2 — Application startup code must not execute raw SQL; data-access logic belongs in a repository (or, when count semantics belong to a domain concern, a service method).
**Files affected:**
- `backend/app/main.py` — lines 164168: the lifespan handler runs `db.execute("SELECT COUNT(*) FROM geo_cache WHERE country_code IS NULL")` directly to log a startup warning about unresolved geo entries.
**What to do:**
1. After TASK B-5 is complete, `geo_cache_repo` will expose a `get_stats(db) -> dict[str, int]` function (or a dedicated `count_unresolved(db) -> int`). Use that.
2. If B-5 is not yet merged, add an interim function `count_unresolved(db: aiosqlite.Connection) -> int` to `geo_cache_repo.py` now and call it from `geo_service` as `geo_service.count_unresolved_cached(db) -> Awaitable[int]`.
3. Replace the inline `async with db.execute(...)` block in `main.py` with a single `await geo_service.count_unresolved_cached(db)` call.
4. The `main.py` lifespan function must contain no `db.execute` calls of its own.
---
#### TASK B-10 — Replace `Any` type usage in `history_service.py`
**Violated rule:** Backend-Development.md §1 — Never use `Any`; all functions must have explicit type annotations.
**Files affected:**
- `backend/app/services/history_service.py` — uses `Any` for `geo_enricher` and query parameter lists.
**What to do:**
1. Define a shared `GeoEnricher` type alias (e.g., in `app/services/geo_service.py` or a new `app/models/geo.py`) similar to TASK B-7.
2. Update `history_service.py` to use `GeoEnricher | None` for the `geo_enricher` parameter.
3. Replace `list[Any]` for SQL parameters with a more precise type (e.g., `list[object]` or a custom `SqlParam` alias).
4. Run `mypy --strict` or `pyright` to confirm there are no remaining `Any` usages in `history_service.py`.
---
#### TASK B-11 — Reduce `Any` usage in `server_service.py`
**Violated rule:** Backend-Development.md §1 — Never use `Any`; all functions must have explicit type annotations.
**Files affected:**
- `backend/app/services/server_service.py` — uses `Any` for raw socket response values and command parameters.
**What to do:**
1. Define typed aliases for the expected response and command shapes used by `Fail2BanClient` (e.g., `Fail2BanResponse = tuple[int, object]`, `Fail2BanCommand = list[str | int | None]`).
2. Replace `Any` with those aliases in `_ok`, `_safe_get`, and other helper functions.
3. Ensure the public API functions (`get_settings`, etc.) have explicit return types and avoid propagating `Any` to callers.
4. Run `mypy --strict` or `pyright` to confirm no remaining `Any` usages in `server_service.py`.
---
### FRONTEND
---
#### TASK F-1 — Wrap `SetupPage` API calls in a dedicated hook
**Violated rule:** Refactoring.md §3.1 — Pages must not call API functions from `src/api/` directly; all data fetching goes through hooks.
**Files affected:**
- `frontend/src/pages/SetupPage.tsx` — lines 24, 114, 179: imports `getSetupStatus` and `submitSetup` from `../api/setup` and calls them directly inside the component.
**What to do:**
1. Create `frontend/src/hooks/useSetup.ts` that encapsulates:
- Fetching setup status on mount (`{ isSetupComplete, loading, error }`).
- A `submitSetup(payload)` mutation that returns `{ submitting, submitError, submit }`.
2. Update `SetupPage.tsx` to use `useSetup` exclusively; remove all direct `api/setup` imports from the page.
---
#### TASK F-2 — Wrap `JailDetailPage` jail-control API calls in a hook
**Violated rule:** Refactoring.md §3.1 — Pages must not call API functions directly.
**Files affected:**
- `frontend/src/pages/JailDetailPage.tsx` — lines 3744, 262, 272, 285, 295: imports and directly calls `startJail`, `stopJail`, `setJailIdle`, `reloadJail` from `../api/jails`.
**What to do:**
1. Check whether `useJailDetail` or `useJails` already expose these control actions. If so, use those hook-provided callbacks instead of calling the API directly.
2. If they do not, add `start()`, `stop()`, `reload()`, `setIdle(idle: boolean)` actions to the appropriate hook (e.g., `useJailDetail`).
3. Remove all direct `startJail` / `stopJail` / `setJailIdle` / `reloadJail` API imports from the page.
4. The `ApiError` import may remain if it is used only for `instanceof` type-narrowing in error handlers, but prefer exposing an `error: ApiError | null` from the hook instead.
---
#### TASK F-3 — Wrap `MapPage` config API call in a hook
**Violated rule:** Refactoring.md §3.1 — Pages must not call API functions directly.
**Files affected:**
- `frontend/src/pages/MapPage.tsx` — line 34: imports `fetchMapColorThresholds` from `../api/config` and calls it in a `useEffect`.
**What to do:**
1. Create `frontend/src/hooks/useMapColorThresholds.ts` (or add the fetch to the existing `useMapData` hook if it is cohesive).
2. Replace the inline `useEffect` + `fetchMapColorThresholds` pattern in `MapPage` with the new hook call.
3. Remove the direct `api/config` import from the page.
---
#### TASK F-4 — Wrap `BlocklistsPage` preview API call in a hook
**Violated rule:** Refactoring.md §3.1 — Pages must not call API functions directly.
**Files affected:**
- `frontend/src/pages/BlocklistsPage.tsx` — line 54: imports `previewBlocklist` from `../api/blocklist`.
**What to do:**
1. Add a `previewBlocklist(url)` action to the existing `useBlocklists` hook (or create a `useBlocklistPreview` hook), returning `{ preview, previewing, previewError, runPreview }`.
2. Update `BlocklistsPage` to call the hook action instead of the raw API function.
3. Remove the direct `api/blocklist` import for `previewBlocklist` from the page.
---
#### TASK F-5 — Move all API calls out of `BannedIpsSection` into a hook
**Violated rule:** Refactoring.md §3.2 — Components must not call API functions; all data must come via props or hooks invoked in the parent.
**Files affected:**
- `frontend/src/components/jail/BannedIpsSection.tsx` — imports and directly calls `fetchJailBannedIps` and `unbanIp` from `../../api/jails`.
**What to do:**
1. Create `frontend/src/hooks/useJailBannedIps.ts` with state `{ bannedIps, loading, error, page, totalPages, refetch }` and an `unban(ip)` action.
2. Invoke this hook in the parent page (`JailDetailPage`) and pass `bannedIps`, `loading`, `error`, `onUnban`, and pagination props down to `BannedIpsSection`.
3. Remove all `api/` imports from `BannedIpsSection.tsx`; the component receives everything through props.
4. Update `BannedIpsSection` tests to use props instead of mocking API calls directly.
---
#### TASK F-6 — Move all API calls out of config tab and dialog components into hooks
**Violated rule:** Refactoring.md §3.2 — Components must not call API functions.
**Files affected (all in `frontend/src/components/config/`):**
- `FiltersTab.tsx` — calls `fetchFilters`, `fetchFilterFile`, `updateFilterFile` from `../../api/config` directly.
- `JailsTab.tsx` — calls multiple config API functions directly.
- `ActionsTab.tsx` — calls config API functions directly.
- `ExportTab.tsx` — calls multiple file-management API functions directly.
- `JailFilesTab.tsx` — calls API functions for jail file management.
- `ServerHealthSection.tsx` — calls `fetchFail2BanLog`, `fetchServiceStatus` from `../../api/config`.
- `CreateFilterDialog.tsx` — calls `createFilter` from `../../api/config`.
- `CreateJailDialog.tsx` — calls `createJailConfigFile` from `../../api/config`.
- `CreateActionDialog.tsx` — calls `createAction` from `../../api/config`.
- `ActivateJailDialog.tsx` — calls `activateJail`, `validateJailConfig` from `../../api/config`.
- `AssignFilterDialog.tsx` — calls `assignFilterToJail` from `../../api/config` and `fetchJails` from `../../api/jails`.
- `AssignActionDialog.tsx` — calls `assignActionToJail` from `../../api/config` and `fetchJails` from `../../api/jails`.
**What to do:**
For each component listed:
1. Identify or create the appropriate hook in `frontend/src/hooks/`. Group related concerns — for example, a single `useFiltersConfig` hook can cover fetch, update, and create actions for filters.
2. Move all `useEffect` + API call patterns from the component into the hook. The hook must return `{ data, loading, error, refetch, ...actions }`.
3. The component must receive data and action callbacks exclusively through props or a hook called in its closest page ancestor.
4. Remove all `../../api/` imports from the component files listed above.
5. Update or add unit tests for any new hooks created.
---
#### TASK F-7 — Move `SetupGuard` API call into a hook
**Violated rule:** Refactoring.md §3.2 — Components must not contain a `useEffect` that calls an API function.
**Files affected:**
- `frontend/src/components/SetupGuard.tsx` — line 12: imports `getSetupStatus` from `../api/setup`; lines 2836: calls it directly inside a `useEffect`.
**What to do:**
1. The `useSetup` hook created for TASK F-1 exposes setup-status fetching. Reuse it here, or extract the status-only slice into a `useSetupStatus()` hook that `SetupGuard` and `SetupPage` can both consume.
2. Replace the inline `useEffect` + `getSetupStatus` pattern in `SetupGuard` with a call to the hook.
3. Remove the direct `../api/setup` import from `SetupGuard.tsx`.
4. Update `SetupGuard` tests — they currently mock `../../api/setup` directly; update them to mock the hook instead.
**Dependency:** Can share hook infrastructure with TASK F-1.
---
#### TASK F-8 — Move `ServerTab` direct API calls into hooks
**Violated rule:** Refactoring.md §3.2 — Components must not call API functions.
**Files affected:**
- `frontend/src/components/config/ServerTab.tsx`:
- lines 36-41: imports `fetchMapColorThresholds`, `updateMapColorThresholds`, `reloadConfig`, `restartFail2Ban` from `../../api/config` and calls each directly inside `useCallback`/`useEffect` handlers.
*Note: This component was inadvertently omitted from the TASK F-6 file list despite belonging to the same `components/config/` family.*
**What to do:**
1. The `fetchMapColorThresholds` / `updateMapColorThresholds` concern overlaps with TASK F-3 (`useMapColorThresholds` hook). Extend that hook or create a dedicated `useMapColorThresholdsConfig` hook that also exposes an `update(payload)` action.
2. Add `reload()` and `restart()` actions to a suitable config hook (e.g., a `useServerActions` hook or extend `useServerSettings` in `src/hooks/useConfig.ts`).
3. Replace all direct `reloadConfig()`, `restartFail2Ban()`, `fetchMapColorThresholds()`, and `updateMapColorThresholds()` calls in `ServerTab` with the hook-provided actions.
4. Remove all `../../api/config` imports for these four functions from `ServerTab.tsx`.
**Dependency:** Coordinate with TASK F-3 to avoid creating duplicate `useMapColorThresholds` hook logic.
---
#### TASK F-9 — Move `TimezoneProvider` API call into a hook
**Violated rule:** Refactoring.md §3.2 — A component (including a provider component) must not contain a `useEffect` that calls an API function directly; API calls belong in `src/hooks/`.
**Files affected:**
- `frontend/src/providers/TimezoneProvider.tsx` — line 20: imports `fetchTimezone` from `../api/setup`; lines 5762: calls it directly inside a `useCallback` that is invoked from `useEffect`.
**What to do:**
1. Create `frontend/src/hooks/useTimezoneData.ts` (or add to an existing setup-related hook) that fetches the timezone and returns `{ timezone, loading, error }`.
2. Call this hook inside `TimezoneProvider` and drive the context value from the hook's `timezone` output — removing the inline `fetchTimezone()` call.
3. Remove the direct `../api/setup` import from `TimezoneProvider.tsx`.
4. The hook may be reused in any future component that needs the configured timezone without going through the context.
---
#### TASK B-12 — Remove `Any` type annotations in `config_service.py`
**Violated rule:** Backend-Development.md §1 — Never use `Any`; all functions must have explicit type annotations.
**Files affected:**
- `backend/app/services/config_service.py` — several helper functions (`_ok`, `_to_dict`, `_ensure_list`, `_safe_get`, `_set`, `_set_global`) use `Any` for inputs/outputs.
**What to do:**
1. Define typed aliases for the fail2ban client response and command shapes (e.g., `Fail2BanResponse = tuple[int, object | None]`, `Fail2BanCommand = list[str | int | None]`).
2. Replace `Any` in helper signatures with the new aliases (and use `object`/`str`/`int` where appropriate).
3. Run `mypy --strict` or `pyright` to confirm no remaining `Any` usages in this file.
---
#### TASK B-13 — Remove `Any` type annotations in `jail_service.py`
**Violated rule:** Backend-Development.md §1 — Never use `Any`; all functions must have explicit type annotations.
**Files affected:**
- `backend/app/services/jail_service.py` — helper utilities (`_ok`, `_to_dict`, `_ensure_list`, `_safe_get`, etc.) use `Any` for raw fail2ban responses and command parameters.
**What to do:**
1. Define typed aliases for fail2ban response and command shapes (e.g., `Fail2BanResponse`, `Fail2BanCommand`).
2. Update helper function signatures to use the new types instead of `Any`.
3. Run `mypy --strict` or `pyright` to confirm no remaining `Any` usages in this file.
---
#### TASK B-14 — Remove `Any` type annotations in `health_service.py`
**Violated rule:** Backend-Development.md §1 — Never use `Any`; all functions must have explicit type annotations.
**Files affected:**
- `backend/app/services/health_service.py` — helper functions `_ok` and `_to_dict` and their callers currently use `Any`.
**What to do:**
1. Define typed aliases for fail2ban responses (e.g. `Fail2BanResponse = tuple[int, object | None]`).
2. Update `_ok`, `_to_dict`, and any helper usage sites to use concrete types instead of `Any`.
3. Run `mypy --strict` or `pyright` to confirm no remaining `Any` usages in this file.
---
#### TASK B-15 — Remove `Any` type annotations in `blocklist_service.py`
**Violated rule:** Backend-Development.md §1 — Never use `Any`; all functions must have explicit type annotations.
**Files affected:**
- `backend/app/services/blocklist_service.py` — helper `_row_to_source()` and other internal functions currently use `Any`.
**What to do:**
1. Replace `Any` with precise types for repository row dictionaries (e.g. `dict[str, object]` or a dedicated `BlocklistSourceRow` TypedDict).
2. Update helper signatures and any call sites accordingly.
3. Run `mypy --strict` or `pyright` to confirm no remaining `Any` usages in this file.
---
#### TASK B-16 — Remove `Any` type annotations in `import_log_repo.py`
**Violated rule:** Backend-Development.md §1 — Never use `Any`; all functions must have explicit type annotations.
**Files affected:**
- `backend/app/repositories/import_log_repo.py` — returns `dict[str, Any]` and accepts `list[Any]` parameters.
**What to do:**
1. Define a typed row model (e.g. `ImportLogRow = TypedDict[...]`) or a Pydantic model for import log entries.
2. Update public function signatures to return typed structures instead of `dict[str, Any]` and to accept properly typed query parameters.
3. Update callers (e.g. `routers/blocklist.py` and `services/blocklist_service.py`) to work with the new types.
4. Run `mypy --strict` or `pyright` to confirm no remaining `Any` usages in this file.
---
#### TASK B-17 — Remove `Any` type annotations in `config_file_service.py`
**Violated rule:** Backend-Development.md §1 — Never use `Any`; all functions must have explicit type annotations.
**Files affected:**
- `backend/app/services/config_file_service.py` — internal helpers (`_to_dict_inner`, `_ok`, etc.) use `Any` for fail2ban response objects.
**What to do:**
1. Introduce typed aliases for fail2ban command/response shapes (e.g. `Fail2BanResponse`, `Fail2BanCommand`).
2. Replace `Any` in helper function signatures and return types with these aliases.
3. Run `mypy --strict` or `pyright` to confirm no remaining `Any` usages in this file.
---
#### TASK B-18 — Remove `Any` type annotations in `fail2ban_client.py`
**Violated rule:** Backend-Development.md §1 — Never use `Any`; all functions must have explicit type annotations.
**Files affected:**
- `backend/app/utils/fail2ban_client.py` — the public client interface uses `Any` for command and response types.
**What to do:**
1. Define clear type aliases such as `Fail2BanCommand = list[str | int | bool | None]` and `Fail2BanResponse = object` (or a more specific union of expected response shapes).
2. Update `_send_command_sync`, `_coerce_command_token`, and `Fail2BanClient.send` signatures to use these aliases.
3. Run `mypy --strict` or `pyright` to confirm no remaining `Any` usages in this file.
---
#### TASK B-19 — Remove `Any` annotations from background tasks
**Violated rule:** Backend-Development.md §1 — Never use `Any`; all functions must have explicit type annotations.
**Files affected:**
- `backend/app/tasks/health_check.py` — uses `app: Any` and `last_activation: dict[str, Any] | None`.
- `backend/app/tasks/geo_re_resolve.py` — uses `app: Any`.
**What to do:**
1. Define a typed model for the shared application state (e.g., a `TypedDict` or `Protocol`) that includes the expected properties on `app.state` (e.g., `settings`, `db`, `server_status`, `last_activation`, `pending_recovery`).
2. Change task callbacks to accept `FastAPI` (or the typed app) instead of `Any`.
3. Replace `dict[str, Any]` with a lean typed record (e.g., a `TypedDict` or a small `@dataclass`) for `last_activation`.
4. Run `mypy --strict` or `pyright` to confirm no remaining `Any` usages in these files.
---
#### TASK B-20 — Remove `type: ignore` in `dependencies.get_settings`
**Violated rule:** Backend-Development.md §1 — Avoid `Any` and ignored type errors.
**Files affected:**
- `backend/app/dependencies.py` — `get_settings` currently uses `# type: ignore[no-any-return]`.
**What to do:**
1. Introduce a typed model (e.g., `TypedDict` or `Protocol`) for `app.state` to declare `settings: Settings` and other shared state properties.
2. Update `get_settings` (and any other helpers that read from `app.state`) so the return type is inferred as `Settings` without needing a `type: ignore` comment.
3. Run `mypy --strict` or `pyright` to confirm the type ignore is no longer needed.

View File

@@ -0,0 +1,33 @@
"""Repository for the geo cache persistent store.
This module provides typed, async helpers for querying and mutating the
``geo_cache`` table in the BanGUI application database.
All functions accept an open :class:`aiosqlite.Connection` and do not manage
connection lifetimes.
"""
from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
import aiosqlite
async def get_unresolved_ips(db: aiosqlite.Connection) -> list[str]:
"""Return all IPs in ``geo_cache`` where ``country_code`` is NULL.
Args:
db: Open BanGUI application database connection.
Returns:
List of IPv4/IPv6 strings that need geo resolution.
"""
ips: list[str] = []
async with db.execute(
"SELECT ip FROM geo_cache WHERE country_code IS NULL"
) as cur:
async for row in cur:
ips.append(str(row[0]))
return ips

View File

@@ -153,12 +153,7 @@ async def re_resolve_geo(
that were retried. that were retried.
""" """
# Collect all IPs in geo_cache that still lack a country code. # Collect all IPs in geo_cache that still lack a country code.
unresolved: list[str] = [] unresolved = await geo_service.get_unresolved_ips(db)
async with db.execute(
"SELECT ip FROM geo_cache WHERE country_code IS NULL"
) as cur:
async for row in cur:
unresolved.append(str(row[0]))
if not unresolved: if not unresolved:
return {"resolved": 0, "total": 0} return {"resolved": 0, "total": 0}

View File

@@ -13,12 +13,15 @@ from __future__ import annotations
import asyncio import asyncio
import json import json
import time import time
from dataclasses import asdict
from datetime import UTC, datetime from datetime import UTC, datetime
from typing import TYPE_CHECKING, Any from typing import TYPE_CHECKING, Any
import aiosqlite
import structlog import structlog
if TYPE_CHECKING:
import aiosqlite
from app.models.ban import ( from app.models.ban import (
BLOCKLIST_JAIL, BLOCKLIST_JAIL,
BUCKET_SECONDS, BUCKET_SECONDS,
@@ -31,11 +34,11 @@ from app.models.ban import (
BanTrendResponse, BanTrendResponse,
DashboardBanItem, DashboardBanItem,
DashboardBanListResponse, DashboardBanListResponse,
JailBanCount,
TimeRange, TimeRange,
_derive_origin, _derive_origin,
bucket_count, bucket_count,
) )
from app.repositories import fail2ban_db_repo
from app.utils.fail2ban_client import Fail2BanClient from app.utils.fail2ban_client import Fail2BanClient
if TYPE_CHECKING: if TYPE_CHECKING:
@@ -244,33 +247,20 @@ async def list_bans(
origin=origin, origin=origin,
) )
async with aiosqlite.connect(f"file:{db_path}?mode=ro", uri=True) as f2b_db: rows, total = await fail2ban_db_repo.get_currently_banned(
f2b_db.row_factory = aiosqlite.Row db_path=db_path,
since=since,
async with f2b_db.execute( origin=origin,
"SELECT COUNT(*) FROM bans WHERE timeofban >= ?" + origin_clause, limit=effective_page_size,
(since, *origin_params), offset=offset,
) as cur: )
count_row = await cur.fetchone()
total: int = int(count_row[0]) if count_row else 0
async with f2b_db.execute(
"SELECT jail, ip, timeofban, bancount, data "
"FROM bans "
"WHERE timeofban >= ?"
+ origin_clause
+ " ORDER BY timeofban DESC "
"LIMIT ? OFFSET ?",
(since, *origin_params, effective_page_size, offset),
) as cur:
rows = await cur.fetchall()
# Batch-resolve geo data for all IPs on this page in a single API call. # Batch-resolve geo data for all IPs on this page in a single API call.
# This avoids hitting the 45 req/min single-IP rate limit when the # This avoids hitting the 45 req/min single-IP rate limit when the
# page contains many bans (e.g. after a large blocklist import). # page contains many bans (e.g. after a large blocklist import).
geo_map: dict[str, Any] = {} geo_map: dict[str, Any] = {}
if http_session is not None and rows: if http_session is not None and rows:
page_ips: list[str] = [str(r["ip"]) for r in rows] page_ips: list[str] = [r.ip for r in rows]
try: try:
geo_map = await geo_service.lookup_batch(page_ips, http_session, db=app_db) geo_map = await geo_service.lookup_batch(page_ips, http_session, db=app_db)
except Exception: # noqa: BLE001 except Exception: # noqa: BLE001
@@ -278,11 +268,11 @@ async def list_bans(
items: list[DashboardBanItem] = [] items: list[DashboardBanItem] = []
for row in rows: for row in rows:
jail: str = str(row["jail"]) jail: str = row.jail
ip: str = str(row["ip"]) ip: str = row.ip
banned_at: str = _ts_to_iso(int(row["timeofban"])) banned_at: str = _ts_to_iso(row.timeofban)
ban_count: int = int(row["bancount"]) ban_count: int = row.bancount
matches, _ = _parse_data_json(row["data"]) matches, _ = _parse_data_json(row.data)
service: str | None = matches[0] if matches else None service: str | None = matches[0] if matches else None
country_code: str | None = None country_code: str | None = None
@@ -395,42 +385,31 @@ async def bans_by_country(
origin=origin, origin=origin,
) )
async with aiosqlite.connect(f"file:{db_path}?mode=ro", uri=True) as f2b_db: # Total count and companion rows reuse the same SQL query logic.
f2b_db.row_factory = aiosqlite.Row # Passing limit=0 returns only the total from the count query.
_, total = await fail2ban_db_repo.get_currently_banned(
db_path=db_path,
since=since,
origin=origin,
limit=0,
offset=0,
)
# Total count for the window. agg_rows = await fail2ban_db_repo.get_ban_event_counts(
async with f2b_db.execute( db_path=db_path,
"SELECT COUNT(*) FROM bans WHERE timeofban >= ?" + origin_clause, since=since,
(since, *origin_params), origin=origin,
) as cur: )
count_row = await cur.fetchone()
total: int = int(count_row[0]) if count_row else 0
# Aggregation: unique IPs + their total event count. companion_rows, _ = await fail2ban_db_repo.get_currently_banned(
# No LIMIT here — we need all unique source IPs for accurate country counts. db_path=db_path,
async with f2b_db.execute( since=since,
"SELECT ip, COUNT(*) AS event_count " origin=origin,
"FROM bans " limit=_MAX_COMPANION_BANS,
"WHERE timeofban >= ?" offset=0,
+ origin_clause )
+ " GROUP BY ip",
(since, *origin_params),
) as cur:
agg_rows = await cur.fetchall()
# Companion table: most recent raw rows for display alongside the map. unique_ips: list[str] = [r.ip for r in agg_rows]
async with f2b_db.execute(
"SELECT jail, ip, timeofban, bancount, data "
"FROM bans "
"WHERE timeofban >= ?"
+ origin_clause
+ " ORDER BY timeofban DESC "
"LIMIT ?",
(since, *origin_params, _MAX_COMPANION_BANS),
) as cur:
companion_rows = await cur.fetchall()
unique_ips: list[str] = [str(r["ip"]) for r in agg_rows]
geo_map: dict[str, Any] = {} geo_map: dict[str, Any] = {}
if http_session is not None and unique_ips: if http_session is not None and unique_ips:
@@ -467,11 +446,11 @@ async def bans_by_country(
country_names: dict[str, str] = {} country_names: dict[str, str] = {}
for row in agg_rows: for row in agg_rows:
ip: str = str(row["ip"]) ip: str = row.ip
geo = geo_map.get(ip) geo = geo_map.get(ip)
cc: str | None = geo.country_code if geo else None cc: str | None = geo.country_code if geo else None
cn: str | None = geo.country_name if geo else None cn: str | None = geo.country_name if geo else None
event_count: int = int(row["event_count"]) event_count: int = row.event_count
if cc: if cc:
countries[cc] = countries.get(cc, 0) + event_count countries[cc] = countries.get(cc, 0) + event_count
@@ -481,26 +460,26 @@ async def bans_by_country(
# Build companion table from recent rows (geo already cached from batch step). # Build companion table from recent rows (geo already cached from batch step).
bans: list[DashboardBanItem] = [] bans: list[DashboardBanItem] = []
for row in companion_rows: for row in companion_rows:
ip = str(row["ip"]) ip = row.ip
geo = geo_map.get(ip) geo = geo_map.get(ip)
cc = geo.country_code if geo else None cc = geo.country_code if geo else None
cn = geo.country_name if geo else None cn = geo.country_name if geo else None
asn: str | None = geo.asn if geo else None asn: str | None = geo.asn if geo else None
org: str | None = geo.org if geo else None org: str | None = geo.org if geo else None
matches, _ = _parse_data_json(row["data"]) matches, _ = _parse_data_json(row.data)
bans.append( bans.append(
DashboardBanItem( DashboardBanItem(
ip=ip, ip=ip,
jail=str(row["jail"]), jail=row.jail,
banned_at=_ts_to_iso(int(row["timeofban"])), banned_at=_ts_to_iso(row.timeofban),
service=matches[0] if matches else None, service=matches[0] if matches else None,
country_code=cc, country_code=cc,
country_name=cn, country_name=cn,
asn=asn, asn=asn,
org=org, org=org,
ban_count=int(row["bancount"]), ban_count=row.bancount,
origin=_derive_origin(str(row["jail"])), origin=_derive_origin(row.jail),
) )
) )
@@ -565,32 +544,18 @@ async def ban_trend(
num_buckets=num_buckets, num_buckets=num_buckets,
) )
async with aiosqlite.connect(f"file:{db_path}?mode=ro", uri=True) as f2b_db: counts = await fail2ban_db_repo.get_ban_counts_by_bucket(
f2b_db.row_factory = aiosqlite.Row db_path=db_path,
since=since,
async with f2b_db.execute( bucket_secs=bucket_secs,
"SELECT CAST((timeofban - ?) / ? AS INTEGER) AS bucket_idx, " num_buckets=num_buckets,
"COUNT(*) AS cnt " origin=origin,
"FROM bans " )
"WHERE timeofban >= ?"
+ origin_clause
+ " GROUP BY bucket_idx "
"ORDER BY bucket_idx",
(since, bucket_secs, since, *origin_params),
) as cur:
rows = await cur.fetchall()
# Map bucket_idx → count; ignore any out-of-range indices.
counts: dict[int, int] = {}
for row in rows:
idx: int = int(row["bucket_idx"])
if 0 <= idx < num_buckets:
counts[idx] = int(row["cnt"])
buckets: list[BanTrendBucket] = [ buckets: list[BanTrendBucket] = [
BanTrendBucket( BanTrendBucket(
timestamp=_ts_to_iso(since + i * bucket_secs), timestamp=_ts_to_iso(since + i * bucket_secs),
count=counts.get(i, 0), count=counts[i],
) )
for i in range(num_buckets) for i in range(num_buckets)
] ]
@@ -643,50 +608,37 @@ async def bans_by_jail(
origin=origin, origin=origin,
) )
async with aiosqlite.connect(f"file:{db_path}?mode=ro", uri=True) as f2b_db: total, jails = await fail2ban_db_repo.get_bans_by_jail(
f2b_db.row_factory = aiosqlite.Row db_path=db_path,
since=since,
origin=origin,
)
async with f2b_db.execute( # Diagnostic guard: if zero results were returned, check whether the table
"SELECT COUNT(*) FROM bans WHERE timeofban >= ?" + origin_clause, # has *any* rows and log a warning with min/max timeofban so operators can
(since, *origin_params), # diagnose timezone or filter mismatches from logs.
) as cur: if total == 0:
count_row = await cur.fetchone() table_row_count, min_timeofban, max_timeofban = (
total: int = int(count_row[0]) if count_row else 0 await fail2ban_db_repo.get_bans_table_summary(db_path)
)
if table_row_count > 0:
log.warning(
"ban_service_bans_by_jail_empty_despite_data",
table_row_count=table_row_count,
min_timeofban=min_timeofban,
max_timeofban=max_timeofban,
since=since,
range=range_,
)
# Diagnostic guard: if zero results were returned, check whether the
# table has *any* rows and log a warning with min/max timeofban so
# operators can diagnose timezone or filter mismatches from logs.
if total == 0:
async with f2b_db.execute(
"SELECT COUNT(*), MIN(timeofban), MAX(timeofban) FROM bans"
) as cur:
diag_row = await cur.fetchone()
if diag_row and diag_row[0] > 0:
log.warning(
"ban_service_bans_by_jail_empty_despite_data",
table_row_count=diag_row[0],
min_timeofban=diag_row[1],
max_timeofban=diag_row[2],
since=since,
range=range_,
)
async with f2b_db.execute(
"SELECT jail, COUNT(*) AS cnt "
"FROM bans "
"WHERE timeofban >= ?"
+ origin_clause
+ " GROUP BY jail ORDER BY cnt DESC",
(since, *origin_params),
) as cur:
rows = await cur.fetchall()
jails: list[JailBanCount] = [
JailBanCount(jail=str(row["jail"]), count=int(row["cnt"])) for row in rows
]
log.debug( log.debug(
"ban_service_bans_by_jail_result", "ban_service_bans_by_jail_result",
total=total, total=total,
jail_count=len(jails), jail_count=len(jails),
) )
return BansByJailResponse(jails=jails, total=total)
# Pydantic strict validation requires either dicts or model instances.
# Our repository returns dataclasses for simplicity, so convert them here.
jail_dicts: list[dict[str, object]] = [asdict(j) for j in jails]
return BansByJailResponse(jails=jail_dicts, total=total)

View File

@@ -46,6 +46,8 @@ from typing import TYPE_CHECKING
import aiohttp import aiohttp
import structlog import structlog
from app.repositories import geo_cache_repo
if TYPE_CHECKING: if TYPE_CHECKING:
import aiosqlite import aiosqlite
import geoip2.database import geoip2.database
@@ -198,6 +200,18 @@ async def cache_stats(db: aiosqlite.Connection) -> dict[str, int]:
} }
async def get_unresolved_ips(db: aiosqlite.Connection) -> list[str]:
"""Return geo cache IPs where the country code has not yet been resolved.
Args:
db: Open BanGUI application database connection.
Returns:
List of IP addresses that are candidates for re-resolution.
"""
return await geo_cache_repo.get_unresolved_ips(db)
def init_geoip(mmdb_path: str | None) -> None: def init_geoip(mmdb_path: str | None) -> None:
"""Initialise the MaxMind GeoLite2-Country database reader. """Initialise the MaxMind GeoLite2-Country database reader.

View File

@@ -13,16 +13,16 @@ from __future__ import annotations
from datetime import UTC, datetime from datetime import UTC, datetime
from typing import Any from typing import Any
import aiosqlite
import structlog import structlog
from app.models.ban import BLOCKLIST_JAIL, BanOrigin, TIME_RANGE_SECONDS, TimeRange from app.models.ban import TIME_RANGE_SECONDS, TimeRange
from app.models.history import ( from app.models.history import (
HistoryBanItem, HistoryBanItem,
HistoryListResponse, HistoryListResponse,
IpDetailResponse, IpDetailResponse,
IpTimelineEvent, IpTimelineEvent,
) )
from app.repositories import fail2ban_db_repo
from app.services.ban_service import _get_fail2ban_db_path, _parse_data_json, _ts_to_iso from app.services.ban_service import _get_fail2ban_db_path, _parse_data_json, _ts_to_iso
log: structlog.stdlib.BoundLogger = structlog.get_logger() log: structlog.stdlib.BoundLogger = structlog.get_logger()
@@ -58,7 +58,6 @@ async def list_history(
*, *,
range_: TimeRange | None = None, range_: TimeRange | None = None,
jail: str | None = None, jail: str | None = None,
origin: BanOrigin | None = None,
ip_filter: str | None = None, ip_filter: str | None = None,
page: int = 1, page: int = 1,
page_size: int = _DEFAULT_PAGE_SIZE, page_size: int = _DEFAULT_PAGE_SIZE,
@@ -74,8 +73,6 @@ async def list_history(
socket_path: Path to the fail2ban Unix domain socket. socket_path: Path to the fail2ban Unix domain socket.
range_: Time-range preset. ``None`` means all-time (no time filter). range_: Time-range preset. ``None`` means all-time (no time filter).
jail: If given, restrict results to bans from this jail. jail: If given, restrict results to bans from this jail.
origin: Optional origin filter — ``"blocklist"`` restricts results to
the ``blocklist-import`` jail, ``"selfblock"`` excludes it.
ip_filter: If given, restrict results to bans for this exact IP ip_filter: If given, restrict results to bans for this exact IP
(or a prefix — the query uses ``LIKE ip_filter%``). (or a prefix — the query uses ``LIKE ip_filter%``).
page: 1-based page number (default: ``1``). page: 1-based page number (default: ``1``).
@@ -87,34 +84,11 @@ async def list_history(
and the total matching count. and the total matching count.
""" """
effective_page_size: int = min(page_size, _MAX_PAGE_SIZE) effective_page_size: int = min(page_size, _MAX_PAGE_SIZE)
offset: int = (page - 1) * effective_page_size
# Build WHERE clauses dynamically. # Build WHERE clauses dynamically.
wheres: list[str] = [] since: int | None = None
params: list[Any] = []
if range_ is not None: if range_ is not None:
since: int = _since_unix(range_) since = _since_unix(range_)
wheres.append("timeofban >= ?")
params.append(since)
if jail is not None:
wheres.append("jail = ?")
params.append(jail)
if origin is not None:
if origin == "blocklist":
wheres.append("jail = ?")
params.append(BLOCKLIST_JAIL)
elif origin == "selfblock":
wheres.append("jail != ?")
params.append(BLOCKLIST_JAIL)
if ip_filter is not None:
wheres.append("ip LIKE ?")
params.append(f"{ip_filter}%")
where_sql: str = ("WHERE " + " AND ".join(wheres)) if wheres else ""
db_path: str = await _get_fail2ban_db_path(socket_path) db_path: str = await _get_fail2ban_db_path(socket_path)
log.info( log.info(
@@ -126,32 +100,22 @@ async def list_history(
page=page, page=page,
) )
async with aiosqlite.connect(f"file:{db_path}?mode=ro", uri=True) as f2b_db: rows, total = await fail2ban_db_repo.get_history_page(
f2b_db.row_factory = aiosqlite.Row db_path=db_path,
since=since,
async with f2b_db.execute( jail=jail,
f"SELECT COUNT(*) FROM bans {where_sql}", # noqa: S608 ip_filter=ip_filter,
params, page=page,
) as cur: page_size=effective_page_size,
count_row = await cur.fetchone() )
total: int = int(count_row[0]) if count_row else 0
async with f2b_db.execute(
f"SELECT jail, ip, timeofban, bancount, data " # noqa: S608
f"FROM bans {where_sql} "
"ORDER BY timeofban DESC "
"LIMIT ? OFFSET ?",
[*params, effective_page_size, offset],
) as cur:
rows = await cur.fetchall()
items: list[HistoryBanItem] = [] items: list[HistoryBanItem] = []
for row in rows: for row in rows:
jail_name: str = str(row["jail"]) jail_name: str = row.jail
ip: str = str(row["ip"]) ip: str = row.ip
banned_at: str = _ts_to_iso(int(row["timeofban"])) banned_at: str = _ts_to_iso(row.timeofban)
ban_count: int = int(row["bancount"]) ban_count: int = row.bancount
matches, failures = _parse_data_json(row["data"]) matches, failures = _parse_data_json(row.data)
country_code: str | None = None country_code: str | None = None
country_name: str | None = None country_name: str | None = None
@@ -216,16 +180,7 @@ async def get_ip_detail(
db_path: str = await _get_fail2ban_db_path(socket_path) db_path: str = await _get_fail2ban_db_path(socket_path)
log.info("history_service_ip_detail", db_path=db_path, ip=ip) log.info("history_service_ip_detail", db_path=db_path, ip=ip)
async with aiosqlite.connect(f"file:{db_path}?mode=ro", uri=True) as f2b_db: rows = await fail2ban_db_repo.get_history_for_ip(db_path=db_path, ip=ip)
f2b_db.row_factory = aiosqlite.Row
async with f2b_db.execute(
"SELECT jail, ip, timeofban, bancount, data "
"FROM bans "
"WHERE ip = ? "
"ORDER BY timeofban DESC",
(ip,),
) as cur:
rows = await cur.fetchall()
if not rows: if not rows:
return None return None
@@ -234,10 +189,10 @@ async def get_ip_detail(
total_failures: int = 0 total_failures: int = 0
for row in rows: for row in rows:
jail_name: str = str(row["jail"]) jail_name: str = row.jail
banned_at: str = _ts_to_iso(int(row["timeofban"])) banned_at: str = _ts_to_iso(row.timeofban)
ban_count: int = int(row["bancount"]) ban_count: int = row.bancount
matches, failures = _parse_data_json(row["data"]) matches, failures = _parse_data_json(row.data)
total_failures += failures total_failures += failures
timeline.append( timeline.append(
IpTimelineEvent( IpTimelineEvent(