Harden preview_log path validation and add regression test
This commit is contained in:
724
Docs/Tasks.md
724
Docs/Tasks.md
@@ -8,404 +8,452 @@ Reference: `Docs/Refactoring.md` for full analysis of each issue.
|
|||||||
|
|
||||||
## Open Issues
|
## Open Issues
|
||||||
|
|
||||||
---
|
### 1. Arbitrary File Read in `preview_log()`
|
||||||
|
|
||||||
### Task 1 — Move `Fail2BanConnectionError` and `Fail2BanProtocolError` to `app/exceptions.py`
|
**Where:** `backend/app/services/log_service.py` — the `preview_log()` function (around line 212). Called from `backend/app/routers/config_misc.py` via `POST /api/config/preview-log`.
|
||||||
|
|
||||||
**Found in:** `backend/app/utils/fail2ban_client.py` lines 128 and 142. Every router that catches these must currently `from app.utils.fail2ban_client import Fail2BanConnectionError`, importing a transport-layer utility directly.
|
**Goal:** The `preview_log()` function must validate the incoming `log_path` against an allowlist of safe directory prefixes before reading the file, exactly the way `read_fail2ban_log()` already does at line 131 using `_SAFE_LOG_PREFIXES`. After the fix, any path outside the allowed directories must be rejected with a `ConfigOperationError`.
|
||||||
|
|
||||||
**Goal:** Move (or re-export) both exception classes into `app/exceptions.py` alongside the rest of the domain error hierarchy. Routers and services should import them from `app.exceptions`, not from a utility module. `fail2ban_client.py` can keep the class definitions and simply re-export from `exceptions`, or import them from there — whichever direction avoids a circular import.
|
**Status:** Completed.
|
||||||
|
|
||||||
**Possible traps and issues:**
|
**Possible traps and issues:**
|
||||||
- `fail2ban_client.py` is in the `utils` layer; `exceptions.py` is in `app`. Moving the definition to `exceptions.py` and importing it back into `fail2ban_client.py` (for raising) is the natural direction, but verify there is no circular import chain (`exceptions` → `utils` → `exceptions`).
|
- The allowlist may need to be broader than `_SAFE_LOG_PREFIXES` (`/var/log`, `/config/log`) because `preview_log` is used with arbitrary log files that fail2ban monitors (e.g. `/var/log/auth.log`, `/var/log/nginx/access.log`). Restricting too tightly could break the regex tester feature.
|
||||||
- All callers across `routers/` and `services/` must be updated to import from the new location. A global grep is needed before starting.
|
- The path must be resolved (`.resolve()`) before checking prefixes, otherwise symlink-based bypasses are possible.
|
||||||
- `main.py` registers global exception handlers for both; that import path must be updated too.
|
- Needs a test that confirms a path like `/etc/shadow` or `/proc/self/environ` is rejected.
|
||||||
|
|
||||||
**Docs changes needed:** Update `Docs/Refactoring.md` to mark this issue resolved. No user-facing API change.
|
**Docs changes needed:** None. This is an internal security hardening; the API contract does not change (invalid paths return an error response instead of file contents).
|
||||||
|
|
||||||
**Why this is needed:** All domain exceptions should live in one registry so that global exception handlers, tests, and consumers have a single import target. Having transport-layer utilities define exceptions that the entire router layer must catch defeats the purpose of `app/exceptions.py`.
|
**Why this is needed:** An authenticated user can currently read the tail of any file readable by the backend process by sending a crafted `log_path` in the request body. This is an OWASP path traversal / arbitrary file read vulnerability. Even though it requires authentication, it violates the principle of least privilege — the endpoint should only read log files, not arbitrary system files.
|
||||||
|
|
||||||
**Status:** Completed ✅
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### Task 2 — Move the five `ConfigFile*` exceptions out of `raw_config_io_service.py`
|
### 2. Missing Repository Protocols for `settings_repo` and `history_archive_repo`
|
||||||
|
|
||||||
**Found in:** `backend/app/services/raw_config_io_service.py` lines 66–100 (`ConfigDirError`, `ConfigFileNotFoundError`, `ConfigFileExistsError`, `ConfigFileWriteError`, `ConfigFileNameError`). `backend/app/routers/file_config.py` lines 55–61 imports them directly from the service.
|
**Where:** `backend/app/repositories/protocols.py` defines protocols for 5 of the 7 repositories. The two missing are `settings_repo.py` and `history_archive_repo.py`.
|
||||||
|
|
||||||
**Goal:** Move all five class definitions into `app/exceptions.py`. Update `raw_config_io_service.py` to import and raise them from there. Update `file_config.py` to import them from `app.exceptions`.
|
**Goal:** Define `SettingsRepository` and `HistoryArchiveRepository` protocols in `protocols.py` that match the public functions of `settings_repo` (`get_setting`, `set_setting`, `delete_setting`, `get_all_settings`) and `history_archive_repo` (`archive_ban_event`, `get_max_timeofban`, `get_archived_history`). Add corresponding dependency providers in `dependencies.py` and typed aliases so these repositories can be injected the same way as the other five.
|
||||||
|
|
||||||
**Possible traps and issues:**
|
**Possible traps and issues:**
|
||||||
- The existing names must be preserved (they are part of the public raise/catch contract); do not rename them.
|
- `settings_repo` is imported directly by `setup_service` and `auth_service` — these call sites need to be updated to accept the protocol type, or at minimum the direct import must remain compatible.
|
||||||
- `file_config.py` currently imports from `app.services.raw_config_io_service` — that import line must change. Check whether any test files also import these classes directly from the service.
|
- `history_archive_repo` is imported inline in `history_service.py` and `ban_service.py` via `from app.repositories.history_archive_repo import ...`. Converting these to dependency injection requires threading the repo through service function signatures, which touches multiple call sites in routers and tasks.
|
||||||
- After the move, confirm that `raw_config_io_service.py` still raises them (not the old local definitions that are now gone).
|
- Changing function signatures is a breaking change for tests that call services directly.
|
||||||
|
|
||||||
**Docs changes needed:** Update `Docs/Refactoring.md`.
|
**Docs changes needed:** Update `Docs/Architekture.md` section 2.2 (Repositories table) to list all 7 repositories as having protocols.
|
||||||
|
|
||||||
**Why this is needed:** Routers should not reach into service internals to import exception types. All domain exceptions belong in the central registry so `main.py` global handlers and any future middleware can catch them uniformly.
|
**Why this is needed:** The other 5 repositories all have protocol definitions, enabling clean test mocking via dependency injection. Without protocols, `settings_repo` and `history_archive_repo` are tightly coupled — tests must monkeypatch the module import rather than injecting a fake. This inconsistency makes the codebase harder to test and violates the project's own dependency inversion principle documented in section 10 of the architecture doc.
|
||||||
|
|
||||||
**Status:** Completed ✅
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### Task 3 — Eliminate the `config_file_service.py` delegation façade
|
### 3. Remove Empty `helpers/` Directory
|
||||||
|
|
||||||
**Found in:** `backend/app/services/config_file_service.py` (1 136 lines). Every public function is a one-liner that delegates to `jail_config_service`, `filter_config_service`, or `action_config_service`. Those three sub-services import `config_file_service` back via lazy imports scattered across ~10 function bodies each, creating a hidden circular dependency.
|
**Where:** `backend/app/helpers/` — contains only `__init__.py` and `__pycache__/`.
|
||||||
|
|
||||||
**Goal:** Delete `config_file_service.py`. Update every caller (routers: `config_misc.py`, `jail_config.py`; services: anything that calls through the façade) to import the appropriate sub-service directly. Remove all lazy-import call sites (`from app.services import config_file_service as _cfs`) in `jail_config_service.py`, `filter_config_service.py`, and `action_config_service.py`.
|
**Goal:** Delete the `backend/app/helpers/` directory entirely. All utility code already lives in `backend/app/utils/`, which is the documented location for helper modules.
|
||||||
|
|
||||||
**Possible traps and issues:**
|
**Possible traps and issues:**
|
||||||
- `config_file_service.py` also re-exports `start_daemon` and `wait_for_fail2ban` from `app.utils`. Find all callers of those two names and update their imports.
|
- Verify no import references `app.helpers` anywhere in the codebase before deleting.
|
||||||
- The lazy imports exist specifically to avoid the circular dependency — simply removing them may surface an import cycle. Trace the actual call graph before deleting the file.
|
- If any generated tooling or IDE configuration references `helpers/`, that needs cleanup too.
|
||||||
- `config_misc.py` router imports `config_file_service` for several operations; it will need multiple new imports to replace the façade.
|
|
||||||
- Large file change — high risk of regression. Comprehensive test run required before and after.
|
|
||||||
|
|
||||||
**Docs changes needed:** Update `Docs/Refactoring.md` and `Docs/Architekture.md` if it mentions this service.
|
**Docs changes needed:** The architecture doc (`Docs/Architekture.md`) section 2.1 lists `helpers/` in the project structure tree. Remove that line.
|
||||||
|
|
||||||
**Status:** Completed ✅
|
**Why this is needed:** An empty package directory with no module inside it is confusing for developers. It suggests functionality should live there, when the project convention is to use `utils/`. Removing it eliminates ambiguity about where new helper code should go.
|
||||||
|
|
||||||
**Why this is needed:** Pure delegation façades add indirection with no abstraction benefit and obscure the true dependencies of the system. The hidden circular dependency via lazy imports is a structural risk — a refactor inside any of the three sub-services could easily break the cycle in unexpected ways.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### Task 4 — Move raw SQL query out of `history_service.py` into a repository
|
### 4. Add Path Validation to `_geoip_reader` Initialization
|
||||||
|
|
||||||
**Found in:** `backend/app/services/history_service.py` line 70: `db.execute("SELECT MAX(timeofban) FROM history_archive")`.
|
**Where:** `backend/app/services/geo_service.py` — the `init_geoip()` function (around line 249) sets the module-level `_geoip_reader` without acquiring `_cache_lock`.
|
||||||
|
|
||||||
**Goal:** Add a `get_max_timeofban() -> datetime | None` function to `backend/app/repositories/history_repo.py` (or the appropriate history archive repository). Replace the inline `db.execute` call in the service with a call to that repository function.
|
**Goal:** Add a clear code comment documenting the startup-only assumption: `init_geoip()` must only be called during application startup (from `startup.py`) before request handling begins. Optionally, add an assertion or guard that prevents double-initialization.
|
||||||
|
|
||||||
**Possible traps and issues:**
|
**Possible traps and issues:**
|
||||||
- Check whether `history_archive` is queried via the same `aiosqlite.Connection` that the repository layer already uses. Confirm the repository file exists and is the right home (there may be separate `history_repo` and `history_archive_repo`).
|
- Wrapping `init_geoip()` in `_cache_lock` would require making it `async`, which changes the call site in `startup.py`. Since `geoip2.database.Reader()` does blocking file I/O, a lock alone is not sufficient — it would also need `run_blocking`.
|
||||||
- The column `timeofban` must be mapped to the correct Python type (likely `datetime`) — match the convention of the surrounding repository functions.
|
- The simplest safe approach is a module-level `bool` flag (`_geoip_initialized`) checked at the top of `init_geoip()`, with a `RuntimeError` if called twice.
|
||||||
- Unit tests for `history_service` that mock the db connection will need to be updated to mock the repository call instead.
|
- Over-engineering this (full lock + async) is not worth it for a function called exactly once.
|
||||||
|
|
||||||
**Docs changes needed:** None beyond `Refactoring.md`.
|
|
||||||
|
|
||||||
**Status:** Completed ✅
|
|
||||||
|
|
||||||
**Why this is needed:** Raw SQL in the service layer bypasses the repository abstraction. It makes the service harder to test (requires a real DB schema) and harder to maintain (schema changes must be tracked in the service, not just the repository).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 5 — Remove the reversed dependency from `blocklist_service.py` to `app.tasks`
|
|
||||||
|
|
||||||
**Found in:** `backend/app/services/blocklist_service.py` line 530: `from app.tasks import blocklist_import as blocklist_import_task`. Services must not import from tasks; tasks are consumers of services, not dependencies of them.
|
|
||||||
|
|
||||||
**Goal:** Invert the dependency. The functionality needed from the task (triggering or querying a scheduled blocklist import) should be expressed as a callable or scheduler reference that is passed into the service function, or the service should expose a pure data/state function that the task calls rather than the service calling into the task.
|
|
||||||
|
|
||||||
**Possible traps and issues:**
|
|
||||||
- Understand what exactly the service does with `blocklist_import_task` — if it schedules or triggers a job, the job scheduler (`app.state.scheduler`) should be passed in as a parameter rather than imported from tasks.
|
|
||||||
- This may require adding a parameter to the affected service function and updating its callers (the router).
|
|
||||||
- If the task import is conditional/lazy, it still violates the dependency rule regardless of when it resolves.
|
|
||||||
|
|
||||||
**Docs changes needed:** Update `Docs/Refactoring.md` and `Docs/Architekture.md` (task→service dependency direction).
|
|
||||||
|
|
||||||
**Status:** Done ✅
|
|
||||||
|
|
||||||
**Why this is needed:** Circular layer dependencies (`service → task → service`) make it impossible to test services in isolation and create hidden initialisation-order coupling that can cause import errors or subtle bugs at startup.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 6 — Extract log reading and service status out of `config_service.py`
|
|
||||||
|
|
||||||
**Found in:** `backend/app/services/config_service.py` line 687 (`read_fail2ban_log`) and line 781 (`get_service_status`). Both are unrelated to configuration management and do not belong in this service.
|
|
||||||
|
|
||||||
**Goal:** Move `read_fail2ban_log` into `log_service.py` (the dedicated log-reading service). Move `get_service_status` into `health_service.py`. Update the calling routers (`config_misc.py` uses both) to import from the correct service.
|
|
||||||
|
|
||||||
**Possible traps and issues:**
|
|
||||||
- Check whether `log_service.py` and `health_service.py` already have similar functions; avoid duplication and reconcile signatures.
|
|
||||||
- `config_misc.py` currently imports `config_service` for these; after the move it will need two additional service imports.
|
|
||||||
- Existing tests targeting `config_service` that cover these functions will need to be moved to the appropriate test file.
|
|
||||||
|
|
||||||
**Docs changes needed:** Update `Docs/Refactoring.md`. Adjust any architecture diagram that shows `config_service` as the home for log/status operations.
|
|
||||||
|
|
||||||
**Why this is needed:** Single-responsibility principle. A configuration management service owning log reading and daemon status checks makes the service harder to reason about and violates the established service split.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 7 — Deduplicate `get_map_color_thresholds` / `update_map_color_thresholds`
|
|
||||||
|
|
||||||
**Found in:** Implementations exist in `backend/app/services/config_service.py`, `backend/app/services/setup_service.py`, and `backend/app/utils/setup_utils.py` (the latter used by both). The router `config_misc.py` calls `setup_service.get_map_color_thresholds()` for ongoing map color operations that have nothing to do with first-run setup.
|
|
||||||
|
|
||||||
**Goal:** Consolidate the single authoritative implementation in one appropriate service — likely a dedicated `settings_service.py` or inside `config_service.py`. Remove the duplicate copies. Update `config_misc.py` to import from the canonical location. Remove `get_map_color_thresholds` / `set_map_color_thresholds` from `setup_utils.py` if they only exist to serve this use case.
|
|
||||||
|
|
||||||
**Possible traps and issues:**
|
|
||||||
- `setup_utils.py` imports from `repositories` (itself a violation — see Task 10). Consolidating here creates a dependency chain; resolve Task 10 either before or as part of this task.
|
|
||||||
- `setup_service.py` is conceptually a first-run service; if callers of the consolidated function include both first-run setup and runtime settings, choose the runtime home and let `setup_service` call it, not the other way.
|
|
||||||
- Tests that mock `setup_service.get_map_color_thresholds` in the router tests will need updating.
|
|
||||||
|
|
||||||
**Docs changes needed:** Update `Docs/Refactoring.md`.
|
|
||||||
|
|
||||||
**Why this is needed:** Triplicated implementation violates DRY and means a change to the threshold schema must be made in three places. Using `setup_service` for ongoing runtime settings is conceptually wrong and misleads maintainers.
|
|
||||||
|
|
||||||
**Status:** Completed ✅
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 8 — Move the `activate_jail` 3-step restart workflow out of `config_misc.py` router
|
|
||||||
|
|
||||||
**Found in:** `backend/app/routers/config_misc.py` lines 193–197: the `restart_fail2ban` handler directly orchestrates `config_file_service.stop_daemon()` → `config_file_service.start_daemon()` → `config_file_service.wait_for_fail2ban()` as a three-step inline sequence.
|
|
||||||
|
|
||||||
**Goal:** Extract this orchestration into a `restart_daemon()` function inside `config_file_service.py` (or a more appropriate service). The router handler should call that single function.
|
|
||||||
|
|
||||||
**Possible traps and issues:**
|
|
||||||
- Error handling differs between the three steps; the new service function must preserve the existing error semantics (what happens if `start_daemon` fails must be the same as before).
|
|
||||||
- If `config_file_service.py` is being deleted (Task 3), coordinate which service owns this function after the façade is removed.
|
|
||||||
- `wait_for_fail2ban` is a polling operation — confirm any timeout behaviour is preserved.
|
|
||||||
|
|
||||||
**Docs changes needed:** Update `Docs/Refactoring.md`.
|
|
||||||
|
|
||||||
**Why this is needed:** Multi-step orchestration workflows are business logic and do not belong in HTTP handler functions. Keeping them in the service layer makes the workflow testable in isolation and reusable if a second endpoint ever needs to restart the daemon.
|
|
||||||
|
|
||||||
**Status:** Completed ✅
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 9 — Move `sign_session_token` call out of `auth.py` router into `auth_service.login()`
|
|
||||||
|
|
||||||
**Found in:** `backend/app/routers/auth.py` line 74 calls `sign_session_token(token, secret)` after calling `auth_service.login()`. The raw token is returned by the service; the signing is performed in the router.
|
|
||||||
|
|
||||||
**Goal:** `auth_service.login()` should return an already-signed token. The router should receive the final cookie value from the service without needing to know about the signing step. Move the `sign_session_token` call into `auth_service.login()`.
|
|
||||||
|
|
||||||
**Possible traps and issues:**
|
|
||||||
- `auth_service.login()` currently has a specific return type; changing what it returns (raw token → signed token) must be reflected in the return type annotation and any tests that assert on the raw token value.
|
|
||||||
- `sign_session_token` is currently a public function exported from `auth_service`. After this change it may become a private implementation detail (`_sign_session_token`). Check for any other callers before hiding it.
|
|
||||||
- Tests that test `login()` in isolation and currently check for the raw token will need to be updated to verify the signed format instead (or mock `sign_session_token` separately).
|
|
||||||
|
|
||||||
**Docs changes needed:** Update `Docs/Refactoring.md`.
|
|
||||||
|
|
||||||
**Why this is needed:** Security-sensitive token construction is business logic. Letting the HTTP layer build cookie values means a future maintainer adding a second login endpoint could forget the signing step, creating a silent security regression.
|
|
||||||
|
|
||||||
**Status:** Completed ✅
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 10 — Remove the repository import from `utils/setup_utils.py`
|
|
||||||
|
|
||||||
**Found in:** `backend/app/utils/setup_utils.py` line 5: `from app.repositories import settings_repo`. The `utils` layer must not import from `repositories`; that is the service layer's job.
|
|
||||||
|
|
||||||
**Goal:** Move any function in `setup_utils.py` that depends on `settings_repo` into `setup_service.py` (which is already allowed to import repositories). The remaining pure utility functions (password hashing, etc.) can stay in `setup_utils.py`.
|
|
||||||
|
|
||||||
**Possible traps and issues:**
|
|
||||||
- `setup_utils.py` is imported by `auth_service.py` (`from app.utils.setup_utils import get_password_hash`). This import is fine and must be preserved; only the repository-dependent functions move.
|
|
||||||
- Identify every function in `setup_utils.py` that directly or indirectly calls `settings_repo`. Move the entire call chain, not just the top-level function.
|
|
||||||
- The functions being moved may already have analogues in `setup_service.py`; check for duplication before moving.
|
|
||||||
|
|
||||||
**Docs changes needed:** Update `Docs/Refactoring.md` and `Docs/Architekture.md` (layer diagram).
|
|
||||||
|
|
||||||
**Why this is needed:** Utils are stateless helpers with no external dependencies. Allowing them to import from the repository layer breaks the architectural contract, makes the utilities untestable without a database, and obscures what is actually a service-layer operation.
|
|
||||||
|
|
||||||
**Status:** Completed ✅
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 11 — Centralise `DbDep` — remove local redefinitions in `blocklist.py` and `geo.py`
|
|
||||||
|
|
||||||
**Found in:** `backend/app/routers/blocklist.py` line 54 redefines `DbDep = Annotated[aiosqlite.Connection, Depends(get_db)]` locally. `backend/app/routers/geo.py` uses the inline form `Annotated[aiosqlite.Connection, Depends(get_db)]` directly in function signatures instead of importing `DbDep`.
|
|
||||||
|
|
||||||
**Goal:** Delete the local redefinitions. Import `DbDep` from `app.dependencies` in both routers.
|
|
||||||
|
|
||||||
**Possible traps and issues:**
|
|
||||||
- Confirm that `DbDep` is already exported from `app/dependencies.py` (verify the exact name used there).
|
|
||||||
- The inline form in `geo.py` may appear in multiple function signatures; all occurrences must be replaced.
|
|
||||||
- No behaviour change — purely a refactor. Run the test suite to confirm.
|
|
||||||
|
|
||||||
**Docs changes needed:** None beyond `Refactoring.md`.
|
|
||||||
|
|
||||||
**Why this is needed:** A single definition of `DbDep` ensures that any future change to the db dependency (e.g. adding row factory configuration) is applied uniformly across all routers.
|
|
||||||
|
|
||||||
**Status:** Completed ✅
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 12 — Complete the protocol injection layer or remove it
|
|
||||||
|
|
||||||
**Found in:** `backend/app/dependencies.py` defines `AuthServiceDep`, `JailServiceDep`, `ConfigServiceDep`, `GeoServiceDep`, `HistoryServiceDep`, `BlocklistServiceDep`, `HealthServiceDep`, `ServerServiceDep` as protocol-typed dependency aliases. Only `routers/auth.py` and `routers/jails.py` actually use these; the other 16 routers import concrete service modules directly.
|
|
||||||
|
|
||||||
**Goal:** Make a deliberate decision and enforce it consistently. Option A: adopt the protocol injection pattern everywhere — update all 16 non-compliant routers to accept their service via the typed `Dep` alias. Option B: acknowledge the pattern is unused overhead and remove the protocol aliases and `cast()` wrappers from `dependencies.py`, letting all routers import concrete services directly (the current de facto standard). Option B is cheaper; Option A makes service substitution in tests easier.
|
|
||||||
|
|
||||||
**Decision:** Option B applied — service protocol aliases and provider wrappers have been removed in favor of direct concrete service imports.
|
|
||||||
|
|
||||||
**Status:** Completed ✅
|
|
||||||
|
|
||||||
**Possible traps and issues:**
|
|
||||||
- Option A requires updating all 16 routers and their test files simultaneously; this is a broad change with high regression risk. Stage it one router at a time.
|
|
||||||
- Option B means deleting `services/protocols.py` (398 lines) and all `cast("…Service", …)` calls in `dependencies.py`. Ensure nothing outside this layer references the Protocol classes (check test files).
|
|
||||||
- `dependencies.py` also has repository protocol aliases (`BlocklistRepository`, `ImportLogRepository`, etc.) — decide whether those follow the same fate.
|
|
||||||
|
|
||||||
**Docs changes needed:** Update `Docs/Architekture.md` to reflect the chosen pattern. Update `Docs/Refactoring.md`.
|
|
||||||
|
|
||||||
**Why this is needed:** Inconsistency between `auth.py`/`jails.py` (uses protocols) and every other router (bypasses them) makes the codebase confusing and the injection layer misleading. This is the most widespread single structural inconsistency in the router layer.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 13 — Move `ban_ip`, `unban_ip`, and `get_active_bans` from `jail_service` to `ban_service`
|
|
||||||
|
|
||||||
**Status:** Completed ✅
|
|
||||||
|
|
||||||
**Found in:** `backend/app/services/jail_service.py` contains `ban_ip`, `unban_ip`, and `get_active_bans`. These operations conceptually belong in `ban_service.py`, which is the declared home for ban management. Routers `bans.py` and `blocklist.py` already import `jail_service` specifically for these functions.
|
|
||||||
|
|
||||||
**Goal:** Move the three functions into `ban_service.py`. Update `bans.py` and `blocklist.py` to import from `ban_service` instead of `jail_service`. Remove the now-redundant `jail_service` import from those routers if it is no longer needed.
|
|
||||||
|
|
||||||
**Possible traps and issues:**
|
|
||||||
- `ban_ip` and `unban_ip` likely use the fail2ban socket through utilities shared with the rest of `jail_service`. Confirm there is no shared private helper that needs to move with them.
|
|
||||||
- `get_active_bans` may overlap with existing functions in `ban_service`. Check for duplication and merge if needed.
|
|
||||||
- If `jail_service` tests cover these functions, move those test cases to the `ban_service` test file.
|
|
||||||
|
|
||||||
**Docs changes needed:** Update `Docs/Architekture.md` service responsibility table. Update `Docs/Refactoring.md`.
|
|
||||||
|
|
||||||
**Why this is needed:** Function placement that contradicts the declared service boundary makes it harder to find behaviour during maintenance and violates the principle of least surprise. The service name should predict where operations live.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 14 — Lift geo-enrichment closure construction out of `history.py` router
|
|
||||||
|
|
||||||
**Found in:** `backend/app/routers/history.py` lines 110, 147, and 189 each build a `_enricher` async closure and pass it to `history_service` calls. The closure captures `geo_service` and `http_session` from the outer scope and adapts the geo lookup interface for the service.
|
|
||||||
|
|
||||||
**Goal:** Move the closure construction into `history_service.py` or into a shared helper in `geo_service.py`. The router should pass the `http_session` and let the service build the enricher internally, or the service should accept `geo_service` as an injected callable.
|
|
||||||
|
|
||||||
**Possible traps and issues:**
|
|
||||||
- `history_service` already accepts an optional `geo_enricher: GeoEnricher | None` parameter — this is the hook. The router's closures are just adapters for that parameter. Either the service gains a higher-level function that takes `http_session` directly, or a factory function in `geo_service` builds the enricher.
|
|
||||||
- The same pattern exists in `geo.py` and `ban_service.py` — a consistent solution should handle all three callsites, not just `history.py`.
|
|
||||||
- Changing the `history_service` function signature will require updating tests.
|
|
||||||
|
|
||||||
**Docs changes needed:** Update `Docs/Refactoring.md`.
|
|
||||||
|
|
||||||
**Status:** Completed ✅
|
|
||||||
|
|
||||||
**Why this is needed:** Adapter/closure construction is glue code that belongs at the service boundary or in a factory, not in the HTTP handler. Routers should not need to understand the geo service's lookup interface to serve history requests.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 15 — Fix stale activation record on failed `activate_jail`
|
|
||||||
|
|
||||||
**Found in:** `backend/app/routers/jail_config.py` line 385. `record_activation(app, name)` is called unconditionally before the service call. When the service raises any exception, no handler clears `last_activation`. For the following 60 seconds, the health-check task will misattribute any fail2ban offline event to this failed activation, potentially creating a spurious `PendingRecovery`.
|
|
||||||
|
|
||||||
**Goal:** Wrap the service call in a try/except that clears `last_activation` on any exception before re-raising. Alternatively, only call `record_activation` after a successful return.
|
|
||||||
|
|
||||||
**Possible traps and issues:**
|
|
||||||
- Moving `record_activation` to after the `await` means the timing is slightly later (by the service call duration). This is acceptable because the 60-second window is generous.
|
|
||||||
- Ensure every exception branch (not just the ones currently handled) triggers the cleanup. Use a `try/finally` to clear the record on failure, not individual `except` blocks.
|
|
||||||
- The unused `activation_time` return value of `record_activation` should be removed if moving the call eliminates the need for it.
|
|
||||||
|
|
||||||
**Docs changes needed:** Update `Docs/Refactoring.md`.
|
|
||||||
|
|
||||||
**Status:** Completed ✅
|
|
||||||
|
|
||||||
**Why this is needed:** A false `PendingRecovery` presented to the user offers a rollback for a jail that was never successfully activated. This could confuse operators into performing a rollback that modifies configuration unnecessarily.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 16 — Add a lock to `geo_service.py` module-level mutable state
|
|
||||||
|
|
||||||
**Found in:** `backend/app/services/geo_service.py` lines 99–110: `_cache: dict`, `_neg_cache: dict`, `_dirty: set`, and `_geoip_reader` are mutable module-level singletons with no concurrency control. Background tasks (`geo_cache_flush`, `geo_re_resolve`) and request handlers all read and write these concurrently.
|
|
||||||
|
|
||||||
**Goal:** Add an `asyncio.Lock` protecting mutations of `_cache`, `_neg_cache`, and `_dirty`. The lock should be acquired for any write and for the copy-then-clear pattern in `flush_dirty`. Read-only accesses that are single-await-free are safe without a lock in the single-threaded asyncio model, but mutation sites must be explicit.
|
|
||||||
|
|
||||||
**Possible traps and issues:**
|
|
||||||
- The comment in `flush_dirty` states that no `await` between copy and clear makes it safe today; a lock is still preferable to ensure the invariant is enforced rather than relied upon implicitly.
|
|
||||||
- Avoid holding the lock across network I/O (e.g. the ip-api.com fetch in `lookup`). Acquire it only around the dict/set mutation itself.
|
|
||||||
- `asyncio.Lock` is not thread-safe if `run_in_executor` is used anywhere in the geo path — verify `_cache` is never read from a thread pool.
|
|
||||||
- Introducing a lock is low risk but adds overhead on every cache write; profile if the geo cache is a hot path.
|
|
||||||
|
|
||||||
**Docs changes needed:** Update `Docs/Refactoring.md`.
|
|
||||||
|
|
||||||
**Status:** Done ✅
|
|
||||||
|
|
||||||
**Why this is needed:** The current safety is implicit and fragile. A future change that adds an `await` inside the critical section (e.g. logging to a remote sink) would silently introduce data loss in the dirty-flush path. An explicit lock documents the intent and makes the safety guarantee unconditional.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 17 — Move crash-detection logic out of the `health_check` task
|
|
||||||
|
|
||||||
**Found in:** `backend/app/tasks/health_check.py` lines 70–113. The `_run_probe_with_resources` function contains a state machine that checks `last_activation` timing, detects `online→offline` transitions, and writes `PendingRecovery` records. This is domain business logic embedded in a scheduling function.
|
|
||||||
|
|
||||||
**Goal:** Extract the crash-detection state machine into a function in `backend/app/utils/runtime_state.py` or into a new `recovery_service.py`. The health-check task should call that function with the previous status, the new status, and the current runtime state — and receive back the updated state without containing the decision logic itself.
|
|
||||||
|
|
||||||
**Possible traps and issues:**
|
|
||||||
- The state machine references `datetime.datetime.now(UTC)` — ensure the extracted function remains deterministic/injectable for testing by accepting `now` as an optional parameter.
|
|
||||||
- `runtime_state.py` already holds `record_activation`, `create_pending_recovery`, `clear_pending_recovery`. The extracted logic is a natural neighbour there.
|
|
||||||
- When moving the logic, preserve the exact 60-second window (`_ACTIVATION_CRASH_WINDOW`) and the guard that prevents overwriting an unresolved record.
|
|
||||||
- Tests for the health check task will need to be refactored: the task becomes trivially thin; the logic can now be unit-tested without a running scheduler.
|
|
||||||
|
|
||||||
**Docs changes needed:** Update `Docs/Refactoring.md`.
|
|
||||||
|
|
||||||
**Status:** Completed ✅
|
|
||||||
|
|
||||||
**Why this is needed:** Business rules about crash attribution timing should not live inside a scheduling artifact. Embedding decision logic in a background job makes it invisible, hard to test without the scheduler, and impossible to reuse from another trigger (e.g. a manual probe endpoint).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 18 — Replace `__import__("datetime")` antipattern in `health_check.py`
|
|
||||||
|
|
||||||
**Found in:** `backend/app/tasks/health_check.py` lines 165–166: `next_run_time=__import__("datetime").datetime.now(tz=__import__("datetime").timezone.utc)`. The `datetime` module is already imported at the top of the file on line 6.
|
|
||||||
|
|
||||||
**Goal:** Replace the `__import__` calls with the already-imported `datetime` name: `next_run_time=datetime.datetime.now(tz=datetime.timezone.utc)`.
|
|
||||||
|
|
||||||
**Possible traps and issues:**
|
|
||||||
- This is a trivial one-line change. Confirm the module-level `import datetime` exists and is not inside a `TYPE_CHECKING` block.
|
|
||||||
- Run the health-check related tests to confirm the scheduler still fires on startup.
|
|
||||||
|
|
||||||
**Docs changes needed:** None.
|
**Docs changes needed:** None.
|
||||||
|
|
||||||
**Status:** Completed ✅
|
**Why this is needed:** A future developer unfamiliar with the startup sequence could call `init_geoip()` from a request handler or background task, introducing a data race on `_geoip_reader`. A guard and comment make the contract explicit and prevent silent bugs.
|
||||||
|
|
||||||
**Why this is needed:** `__import__()` is an implementation detail of Python's import system and should never appear in application code. It obscures the dependency, bypasses linting checks, and signals that the code was written reactively to solve a circular import that no longer exists.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### Task 19 — Remove the `setup_service` misuse in `config_misc.py` for runtime map-color settings
|
### 5. Add Concurrency Comment / Guard to `RuntimeState`
|
||||||
|
|
||||||
**Found in:** `backend/app/routers/config_misc.py` uses `setup_service.get_map_color_thresholds()` and `setup_service.update_map_color_thresholds()` for the ongoing `/api/config/map-colors` endpoints. `setup_service` is intended for first-run setup only.
|
**Where:** `backend/app/utils/runtime_state.py` — the `RuntimeState` dataclass (line 47) and its mutation functions (`process_health_probe_result`, `record_activation`, `clear_pending_recovery`, etc.).
|
||||||
|
|
||||||
**Goal:** After Task 7 consolidates the map-color functions into the canonical runtime service, update `config_misc.py` to call that service instead of `setup_service`. Ensure `setup_service` no longer exports these functions publicly.
|
**Goal:** Add a module-level docstring section explicitly documenting the concurrency model: `RuntimeState` is safe only under single-worker asyncio with cooperative scheduling. Mutations must not span `await` points (no read-modify-write across suspension). If multi-worker deployment is ever needed, `RuntimeState` must be replaced with a shared backend (Redis, shared memory).
|
||||||
|
|
||||||
**Possible traps and issues:**
|
**Possible traps and issues:**
|
||||||
- This task is a follow-on to Task 7; do not attempt it before Task 7 is complete.
|
- Adding locks to `RuntimeState` would be overkill for the current single-worker design and would complicate every read path.
|
||||||
- If `setup_service` still needs these functions during first-run setup, it should call the canonical service rather than owning the implementation.
|
- The real risk is a future developer adding an `await` between reading and writing a `RuntimeState` attribute in a new function, introducing a TOCTOU race. Documentation is the correct mitigation here, not locks.
|
||||||
- Router tests that mock `setup_service.get_map_color_thresholds` will need updating.
|
- `process_health_probe_result()` already does read-then-write on `server_status` and `pending_recovery`, but since it runs in a background task that completes without yielding between those operations, it is safe today.
|
||||||
|
|
||||||
**Docs changes needed:** Update `Docs/Refactoring.md`.
|
**Docs changes needed:** Update `Docs/Architekture.md` section 6 (Authentication & Session Management) where the session cache is already documented as process-local. Add a parallel note about `RuntimeState` in the same section or in a new "Concurrency Model" subsection.
|
||||||
|
|
||||||
**Status:** Completed ✅
|
**Why this is needed:** The safety of `RuntimeState` depends on an implicit assumption (single-worker, no `await` between read and write) that is not written down anywhere. If the deployment model changes or a developer modifies state-mutating functions, silent data races could appear with no warning. Explicit documentation prevents this.
|
||||||
|
|
||||||
**Why this is needed:** Using a first-run setup service for ongoing runtime operations misleads developers into modifying the wrong service when behaviour needs to change. It also creates an implicit dependency between setup state and runtime configuration.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### Task 20 — Add global exception handlers for domain exceptions in `main.py`
|
### 6. `geo_service` Calls `db.commit()` Directly Instead of Deferring to Repository
|
||||||
|
|
||||||
**Found in:** `backend/app/main.py`. Domain exceptions raised by services (`JailNotFoundError`, `ConfigValidationError`, `ConfigWriteError`, `FilterNotFoundError`, etc.) are caught individually with repetitive try/except blocks in each router handler. There are no global `@app.exception_handler` registrations for domain exceptions beyond `Fail2BanConnectionError` and `Fail2BanProtocolError`.
|
**Where:** `backend/app/services/geo_service.py` — approximately 5 locations (lines 417, 443, 455, 637, 813) where the service calls `await db.commit()` directly after repository operations.
|
||||||
|
|
||||||
**Goal:** Register `@app.exception_handler` entries in `main.py` for the most commonly caught domain exception classes (at minimum: `JailNotFoundError` → 404, `ConfigValidationError` → 400, `ConfigWriteError` → 500). This allows router handlers to let these exceptions propagate naturally, removing boilerplate try/except blocks.
|
**Goal:** Move transaction commit calls into the repository layer or into a unit-of-work pattern so that the service layer does not manage database transaction boundaries. The repository functions (`upsert_entry`, `bulk_upsert_entries`, etc.) should commit internally, or the caller should use an explicit context manager that commits on exit.
|
||||||
|
|
||||||
**Possible traps and issues:**
|
**Possible traps and issues:**
|
||||||
- Not all routers want the default status code; some endpoints return 409 or 400 for conditions that another endpoint maps to 500. Global handlers are only appropriate for exceptions with a consistent HTTP mapping across all consumers.
|
- The current pattern exists because `geo_service` writes to the DB from both request handlers and background tasks, and commit timing differs between the two contexts. A blanket "always commit in the repo" approach could cause excessive commits during batch operations (e.g., `lookup_batch` resolves 100 IPs — committing after each `upsert_entry` instead of once at the end would be 100x more disk I/O).
|
||||||
- Introduce global handlers incrementally: add the handler first, then remove the try/except from one router at a time and verify the test suite still passes.
|
- A pragmatic middle ground: keep `db.commit()` in the service for batch operations, but document why this deviates from the general pattern. Alternatively, add a `bulk_upsert_and_commit` repo method that handles both.
|
||||||
- Ensure the handler response format matches the existing `{"detail": "…"}` convention so the frontend does not need updating.
|
- Changing commit semantics can cause data loss if the commit is moved but an exception prevents it from running.
|
||||||
- `HTTPException` must remain a local catch and not be accidentally swallowed by a broad domain handler.
|
|
||||||
|
|
||||||
**Docs changes needed:** Update `Docs/Refactoring.md`.
|
**Docs changes needed:** None required, but a comment in the architecture doc section 2.2 (Repositories) noting that geo cache bulk operations are an intentional exception to the "repositories own transactions" rule would help.
|
||||||
|
|
||||||
**Status:** Completed ✅
|
**Why this is needed:** Every other service delegates full DB lifecycle to the repository. `geo_service` is the only one calling `db.commit()` directly. This inconsistency makes it harder to reason about transaction boundaries and could confuse developers who expect the repository to handle commits.
|
||||||
|
|
||||||
**Why this is needed:** The same try/except→HTTPException conversion is duplicated across every router endpoint. Global handlers reduce boilerplate, make error mapping auditable in one place, and prevent inconsistencies where the same exception produces different status codes in different routes.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### Task 21 — Restrict tasks to opening their own DB connections only where justified; document the pattern
|
### 7. Consider Splitting Large Service and Router Files
|
||||||
|
|
||||||
**Found in:** `backend/app/tasks/blocklist_import.py`, `geo_cache_flush.py`, `geo_re_resolve.py`, and `history_sync.py`. Each task calls `open_db(settings.database_path)` directly rather than receiving a DB connection via dependency injection. This is structurally inconsistent with the request-level `get_db` dependency used by routers.
|
**Where:**
|
||||||
|
- `backend/app/services/jail_service.py` — 1287 lines
|
||||||
|
- `backend/app/services/ban_service.py` — 1068 lines
|
||||||
|
- `backend/app/services/action_config_service.py` — 949 lines
|
||||||
|
- `backend/app/services/raw_config_io_service.py` — 981 lines
|
||||||
|
- `backend/app/routers/file_config.py` — 814 lines
|
||||||
|
- `backend/app/routers/jail_config.py` — 763 lines
|
||||||
|
|
||||||
**Goal:** Document in `Docs/Architekture.md` that background tasks are intentionally outside FastAPI's dependency injection scope and must manage their own DB connections. Add a short shared helper (e.g. `tasks/_db.py` or reuse `open_db`) with a consistent context-manager pattern so all four tasks open/close connections the same way. If a task already has correct cleanup (try/finally close), confirm it and leave it; do not change what works.
|
**Goal:** These files are cohesive today but trending toward sizes where navigation becomes difficult. `raw_config_io_service.py` in particular contains near-identical CRUD logic for three config file types (jail, filter, action). The goal is not to split them now, but to establish a size threshold (e.g., 800 lines) and plan extraction points:
|
||||||
|
- `raw_config_io_service.py` → extract shared `_list_conf_files`, `_read_conf_file`, `_write_conf_file`, `_create_conf_file` into a base helper, reducing the per-type functions to thin wrappers.
|
||||||
|
- `file_config.py` router → split into `jail_file_config.py`, `filter_file_config.py`, `action_file_config.py` (matching the existing `jail_config.py` / `filter_config.py` / `action_config.py` split).
|
||||||
|
- `ban_service.py` → extract aggregation/statistics functions (`ban_trend`, `bans_by_country`, `bans_by_jail`) into a `ban_stats_service.py`.
|
||||||
|
|
||||||
**Possible traps and issues:**
|
**Possible traps and issues:**
|
||||||
- Do not attempt to force APScheduler background jobs through FastAPI's DI system — this is not supported without a running request context and would require thread-unsafe hacks.
|
- Splitting services changes import paths, which breaks any test that imports from the old location.
|
||||||
- Focus on consistency: all tasks should use the same open/close pattern and handle exceptions the same way.
|
- Moving functions between modules can introduce circular imports if the extracted module needs to call back into the original.
|
||||||
- If a task currently leaks a connection on exception, fix the cleanup; if it is correct, only add the documentation.
|
- Splitting too early (before the file is genuinely hard to navigate) adds indirection without benefit.
|
||||||
|
- `raw_config_io_service.py` deduplication looks tempting but the per-type functions have subtle differences in path resolution and validation that a generic helper must account for.
|
||||||
|
|
||||||
**Docs changes needed:** Add a "Background Tasks and Database Access" section to `Docs/Architekture.md` explaining why tasks own their connections and how to write a new task.
|
**Docs changes needed:** Update `Docs/Architekture.md` section 2.1 (Project Structure) and section 2.2 (Module Purposes) to reflect any new modules created.
|
||||||
|
|
||||||
**Status:** Completed ✅
|
**Why this is needed:** Large files slow down code review, increase merge conflict probability, and make it harder to find relevant code. Proactively identifying split points before the files grow further ensures the architecture stays navigable. This is a maintenance concern, not a correctness issue.
|
||||||
|
|
||||||
**Why this is needed:** Without explanation, the inconsistency between router DI-provided connections and task-managed connections looks like an oversight. Documentation prevents future developers from incorrectly trying to inject a DB connection into a task via `Depends`, which would fail silently at runtime.
|
---
|
||||||
|
|
||||||
|
### 8. Split `api/config.ts` into Domain-Specific API Modules
|
||||||
|
|
||||||
|
**Where:** `frontend/src/api/config.ts` — a single file that currently bundles socket-based jail config, file-based config (jail.d / filter.d / action.d), server settings, log preview, regex testing, and jail activation/deactivation. The `endpoints.ts` file references `ENDPOINTS.health` but no API module calls it.
|
||||||
|
|
||||||
|
**Goal:** Break `api/config.ts` into the modules the architecture specifies: `api/file_config.ts` (all file-based read/write for jail.d, filter.d, action.d), `api/server.ts` (server settings, log level, syslog socket, DB path, purge age), and `api/health.ts` (the health check call using `ENDPOINTS.health`). The socket-based jail/filter config functions can remain in `api/config.ts` but should be limited to that domain. Additionally, ban-related functions currently spread across `api/dashboard.ts` and `api/jails.ts` should be consolidated into `api/bans.ts`, and IP geolocation calls from `api/jails.ts` should move to `api/geo.ts`.
|
||||||
|
|
||||||
|
**Possible traps and issues:**
|
||||||
|
- Every hook that imports from `api/config.ts` needs its import path updated. `useConfig.ts` is the primary consumer; its six internal hooks already map to the split domains, making the update mechanical once the modules are created.
|
||||||
|
- `api/health.ts` currently has no consumer — the `ENDPOINTS.health` constant exists but is unused. Adding the module is straightforward but a corresponding `useHealth` hook (or integration into `useServerStatus`) is needed for it to be called.
|
||||||
|
- Moving functions between modules changes the public surface; any test that imports API functions by path will fail and must be updated.
|
||||||
|
- Do not create stub files just to satisfy the architecture map — only create a module when there is at least one function to put in it.
|
||||||
|
|
||||||
|
**Docs changes needed:** `Docs/Architekture.md` section 3.2 (API Layer table) already lists the target modules; no text changes needed beyond verifying the table matches once the split is done.
|
||||||
|
|
||||||
|
**Why this is needed:** `api/config.ts` currently serves five conceptually unrelated domains. Developers looking for server settings functions or health check calls have no obvious place to find them, and the file will keep growing as new endpoints are added. Splitting it into focused modules makes the API layer match the documented architecture and makes new additions predictable.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 9. Move Hooks Out of Provider Files
|
||||||
|
|
||||||
|
**Where:** `frontend/src/providers/AuthProvider.tsx` — exports `useAuth()` (defined at the bottom of the file, around line 111). `frontend/src/providers/TimezoneProvider.tsx` — exports `useTimezone()` (defined at the bottom of the file, around line 70).
|
||||||
|
|
||||||
|
**Goal:** Move `useAuth()` into its own file `frontend/src/hooks/useAuth.ts` and `useTimezone()` into `frontend/src/hooks/useTimezone.ts`. Each provider file should only define and export the context object and the provider component. The hooks should import the context from the provider file and re-export it from the `hooks/` directory.
|
||||||
|
|
||||||
|
**Possible traps and issues:**
|
||||||
|
- Every file that currently imports `useAuth` from `providers/AuthProvider` must update its import path. Run a global search for `from "../providers/AuthProvider"` and `from "../../providers/AuthProvider"` to find all consumers before moving.
|
||||||
|
- The context object (`AuthContext`, `TimezoneContext`) must remain exported from the provider file so the hook can import it. Do not move the context — only the hook function.
|
||||||
|
- The architecture rule is "each hook in its own file under `hooks/`". The hook file should be a thin wrapper that calls `useContext(AuthContext)` and throws if outside the provider — identical logic to what exists today, just at a different path.
|
||||||
|
|
||||||
|
**Docs changes needed:** None. The architecture doc already lists `hooks/useAuth.ts` as the canonical location.
|
||||||
|
|
||||||
|
**Why this is needed:** The rule is that all custom hooks live under `hooks/`, each in their own file. Hooks buried at the bottom of provider files are hard to discover and violate the separation between "provider component" (what wraps the app) and "consumer hook" (what components call). Developers looking for `useAuth` will search `hooks/` and not find it.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 10. Split Multi-Hook Files into Single-Hook Files
|
||||||
|
|
||||||
|
**Where:**
|
||||||
|
- `frontend/src/hooks/useBlocklist.ts` — defines `useBlocklists`, `useSchedule`, `useImportLog`, `useRunImport` (four hooks in one file).
|
||||||
|
- `frontend/src/hooks/useConfig.ts` — defines `useJailConfigs`, `useJailConfigDetail`, `useGlobalConfig`, `useServerSettings`, `useRegexTester`, `useLogPreview` (six hooks in one file).
|
||||||
|
- `frontend/src/hooks/useJails.ts` — defines multiple hooks in one file.
|
||||||
|
|
||||||
|
**Goal:** Split each multi-hook file so that every hook lives in its own file following the `hooks/<hookName>.ts` naming convention. Create: `useBlocklists.ts`, `useSchedule.ts`, `useImportLog.ts`, `useRunImport.ts`, `useJailConfigs.ts`, `useJailConfigDetail.ts`, `useGlobalConfig.ts`, `useServerSettings.ts`, `useRegexTester.ts`, `useLogPreview.ts`, and whatever hooks are in `useJails.ts`. If hooks share internal utilities or types, extract those to a helper module — do not inline them in each new file.
|
||||||
|
|
||||||
|
**Possible traps and issues:**
|
||||||
|
- All existing import sites must be updated. Each page and component imports specific hooks by name from these files; the import path changes but the imported name stays the same.
|
||||||
|
- Hooks within the same file may share local helper functions or state logic that is not currently exported. Those helpers must be extracted into a shared internal module (e.g., `hooks/_configHelpers.ts`) or duplicated if they are truly trivial.
|
||||||
|
- After splitting, `useConfig.ts` and `useBlocklist.ts` themselves might become barrel re-export files. Decide upfront whether to keep them as barrels (for backward-compatible imports) or delete them entirely and update all import sites.
|
||||||
|
- `useConfig.ts` is the most complex — `useJailConfigDetail` may depend on state set up by `useJailConfigs`. If they share state, they must either be kept in the same file or refactored to communicate via props/context.
|
||||||
|
|
||||||
|
**Docs changes needed:** `Docs/Architekture.md` section 3.2 (Hooks table) already lists individual hooks; no text update needed, but verify the table is complete after the split.
|
||||||
|
|
||||||
|
**Why this is needed:** The rule is one hook per file. When six hooks are crammed into one file, the file grows to hundreds of lines and becomes a grab-bag. A developer looking for `useLogPreview` has no indication it lives inside `useConfig.ts`. Single-file hooks are immediately discoverable and separately testable.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 11. Remove Direct API Calls from Components
|
||||||
|
|
||||||
|
**Where:** Eleven component files call the API layer directly, bypassing the required hooks layer:
|
||||||
|
- `frontend/src/components/config/JailsTab.tsx` (~lines 36–43): imports and calls `addLogPath`, `deactivateJail`, `deleteJailLocalOverride`, `deleteLogPath`, `fetchInactiveJails`, `fetchJailConfigFileContent`, `updateJailConfigFile`, `validateJailConfig`.
|
||||||
|
- `frontend/src/components/config/FiltersTab.tsx` (~74, ~82, ~130): `fetchFilterFile`, `updateFilterFile`, `fetchFilters`.
|
||||||
|
- `frontend/src/components/config/ActionsTab.tsx` (~100–200): `fetchActionFile`, `updateActionFile`, `removeActionFromJail`, `fetchActions`.
|
||||||
|
- `frontend/src/components/config/ConfFilesTab.tsx` (~60): `fetchList` inside a `useEffect`.
|
||||||
|
- `frontend/src/components/config/ActivateJailDialog.tsx` (~45, ~80): `validateJailConfig`, `activateJail`.
|
||||||
|
- `frontend/src/components/config/AssignActionDialog.tsx` (~30, ~70): `fetchJails`, `assignActionToJail`.
|
||||||
|
- `frontend/src/components/config/AssignFilterDialog.tsx`: `fetchJails`, `assignFilterToJail`.
|
||||||
|
- `frontend/src/components/config/CreateActionDialog.tsx` (~40): `createAction`.
|
||||||
|
- `frontend/src/components/config/CreateFilterDialog.tsx`: `createFilter`.
|
||||||
|
- `frontend/src/components/config/CreateJailDialog.tsx`: `createJailConfigFile`.
|
||||||
|
- `frontend/src/components/config/ServerHealthSection.tsx` (~45, ~80): `fetchServiceStatus`, `fetchFail2BanLog`.
|
||||||
|
- `frontend/src/components/blocklist/BlocklistSourcesSection.tsx` (~90): `fetchPreview` inside `PreviewDialog`.
|
||||||
|
|
||||||
|
**Goal:** Each component must receive its data and mutation callbacks via props or via a hook — it must never import from `api/` directly. For each component above: identify which operations it performs, create or extend the appropriate hook in `hooks/`, and replace the inline API call with a hook call or a callback prop. Dialog components (`ActivateJailDialog`, `AssignActionDialog`, etc.) are the trickiest — they typically need a single async action; these can accept an `onConfirm: () => Promise<void>` callback prop where the caller (the parent tab) provides the hook-backed implementation.
|
||||||
|
|
||||||
|
**Possible traps and issues:**
|
||||||
|
- `ServerHealthSection` combines polling, log streaming, and a one-shot reload into a single component body. The best split is a dedicated `useServerHealth` hook that owns the polling state and exposes `{ status, log, refresh, lineCount, setLineCount }`.
|
||||||
|
- Dialog components that call `validateJailConfig` during `useEffect` on open (like `ActivateJailDialog`) need the validation result as a prop or via a passed-in async callback — not fetched internally.
|
||||||
|
- Converting `fetchJails` calls inside dialogs to a prop means the parent tab must pass the jail list down. This is the correct data-flow direction per the architecture.
|
||||||
|
- `ConfFilesTab` does a `useEffect` fetch that is essentially a custom hook. Extracting it into `useConfFiles` is the minimal fix.
|
||||||
|
- After this change, many components will become purely presentational, which is the goal.
|
||||||
|
|
||||||
|
**Docs changes needed:** None. This task brings the code into conformance with the documented rules ("Components receive data via props only — they never call the API directly").
|
||||||
|
|
||||||
|
**Why this is needed:** Components that call the API directly are impossible to unit-test without mocking the network, cannot be reused with alternate data sources, and violate the layered architecture. The hook layer exists precisely to own data fetching so that the component layer stays declarative and testable.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 12. Add AbortController Cleanup to All async `useEffect` Calls
|
||||||
|
|
||||||
|
**Where:** Multiple hooks and components create async fetch calls inside `useEffect` without an `AbortController`:
|
||||||
|
- `frontend/src/hooks/useBans.ts`: `doFetch` has no abort, allowing stale in-flight requests to race.
|
||||||
|
- `frontend/src/hooks/useActionConfig.ts`, `useFilterConfig.ts`, `useJailFileConfig.ts`: each passes an `AbortSignal` placeholder to `useConfigItem` but the underlying `fetchFn` callback ignores it — abort is silently dropped.
|
||||||
|
- `frontend/src/hooks/useBlocklist.ts` (`useSchedule`): `fetchSchedule` called with no abort.
|
||||||
|
- `frontend/src/hooks/useMapColorThresholds.ts`: `load` called with no abort.
|
||||||
|
- `frontend/src/hooks/useTimezoneData.ts`: `fetchTimezone()` with no cleanup returned from `useEffect`.
|
||||||
|
- `frontend/src/components/config/ConfFilesTab.tsx` (~60): `useEffect` fetch, no cleanup.
|
||||||
|
- `frontend/src/components/config/ActivateJailDialog.tsx` (~45): `validateJailConfig` in `useEffect`, no cleanup.
|
||||||
|
- `frontend/src/components/config/AssignActionDialog.tsx` (~30) and `AssignFilterDialog.tsx`: `fetchJails` in `useEffect`, no cleanup.
|
||||||
|
- `frontend/src/components/blocklist/BlocklistScheduleSection.tsx` (~60): `setTimeout` return not stored in a ref — cannot be cancelled on unmount.
|
||||||
|
|
||||||
|
**Goal:** Every `useEffect` that initiates an async operation must create an `AbortController`, pass its `signal` to the fetch call, and return a cleanup function that calls `controller.abort()`. For `setTimeout` usage in components, store the timeout ID in a `useRef` and clear it in the cleanup. For the `useConfigItem` pattern, the `fetchFn` callback type must accept a `signal: AbortSignal` parameter and pass it through to the underlying API call.
|
||||||
|
|
||||||
|
**Possible traps and issues:**
|
||||||
|
- `useConfigItem` is a shared internal hook consumed by `useActionConfig`, `useFilterConfig`, and `useJailFileConfig`. Changing its `fetchFn` signature to require a `signal` parameter means updating all three consumers simultaneously — make the change in one commit to avoid half-broken states.
|
||||||
|
- `AbortError` must be caught and silently ignored (it is not a real error — it is expected on unmount). Use `if (err instanceof DOMException && err.name === "AbortError") return;` in the catch block.
|
||||||
|
- Fetch calls inside dialog components that are currently being fixed (Task 11) will migrate to hooks anyway — those abort fixes can be done as part of Task 11 rather than separately.
|
||||||
|
- `BlocklistScheduleSection`'s `setTimeout` replacement with `useRef` is a one-liner but easy to forget to clear in the cleanup function.
|
||||||
|
|
||||||
|
**Docs changes needed:** None.
|
||||||
|
|
||||||
|
**Why this is needed:** Without abort cleanup, a component that unmounts mid-fetch will still call `setState` on the unmounted component, producing React "can't perform state update on unmounted component" warnings and, more critically, stale responses from a fast-then-slow request sequence can silently overwrite fresher data. This is a reliability issue in any view that mounts and unmounts frequently (dialogs, paginated tabs).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 13. Split Multi-Component Files to One Component Per File
|
||||||
|
|
||||||
|
**Where:**
|
||||||
|
- `frontend/src/pages/JailsPage.tsx`: defines `JailOverviewSection`, `BanUnbanForm`, `IpLookupSection`, and `JailsPage` in one file.
|
||||||
|
- `frontend/src/pages/JailDetailPage.tsx`: defines `CodeList`, `JailInfoSection`, `PatternsSection`, `BantimeEscalationSection`, and `JailDetailPage` — five components in one file.
|
||||||
|
- `frontend/src/pages/BlocklistsPage.tsx`: defines `ImportResultDialog` and `BlocklistsPage` in one file.
|
||||||
|
- `frontend/src/components/config/FiltersTab.tsx`: defines `FilterDetail` and `FiltersTab`.
|
||||||
|
- `frontend/src/components/config/ActionsTab.tsx`: defines `ActionDetail` and `ActionsTab`.
|
||||||
|
- `frontend/src/components/config/JailFileForm.tsx`: defines `StringListEditor`, `KVEditor`, `JailSectionPanel`, `JailFileFormInner`, and the outer form component.
|
||||||
|
- `frontend/src/components/config/FilterForm.tsx`: defines `KVEditor` and `FilterFormEditor`.
|
||||||
|
- `frontend/src/components/config/ActionForm.tsx`: defines `KVEditor`, `CommandField`, and `ActionFormEditor`.
|
||||||
|
- `frontend/src/components/blocklist/BlocklistSourcesSection.tsx`: defines `SourceFormDialog`, `PreviewDialog`, and `BlocklistSourcesSection`.
|
||||||
|
|
||||||
|
**Goal:** Move each sub-component to its own file following the rule "one component per file, filename matches component name". Sub-components that are only used by one parent and have no reuse value outside of it can live in a co-located subdirectory alongside the parent (e.g., `pages/jails/BanUnbanForm.tsx`). Dialog sub-components extracted from blocklists should go into `components/blocklist/`. `KVEditor` is duplicated in `FilterForm.tsx` and `ActionForm.tsx` — after extraction it should become a single shared `components/config/KVEditor.tsx`.
|
||||||
|
|
||||||
|
**Possible traps and issues:**
|
||||||
|
- `JailFileForm.tsx` is the most complex with five components sharing local type definitions. Extract common prop types to a `types/` file or the parent's file before splitting.
|
||||||
|
- `ImportResultDialog` in `BlocklistsPage.tsx` must be replaced with a Fluent UI `<Dialog>` (see Task 14) — do Tasks 13 and 14 for that file in the same pass to avoid touching the file twice.
|
||||||
|
- `KVEditor` appears identically in both `FilterForm.tsx` and `ActionForm.tsx`. After extracting it to a shared file, delete both inlined copies and import from the shared location.
|
||||||
|
- `CodeList`, `JailInfoSection`, etc. in `JailDetailPage.tsx` are not reused elsewhere. They can live in a `pages/jail/` subfolder. Ensure the main page import path stays valid.
|
||||||
|
|
||||||
|
**Docs changes needed:** `Docs/Architekture.md` section 3.1 project structure tree would eventually need updating if new subdirectories are created under `pages/` or `components/`. Update section 3.2 Components table to list any newly extracted reusable components.
|
||||||
|
|
||||||
|
**Why this is needed:** Files with multiple components grow unbounded as features are added. A developer looking for `BanUnbanForm` finds no file by that name — they have to search and discover it is buried hundreds of lines into `JailsPage.tsx`. The single-component-per-file rule exists so that every component is immediately discoverable by name in the file tree.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 14. Replace Custom Modal Overlay with Fluent UI `<Dialog>`
|
||||||
|
|
||||||
|
**Where:** `frontend/src/pages/BlocklistsPage.tsx` — the `ImportResultDialog` component (lines ~23–43) is a hand-built modal using `position: fixed`, a `rgba(0,0,0,0.5)` backdrop `<div>`, and custom inline styles. `frontend/src/components/ErrorBoundary.tsx` (line ~50) uses a raw `<button type="button">` element with inline styles for the reload action in the error fallback UI.
|
||||||
|
|
||||||
|
**Goal:** Replace `ImportResultDialog` with a standard Fluent UI `<Dialog>` from `@fluentui/react-components`, using `<DialogSurface>`, `<DialogTitle>`, `<DialogBody>`, and `<DialogActions>` slots. For `ErrorBoundary`, replace the raw `<button>` with a Fluent UI `<Button>` and remove all inline `style` props from the fallback UI, replacing them with `makeStyles` rules.
|
||||||
|
|
||||||
|
**Possible traps and issues:**
|
||||||
|
- `ImportResultDialog` uses `position: fixed` scroll-locking that Fluent UI `<Dialog>` handles natively. After the switch, verify that the dialog renders above the navigation sidebar (check the z-index layer order configured in `theme/tokens.ts`).
|
||||||
|
- The `ErrorBoundary` is a class component (required because `componentDidCatch` is a class lifecycle method). Fluent UI `<Button>` and `makeStyles` work fine from class components — `makeStyles` returns a hook but it can be called in a wrapper functional component that renders the fallback instead of inline in the class `render()`.
|
||||||
|
- After `ImportResultDialog` is replaced, the parent `BlocklistsPage` should pass `open` and `onDismiss` props to the Fluent dialog in the same way it currently controls the hand-built overlay.
|
||||||
|
- This fix should be done in the same pass as Task 13 for `BlocklistsPage` to avoid redundant edits to the same file.
|
||||||
|
|
||||||
|
**Docs changes needed:** None.
|
||||||
|
|
||||||
|
**Why this is needed:** The architecture explicitly forbids building custom modal overlays ("Use Fluent UI `<Dialog>` for modals and confirmations — never build custom modal overlays"). Hand-built modals bypass Fluent UI's accessibility tree (focus trap, ARIA roles, portal rendering) and duplicate boilerplate that the library provides. A Fluent `<Dialog>` is keyboard-accessible, screen-reader-compatible, and inherits the theme automatically.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 15. Replace Inline Styles and Hardcoded Values with `makeStyles` and Design Tokens
|
||||||
|
|
||||||
|
**Where:** Inline `style={...}` props and hardcoded pixel/rem values appear throughout the codebase. Major concentrations:
|
||||||
|
- `frontend/src/pages/JailsPage.tsx`: `fontFamily: "Consolas, 'Courier New', monospace"`, `fontSize: "0.85rem"`, `marginLeft: "8px"`, `gap: "6px"`.
|
||||||
|
- `frontend/src/pages/JailDetailPage.tsx`: multiple `style={{ color: tokens.colorNeutralForeground3 }}`, `fontFamily: "Consolas, ..."`, and `marginTop: tokens.spacingVerticalS` as inline props.
|
||||||
|
- `frontend/src/pages/HistoryPage.tsx`: hardcoded `fontSize` strings in `makeStyles` (`"0.85rem"`, `"0.75rem"`, `"0.9rem"`, `"0.9em"`), inline styles on timeline items.
|
||||||
|
- `frontend/src/pages/MapPage.tsx`: loading spinner, filter bar, pagination footer — all inline.
|
||||||
|
- `frontend/src/components/config/RawConfigSection.tsx`: `<textarea style={{ fontFamily:"monospace", fontSize:"0.85rem", borderLeft:"3px solid…" }}>`.
|
||||||
|
- `frontend/src/components/config/AutoSaveIndicator.tsx`: `style={{ minWidth: 80 }}`, `style={{ opacity, transform, transition }}`.
|
||||||
|
- `frontend/src/components/config/JailFileForm.tsx`, `FilterForm.tsx`, `ActionForm.tsx`: `style={{ gap: 4 }}`, `style={{ marginBottom: 8 }}`, `style={{ width: 140 }}`, etc.
|
||||||
|
- `frontend/src/components/config/blocklistStyles.ts`: `fontSize: "12px"` (×2), `gap: "2px"`.
|
||||||
|
- `frontend/src/components/blocklist/BlocklistScheduleSection.tsx` and `BlocklistSourcesSection.tsx`: multiple inline style props with hardcoded pixel strings.
|
||||||
|
- `frontend/src/components/jail/BannedIpsSection.tsx`: `style={{ color: … }}`, `style={{ minWidth:"80px" }}`.
|
||||||
|
- `frontend/src/components/BanTrendChart.tsx`, `JailDistributionChart.tsx`, `TopCountriesBarChart.tsx`, `TopCountriesPieChart.tsx`: Recharts tooltip divs with `fontSize: "13px"`, `borderRadius: "4px"`, `padding: "8px 12px"` duplicated across all four files.
|
||||||
|
- `frontend/src/components/WorldMap.tsx`: `strokeWidth: 0.75`, `fontSize: "9px"` in `makeStyles`, and tooltip positioned with inline `style={{ left: tooltip.x + 12, top: tooltip.y + 12 }}`.
|
||||||
|
- `frontend/src/components/SetupGuard.tsx`: spinner container with inline flex styles.
|
||||||
|
|
||||||
|
**Goal:** All custom styling must use `makeStyles` from `@fluentui/react-components`. All size and colour values must use design tokens (`tokens.spacingVerticalM`, `tokens.fontSizeBase200`, `tokens.fontFamilyMonospace`, etc.) rather than literal pixel or rem strings. For the four chart tooltip components, extract a shared `ChartTooltip` wrapper component (or a `chartTooltipStyle` constant in `utils/chartTheme.ts`) to eliminate the duplication. Hardcoded values inside existing `makeStyles` calls (`"0.85rem"`, `"12px"`, `"9px"`) must be replaced with the appropriate token.
|
||||||
|
|
||||||
|
**Possible traps and issues:**
|
||||||
|
- `tokens.fontFamilyMonospace` exists in Fluent UI v9 and should replace every `fontFamily: "Consolas, 'Courier New', monospace"` and `fontFamily: "monospace"` occurrence.
|
||||||
|
- For Recharts chart elements (`tick`, `activeDot`, `outerRadius`, `margin`), Recharts does not accept Griffel class names — those numeric props must use named constants defined once in `utils/chartTheme.ts` (e.g., `CHART_TICK_FONT_SIZE = 12`). This is the correct approach because Recharts bypasses the DOM styling entirely.
|
||||||
|
- The `WorldMap` tooltip position (`left: x + 12, top: y + 12`) is a dynamic inline style driven by runtime coordinates — this one is acceptable as a genuine dynamic value and does not need `makeStyles`. Document the exemption with a comment.
|
||||||
|
- `AutoSaveIndicator`'s `opacity`, `transform`, and `transition` for the save-state animation may be better expressed as Griffel `selectors` or via `mergeClasses` with conditional class names.
|
||||||
|
- `minWidth: 80` on `AutoSaveIndicator` and `minWidth: "80px"` on `BannedIpsSection` `<Dropdown>` should use a named `makeStyles` constant rather than a raw number, since there is no exact Fluent token for these values.
|
||||||
|
|
||||||
|
**Docs changes needed:** None.
|
||||||
|
|
||||||
|
**Why this is needed:** Inline styles bypass Griffel's atomic CSS engine, produce non-deduplicatable style nodes, and cannot respond to theme changes. Hardcoded pixel values break when the user switches to a different Fluent UI theme with different spacing scales. The project rule states that `makeStyles` with design tokens is the only permitted styling mechanism.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 16. Deduplicate `TimeRange` Type
|
||||||
|
|
||||||
|
**Where:** `types/ban.ts` line ~13, `types/map.ts` line ~6, and `types/history.ts` line ~7 each contain an identical type alias: `export type TimeRange = "24h" | "7d" | "30d" | "365d";`.
|
||||||
|
|
||||||
|
**Goal:** Keep the single definition in `types/ban.ts` (it is the most general domain) and replace the duplicate definitions in `types/map.ts` and `types/history.ts` with `import type { TimeRange } from "./ban"` followed by a re-export if consumers import `TimeRange` from those files. Verify that all import sites resolve correctly after the change.
|
||||||
|
|
||||||
|
**Possible traps and issues:**
|
||||||
|
- Search for all `import { TimeRange }` and `import type { TimeRange }` occurrences to identify every consumer. If a consumer imports from `types/history` or `types/map`, either update that import to point to `types/ban`, or re-export from the domain file with `export type { TimeRange } from "./ban"` to maintain backward-compatible imports without duplication.
|
||||||
|
- `TIME_RANGE_LABELS` in `types/ban.ts` (a runtime object mapping `TimeRange` keys to display strings) must also move to `utils/` as part of Task 17 — coordinate these two changes to avoid touching `types/ban.ts` twice.
|
||||||
|
|
||||||
|
**Docs changes needed:** None.
|
||||||
|
|
||||||
|
**Why this is needed:** Three identical type definitions will diverge the moment someone changes the set of allowed time ranges in one file but forgets the others. The project rule is "define it once, import everywhere."
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 17. Move Runtime Constants Out of `types/`
|
||||||
|
|
||||||
|
**Where:** `frontend/src/types/ban.ts` — exports `BAN_ORIGIN_FILTER_LABELS` (a `Record<string, string>` mapping ban origin keys to display strings) and `TIME_RANGE_LABELS` (a `Record<TimeRange, string>` mapping time-range keys to display strings), both declared with `as const`. These are runtime JavaScript objects, not type declarations.
|
||||||
|
|
||||||
|
**Goal:** Move `BAN_ORIGIN_FILTER_LABELS` and `TIME_RANGE_LABELS` to `frontend/src/utils/constants.ts` (create the file if it does not exist, or add to an existing constants utility). Update all import sites. The `types/ban.ts` file should contain only `interface`, `type`, and `enum` declarations with no runtime-evaluatable code.
|
||||||
|
|
||||||
|
**Possible traps and issues:**
|
||||||
|
- Search for every file that imports `BAN_ORIGIN_FILTER_LABELS` or `TIME_RANGE_LABELS` from `types/ban` and update the import path to `utils/constants`.
|
||||||
|
- If `utils/constants.ts` does not yet exist, create it. If it already exists, append the constants to it — do not create a second constants file.
|
||||||
|
- After the move, `types/ban.ts` should not need to import from `utils/` — the dependency direction is `utils/` → `types/` (utils import types), never the reverse.
|
||||||
|
|
||||||
|
**Docs changes needed:** None.
|
||||||
|
|
||||||
|
**Why this is needed:** The `types/` directory is meant to contain zero runtime code — only TypeScript type declarations that are erased at compile time. A runtime `const` object in a types file is confusing, breaks the "no runtime code in types/" rule, and pollutes the bundle with a lookup table alongside pure type transpile artifacts.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 18. Fix Type-Unsafe Casts and Missing `import type` Qualifiers
|
||||||
|
|
||||||
|
**Where:**
|
||||||
|
- `frontend/src/pages/BlocklistsPage.tsx` (~line 50): `const safeUseBlocklistStyles = useBlocklistStyles as unknown as () => { root: string }` — a double cast that discards the real return type.
|
||||||
|
- `frontend/src/hooks/useDashboardCountryData.ts`: `setBans(data.bans as DashboardBanItem[])` — an unchecked cast on an API response.
|
||||||
|
- `frontend/src/pages/HistoryPage.tsx` (~line 31): `TableColumnDefinition` is a TypeScript-only type imported without the `type` modifier in a mixed import from `@fluentui/react-components`.
|
||||||
|
- `frontend/src/components/jail/BannedIpsSection.tsx` (~line 5): `TableColumnDefinition` inline-typed in a value import.
|
||||||
|
- `frontend/src/components/BanTable.tsx` (~line 4): same pattern.
|
||||||
|
|
||||||
|
**Goal:** Remove the `as unknown as` cast in `BlocklistsPage.tsx` — if `useBlocklistStyles` has the wrong return type signature, fix the `makeStyles` definition rather than casting around it. Replace `data.bans as DashboardBanItem[]` in `useDashboardCountryData.ts` with a type guard function that validates the shape at runtime (or remove the cast if TypeScript already infers the correct type from a typed API call). Convert all mixed value+type imports that include pure types to either use the inline `type` modifier (`import { type TableColumnDefinition, createTableColumn }`) or a separate `import type { TableColumnDefinition }` statement.
|
||||||
|
|
||||||
|
**Possible traps and issues:**
|
||||||
|
- The `as unknown as` cast in `BlocklistsPage.tsx` likely exists because `blocklistStyles.ts` exports `useBlocklistStyles` with a type signature that does not match how it is consumed. The fix is to correct the `makeStyles` definitions in `blocklistStyles.ts` so the return type is accurate — then the cast is unnecessary.
|
||||||
|
- A runtime type guard for `DashboardBanItem[]` does not need to be exhaustive — checking that `data.bans` is an array and that the first element has an `ip` string field is sufficient to catch serialization bugs without adding heavy validation overhead.
|
||||||
|
- The `import type` change is a zero-risk change that only affects the TypeScript compiler output (unused import elimination); it never changes runtime behavior.
|
||||||
|
|
||||||
|
**Docs changes needed:** None.
|
||||||
|
|
||||||
|
**Why this is needed:** `as unknown as T` is the TypeScript equivalent of `any` — it silences the type checker without any safety guarantee. If the actual runtime value does not match `T`, the error is deferred to a cryptic downstream failure. The `import type` issue is a code hygiene violation — it prevents tree-shaking tools from safely dropping type-only imports and misleads readers into thinking a type has a runtime value.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 19. Move Utility Functions Out of Page Modules
|
||||||
|
|
||||||
|
**Where:** `frontend/src/pages/HistoryPage.tsx` — the `areHistoryQueriesEqual()` function (lines ~135–146) is a pure comparison utility defined inside the page module. It takes two query objects and returns a boolean — it has no React dependency and no side effects.
|
||||||
|
|
||||||
|
**Goal:** Move `areHistoryQueriesEqual` to `frontend/src/utils/` (either into a new `queryUtils.ts` or an existing utility file if a suitable one exists). Import it back into `HistoryPage.tsx` from that location. Verify the function has no implicit dependency on page-local types — if it relies on `HistoryQuery` from `types/history.ts`, it can still live in utils by importing that type.
|
||||||
|
|
||||||
|
**Possible traps and issues:**
|
||||||
|
- This is a mechanical move with no behavioral change. The only risk is a stale import if the function is used in more than one place (confirm with a search before moving).
|
||||||
|
- If similar utility functions exist in other page files (a search for `function` declarations inside page files is worthwhile), extract them in the same pass.
|
||||||
|
|
||||||
|
**Docs changes needed:** None.
|
||||||
|
|
||||||
|
**Why this is needed:** The architecture rule is that pages contain "no business logic — only orchestration." A pure utility function with no JSX and no React dependency does not belong in a page module. It is invisible to other developers who might need the same comparison, cannot be independently unit-tested, and adds noise to the page's module surface.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 20. Relocate Misplaced Files
|
||||||
|
|
||||||
|
**Where:**
|
||||||
|
- `frontend/src/data/isoNumericToAlpha2.ts` — a pure static lookup table (`Record<number, string>`) with no React dependency. The `data/` directory is not in the architecture specification.
|
||||||
|
- `frontend/src/api/map.test.ts` — a test file sitting directly in `api/` among source modules.
|
||||||
|
- `frontend/src/theme/commonStyles.ts` — exports `useCommonSectionStyles` and `useCardStyles`, which are `makeStyles`-based component style hooks, not theme token definitions.
|
||||||
|
- `frontend/src/components/config/configStyles.ts` — a standalone `makeStyles` file for config components instead of styles being co-located in each component file.
|
||||||
|
- `frontend/src/components/blocklist/blocklistStyles.ts` — same issue.
|
||||||
|
|
||||||
|
**Goal:**
|
||||||
|
- Move `data/isoNumericToAlpha2.ts` to `utils/isoNumericToAlpha2.ts`. Delete the `data/` directory.
|
||||||
|
- Move `api/map.test.ts` to `api/__tests__/map.test.ts` (create the `__tests__` directory if needed).
|
||||||
|
- Move `theme/commonStyles.ts`: if the styles in it are shared across multiple components with no better home, move them to the component directory that uses them most, or rename the file to `components/commonStyles.ts`. The `theme/` directory must contain only `customTheme.ts` and token-related files.
|
||||||
|
- For `configStyles.ts` and `blocklistStyles.ts`: inline each exported `makeStyles` hook into the component file that uses it, or keep the file in the same directory as the components but document that it is a style helper rather than a theme definition. The architecture rule is "co-locate styles in the same file as the component" — a shared styles file is acceptable only if documented as an explicit exception for styles used by multiple components in the same subdirectory.
|
||||||
|
|
||||||
|
**Possible traps and issues:**
|
||||||
|
- Moving `isoNumericToAlpha2.ts` changes its import path in `WorldMap.tsx` and any other consumer — update all import sites.
|
||||||
|
- Moving `api/map.test.ts` to a `__tests__` subdirectory requires the test runner config (`vitest.config.ts`) to include `api/__tests__/` if it only scans `__tests__/` top-level directories — verify the glob pattern first.
|
||||||
|
- `theme/commonStyles.ts` is imported by multiple components. Moving it requires updating all consumers. Verify by grepping for `commonStyles` before touching it.
|
||||||
|
- `configStyles.ts` is imported by several files in `components/config/`. Moving its contents inline into each component file means duplicating styles that happen to be identical — evaluate whether a shared file is a justified exception before breaking it up.
|
||||||
|
|
||||||
|
**Docs changes needed:** `Docs/Architekture.md` section 3.1 (Project Structure) shows `theme/` with two files; update to reflect the correct contents after the move.
|
||||||
|
|
||||||
|
**Why this is needed:** Files placed in the wrong directory mislead developers about where to find things and where to put new related code. A `data/` directory that is not in the specification is a dead-end — future developers may put unrelated lookup tables there, or avoid it entirely because its purpose is unclear. Test files mixed alongside source modules are harder to exclude from coverage and harder to find.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 21. Remove Test Scaffolding from Production Hooks
|
||||||
|
|
||||||
|
**Where:** `frontend/src/hooks/useMapData.ts` — exports `getLastArgs()` and `setMapData()` as named exports. These functions are test-only scaffolding (they expose internal state for assertion in tests) and have no legitimate use in production code.
|
||||||
|
|
||||||
|
**Goal:** Remove `getLastArgs` and `setMapData` from `useMapData.ts`. If the tests that use them cannot function without this scaffolding, refactor those tests to test the hook's public return value and side effects via `renderHook` from React Testing Library instead of reading internal state directly.
|
||||||
|
|
||||||
|
**Possible traps and issues:**
|
||||||
|
- Find every test that imports `getLastArgs` or `setMapData` from `useMapData` and rewrite them before removing the exports. Removing the exports first will cause a type error that points to the test files.
|
||||||
|
- `renderHook` from `@testing-library/react` is the correct tool for testing hook return values without exposing internals.
|
||||||
|
- If `setMapData` is used to inject test data, the test should instead mock the API layer (`api/map.ts`) so that when `useMapData` calls the API, it gets controlled data — no internal exposure required.
|
||||||
|
|
||||||
|
**Docs changes needed:** None.
|
||||||
|
|
||||||
|
**Why this is needed:** Test scaffolding exported from production hooks ships in the production bundle, pollutes the module's public API, and creates a false dependency between the hook's internal implementation and its tests. If the internal structure of `useMapData` changes, tests that rely on `getLastArgs`/`setMapData` will break even if the observable behavior is unchanged. Testing via the public interface (return values, rendered output) makes refactoring safe.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 22. Surface Error State Instead of `console.warn` in `useSetup`
|
||||||
|
|
||||||
|
**Where:** `frontend/src/hooks/useSetup.ts` (~line 60) — a `console.warn("Setup status check failed:", ...)` is the only response to a failed setup status fetch. The `error` state variable is not set.
|
||||||
|
|
||||||
|
**Goal:** Replace the `console.warn` with `setError(err instanceof Error ? err.message : "Unknown error")` so that the error is surfaced to the consuming component via the hook's public interface. Remove the `console.warn` entirely — the hook already has an `error` state for this purpose.
|
||||||
|
|
||||||
|
**Possible traps and issues:**
|
||||||
|
- Verify that `SetupPage.tsx` (or whatever consumes `useSetup`) already renders the `error` state. If it does not, add an error message display (a Fluent UI `<MessageBar intent="error">`) so the surfaced error is actually visible to the user.
|
||||||
|
- `console.warn` in hooks that ship to production will produce console noise for end users and may appear in error monitoring tools as unexpected warnings. The rule is to handle errors in state, not in the console.
|
||||||
|
|
||||||
|
**Docs changes needed:** None.
|
||||||
|
|
||||||
|
**Why this is needed:** A silent `console.warn` means the first sign of a setup check failure is a spinner that never resolves — the user has no feedback and no actionable message. Error state in hooks exists precisely to enable the UI layer to display a meaningful error to the user.
|
||||||
|
|||||||
@@ -7,9 +7,10 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
import re
|
import re
|
||||||
import structlog
|
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import TypeVar, cast
|
from typing import cast
|
||||||
|
|
||||||
|
import structlog
|
||||||
|
|
||||||
from app.exceptions import ConfigOperationError
|
from app.exceptions import ConfigOperationError
|
||||||
from app.models.config import (
|
from app.models.config import (
|
||||||
@@ -42,7 +43,7 @@ _SAFE_LOG_PREFIXES: tuple[str, ...] = ("/var/log", "/config/log")
|
|||||||
def _ok(response: object) -> object:
|
def _ok(response: object) -> object:
|
||||||
"""Extract the payload from a fail2ban ``(return_code, data)`` response."""
|
"""Extract the payload from a fail2ban ``(return_code, data)`` response."""
|
||||||
try:
|
try:
|
||||||
code, data = cast(Fail2BanResponse, response)
|
code, data = cast("Fail2BanResponse", response)
|
||||||
except (TypeError, ValueError) as exc:
|
except (TypeError, ValueError) as exc:
|
||||||
raise ValueError(
|
raise ValueError(
|
||||||
f"Unexpected fail2ban response shape: {response!r}"
|
f"Unexpected fail2ban response shape: {response!r}"
|
||||||
@@ -80,16 +81,13 @@ async def _safe_get(
|
|||||||
return default
|
return default
|
||||||
|
|
||||||
|
|
||||||
T = TypeVar("T")
|
|
||||||
|
|
||||||
|
|
||||||
async def _safe_get_typed(
|
async def _safe_get_typed(
|
||||||
client: Fail2BanClient,
|
client: Fail2BanClient,
|
||||||
command: list[str],
|
command: list[str],
|
||||||
default: T,
|
default: object | None = None,
|
||||||
) -> T:
|
) -> object | None:
|
||||||
"""Send a command and return the result typed as ``default``'s type."""
|
"""Send a command and return a typed result or *default* if it fails."""
|
||||||
return cast("T", await _safe_get(client, command, default))
|
return await _safe_get(client, command, default)
|
||||||
|
|
||||||
|
|
||||||
async def read_fail2ban_log(
|
async def read_fail2ban_log(
|
||||||
@@ -210,7 +208,31 @@ async def preview_log(req: LogPreviewRequest) -> LogPreviewResponse:
|
|||||||
)
|
)
|
||||||
|
|
||||||
path = Path(req.log_path)
|
path = Path(req.log_path)
|
||||||
if not path.is_file():
|
try:
|
||||||
|
resolved = path.resolve(strict=False)
|
||||||
|
except (ValueError, OSError) as exc:
|
||||||
|
return LogPreviewResponse(
|
||||||
|
lines=[],
|
||||||
|
total_lines=0,
|
||||||
|
matched_count=0,
|
||||||
|
regex_error=f"Cannot resolve log path {req.log_path!r}: {exc}",
|
||||||
|
)
|
||||||
|
|
||||||
|
if not any(
|
||||||
|
resolved.is_relative_to(Path(prefix))
|
||||||
|
for prefix in _SAFE_LOG_PREFIXES
|
||||||
|
):
|
||||||
|
return LogPreviewResponse(
|
||||||
|
lines=[],
|
||||||
|
total_lines=0,
|
||||||
|
matched_count=0,
|
||||||
|
regex_error=(
|
||||||
|
f"Log path {str(resolved)!r} is outside the allowed directory. "
|
||||||
|
"Only paths under /var/log or /config/log are permitted."
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
if not resolved.is_file():
|
||||||
return LogPreviewResponse(
|
return LogPreviewResponse(
|
||||||
lines=[],
|
lines=[],
|
||||||
total_lines=0,
|
total_lines=0,
|
||||||
@@ -221,7 +243,7 @@ async def preview_log(req: LogPreviewRequest) -> LogPreviewResponse:
|
|||||||
try:
|
try:
|
||||||
raw_lines = await run_blocking(
|
raw_lines = await run_blocking(
|
||||||
_read_tail_lines,
|
_read_tail_lines,
|
||||||
str(path),
|
str(resolved),
|
||||||
req.num_lines,
|
req.num_lines,
|
||||||
)
|
)
|
||||||
except OSError as exc:
|
except OSError as exc:
|
||||||
|
|||||||
@@ -592,13 +592,28 @@ class TestPreviewLog:
|
|||||||
|
|
||||||
assert result.regex_error is not None
|
assert result.regex_error is not None
|
||||||
|
|
||||||
|
async def test_rejects_log_paths_outside_allowed_directories(self) -> None:
|
||||||
|
"""preview_log rejects files outside the configured safe log directories."""
|
||||||
|
req = LogPreviewRequest(
|
||||||
|
log_path="/etc/passwd",
|
||||||
|
fail_regex=r"root",
|
||||||
|
)
|
||||||
|
result = await config_service.preview_log(req)
|
||||||
|
|
||||||
|
assert result.regex_error is not None
|
||||||
|
assert "outside the allowed directory" in result.regex_error
|
||||||
|
|
||||||
async def test_matches_lines_in_file(self, tmp_path: Any) -> None:
|
async def test_matches_lines_in_file(self, tmp_path: Any) -> None:
|
||||||
"""preview_log correctly identifies matching and non-matching lines."""
|
"""preview_log correctly identifies matching and non-matching lines."""
|
||||||
log_file = tmp_path / "test.log"
|
log_file = tmp_path / "test.log"
|
||||||
log_file.write_text("FAIL login from 1.2.3.4\nOK normal line\nFAIL from 5.6.7.8\n")
|
log_file.write_text("FAIL login from 1.2.3.4\nOK normal line\nFAIL from 5.6.7.8\n")
|
||||||
|
|
||||||
req = LogPreviewRequest(log_path=str(log_file), fail_regex=r"FAIL")
|
req = LogPreviewRequest(log_path=str(log_file), fail_regex=r"FAIL")
|
||||||
result = await config_service.preview_log(req)
|
with patch(
|
||||||
|
"app.services.log_service._SAFE_LOG_PREFIXES",
|
||||||
|
(str(tmp_path),),
|
||||||
|
):
|
||||||
|
result = await config_service.preview_log(req)
|
||||||
|
|
||||||
assert result.total_lines == 3
|
assert result.total_lines == 3
|
||||||
assert result.matched_count == 2
|
assert result.matched_count == 2
|
||||||
@@ -612,7 +627,11 @@ class TestPreviewLog:
|
|||||||
log_path=str(log_file),
|
log_path=str(log_file),
|
||||||
fail_regex=r"from (\d+\.\d+\.\d+\.\d+)",
|
fail_regex=r"from (\d+\.\d+\.\d+\.\d+)",
|
||||||
)
|
)
|
||||||
result = await config_service.preview_log(req)
|
with patch(
|
||||||
|
"app.services.log_service._SAFE_LOG_PREFIXES",
|
||||||
|
(str(tmp_path),),
|
||||||
|
):
|
||||||
|
result = await config_service.preview_log(req)
|
||||||
|
|
||||||
matched = [ln for ln in result.lines if ln.matched]
|
matched = [ln for ln in result.lines if ln.matched]
|
||||||
assert len(matched) == 1
|
assert len(matched) == 1
|
||||||
@@ -624,7 +643,11 @@ class TestPreviewLog:
|
|||||||
log_file.write_text("\n".join(f"line {i}" for i in range(500)) + "\n")
|
log_file.write_text("\n".join(f"line {i}" for i in range(500)) + "\n")
|
||||||
|
|
||||||
req = LogPreviewRequest(log_path=str(log_file), fail_regex=r"line", num_lines=50)
|
req = LogPreviewRequest(log_path=str(log_file), fail_regex=r"line", num_lines=50)
|
||||||
result = await config_service.preview_log(req)
|
with patch(
|
||||||
|
"app.services.log_service._SAFE_LOG_PREFIXES",
|
||||||
|
(str(tmp_path),),
|
||||||
|
):
|
||||||
|
result = await config_service.preview_log(req)
|
||||||
|
|
||||||
assert result.total_lines <= 50
|
assert result.total_lines <= 50
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user