- Create PaginationMetadata model with computed derived fields (total_pages, has_next_page, has_prev_page)
- Update PaginatedListResponse to embed pagination metadata in a separate 'pagination' object
- Add create_pagination_metadata() factory function in utils/pagination.py for consistent computation
- Update all paginated service functions to use new structure:
- history_service.list_history()
- blocklist_service.get_import_logs()
- jail_service.get_jail_banned_ips()
- ban_mappers.map_domain_dashboard_ban_list_to_response()
- Update response model docstrings with new structure examples
- Update Backend-Development.md documentation with new pagination patterns
- Update test fixtures to work with new response structure
Response shape changes from:
{"items": [...], "total": 100, "page": 1, "page_size": 50}
To:
{"items": [...], "pagination": {"page": 1, "page_size": 50, "total": 100, "total_pages": 2, "has_next_page": true, "has_prev_page": false}}
Benefits:
- Frontend receives all pagination state needed for UI controls
- No need for frontend to calculate total_pages or page navigation logic
- Consolidated pagination metadata reduces field sprawl
- OpenAPI schema automatically reflects changes
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit standardizes how API responses are wrapped, solving issue #24.
Problem:
- Inconsistent response envelopes (jails vs items vs bans vs no wrapper)
- Frontend required multiple field name variants
- Integration bugs from branching logic
- No clear pattern for different response types
Solution:
- Created response.py with base classes: PaginatedListResponse,
CollectionResponse, CommandResponse
- Standardized all list/collection responses to use 'items' field
- Domain-specific field names for detail and aggregation responses
- Updated all backends routers and mappers
- Updated frontend types and hooks to match
Changes:
Backend:
- backend/app/models/response.py (new): Base response models
- backend/app/models/ban.py: Updated responses to inherit from bases
- backend/app/models/jail.py: Updated JailListResponse, JailCommandResponse
- backend/app/models/config.py: Updated collection responses
- backend/app/services/jail_service.py: Updated return statements
- backend/app/mappers/ban_mappers.py: Updated 'bans' to 'items'
- backend/tests/test_mappers/test_ban_mappers.py: Updated tests
Frontend:
- frontend/src/types/jail.ts: Updated response interfaces
- frontend/src/types/config.ts: Updated response interfaces
- frontend/src/hooks/useActiveBans.ts: Updated selector
- frontend/src/hooks/useJailList.ts: Updated selector
- frontend/src/hooks/useJailConfigs.ts: Updated selector
- frontend/src/hooks/useConfigActiveStatus.ts: Updated field access
- frontend/src/hooks/useJailAdmin.ts: Updated field access
Documentation:
- Docs/Backend-Development.md: Added § 4.1 API Response Envelope Policy
The policy defines:
1. Paginated lists use PaginatedListResponse (items, total, page, page_size)
2. Non-paginated collections use CollectionResponse (items, total)
3. Detail responses use entity-specific field names (jail, status, settings)
4. Command responses use CommandResponse (message, success, optional target)
5. Aggregations use domain-specific fields (jails, countries, buckets, bans)
All responses now follow one of these patterns, reducing frontend complexity.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Convert inconsistent modeling style to standardized Pydantic models for all
external-facing data structures while maintaining TypedDict compatibility where
appropriate for internal layer-private structures.
Changes:
- Converted IpLookupResult TypedDict to use IpLookupResponse Pydantic model
in jail_service.lookup_ip() for consistency with routers
- Added GeoCacheEntry Pydantic model for geo cache repository rows
- Converted GeoCacheRow TypedDict to use GeoCacheEntry alias
- Converted ImportLogRow TypedDict to use ImportLogEntry alias
- Updated routers and services to work with Pydantic models
- Updated all tests to use Pydantic model field access (attributes)
instead of dict subscripting
Documentation:
- Added 'Model Type Usage by Layer' section to Backend-Development.md
- Defines when TypedDict is allowed (internal structures) vs Pydantic
(external-facing, cross-boundary data)
- Provides clear guidance on modeling conventions per layer
Benefits:
- Consistent validation and serialization behavior
- Better IDE support and type checking
- Clearer separation of concerns by layer
- Reduced maintenance cost from mixed validation approaches
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
TASK-004: Replace module-level mutable runtime flags in service layer with
injected state holder, eliminating hidden global state and improving testability
and synchronization boundaries.
Changes:
- Create JailServiceState dataclass in app/utils/runtime_state.py to hold
backend capability cache and synchronization lock
- Add JailServiceState as a field in RuntimeState (with default_factory)
- Remove module-level _backend_cmd_supported and _backend_cmd_lock from
jail_service.py
- Refactor _check_backend_cmd_supported() to accept state parameter
- Inject JailServiceState into list_jails() and _fetch_jail_summary() via
parameters
- Add get_jail_service_state() dependency provider in app/dependencies.py
- Add JailServiceStateDep type alias for router injection
- Update jails router to receive and pass state to service functions
- Update all tests to use jail_service_state fixture and pass state to functions
- Remove duplicate _MAX_PAGE_SIZE constant definition
- Document mutable state management in Backend-Development.md
- Update Architecture.md to describe JailServiceState and state nesting pattern
Benefits:
- Eliminates global mutable state and associated race conditions
- Makes state visible to callers (not hidden in module scope)
- Enables test isolation (each test gets fresh state)
- Prepares codebase for multi-worker deployments (state can be extracted to
shared backend)
- Synchronization boundaries are now explicit (state.get_backend_cmd_lock())
Compliance:
- All tests pass (17 passed in TestListJails, TestGetJail, TestLockInitialization)
- No ruff linting errors
- Type-safe: JailServiceState properly typed with asyncio.Lock, bool | None
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove hidden cross-service coupling by making dependencies explicit through
dependency injection while maintaining backward compatibility via lazy imports.
Key changes:
- history_service and ban_service: Removed direct module-level imports of
fail2ban_metadata_service, added optional service parameters to functions
- Added get_fail2ban_metadata_service() provider to dependencies.py
- Updated history router to inject Fail2BanMetadataService dependency
- history_service functions now use lazy imports in fallback paths for
backward compatibility when service is not explicitly injected
- All test patches updated to use internal _get_fail2ban_db_path() helper
- jail_config_service and jail_service already follow best practices
This pattern prevents circular imports, makes services testable via explicit
mocking, and documents service dependencies clearly.
Fixes: Instructions.md #2 - Hidden cross-service coupling
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Instead of returning a bound method (geo_cache.lookup_batch), now inject
the GeoCache instance directly into routers and services. This provides
proper runtime isolation since T-04 made GeoCache a proper object.
Changes:
- Remove get_geo_batch_lookup() dependency provider
- Add GeoCacheDep type alias for injecting GeoCache instances
- Update all routers (bans, blocklist, dashboard, jails) to use GeoCacheDep
- Update ban_service, blocklist_service, jail_service to accept GeoCache
- Update service protocols to match new signatures
- Update docstrings to reference GeoCache methods instead of module functions
All callers now call geo_cache.lookup_batch(...) directly instead of
geo_batch_lookup(...), providing real dependency injection with proper
testing isolation.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Refactor action_config_service, filter_config_service, jail_config_service, and jail_service
- Add jail_socket utility module for socket communication
- Update test_jail_service with new test cases
- Update architecture and task documentation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Consolidate duplicate _ok(), _to_dict(), ensure_list(), and is_not_found_error()
functions from 6 service modules into a single canonical implementation at
backend/app/utils/fail2ban_response.py.
Changes:
- Create fail2ban_response.py with canonical implementations
- Remove local duplicates from: ban_service, jail_service, config_service,
health_service, server_service, config_file_utils
- Update all imports to use shared module
- Add comprehensive docstrings and examples
- Update Architecture.md and Backend-Development.md documentation
Benefits:
- Single source of truth for response parsing logic
- Eliminates code duplication across service layer
- Improves maintainability and consistency
- Enables centralized bug fixes and improvements
Tests: All 228 service tests passing, no regressions
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add TYPE_CHECKING guards for runtime-expensive imports (aiohttp, aiosqlite)
- Reorganize imports to follow PEP 8 conventions
- Convert TypeAlias to modern PEP 695 type syntax (where appropriate)
- Use Sequence/Mapping from collections.abc for type hints (covariant)
- Replace string literals with cast() for improved type inference
- Fix casting of Fail2BanResponse and TypedDict patterns
- Add IpLookupResult TypedDict for precise return type annotation
- Reformat overlong lines for readability (120 char limit)
- Add asyncio_mode and filterwarnings to pytest config
- Update test fixtures with improved type hints
This improves mypy type checking and makes type relationships explicit.
- jail_service.restart(): replace invalid ["restart"] socket command with
["stop"], matching fail2ban transmitter protocol. The daemon is now
stopped via socket; the caller starts it via subprocess.
- config_file_service: expose _start_daemon and _wait_for_fail2ban as
public start_daemon / wait_for_fail2ban functions.
- restart_fail2ban router: orchestrate stop (socket) → start (subprocess)
→ probe (socket). Returns 204 on success, 503 when fail2ban does not
come back within 10 s. Catches JailOperationError → 409.
- reload_fail2ban router: add JailOperationError catch → 409 Conflict,
consistent with other jail control endpoints.
- Tests: add TestJailControls.test_restart_* (3 cases), TestReloadFail2ban
502/409 cases, TestRestartFail2ban (5 cases), TestRollbackJail (6
integration tests verifying file-write, subprocess invocation, socket-
probe truthiness, active_jails count, and offline-at-call-time).
This commit implements fixes for three independent bugs in the fail2ban configuration and integration layer:
1. Task 1: Detect UnknownJailException and prevent silent failures
- Added JailNotFoundError detection in jail_service.reload_all()
- Enhanced error handling in config_file_service to catch JailNotFoundError
- Added specific error message with logpath validation hints
- Added rollback test for this scenario
2. Task 2: Fix iptables-allports exit code 4 (xtables lock contention)
- Added global banaction setting in jail.conf with -w 5 lockingopt
- Removed redundant per-jail banaction overrides from bangui-sim and blocklist-import
- Added production compose documentation note
3. Task 3: Suppress log noise from unsupported backend/idle commands
- Implemented capability detection to cache command support status
- Double-check locking to minimize lock contention
- Avoids sending unsupported get <jail> backend/idle commands
- Returns default values without socket calls when unsupported
All changes include comprehensive tests and maintain backward compatibility.
Adds ability to reload or restart fail2ban service from the Server tab UI.
Backend changes:
- Add new restart() method to jail_service.py that sends 'restart' command
- Add new POST /api/config/restart endpoint in config router
- Endpoint returns 204 on success, 502 if fail2ban unreachable
- Includes structured logging via 'fail2ban_restarted' log entry
Frontend changes:
- Add configRestart endpoint to endpoints.ts
- Add restartFail2Ban() API function in config.ts API module
- Import ArrowSync24Regular icon from Fluent UI
- Add reload and restart button handlers to ServerTab
- Display 'Reload fail2ban' and 'Restart fail2ban' buttons in action row
- Show loading spinner during operation
- Display success/error MessageBar with appropriate feedback
- Update ServerTab docstring to document new buttons
All 115 frontend tests pass.
Stage 1.1-1.3: reload_all include/exclude_jails params already implemented;
added keyword-arg assertions in router and service tests.
Stage 2.1/6.1: _send_command_sync retry loop (3 attempts, 150ms exp backoff)
retrying on EAGAIN/ECONNREFUSED/ENOBUFS; immediate raise on all other errors.
Stage 2.2: asyncio.Lock at module level in jail_service.reload_all to
serialize concurrent reload--all commands.
Stage 3.1: activate_jail re-queries _get_active_jail_names after reload;
returns active=False with descriptive message if jail did not start.
Stage 4.1/6.2: asyncio.Semaphore (max 10) in Fail2BanClient.send, lazy-
initialized; logs fail2ban_command_waiting_semaphore at debug when waiting.
Stage 5.1/5.2: unit tests asserting reload_all is called with include_jails
and exclude_jails; activation verification happy/sad path tests.
Stage 6.3: TestSendCommandSyncRetry (5 cases) + TestFail2BanClientSemaphore
concurrency test.
Stage 7.1-7.3: _since_unix uses time.time(); bans_by_jail debug logging with
since_iso; diagnostic warning when total==0 despite table rows; unit test
verifying the warning fires for stale data.
- Backend: Add BantimeEscalation + BantimeEscalationUpdate Pydantic models
to app/models/config.py; add bantime_escalation field to Jail in jail.py
- Backend: jail_service.get_jail_detail() fetches 7 bantime.* socket commands
(increment, factor, formula, multipliers, maxtime, rndtime, overalljails)
and populates bantime_escalation on the returned Jail object
- Backend: config_service.get_jail_config() fetches same 7 commands;
update_jail_config() writes escalation fields when provided
- Frontend: Add BantimeEscalation + BantimeEscalationUpdate interfaces to
types/config.ts; extend JailConfig + JailConfigUpdate; extend Jail in
types/jail.ts
- Frontend: JailDetailPage.tsx adds BantimeEscalationSection component that
renders only when increment is enabled (shows factor, formula, multipliers,
max_time, rnd_time, overall_jails)
- Frontend: ConfigPage.tsx JailAccordionPanel adds full escalation edit form
(Switch for enable/disable, number inputs for factor/max_time/rnd_time,
text inputs for formula/multipliers, Switch for overall_jails);
handleSave includes bantime_escalation in the JailConfigUpdate payload
- Tests: Update ConfigPageLogPath.test.tsx mock to include bantime_escalation:null
- Docs: Mark Task 6 as DONE in Tasks.md
- Send fail2ban's `unban --all` command via new `unban_all_ips()` service
function; returns the count of unbanned IPs
- Add `UnbanAllResponse` Pydantic model (message + count)
- Add `DELETE /api/bans/all` router endpoint; handles 502 on socket error
- Frontend: `bansAll` endpoint constant, `unbanAllBans()` API call,
`UnbanAllResponse` type, `unbanAll` action in `useActiveBans` hook
- JailsPage: "Clear All Bans" button (visible when bans > 0) with a
Fluent UI confirmation Dialog before executing the operation
- 7 new tests (3 service, 4 router); 440 total pass, 82% coverage
Task 1 — fix Stop/Reload Jail returning 404
Root cause: reload_jail and reload_all sent an empty config stream
(["reload", name, [], []]). In fail2ban's reload protocol the end-of-
reload phase deletes every jail still in reload_state — i.e. every jail
that received no configuration commands. An empty stream means *all*
affected jails are silently removed from the daemon's runtime, causing
everything touching those jails afterwards (including stop) to receive
UnknownJailException → HTTP 404.
Fixes:
- reload_jail: send ["start", name] in the config stream; startJail()
removes the jail from reload_state so the end phase commits instead of
deletes, and un-idles the jail.
- reload_all: fetch current jail list first, build a ["start", name]
entry for every active jail, then send reload --all with that stream.
- stop_jail: made idempotent — if the jail is already gone (not-found
error) the operation silently succeeds (200 OK) rather than returning
404, matching the user expectation that stop = ensure-stopped.
- Router: removed dead JailNotFoundError handler from stop endpoint.
391 tests pass (2 new), ruff clean, mypy clean (pre-existing
config.py error unchanged).
Task 2 — access list simulator
- Docker/simulate_accesses.sh: writes fake HTTP-scan log lines in
custom format (bangui-access: http scan from <IP> ...) to
Docker/logs/access.log so the bangui-access jail detects them.
- fail2ban/filter.d/bangui-access.conf: failregex matching the above.
- fail2ban/jail.d/bangui-access.conf: polling jail on access.log,
same settings as bangui-sim (maxretry=3, bantime=60s).
- .gitignore: whitelist new bangui-access.conf files.
- Docker/fail2ban-dev-config/README.md: added "Testing the Access
List Feature" section with step-by-step instructions and updated
Configuration Reference + Troubleshooting.
_is_not_found_error in jail_service did not match the concatenated form
'unknownjailexception' that fail2ban produces when it serialises
UnknownJailException, so JailOperationError was raised instead of
JailNotFoundError and every ban attempt in the import loop failed
individually, skipping all 27 840 IPs before returning an error.
Two changes:
- Add 'unknownjail' to the phrase list in _is_not_found_error so that
UnknownJailException is correctly mapped to JailNotFoundError.
- In blocklist_service.import_source, catch JailNotFoundError explicitly
and break out of the loop immediately with a warning log instead of
retrying on every IP.
