Commit Graph

610 Commits

Author SHA1 Message Date
aa717a28f8 fix(e2e): resolve blocklist import test failures
auth.resource:
- add Login Via HTTP keyword for RequestsLibrary auth (CSRF-aware)
- fix session_duration_minutes type: bare int → ${60}
- add Process library import to common.resource

03_blocklist_import.robot:
- fix selector to button[data-testid] (was matching all buttons)
- use GET/POST On Session with auth session for blocklist API calls
- fix log response key: entries → items
- fix enabled=true → ${TRUE} for boolean type
- fix ${len(sources)} → Get Length keyword
- make Ensure Blocklist Source Exists accept session argument
- replace strict error assertion with specific error banner check
- add graceful Terminate Process teardown

02_ban_records.robot:
- add Process library import

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-08 08:07:39 +02:00
e4c3ae718c fix(backend): relax SSRF validation for loopback in dev, graceful metrics/regexploit fallback
- ip_utils: allow loopback (127.0.0.1) in dev mode (BANGUI_LOG_LEVEL=debug)
  so e2e tests can reach a mock HTTP server on the host
- metrics: make all operations no-ops when prometheus_client not installed
- regex_validator: graceful fallback when regexploit not installed
- geo_cache: use attribute access instead of dict subscript for typed rows
- rate_limit: support bucket_override parameter for per-endpoint rate limits
- ban_service: construct DomainActiveBan explicitly instead of model_copy

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-08 08:07:13 +02:00
d4bab89cf3 fix(e2e): resolve SPA auth race conditions in Robot tests
- Rework Login As Admin: use sessionStorage flag + relative fetch login + polling loop
- Add data-testid to JailDetailPage error render path
- Add Collections library import for Get From List keyword
- Fix /jails API response extraction (returns {items, total} not plain list)
- Change Close Context to Close Browser for proper browser cleanup
- Add domcontentloaded + Sleep + polling to Config test to avoid premature timeout

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-06 06:53:09 +02:00
48ef85bec5 backup 2026-05-05 19:51:14 +02:00
17ba07b592 refactor(e2e): replace HttpLibrary with RequestsLibrary
- Swap HttpLibrary for RequestsLibrary in common.resource
- Add robotframework-requests to requirements
- Remove backend health check from suite setup (setup moved to individual tests)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-05 19:11:36 +02:00
481f32bb85 backup 2026-05-05 18:47:56 +02:00
d25b56e7e1 backup 2026-05-04 13:13:01 +02:00
48d57c31e1 backup 2026-05-04 13:12:57 +02:00
e41831447f docs: update documentation and e2e tests
- Add configuration docs for database and rate limiting
- Remove completed tasks from tracking list
- Update testing requirements with new test patterns
- Enhance web development docs with frontend guidelines
- Expand page loading and ban records e2e test coverage
2026-05-04 08:34:18 +02:00
23c3a0d9e6 feat: add e2e test suite with Robot Framework
Add e2e/ dir with Robot Framework tests for page loading, ban records,
blocklist import, config edit. Add requirements.txt. Update Makefile with
test commands. Update .gitignore, backend docs, testing requirements docs.
2026-05-04 08:29:12 +02:00
5fa67d3288 backup
Co-authored-by: Copilot <copilot@github.com>
2026-05-04 08:16:20 +02:00
744275d17f backup 2026-05-04 07:20:20 +02:00
58173bd6a9 backup 2026-05-04 07:20:16 +02:00
69e1726045 Refactor data fetching hooks, add page size lint test
- Simplify useFetchData: remove unused URL building logic
- Add usePolledData initial implementation
- Add router page_size param validation test
- Update API reference docs
- Clean up tasks doc
2026-05-04 06:48:24 +02:00
0a3f9c6c16 refactor(backend): external logging metrics, required mode, health checks
- Add external_logging_init_failures counter
- Add external_log_required flag, raise if init fails and required
- Health endpoint: add external_logging status check
- Blocklist service: enrich with metadata fields, update import logic
- Health check task: add runtime_state dependency, fix return typing
- Metrics: add Histogram for request latencies
- Frontend: align BlocklistImportLogSection props
- Docs: update deployment guide, remove stale tasks

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-04 03:45:13 +02:00
42e177e6ea feat(frontend): add ignoreCancellation option for background tasks
Allow useNavigationAbortSignal to opt out of navigation-based abort
for long-lived background tasks like polling. Set ignoreCancellation: true
to keep requests alive across route changes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-04 02:57:56 +02:00
eb339efcfd Add Kubernetes liveness/readiness probes and middleware order validation
- Split /health into /health/live (liveness) and /health/ready (readiness)
  following Kubernetes conventions. Combined /health retained for backward
  compatibility with existing Docker HEALTHCHECK definitions.
- Add ReadyCheck and ReadyResponse models for structured readiness output.
- Add _assert_middleware_order() startup check enforcing:
  RateLimit → Csrf → CorrelationId middleware chain.
- Register CorrelationIdMiddleware, CsrfMiddleware, RateLimitMiddleware
  in create_app() with documented required order (reverse of processing).
- Add correlation.py, csrf.py, rate_limit.py middleware modules.
- Add health probe tests in test_health_probes.py.
- Update test_main.py with middleware order assertion tests.
- Update frontend useFetchData hook tests.
- Docs: update Deployment.md with Kubernetes probe config examples.
2026-05-04 02:42:09 +02:00
65fe747cba feat(backend): add deprecation middleware and API versioning support
- Add deprecation middleware for warning headers on sunset endpoints
- Add jails_v2 router for API v2 migration path
- Update CI workflow with new test coverage
- Update API versioning documentation
- Remove completed tasks from Tasks.md
2026-05-04 00:03:52 +02:00
c8b48b5b65 fix(api): correlation ID survives HMR; fix endpoint template literal typos
- client.ts: store correlation ID in sessionStorage so HMR (module re-eval)
  does not generate a new ID mid-session; add clearSessionCorrelationId()
- endpoints.ts: fix 3 template literal trailing-quote bugs (missing ')' chars);
  replace template literals with string concat for encodeURIComponent calls
- AuthProvider.tsx: call clearSessionCorrelationId() on logout
- App.tsx: reorder ThemeProvider import before AuthProvider per PROVIDER_ORDER.md;
  indent Routes inside AuthProvider to match expected tree structure
- Tasks.md: update task status
- providerTreeOrder.test.tsx: add integration tests for provider nesting order

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 23:35:18 +02:00
fc57c83f79 refactor: split pagination logic from response models
- Extract pagination logic to separate util module
- Update response models to use new pagination util
- Fix pagination calculation edge cases

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 22:57:21 +02:00
b2747381ec Remove Issue #51 from Tasks.md (type-enforcement tracked elsewhere) 2026-05-03 22:47:24 +02:00
edebf1a339 feat(services): add ErrorContract enum and PartialResult type
Add typed wrappers for error handling patterns in error_handling.py:

- ErrorContract(enum): machine-checkable pattern selector with
  from_value() helper and string constants matching the existing
  ABORT_ON_ERROR/RETURN_DEFAULT/PARTIAL_RESULT module-level values
- ErrorEntry: typed error container for PARTIAL_RESULT (context + cause)
- PartialResult[T]: typed result wrapper for PARTIAL_RESULT operations

Existing string constants preserved for backward compat.
Updated module docstring with type annotation table and examples.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 22:46:47 +02:00
a2afec2d1e Remove completed Issue #50 navigation abort signal task
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 22:43:19 +02:00
52a70c3eea Add import-linter boundary to forbid routers importing app.dependencies
Issue #51: Enforce repository boundary at CI level using import-linter.
Contract 'forbid_router_db_import' checks that app.routers never imports
app.dependencies directly, keeping the DB access path through service
contexts only.

- Add import-linter>=2.0.0 to dev dependencies (backend/pyproject.toml)
- Configure [tool.importlinter] with package_root and root_packages
- Add [[tool.importlinter.contracts]] with type='forbidden', source
  app.routers, forbidden app.dependencies
- Add 'Import Boundary' CI job (import-linter)
- Document import-linter in CONTRIBUTING.md code quality table

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 22:42:46 +02:00
3376009903 fix(nav): move AbortController creation synchronously in render
Previously the AbortController was created inside useEffect, which runs
after render. This meant requests initiated during render received
the previous cycle's (possibly aborted) signal and were cancelled
immediately instead of completing.

Now the controller is created synchronously when pathname changes, before
any request can be initiated on the new route. The old controller is
aborted in the same conditional block, before the new one is created.

Side effect: removed resolved Issue #49 from Tasks.md.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 22:37:07 +02:00
7fcfc14199 fix(auth): dedupe handler + error utils refactor
- Add 401/403 dedup guard to API client to prevent double logout
- Extract fetchError util: isAuthError + getErrorMessage
- AuthProvider uses new error utils, removes duplicate logic
- Remove completed task docs from Tasks.md

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 22:13:12 +02:00
dafe8d61e2 feat(security): add CSRF header constants and security-headers endpoint
Move X-BanGUI-Request header name/value to backend/app/utils/constants.py as single source of truth. Add GET /api/v1/config/security-headers endpoint. Update csrf middleware, frontend api client, and docs to use shared constants.
2026-05-03 22:06:43 +02:00
cee3daffc1 fix: enforce PRAGMA query_only on fail2ban DB and refactor CSRF cookie name
- Add _acquire_readonly_connection() that applies PRAGMA query_only=ON after connect
- Verify PRAGMA value back to catch URI flag bypasses
- Wrap in async context manager _readonly_connection() used by all repo methods
- Replace hardcoded '_SESSION_COOKIE_NAME' in CSRF middleware with import from
  app.utils.constants
- Remove completed Issues #45 and #46 from Docs/Tasks.md (Issue #46 now fixed,
  #45 cache invalidation deferred to auth refactor branch)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 21:47:42 +02:00
1c3dff31e8 feat(rate-limiting): add per-bucket limits and startup validation
- Add per-bucket rate limit config (ban, unban, import, config, jail, filter, action)
- Add process-local warning at startup for multi-worker deployments
- Document Redis migration path for shared state across workers
- Remove Issue #42 from Tasks.md (resolved)
2026-05-03 20:53:21 +02:00
c3cd1574dc fix(auth): invalidate session cache on login
Stale sessions from a stolen device could be reused up to the cache
TTL after a legitimate user re-logs in, because login never cleared
the existing cache entry.

Changes:
- Add invalidate_by_user(user_id) to SessionCache protocol
- InMemorySessionCache maintains a user_id -> set[token] index to
  support O(1) invalidation of all sessions for a given user
- NoOpSessionCache stub updated for API compatibility
- auth_service.login() now returns the Session object alongside
  signed_token and expires_at
- login router calls session_cache.invalidate_by_user(session.id)
  immediately after successful authentication

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 20:51:51 +02:00
ae9313568e feat: enforce single-worker at startup
Fail with RuntimeError when WEB_CONCURRENCY or BANGUI_WORKERS > 1.

In-memory session cache, rate-limit windows, and runtime state are
process-local. Multi-worker silently causes stale limits, ghost sessions,
inconsistent status.

Skipped when TESTING=1.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 20:33:23 +02:00
e1a6491ac2 docs: add API reference and database schema docs
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 19:50:36 +02:00
4d09d2538d docs: Add security best practices to Deployment.md
- Secrets management via environment variables
- Container security hardening (non-root user, filesystem permissions, capabilities)
- Network security and TLS termination guidance
- Prune obsolete task tracking from Tasks.md

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 19:48:52 +02:00
624f869f5b docs: add CI workflow and testing requirements documentation
- Add GitHub Actions CI pipeline with pytest, ruff, mypy
- Expand Tasks.md with implementation tracking and testing criteria
- Update CONTRIBUTING.md with CI requirements
- Add Testing-Requirements.md with coverage thresholds and PR checks

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 19:42:50 +02:00
497d7cab41 docs: clean up completed accessibility issue, expand WCAG guidelines
- Remove Issue #31 (weak password validation) from Tasks.md
- Mark Issue #32 (accessibility) done in Tasks.md
- Expand Web-Development.md §14 with WCAG compliance rules, ARIA guide, keyboard nav, form accessibility, testing

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 18:53:58 +02:00
c96b87ee8b feat: reject common passwords in SetupRequest
- Add ~75 common plaintext passwords to setup.py validator
- Check case-insensitively; passes complexity but blocked
- Add tests: reject common, accept unique, short common fail on length
- Update Security.md docs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 18:25:17 +02:00
96525573fa Normalise IP addresses across backend
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 18:19:41 +02:00
85d05ee582 docs: add editorconfig setup and remove completed task
- Add EditorConfig section to CONTRIBUTING.md with IDE plugins table
- Remove Issue #28 from Tasks.md (pre-commit hooks now documented)
2026-05-03 18:07:33 +02:00
5f0ab40816 refactor(backend): clean up models setup, improve ip utils, add adr docs
- Extract ADR documents for architectural decisions (SQLite, FastAPI, React, APScheduler, Scheduler)
- Refactor setup.py: improve code structure and readability
- Add IP validation utilities with test coverage
- Update frontend components (BanTable, HistoryPage)
- Add pre-commit hooks and CONTRIBUTING.md
- Add .editorconfig for consistent coding standards
2026-05-03 18:04:45 +02:00
2f9fc8076d refactor(backend): clean up jail service, add error handling service
- Extract jail status/processing to helper functions
- Add error_handling.py service for centralized error handling
- Update config.py with validation and defaults
- Update .env.example with all config options
- Remove obsolete Tasks.md, add Service-Development.md
- Minor fixes across routers and services

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 17:40:37 +02:00
2df029f7e8 refactor(ban_service): extract _bans_by_country_load_data helper
Break up long function into focused helper. Load data logic separate from aggregation.
2026-05-03 17:00:34 +02:00
5058a50143 Refactor backend: fix geo cache cleanup, scheduler heartbeat, correlation middleware; update docs 2026-05-03 16:02:40 +02:00
896751ada9 fix: handle socket close errors properly in PapertrailLogHandler
- Replace contextlib.suppress with try/except + warning log
- Add test for fail2ban client
- Remove stale Issue #21 from Tasks.md (indexes)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 12:25:14 +02:00
Copilot
22db607875 Add fail2ban DB index management and socket-based path resolution
- New get_fail2ban_db_path() in setup_service resolves DB path from configured socket path
- New ensure_fail2ban_indexes() creates missing performance indexes on bans table
- Call ensure_fail2ban_indexes on every startup before first ban query
- Remove completed tasks from Docs/Tasks.md
- Update Docs/PERFORMANCE.md with index findings
2026-05-03 12:17:31 +02:00
0133489920 Update observability docs and task utilities
- Add Observability.md documentation
- Standardize task logging with correlation_id support
- Add log_sanitizer utility for PII masking
- Update Tasks.md tracking
- Update geo_cache tasks and other task modules with correlation_id

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 11:52:09 +02:00
7b93499551 Refactor config loading and add status code docs
- Move config loading to dedicated ConfigLoader class with validation
- Add DATABASE_MIGRATIONS.md content to TROUBLESHOOTING.md
- Add API_STATUS_CODES.md documenting all API response codes
- Update runner.csx to use new config structure
- Add check_responses.py validation script
- Update config tests for new structure

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 11:52:01 +02:00
8f26776bb3 docs: add OpenAPI responses={} to all router endpoints
Add explicit HTTP status code documentation to every endpoint
across 15 router files. Each endpoint now declares all possible
response codes (200/201/204/400/401/404/409/429/502/503) with
descriptions so frontend can distinguish error types.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 01:12:08 +02:00
7ad885d276 refactor: separate config service from jail config service
- Split config_service.py into config_service.py and jail_config_service.py
- Update Docs/Tasks.md, Security.md, TROUBLESHOOTING.md
2026-05-03 01:05:18 +02:00
881cfbdd71 fix: replace broad except Exception with specific exception types
- jail_service: catch ValueError (fail2ban protocol error) instead of Exception
- health.py: catch AttributeError (not OSError/TypeError) for defensive checks
- ban_service: re-raise programming errors in geo lookup handlers
- server_service: catch Fail2BanConnectionError, Fail2BanProtocolError, ValueError
- config_writer: catch OSError instead of Exception

Programming errors now bubble to global handler instead of being silently caught.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 00:54:44 +02:00
bd6170722a feat(geo): add cache hit/miss metrics and prewarm support
- Add _hits/_misses counters to GeoCache for cache hit/miss ratio tracking
- Reset counters on clear()
- Count hits before misses in lookup_batch() to avoid interleaving
- Add synchronous prewarm() using asyncio.create_task for fire-and-forget
- Add hits/misses fields to GeoCacheStatsResponse model
- Add TestCacheMetrics (5 tests), TestPrewarm (3 tests), TestLargeBanList (2 tests)
- Fix _make_async_db() mock: db.execute is not async, returns ctx manager
- Move collections.abc to TYPE_CHECKING block (TC003)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 00:35:47 +02:00