Commit Graph

634 Commits

Author SHA1 Message Date
3af8f0571b feat: graceful shutdown and WAL cleanup
Some checks failed
CI / Backend Tests (push) Has been cancelled
CI / Lint (push) Has been cancelled
CI / Type Check (push) Has been cancelled
CI / Import Boundary (push) Has been cancelled
CI / OpenAPI Breaking Changes (push) Has been cancelled
CI / OpenAPI Baseline Commit (push) Has been cancelled
- Add stop_grace_period to backend container for graceful shutdown
- Document WAL mode rationale and orphaned file cleanup in db.py
- Handle database close errors gracefully in lifespan
- Clean up orphaned WAL files during startup before opening DB
- Reorder imports and fix formatting in startup.py
2026-05-24 22:05:34 +02:00
d5a78a251a Remove Tasks.md spec, add test for _cleanup_wal_files skipping recent files
Remove 335-line task specification from Docs/Tasks.md.
Add test confirming _cleanup_wal_files skips recently-modified WAL/SHM files.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-24 22:05:34 +02:00
904db63fa2 Add tests for since timestamp accuracy in ban_service
- test_since_unix_returns_utc_epoch: validates since_unix('24h') returns UTC epoch
- test_ban_trend_since_is_within_expected_range: validates 23h-ago ban falls in 24h+slack window

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-24 22:05:34 +02:00
d737a1c319 Add logging duplication tests
- test_logging_configuration_no_duplicate_handlers: verify create_app() twice leaves ≤1 StreamHandler
- test_uvicorn_access_logs_go_through_root_handler: verify uvicorn.access can emit JSON via JSONFormatter
- test_external_logging_processor_queues_record: verify _external_logging_processor queues to handler
- test_plain_text_logs_not_emitted_after_startup: verify app.db emits JSON not plain text

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-24 22:05:34 +02:00
9e765c6cb7 Add granular DB error types with retry logic
New exceptions: DatabaseBusyError, DatabasePermissionDeniedError,
DatabasePathInvalidError, DatabaseCorruptedError, DatabaseUnavailableError.

open_db creates parent directory if missing. Catches all aiosqlite errors
and maps to specific exception types.

get_db retries up to 3x on locked database with backoff.
Propagates specific exceptions instead of generic HTTPException.

Tests for all new error types and retry behavior.
2026-05-24 22:05:34 +02:00
ecb8542496 docs: add comprehensive task backlog and bump version to rc.5
- Document database error handling, logging duplication, ban service
timestamp, and orphaned SQLite file issues in Tasks.md
- Bump backend version from 0.9.19-rc.4 to 0.9.19-rc.5
2026-05-24 22:05:34 +02:00
97f4df4a61 chore: release v0.9.19-rc.5 2026-05-24 22:05:34 +02:00
44542b93c0 chore(release): bump version to 0.9.19-rc.4
- Add production Docker Compose configuration

- Add check_auth.py diagnostic script for session 401 debugging
2026-05-24 22:05:34 +02:00
01a4215f60 chore: release v0.9.19-rc.4 2026-05-24 22:05:34 +02:00
bc49b7cd5b fix(db): fix migration failures when upgrading from 0.8.0 schema
Migration 1: remove idx_sessions_token_hash from _SCHEMA_STATEMENTS.
The legacy schema has sessions.token (not token_hash). The IF NOT EXISTS
guard only prevents duplicate index names — it still requires the column
to exist. Migration 2 drops and rebuilds sessions with token_hash anyway,
so creating the index in migration 1 was redundant.

Migration 3: replace ALTER TABLE ADD COLUMN with a table rebuild.
SQLite rejects ALTER TABLE ADD COLUMN NOT NULL DEFAULT <expression> when
the table already contains rows. The old DB has ~181k geo_cache rows, so
the ALTER always failed. Rebuild copies existing rows with last_seen set
to cached_at as a reasonable approximation of last-seen time.
2026-05-24 22:05:34 +02:00
fa4fe4bbdf chore: release v0.9.19-rc.3 2026-05-24 22:05:34 +02:00
ee0fe9c695 fix(auth): suppress misleading 502 warning during session validation
A 502 Bad Gateway is a server/gateway error, not a network error.
Logging it as a 'Session validation network error' is noisy and
misleading during startup when nginx is temporarily unreachable.

Silently skip the console.warn for 5xx errors in handleValidationError
while keeping the warning for actual network errors.
2026-05-24 22:05:34 +02:00
551db0bb9c chore: release v0.9.19-rc.2 2026-05-24 22:05:34 +02:00
4a649e7347 chore: bump to v0.9.19-rc.1 and add local OpenAPI build support
- Add release candidate (rc) support to release.sh with latestRC tagging
- Bump VERSION, backend pyproject.toml, and frontend package.json to 0.9.19-rc.1
- Add local frontend/openapi.json so build no longer needs running backend
- Update generate:types and validate-types.sh to use local openapi.json
- Fix frontend tests: remove unused imports/variables and update mock data
2026-05-24 22:05:34 +02:00
025c82a982 Merge pull request 'refactoring-backend' (#3) from refactoring-backend into main
Some checks failed
CI / Backend Tests (push) Has been cancelled
CI / Lint (push) Has been cancelled
CI / Type Check (push) Has been cancelled
CI / Import Boundary (push) Has been cancelled
CI / OpenAPI Breaking Changes (push) Has been cancelled
CI / OpenAPI Baseline Commit (push) Has been cancelled
Reviewed-on: #3
2026-05-20 20:23:46 +02:00
83b2cb67b1 backup
Some checks failed
CI / Backend Tests (pull_request) Has been cancelled
CI / Lint (pull_request) Has been cancelled
CI / Type Check (pull_request) Has been cancelled
CI / Import Boundary (pull_request) Has been cancelled
CI / OpenAPI Breaking Changes (pull_request) Has been cancelled
CI / OpenAPI Baseline Commit (pull_request) Has been cancelled
2026-05-20 20:18:58 +02:00
7308ff88d6 fix(rate-limit): stop double-counting requests in middleware
Multiple RateLimitMiddleware instances were each calling
check_allowed() on every request, halving the effective global
limit (200 req/min became ~100). Added path_prefixes and skip_paths
so each instance only checks the paths it owns.

- Auth middleware scoped to /api/v1/auth/login and /api/v1/setup
- History middleware scoped to /api/v1/history
- Global middleware skips auth and history paths
- Updated tests to match single-count behavior
2026-05-15 23:04:02 +02:00
77df5d5d65 fixed tests 2026-05-15 20:41:05 +02:00
96ce516ecf fix(logging): resolve logging_compat keyword arg conflicts
- Fix logging_compat._log() to handle extra keyword arguments properly
- Update config.py, main.py, and test_bans.py for compatibility
- Update Tasks.md and runner.csx
2026-05-10 15:54:00 +02:00
7ec80fdeec refactor(logging): replace structlog with stdlib logging compat layer
- Remove structlog dependency from backend/pyproject.toml
- Add app.utils.logging_compat shim for keyword-arg logging API
- Add app.utils.json_formatter for JSON log output with extra fields
- Update all backend modules to use logging_compat.get_logger()
- Update docstrings in log_sanitizer.py and json_formatter.py
- Update test comment in test_async_utils.py
- Record 406 failing tests in Docs/Tasks.md for tracking
2026-05-10 13:37:54 +02:00
7790736918 feat(jail-config): add banaction and banaction_allports to blocklist config
Adds iptables-multiport and iptables-allports ban actions to the blocklist-import jail configuration and updates the corresponding test assertions.
2026-05-10 09:35:33 +02:00
79df1aa493 backup 2026-05-10 08:48:42 +02:00
cc9d3220c9 docs(e2e): add debugging notes and fix incorrect login example
Document lessons learned from debugging blocklist import tests:
- RequestsLibrary vs Browser library auth isolation
- CSRF header requirement
- Robot variable type rules
- network_mode: host implications
- SSRF protection behavior
- API response key discrepancies

Also fix API login example: backend accepts plaintext passwords,
not SHA256-hashed as previously documented.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-08 08:11:08 +02:00
8fc1989cc4 fix(docker): use host network mode for e2e mock server access
Both backend and frontend now use network_mode=host so the backend
can reach a mock HTTP server on the host's loopback interface during
e2e tests. VITE_BACKEND_URL env var set so frontend proxy reaches
host backend at localhost:8000.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-08 08:07:53 +02:00
aa717a28f8 fix(e2e): resolve blocklist import test failures
auth.resource:
- add Login Via HTTP keyword for RequestsLibrary auth (CSRF-aware)
- fix session_duration_minutes type: bare int → ${60}
- add Process library import to common.resource

03_blocklist_import.robot:
- fix selector to button[data-testid] (was matching all buttons)
- use GET/POST On Session with auth session for blocklist API calls
- fix log response key: entries → items
- fix enabled=true → ${TRUE} for boolean type
- fix ${len(sources)} → Get Length keyword
- make Ensure Blocklist Source Exists accept session argument
- replace strict error assertion with specific error banner check
- add graceful Terminate Process teardown

02_ban_records.robot:
- add Process library import

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-08 08:07:39 +02:00
e4c3ae718c fix(backend): relax SSRF validation for loopback in dev, graceful metrics/regexploit fallback
- ip_utils: allow loopback (127.0.0.1) in dev mode (BANGUI_LOG_LEVEL=debug)
  so e2e tests can reach a mock HTTP server on the host
- metrics: make all operations no-ops when prometheus_client not installed
- regex_validator: graceful fallback when regexploit not installed
- geo_cache: use attribute access instead of dict subscript for typed rows
- rate_limit: support bucket_override parameter for per-endpoint rate limits
- ban_service: construct DomainActiveBan explicitly instead of model_copy

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-08 08:07:13 +02:00
d4bab89cf3 fix(e2e): resolve SPA auth race conditions in Robot tests
- Rework Login As Admin: use sessionStorage flag + relative fetch login + polling loop
- Add data-testid to JailDetailPage error render path
- Add Collections library import for Get From List keyword
- Fix /jails API response extraction (returns {items, total} not plain list)
- Change Close Context to Close Browser for proper browser cleanup
- Add domcontentloaded + Sleep + polling to Config test to avoid premature timeout

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-06 06:53:09 +02:00
48ef85bec5 backup 2026-05-05 19:51:14 +02:00
17ba07b592 refactor(e2e): replace HttpLibrary with RequestsLibrary
- Swap HttpLibrary for RequestsLibrary in common.resource
- Add robotframework-requests to requirements
- Remove backend health check from suite setup (setup moved to individual tests)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-05 19:11:36 +02:00
481f32bb85 backup 2026-05-05 18:47:56 +02:00
d25b56e7e1 backup 2026-05-04 13:13:01 +02:00
48d57c31e1 backup 2026-05-04 13:12:57 +02:00
e41831447f docs: update documentation and e2e tests
- Add configuration docs for database and rate limiting
- Remove completed tasks from tracking list
- Update testing requirements with new test patterns
- Enhance web development docs with frontend guidelines
- Expand page loading and ban records e2e test coverage
2026-05-04 08:34:18 +02:00
23c3a0d9e6 feat: add e2e test suite with Robot Framework
Add e2e/ dir with Robot Framework tests for page loading, ban records,
blocklist import, config edit. Add requirements.txt. Update Makefile with
test commands. Update .gitignore, backend docs, testing requirements docs.
2026-05-04 08:29:12 +02:00
5fa67d3288 backup
Co-authored-by: Copilot <copilot@github.com>
2026-05-04 08:16:20 +02:00
744275d17f backup 2026-05-04 07:20:20 +02:00
58173bd6a9 backup 2026-05-04 07:20:16 +02:00
69e1726045 Refactor data fetching hooks, add page size lint test
- Simplify useFetchData: remove unused URL building logic
- Add usePolledData initial implementation
- Add router page_size param validation test
- Update API reference docs
- Clean up tasks doc
2026-05-04 06:48:24 +02:00
0a3f9c6c16 refactor(backend): external logging metrics, required mode, health checks
- Add external_logging_init_failures counter
- Add external_log_required flag, raise if init fails and required
- Health endpoint: add external_logging status check
- Blocklist service: enrich with metadata fields, update import logic
- Health check task: add runtime_state dependency, fix return typing
- Metrics: add Histogram for request latencies
- Frontend: align BlocklistImportLogSection props
- Docs: update deployment guide, remove stale tasks

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-04 03:45:13 +02:00
42e177e6ea feat(frontend): add ignoreCancellation option for background tasks
Allow useNavigationAbortSignal to opt out of navigation-based abort
for long-lived background tasks like polling. Set ignoreCancellation: true
to keep requests alive across route changes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-04 02:57:56 +02:00
eb339efcfd Add Kubernetes liveness/readiness probes and middleware order validation
- Split /health into /health/live (liveness) and /health/ready (readiness)
  following Kubernetes conventions. Combined /health retained for backward
  compatibility with existing Docker HEALTHCHECK definitions.
- Add ReadyCheck and ReadyResponse models for structured readiness output.
- Add _assert_middleware_order() startup check enforcing:
  RateLimit → Csrf → CorrelationId middleware chain.
- Register CorrelationIdMiddleware, CsrfMiddleware, RateLimitMiddleware
  in create_app() with documented required order (reverse of processing).
- Add correlation.py, csrf.py, rate_limit.py middleware modules.
- Add health probe tests in test_health_probes.py.
- Update test_main.py with middleware order assertion tests.
- Update frontend useFetchData hook tests.
- Docs: update Deployment.md with Kubernetes probe config examples.
2026-05-04 02:42:09 +02:00
65fe747cba feat(backend): add deprecation middleware and API versioning support
- Add deprecation middleware for warning headers on sunset endpoints
- Add jails_v2 router for API v2 migration path
- Update CI workflow with new test coverage
- Update API versioning documentation
- Remove completed tasks from Tasks.md
2026-05-04 00:03:52 +02:00
c8b48b5b65 fix(api): correlation ID survives HMR; fix endpoint template literal typos
- client.ts: store correlation ID in sessionStorage so HMR (module re-eval)
  does not generate a new ID mid-session; add clearSessionCorrelationId()
- endpoints.ts: fix 3 template literal trailing-quote bugs (missing ')' chars);
  replace template literals with string concat for encodeURIComponent calls
- AuthProvider.tsx: call clearSessionCorrelationId() on logout
- App.tsx: reorder ThemeProvider import before AuthProvider per PROVIDER_ORDER.md;
  indent Routes inside AuthProvider to match expected tree structure
- Tasks.md: update task status
- providerTreeOrder.test.tsx: add integration tests for provider nesting order

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 23:35:18 +02:00
fc57c83f79 refactor: split pagination logic from response models
- Extract pagination logic to separate util module
- Update response models to use new pagination util
- Fix pagination calculation edge cases

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 22:57:21 +02:00
b2747381ec Remove Issue #51 from Tasks.md (type-enforcement tracked elsewhere) 2026-05-03 22:47:24 +02:00
edebf1a339 feat(services): add ErrorContract enum and PartialResult type
Add typed wrappers for error handling patterns in error_handling.py:

- ErrorContract(enum): machine-checkable pattern selector with
  from_value() helper and string constants matching the existing
  ABORT_ON_ERROR/RETURN_DEFAULT/PARTIAL_RESULT module-level values
- ErrorEntry: typed error container for PARTIAL_RESULT (context + cause)
- PartialResult[T]: typed result wrapper for PARTIAL_RESULT operations

Existing string constants preserved for backward compat.
Updated module docstring with type annotation table and examples.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 22:46:47 +02:00
a2afec2d1e Remove completed Issue #50 navigation abort signal task
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 22:43:19 +02:00
52a70c3eea Add import-linter boundary to forbid routers importing app.dependencies
Issue #51: Enforce repository boundary at CI level using import-linter.
Contract 'forbid_router_db_import' checks that app.routers never imports
app.dependencies directly, keeping the DB access path through service
contexts only.

- Add import-linter>=2.0.0 to dev dependencies (backend/pyproject.toml)
- Configure [tool.importlinter] with package_root and root_packages
- Add [[tool.importlinter.contracts]] with type='forbidden', source
  app.routers, forbidden app.dependencies
- Add 'Import Boundary' CI job (import-linter)
- Document import-linter in CONTRIBUTING.md code quality table

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 22:42:46 +02:00
3376009903 fix(nav): move AbortController creation synchronously in render
Previously the AbortController was created inside useEffect, which runs
after render. This meant requests initiated during render received
the previous cycle's (possibly aborted) signal and were cancelled
immediately instead of completing.

Now the controller is created synchronously when pathname changes, before
any request can be initiated on the new route. The old controller is
aborted in the same conditional block, before the new one is created.

Side effect: removed resolved Issue #49 from Tasks.md.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 22:37:07 +02:00
7fcfc14199 fix(auth): dedupe handler + error utils refactor
- Add 401/403 dedup guard to API client to prevent double logout
- Extract fetchError util: isAuthError + getErrorMessage
- AuthProvider uses new error utils, removes duplicate logic
- Remove completed task docs from Tasks.md

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-03 22:13:12 +02:00