15 Commits

Author SHA1 Message Date
3af8f0571b feat: graceful shutdown and WAL cleanup
Some checks failed
CI / Backend Tests (push) Has been cancelled
CI / Lint (push) Has been cancelled
CI / Type Check (push) Has been cancelled
CI / Import Boundary (push) Has been cancelled
CI / OpenAPI Breaking Changes (push) Has been cancelled
CI / OpenAPI Baseline Commit (push) Has been cancelled
- Add stop_grace_period to backend container for graceful shutdown
- Document WAL mode rationale and orphaned file cleanup in db.py
- Handle database close errors gracefully in lifespan
- Clean up orphaned WAL files during startup before opening DB
- Reorder imports and fix formatting in startup.py
2026-05-24 22:05:34 +02:00
d5a78a251a Remove Tasks.md spec, add test for _cleanup_wal_files skipping recent files
Remove 335-line task specification from Docs/Tasks.md.
Add test confirming _cleanup_wal_files skips recently-modified WAL/SHM files.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-24 22:05:34 +02:00
904db63fa2 Add tests for since timestamp accuracy in ban_service
- test_since_unix_returns_utc_epoch: validates since_unix('24h') returns UTC epoch
- test_ban_trend_since_is_within_expected_range: validates 23h-ago ban falls in 24h+slack window

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-24 22:05:34 +02:00
d737a1c319 Add logging duplication tests
- test_logging_configuration_no_duplicate_handlers: verify create_app() twice leaves ≤1 StreamHandler
- test_uvicorn_access_logs_go_through_root_handler: verify uvicorn.access can emit JSON via JSONFormatter
- test_external_logging_processor_queues_record: verify _external_logging_processor queues to handler
- test_plain_text_logs_not_emitted_after_startup: verify app.db emits JSON not plain text

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-24 22:05:34 +02:00
9e765c6cb7 Add granular DB error types with retry logic
New exceptions: DatabaseBusyError, DatabasePermissionDeniedError,
DatabasePathInvalidError, DatabaseCorruptedError, DatabaseUnavailableError.

open_db creates parent directory if missing. Catches all aiosqlite errors
and maps to specific exception types.

get_db retries up to 3x on locked database with backoff.
Propagates specific exceptions instead of generic HTTPException.

Tests for all new error types and retry behavior.
2026-05-24 22:05:34 +02:00
ecb8542496 docs: add comprehensive task backlog and bump version to rc.5
- Document database error handling, logging duplication, ban service
timestamp, and orphaned SQLite file issues in Tasks.md
- Bump backend version from 0.9.19-rc.4 to 0.9.19-rc.5
2026-05-24 22:05:34 +02:00
97f4df4a61 chore: release v0.9.19-rc.5 2026-05-24 22:05:34 +02:00
44542b93c0 chore(release): bump version to 0.9.19-rc.4
- Add production Docker Compose configuration

- Add check_auth.py diagnostic script for session 401 debugging
2026-05-24 22:05:34 +02:00
01a4215f60 chore: release v0.9.19-rc.4 2026-05-24 22:05:34 +02:00
bc49b7cd5b fix(db): fix migration failures when upgrading from 0.8.0 schema
Migration 1: remove idx_sessions_token_hash from _SCHEMA_STATEMENTS.
The legacy schema has sessions.token (not token_hash). The IF NOT EXISTS
guard only prevents duplicate index names — it still requires the column
to exist. Migration 2 drops and rebuilds sessions with token_hash anyway,
so creating the index in migration 1 was redundant.

Migration 3: replace ALTER TABLE ADD COLUMN with a table rebuild.
SQLite rejects ALTER TABLE ADD COLUMN NOT NULL DEFAULT <expression> when
the table already contains rows. The old DB has ~181k geo_cache rows, so
the ALTER always failed. Rebuild copies existing rows with last_seen set
to cached_at as a reasonable approximation of last-seen time.
2026-05-24 22:05:34 +02:00
fa4fe4bbdf chore: release v0.9.19-rc.3 2026-05-24 22:05:34 +02:00
ee0fe9c695 fix(auth): suppress misleading 502 warning during session validation
A 502 Bad Gateway is a server/gateway error, not a network error.
Logging it as a 'Session validation network error' is noisy and
misleading during startup when nginx is temporarily unreachable.

Silently skip the console.warn for 5xx errors in handleValidationError
while keeping the warning for actual network errors.
2026-05-24 22:05:34 +02:00
551db0bb9c chore: release v0.9.19-rc.2 2026-05-24 22:05:34 +02:00
4a649e7347 chore: bump to v0.9.19-rc.1 and add local OpenAPI build support
- Add release candidate (rc) support to release.sh with latestRC tagging
- Bump VERSION, backend pyproject.toml, and frontend package.json to 0.9.19-rc.1
- Add local frontend/openapi.json so build no longer needs running backend
- Update generate:types and validate-types.sh to use local openapi.json
- Fix frontend tests: remove unused imports/variables and update mock data
2026-05-24 22:05:34 +02:00
025c82a982 Merge pull request 'refactoring-backend' (#3) from refactoring-backend into main
Some checks failed
CI / Backend Tests (push) Has been cancelled
CI / Lint (push) Has been cancelled
CI / Type Check (push) Has been cancelled
CI / Import Boundary (push) Has been cancelled
CI / OpenAPI Breaking Changes (push) Has been cancelled
CI / OpenAPI Baseline Commit (push) Has been cancelled
Reviewed-on: #3
2026-05-20 20:23:46 +02:00
4 changed files with 28 additions and 10 deletions

View File

@@ -48,6 +48,7 @@ services:
target: runtime target: runtime
container_name: bangui-backend container_name: bangui-backend
restart: unless-stopped restart: unless-stopped
stop_grace_period: 30s # Give lifespan 30s to complete before SIGKILL
depends_on: depends_on:
fail2ban: fail2ban:
condition: service_healthy condition: service_healthy

View File

@@ -274,7 +274,18 @@ CREATE INDEX IF NOT EXISTS idx_import_log_source_id_desc
async def _configure_connection(db: aiosqlite.Connection) -> None: async def _configure_connection(db: aiosqlite.Connection) -> None:
"""Apply hardening pragmas to a newly-opened SQLite connection.""" """Apply hardening pragmas to a newly-opened SQLite connection.
WAL mode is intentionally kept despite the risk of orphaned ``.wal``/``.shm``
files after unclean shutdowns. The benefits for concurrent readers
(readers do not block writers) outweigh the cleanup overhead, especially
under load. BanGUI runs as a single worker, but multiple concurrent HTTP
requests can still issue overlapping reads; DELETE mode would serialize
those reads behind any write, degrading API performance.
Orphaned files are handled by :func:`_cleanup_wal_files`, which is called
during startup before the database is opened.
"""
await db.execute("PRAGMA journal_mode=WAL;") await db.execute("PRAGMA journal_mode=WAL;")
await db.execute("PRAGMA foreign_keys=ON;") await db.execute("PRAGMA foreign_keys=ON;")
await db.execute("PRAGMA busy_timeout=5000;") await db.execute("PRAGMA busy_timeout=5000;")

View File

@@ -318,7 +318,12 @@ async def _lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
log.error("scheduler_lock_release_failed", error=str(e)) log.error("scheduler_lock_release_failed", error=str(e))
# 6. Close the database connection. # 6. Close the database connection.
await startup_db.close() try:
await startup_db.close()
log.debug("database_connection_closed")
except Exception as exc:
log.error("database_connection_close_failed", error=str(exc))
log.info("bangui_shut_down") log.info("bangui_shut_down")

View File

@@ -26,10 +26,9 @@ from pathlib import Path
from typing import TYPE_CHECKING, Any from typing import TYPE_CHECKING, Any
import aiohttp import aiohttp
from app.utils.logging_compat import get_logger
from apscheduler.schedulers.asyncio import AsyncIOScheduler # type: ignore[import-untyped] from apscheduler.schedulers.asyncio import AsyncIOScheduler # type: ignore[import-untyped]
from app.db import init_db, open_db from app.db import _cleanup_wal_files, init_db, open_db
from app.services import setup_service from app.services import setup_service
from app.services.dns_validated_connector import create_dns_validated_socket_factory from app.services.dns_validated_connector import create_dns_validated_socket_factory
from app.services.geo_cache import GeoCache from app.services.geo_cache import GeoCache
@@ -48,6 +47,7 @@ from app.tasks import (
from app.utils.async_utils import run_blocking from app.utils.async_utils import run_blocking
from app.utils.fail2ban_db_utils import ensure_fail2ban_indexes from app.utils.fail2ban_db_utils import ensure_fail2ban_indexes
from app.utils.jail_config import ensure_jail_configs from app.utils.jail_config import ensure_jail_configs
from app.utils.logging_compat import get_logger
from app.utils.runtime_state import set_runtime_settings from app.utils.runtime_state import set_runtime_settings
from app.utils.scheduler_lock import ( from app.utils.scheduler_lock import (
acquire_scheduler_lock, acquire_scheduler_lock,
@@ -98,9 +98,7 @@ def _check_single_worker_mode() -> None:
"See Docs/Architekture.md § Deployment Constraints for details." "See Docs/Architekture.md § Deployment Constraints for details."
) )
except ValueError as e: except ValueError as e:
raise RuntimeError( raise RuntimeError(f"BANGUI_WORKERS environment variable must be an integer, got: {workers_env}") from e
f"BANGUI_WORKERS environment variable must be an integer, got: {workers_env}"
) from e
async def _ensure_database_schema(database_path: str) -> None: async def _ensure_database_schema(database_path: str) -> None:
@@ -333,6 +331,11 @@ async def _stage_init_database(app: FastAPI, settings: Settings) -> Any:
log.debug("database_directory_ensured", directory=str(db_path.parent)) log.debug("database_directory_ensured", directory=str(db_path.parent))
# Clean up orphaned WAL files from previous unclean shutdowns before
# opening the database. This prevents stale .wal/.shm files from
# interfering with startup or triggering misleading warnings.
await _cleanup_wal_files(settings.database_path)
original_db_path = db_path.resolve() original_db_path = db_path.resolve()
startup_db = await open_db(settings.database_path) startup_db = await open_db(settings.database_path)
@@ -357,9 +360,7 @@ async def _stage_init_database(app: FastAPI, settings: Settings) -> Any:
if f2b_db_path: if f2b_db_path:
await run_blocking(ensure_fail2ban_indexes, f2b_db_path) await run_blocking(ensure_fail2ban_indexes, f2b_db_path)
persisted_runtime_settings = ( persisted_runtime_settings = await setup_service.get_persisted_runtime_settings(runtime_db)
await setup_service.get_persisted_runtime_settings(runtime_db)
)
finally: finally:
await runtime_db.close() await runtime_db.close()