diff --git a/Docs/API-Reference.md b/Docs/API-Reference.md new file mode 100644 index 0000000..e1c8539 --- /dev/null +++ b/Docs/API-Reference.md @@ -0,0 +1,1338 @@ +# BanGUI API Reference + +Complete reference for the BanGUI REST API. All endpoints require authentication unless noted as public. + +Base URL: `http://{host}:8000` + +**Authentication** — All protected endpoints require a valid session cookie (`bangui_session`) or `Authorization: Bearer ` header. + +--- + +## Public Endpoints + +### `GET /api/v1/health` + +Health check. No auth required. + +**Response `200`** +```json +{ + "status": "ok", + "fail2ban": "online", + "database": "ok", + "scheduler": "running", + "cache": "initialised", + "components": [] +} +``` + +| Field | Description | +|---|---| +| `status` | `ok`, `degraded`, or `unavailable` | +| `fail2ban` | `online` or `offline` | +| `database` | `ok` or `error` | +| `scheduler` | `running`, `stopped`, or `unknown` | +| `cache` | `initialised` or `uninitialised` | +| `components` | List of unhealthy components (empty when `ok`) | + +**Response `503**` — fail2ban offline. + +--- + +### `GET /api/v1/setup` + +Check whether initial setup has been completed. + +**Response `200`** +```json +{ "completed": true } +``` + +--- + +### `POST /api/v1/setup` + +Run the first-run setup wizard. + +**Request** +```json +{ + "master_password": "Hallo123!", + "database_path": "/var/lib/fail2ban/fail2ban.sqlite3", + "fail2ban_socket": "/var/run/fail2ban/fail2ban.sock", + "timezone": "Europe/Berlin", + "session_duration_minutes": 480 +} +``` + +| Field | Type | Required | Description | +|---|---|---|---| +| `master_password` | string | Yes | Min 8 chars, uppercase + number + special (`!@#$%^&*()`) | +| `database_path` | string | No | Path to fail2ban DB (default: `/var/lib/fail2ban/fail2ban.sqlite3`) | +| `fail2ban_socket` | string | No | Path to fail2ban socket (default: `/var/run/fail2ban/fail2ban.sock`) | +| `timezone` | string | No | IANA timezone (default: `UTC`) | +| `session_duration_minutes` | int | No | Session TTL in minutes (default: `480`) | + +**Response `201`** — Setup completed. + +**Response `409`** — Setup already completed. + +--- + +### `GET /api/v1/setup/timezone` + +Returns the configured IANA timezone. + +**Response `200`** +```json +{ "timezone": "Europe/Berlin" } +``` + +--- + +### `GET /metrics` + +Prometheus metrics endpoint. No auth required. Returns OpenMetrics text format. + +--- + +## Auth + +### `POST /api/v1/auth/login` + +Authenticate with the master password. + +**Request** +```json +{ "password": "Hallo123!" } +``` + +> Note: The frontend SHA256-hashes the password before sending. The backend expects the already-hashed value. + +**Response `200`** — Sets `bangui_session` cookie. +```json +{ "expires_at": "2024-12-25T10:00:00Z" } +``` + +**Response `401`** — Invalid password. + +**Response `429`** — Too many login attempts (exponential backoff delay). Includes `Retry-After` header. + +**Response `503`** — Setup not complete. + +--- + +### `GET /api/v1/auth/session` + +Validate the current session. + +**Response `200`** +```json +{ "valid": true } +``` + +**Response `401`** — Session missing, expired, or invalid. + +--- + +### `POST /api/v1/auth/logout` + +Revoke the current session. + +**Response `200`** +```json +{} +``` + +Session cookie is cleared. Idempotent — returns `200` even if no session present. + +--- + +## Dashboard + +### `GET /api/v1/dashboard/status` + +Returns the cached fail2ban server health snapshot (refreshed every 30 seconds). + +**Response `200`** +```json +{ + "status": { + "version": "0.12.1", + "online": true, + "uptime": 86400, + "jail_count": 3 + } +} +``` + +| Field | Type | Description | +|---|---|---| +| `version` | string | fail2ban server version | +| `online` | bool | Whether fail2ban daemon is reachable | +| `uptime` | int | Daemon uptime in seconds | +| `jail_count` | int | Number of configured jails | + +**Response `401`** — Not authenticated. + +**Response `502`** — fail2ban unreachable. + +--- + +### `GET /api/v1/dashboard/bans` + +Paginated list of recent bans with geo enrichment. + +**Query Parameters** + +| Param | Type | Default | Description | +|---|---|---|---| +| `range` | `TimeRange` | `24h` | Time window: `24h`, `7d`, `30d`, `365d` | +| `source` | string | `fail2ban` | Data source: `fail2ban` or `archive` | +| `page` | int | `1` | 1-based page number | +| `page_size` | int | `25` | Items per page (max 500) | +| `origin` | string | null | Filter: `blocklist` or `selfblock` | + +**Response `200`** +```json +{ + "items": [ + { + "ip": "1.2.3.4", + "jail": "sshd", + "banned_at": "2024-12-25T08:00:00Z", + "expires_at": "2024-12-26T08:00:00Z", + "country": "US", + "asn": "AS15169", + "org": "Google LLC" + } + ], + "total": 150, + "page": 1, + "page_size": 25 +} +``` + +--- + +### `GET /api/v1/dashboard/bans/by-country` + +Ban counts aggregated by country. + +**Query Parameters** — Same as `/bans` plus optional `country_code` filter. + +**Response `200`** +```json +{ + "countries": { "US": 45, "CN": 32, "BR": 18 }, + "total": 150, + "items": [...] +} +``` + +--- + +### `GET /api/v1/dashboard/bans/trend` + +Ban counts grouped into time buckets for charts. + +**Bucket sizes:** +- `24h` → 1-hour buckets (24 total) +- `7d` → 6-hour buckets (28 total) +- `30d` → 1-day buckets (30 total) +- `365d` → 7-day buckets (~53 total) + +**Query Parameters** — Same as `/bans`. + +**Response `200`** +```json +{ + "buckets": [ + { "ts": "2024-12-24T00:00:00Z", "count": 12 }, + { "ts": "2024-12-24T01:00:00Z", "count": 8 } + ], + "bucket_size": "1h", + "total": 150 +} +``` + +--- + +### `GET /api/v1/dashboard/bans/by-jail` + +Ban counts grouped by jail. + +**Query Parameters** — Same as `/bans`. + +**Response `200`** +```json +{ + "jails": [ + { "jail": "sshd", "count": 120 }, + { "jail": "nginx-http-auth", "count": 30 } + ], + "total": 150 +} +``` + +--- + +## Bans + +### `GET /api/v1/bans/active` + +List all currently banned IPs across all jails. + +**Response `200`** +```json +{ + "items": [ + { + "ip": "1.2.3.4", + "jail": "sshd", + "banned_at": "2024-12-25T08:00:00Z", + "expires_at": "2024-12-26T08:00:00Z", + "country": "US" + } + ], + "total": 42 +} +``` + +**Response `401`** — Not authenticated. + +**Response `502`** — fail2ban unreachable. + +--- + +### `POST /api/v1/bans` + +Ban an IP address in a specific jail. + +**Request** +```json +{ "jail": "sshd", "ip": "5.6.7.8" } +``` + +**Response `201`** +```json +{ "message": "IP '5.6.7.8' banned in jail 'sshd'.", "jail": "sshd" } +``` + +**Response `400`** — Invalid IP address. + +**Response `404`** — Jail not found. + +**Response `409`** — Ban command failed in fail2ban. + +**Response `429`** — Rate limit exceeded (10 ban requests/minute per IP). + +**Response `502`** — fail2ban unreachable. + +--- + +### `DELETE /api/v1/bans` + +Unban an IP from one or all jails. + +**Request** +```json +{ "ip": "5.6.7.8", "jail": "sshd", "unban_all": false } +``` + +| Field | Type | Required | Description | +|---|---|---|---| +| `ip` | string | Yes | IP address to unban | +| `jail` | string | No | Specific jail to unban from | +| `unban_all` | bool | No | `true` = unban from all jails (default: `false` if `jail` omitted) | + +**Response `200`** +```json +{ "message": "IP '5.6.7.8' unbanned from jail 'sshd'.", "jail": "sshd" } +``` + +**Response `404`** — Jail not found. + +**Response `429`** — Rate limit exceeded (10 unban requests/minute per IP). + +--- + +### `DELETE /api/v1/bans/all` + +Unban every currently banned IP across all jails. + +**Response `200`** +```json +{ "message": "All bans cleared. 42 IP addresses unbanned.", "count": 42 } +``` + +--- + +## History + +### `GET /api/v1/history` + +Paginated historical ban records. + +**Query Parameters** + +| Param | Type | Default | Description | +|---|---|---|---| +| `range` | `TimeRange` | null | Time filter: `24h`, `7d`, `30d`, `365d` (null = all-time) | +| `jail` | string | null | Filter by jail name (exact match) | +| `ip` | string | null | Filter by IP prefix | +| `origin` | string | null | Filter: `blocklist` or `selfblock` | +| `source` | string | `fail2ban` | `fail2ban` or `archive` | +| `page` | int | `1` | 1-based page number | +| `page_size` | int | `25` | Items per page (max 500) | + +**Response `200`** +```json +{ + "items": [ + { + "ip": "1.2.3.4", + "jail": "sshd", + "banned_at": "2024-12-25T08:00:00Z", + "unbanned_at": "2024-12-26T08:00:00Z", + "origin": "selfblock", + "country": "US" + } + ], + "total": 500, + "page": 1, + "page_size": 25 +} +``` + +--- + +### `GET /api/v1/history/archive` + +Same as `/history` but reads from the archive database. + +**Query Parameters** — Same as `/history` (no `origin` filter). + +--- + +### `GET /api/v1/history/{ip}` + +Complete ban timeline for a single IP. + +**Response `200`** +```json +{ + "ip": "1.2.3.4", + "country": "US", + "total_bans": 5, + "timeline": [ + { + "jail": "sshd", + "banned_at": "2024-12-25T08:00:00Z", + "unbanned_at": "2024-12-26T08:00:00Z", + "origin": "selfblock" + } + ] +} +``` + +**Response `404`** — No history found for this IP. + +--- + +## Jails + +### `GET /api/v1/jails` + +List all active fail2ban jails. + +**Response `200`** +```json +{ + "items": [ + { + "name": "sshd", + "enabled": true, + "currently_banned": 12, + "total_bans": 150, + "failed_attempts": 320, + "find_time": 600, + "ban_time": 86400, + "max_retries": 5, + "backend": "polling", + "idle": false + } + ], + "total": 3 +} +``` + +--- + +### `GET /api/v1/jails/{name}` + +Full detail for a single jail. + +**Response `200`** +```json +{ + "name": "sshd", + "enabled": true, + "log_paths": ["/var/log/auth.log"], + "fail_regex": ["^%(__prefix_line)sFailed publickey forInvalid user"], + "ignore_regex": [], + "date_pattern": null, + "log_encoding": "UTF-8", + "actions": ["iptables"], + "find_time": 600, + "ban_time": 86400, + "max_retries": 5, + "ignore_list": ["192.168.1.1"], + "ignore_self": true, + "currently_banned": 12, + "total_bans": 150, + "failed_attempts": 320 +} +``` + +**Response `404`** — Jail not found. + +--- + +### `POST /api/v1/jails/{name}/start` + +Start a stopped jail. + +**Response `200`** +```json +{ "message": "Jail 'sshd' started.", "jail": "sshd" } +``` + +--- + +### `POST /api/v1/jails/{name}/stop` + +Stop a running jail. + +**Response `200`** +```json +{ "message": "Jail 'sshd' stopped.", "jail": "sshd" } +``` + +--- + +### `POST /api/v1/jails/{name}/idle` + +Toggle jail idle mode. + +**Request body** +```json +{ "on": true } +``` + +**Response `200`** +```json +{ "message": "Jail 'sshd' idle mode turned on.", "jail": "sshd" } +``` + +--- + +### `POST /api/v1/jails/{name}/reload` + +Reload a single jail. + +**Response `200`** +```json +{ "message": "Jail 'sshd' reloaded.", "jail": "sshd" } +``` + +--- + +### `POST /api/v1/jails/reload-all` + +Reload all fail2ban jails. + +**Response `200`** +```json +{ "message": "All jails reloaded successfully.", "jail": "*" } +``` + +--- + +### `GET /api/v1/jails/{name}/ignoreip` + +Get the ignore (whitelist) list for a jail. + +**Response `200`** +```json +{ "items": ["192.168.1.0/24", "10.0.0.1"], "total": 2 } +``` + +--- + +### `POST /api/v1/jails/{name}/ignoreip` + +Add an IP or CIDR to the ignore list. + +**Request** +```json +{ "ip": "192.168.1.100" } +``` + +**Response `201`** +```json +{ "message": "IP '192.168.1.100' added to ignore list of jail 'sshd'.", "jail": "sshd" } +``` + +**Response `400`** — Invalid IP or network. + +--- + +### `DELETE /api/v1/jails/{name}/ignoreip` + +Remove an IP or CIDR from the ignore list. + +**Request** +```json +{ "ip": "192.168.1.100" } +``` + +**Response `200`** +```json +{ "message": "IP '192.168.1.100' removed from ignore list of jail 'sshd'.", "jail": "sshd" } +``` + +--- + +### `POST /api/v1/jails/{name}/ignoreself` + +Toggle the `ignoreself` option (ban server's own IP). + +**Request** +```json +{ "on": true } +``` + +**Response `200`** +```json +{ "message": "ignoreself enabled for jail 'sshd'.", "jail": "sshd" } +``` + +--- + +### `GET /api/v1/jails/{name}/banned` + +Paginated currently-banned IPs for a specific jail. + +**Query Parameters** + +| Param | Type | Default | Description | +|---|---|---|---| +| `page` | int | `1` | 1-based page number | +| `page_size` | int | `25` | Items per page (max 100) | +| `search` | string | null | Case-insensitive substring filter on IP | + +**Response `200`** +```json +{ + "items": [ + { + "ip": "1.2.3.4", + "banned_at": "2024-12-25T08:00:00Z", + "expires_at": "2024-12-26T08:00:00Z", + "country": "US", + "asn": "AS15169", + "org": "Google LLC" + } + ], + "total": 12, + "page": 1, + "page_size": 25 +} +``` + +--- + +## Config + +### `GET /api/v1/config/global` + +Get global fail2ban settings. + +**Response `200`** +```json +{ + "loglevel": "INFO", + "logtarget": "/var/log/fail2ban.log", + "syslog_socket": "auto", + "db_file": "/var/lib/fail2ban/fail2ban.sqlite3", + "db_purge_age": 86400 +} +``` + +--- + +### `PUT /api/v1/config/global` + +Update global fail2ban settings. + +**Request** — All fields optional (only non-null fields written): +```json +{ + "loglevel": "DEBUG", + "logtarget": "/var/log/fail2ban.log", + "db_purge_age": 604800 +} +``` + +| Field | Type | Description | +|---|---|---| +| `loglevel` | string | `CRITICAL`, `ERROR`, `WARNING`, `NOTICE`, `INFO`, `DEBUG` | +| `logtarget` | string | `STDOUT`, `STDERR`, `SYSLOG`, or a file path | +| `db_purge_age` | int | Seconds before old ban records are purged | + +**Response `204`** — Updated. + +**Response `400`** — `logtarget` invalid (not in allowed directories). + +**Response `429`** — Rate limit exceeded (10 updates/minute per IP). + +--- + +### `POST /api/v1/config/reload` + +Trigger a full fail2ban reload. + +**Response `204`** + +--- + +### `POST /api/v1/config/restart` + +Restart the fail2ban service (stop + start). + +**Response `204`** + +**Response `503`** — fail2ban did not come back online within 10 seconds. + +--- + +### `POST /api/v1/config/regex-test` + +Test a fail regex pattern against a sample log line (stateless, no fail2ban call). + +**Request** +```json +{ "pattern": "^%(__prefix_line)sFailed publickey", "sample": "Dec 25 08:00:01 server sshd[123]: Failed publickey for user admin from 1.2.3.4" } +``` + +**Response `200`** +```json +{ "matched": true, "groups": ["Dec 25 08:00:01", "server", "123", "admin", "1.2.3.4"] } +``` + +--- + +### `POST /api/v1/config/preview-log` + +Read a log file and test a regex against each line. + +**Request** +```json +{ + "path": "/var/log/auth.log", + "pattern": "^Failed publickey", + "lines": 50 +} +``` + +**Response `200`** +```json +{ + "lines": [ + { "line": "Dec 25 08:00:01 server sshd[123]: Failed publickey...", "matched": true, "groups": [...] }, + { "line": "Dec 25 08:00:02 server sshd[456]: Accepted publickey...", "matched": false } + ] +} +``` + +--- + +### `GET /api/v1/config/map-color-thresholds` + +Get map color threshold configuration. + +**Response `200`** +```json +{ + "thresholds": [ + { "count": 0, "color": "#4ade80" }, + { "count": 10, "color": "#facc15" }, + { "count": 50, "color": "#f97316" }, + { "count": 200, "color": "#ef4444" } + ] +} +``` + +--- + +### `PUT /api/v1/config/map-color-thresholds` + +Update map color thresholds. + +**Request** +```json +{ + "thresholds": [ + { "count": 0, "color": "#4ade80" }, + { "count": 100, "color": "#facc15" } + ] +} +``` + +> Thresholds must be strictly ascending by `count`. + +**Response `200`** — Updated thresholds. + +**Response `400`** — Thresholds not properly ordered. + +--- + +### `GET /api/v1/config/fail2ban-log` + +Read the tail of the fail2ban daemon log file. + +**Query Parameters** + +| Param | Type | Default | Description | +|---|---|---|---| +| `lines` | int | `200` | Number of tail lines (1–2000) | +| `filter` | string | null | Plain-text substring filter | + +**Response `200`** +```json +{ + "lines": ["2024-12-25 08:00:01,000 INFO ...", "2024-12-25 08:00:02,000 WARNING ..."], + "count": 2 +} +``` + +--- + +### `GET /api/v1/config/service-status` + +Fail2ban service health with log configuration. + +**Response `200`** +```json +{ + "online": true, + "version": "0.12.1", + "loglevel": "INFO", + "logtarget": "/var/log/fail2ban.log" +} +``` + +--- + +## Filters + +### `GET /api/v1/config/filters` + +List all available filters with active/inactive status. + +**Response `200`** +```json +{ + "items": [ + { + "name": "sshd", + "active": true, + "used_by_jails": ["sshd"], + "source_file": "/etc/fail2ban/filter.d/sshd.conf", + "has_local_override": false, + "failregex": ["^%(__prefix_line)sFailed publickey"], + "ignoreregex": [], + "date_pattern": null, + "journalmatch": null + } + ], + "total": 12 +} +``` + +Active filters (used by running jails) are listed first, sorted alphabetically. Inactive filters follow. + +--- + +### `GET /api/v1/config/filters/{name}` + +Full detail for a single filter. + +**Response `200`** — FilterConfig object (same shape as list item). + +**Response `404`** — Filter not found in `filter.d/`. + +--- + +### `POST /api/v1/config/filters` + +Create a new user-defined filter. + +**Request** +```json +{ + "name": "nginx-404", + "failregex": ["^\\s*\\S+ \\S+ \\S+ GET /nonexistent"], + "ignoreregex": null, + "date_pattern": null, + "journalmatch": null +} +``` + +**Response `201`** — Created FilterConfig object. + +**Response `409`** — Filter with this name already exists. + +**Response `422`** — Regex failed to compile. + +**Response `429`** — Rate limit exceeded (5 creates/minute per IP). + +--- + +### `PUT /api/v1/config/filters/{name}` + +Update a filter's `.local` override. Only non-null fields are written. + +**Request** +```json +{ + "failregex": ["^new pattern here"], + "ignoreregex": null +} +``` + +**Query Parameter** — `reload` (bool, default `false`) — trigger fail2ban reload after writing. + +**Response `200`** — Updated FilterConfig object. + +**Response `422`** — Regex failed to compile. + +**Response `429`** — Rate limit exceeded (10 updates/minute per IP). + +--- + +### `DELETE /api/v1/config/filters/{name}` + +Delete a user-created filter's `.local` file. Shipped `.conf`-only filters cannot be deleted. + +**Response `204`** + +**Response `409`** — Filter is a shipped default (conf-only). + +--- + +## Actions + +### `GET /api/v1/config/actions` + +List all available actions with active/inactive status. + +**Response `200`** +```json +{ + "actions": [ + { + "name": "iptables", + "active": true, + "used_by_jails": ["sshd", "nginx-http-auth"], + "source_file": "/etc/fail2ban/action.d/iptables.conf", + "has_local_override": false, + "start_command": "iptables -N f2b-sshd...", + "stop_command": "iptables -X f2b-sshd...", + "check_command": "iptables -L f2b-sshd -n", + "ban_action": "iptables -I f2b-sshd...", + "unban_action": "iptables -D f2b-sshd..." + } + ], + "total": 8 +} +``` + +--- + +### `GET /api/v1/config/actions/{name}` + +Full detail for a single action. + +**Response `200`** — ActionConfig object (same shape as list item). + +--- + +### `POST /api/v1/config/actions` + +Create a new user-defined action. + +**Request** +```json +{ + "name": "my-custom-action", + "start_command": "echo 'starting'", + "stop_command": "echo 'stopping'", + "check_command": "echo 'checking'", + "ban_action": "echo 'banning'", + "unban_action": "echo 'unbanning'" +} +``` + +**Response `201`** — Created ActionConfig object. + +**Response `409`** — Action with this name already exists. + +--- + +### `PUT /api/v1/config/actions/{name}` + +Update an action's `.local` override. + +**Request** — All fields optional: +```json +{ "ban_action": "new ban command here" } +``` + +**Query Parameter** — `reload` (bool, default `false`). + +**Response `200`** — Updated ActionConfig object. + +--- + +### `DELETE /api/v1/config/actions/{name}` + +Delete a user-created action's `.local` file. + +**Response `204`** + +**Response `409`** — Action is a shipped default (conf-only). + +--- + +## Geo + +### `GET /api/v1/geo/lookup/{ip}` + +Ban status and geo info for an IP. + +**Response `200`** +```json +{ + "ip": "1.2.3.4", + "banned": true, + "jails": ["sshd", "nginx-http-auth"], + "country": "US", + "country_name": "United States", + "region": "North America", + "city": "Mountain View", + "isp": "Google LLC", + "asn": "AS15169", + "org": "Google LLC", + "last_ban": "2024-12-25T08:00:00Z", + "total_bans": 3 +} +``` + +**Response `400`** — Invalid IP address. + +--- + +### `GET /api/v1/geo/stats` + +Geo cache diagnostic counters. + +**Response `200`** +```json +{ + "total": 1500, + "resolved": 1480, + "failed": 20, + "cache_size": 1480 +} +``` + +--- + +### `POST /api/v1/geo/re-resolve` + +Re-resolve all IPs with failed geo lookups. + +**Response `200`** +```json +{ + "total": 20, + "resolved": 18, + "failed": 2 +} +``` + +--- + +## Blocklists + +### `GET /api/v1/blocklists` + +List all blocklist sources. + +**Response `200`** +```json +{ + "sources": [ + { + "id": 1, + "name": "Country Block List", + "url": "https://example.com/blocklist.txt", + "enabled": true, + "last_import_at": "2024-12-25T08:00:00Z", + "last_import_succeeded": true, + "last_import_ban_count": 45 + } + ], + "total": 1 +} +``` + +--- + +### `POST /api/v1/blocklists` + +Add a new blocklist source. + +**Request** +```json +{ "name": "Spamhaus DROP", "url": "https://www.spamhaus.org/drop/drop.txt", "enabled": true } +``` + +**Response `201`** — Created BlocklistSource object. + +**Response `400`** — URL validation failed. + +--- + +### `GET /api/v1/blocklists/{source_id}` + +Get a single blocklist source. + +--- + +### `PUT /api/v1/blocklists/{source_id}` + +Update a blocklist source. + +**Request** — All fields optional: +```json +{ "name": "New Name", "enabled": false } +``` + +--- + +### `DELETE /api/v1/blocklists/{source_id}` + +Delete a blocklist source. + +**Response `204`** + +--- + +### `POST /api/v1/blocklists/import` + +Trigger an immediate import of all enabled blocklist sources. + +**Response `200`** +```json +{ + "started_at": "2024-12-25T10:00:00Z", + "sources": [ + { + "id": 1, + "name": "Spamhaus DROP", + "url": "https://www.spamhaus.org/drop/drop.txt", + "imported": 45, + "skipped": 3, + "failed": false, + "error": null + } + ], + "total_imported": 45, + "total_skipped": 3, + "total_failed": 0 +} +``` + +**Response `429`** — Rate limit exceeded (1 import/hour per IP). + +--- + +### `GET /api/v1/blocklists/schedule` + +Get the current import schedule. + +**Response `200`** +```json +{ + "enabled": true, + "interval_hours": 24, + "next_run_at": "2024-12-26T08:00:00Z" +} +``` + +--- + +### `PUT /api/v1/blocklists/schedule` + +Update the import schedule. + +**Request** +```json +{ "enabled": true, "interval_hours": 12 } +``` + +**Response `200`** — Updated ScheduleInfo. + +--- + +### `GET /api/v1/blocklists/log` + +Paginated import log. + +**Query Parameters** + +| Param | Type | Default | Description | +|---|---|---|---| +| `source_id` | int | null | Filter by source | +| `page` | int | `1` | 1-based page | +| `page_size` | int | `25` | Items per page (max 500) | + +**Response `200`** +```json +{ + "items": [ + { + "id": 1, + "source_id": 1, + "source_name": "Spamhaus DROP", + "started_at": "2024-12-25T08:00:00Z", + "completed_at": "2024-12-25T08:01:23Z", + "imported": 45, + "skipped": 3, + "failed": false, + "error": null + } + ], + "total": 50, + "page": 1, + "page_size": 25 +} +``` + +--- + +### `GET /api/v1/blocklists/{source_id}/preview` + +Preview the contents of a blocklist source (downloads and samples first ~20 lines). + +**Response `200`** +```json +{ + "url": "https://example.com/blocklist.txt", + "validated_lines": ["1.2.3.4", "5.6.7.8"], + "invalid_lines": ["not-an-ip"], + "total_valid": 2, + "total_invalid": 1 +} +``` + +**Response `502`** — URL could not be reached. + +--- + +## Server + +### `GET /api/v1/server/settings` + +Get fail2ban server-level settings. + +**Response `200`** +```json +{ + "loglevel": "INFO", + "logtarget": "/var/log/fail2ban.log", + "syslog_socket": "auto", + "db_file": "/var/lib/fail2ban/fail2ban.sqlite3", + "db_purge_age": 86400, + "max_matches": 100 +} +``` + +--- + +### `PUT /api/v1/server/settings` + +Update fail2ban server-level settings. + +**Request** — All fields optional: +```json +{ "loglevel": "DEBUG", "max_matches": 200 } +``` + +**Response `204`** + +**Response `400`** — fail2ban rejected a setting. + +--- + +### `POST /api/v1/server/flush-logs` + +Flush and re-open fail2ban log files (after log rotation). + +**Response `200`** +```json +{ "message": "Success: 1 log(s) flushed" } +``` + +--- + +## Common Types + +### `TimeRange` + +``` +"24h" | "7d" | "30d" | "365d" +``` + +### `BanOrigin` + +``` +"blocklist" | "selfblock" +``` + +### `Source` + +``` +"fail2ban" | "archive" +``` + +--- + +## Status Codes + +| Code | Meaning | +|---|---| +| `200` | OK | +| `201` | Created | +| `204` | No Content | +| `400` | Bad Request — invalid input | +| `401` | Unauthorized — session missing, expired, or invalid | +| `404` | Not Found | +| `409` | Conflict | +| `422` | Unprocessable Entity — validation failed | +| `429` | Too Many Requests — rate limit exceeded | +| `502` | Bad Gateway — fail2ban unreachable | +| `503` | Service Unavailable | diff --git a/Docs/DATABASE_SCHEMA.md b/Docs/DATABASE_SCHEMA.md new file mode 100644 index 0000000..9f73d89 --- /dev/null +++ b/Docs/DATABASE_SCHEMA.md @@ -0,0 +1,347 @@ +# Database Schema Documentation + +BanGUI uses two SQLite databases: + +| Database | Purpose | Location | +|---|---|---| +| **BanGUI app DB** | Own configuration, sessions, blocklist sources, import logs, geo cache | `bangui.db` | +| **fail2ban DB** | fail2ban's internal ban/jail data (read-only) | Configured via `FAIL2BAN_DB` env var | + +--- + +## 1. BanGUI Application Schema + +Single source of truth: `backend/app/db.py`. + +### 1.1 `settings` + +Key-value store for application configuration. + +| Column | Type | Constraints | +|---|---|---| +| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT | +| `key` | TEXT | NOT NULL UNIQUE | +| `value` | TEXT | NOT NULL | +| `created_at` | TEXT | NOT NULL DEFAULT ISO 8601 | +| `updated_at` | TEXT | NOT NULL DEFAULT ISO 8601 | + +**Indexes:** PK only. + +**Purpose:** Stores app-wide settings (e.g., timezone, UI preferences). All settings access goes through `settings_repo` / `settings_service`. + +--- + +### 1.2 `sessions` + +Session tokens for web authentication. + +| Column | Type | Constraints | +|---|---|---| +| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT | +| `token_hash` | TEXT | NOT NULL UNIQUE | +| `created_at` | TEXT | NOT NULL DEFAULT ISO 8601 | +| `expires_at` | TEXT | NOT NULL | + +**Indexes:** `idx_sessions_token_hash` (UNIQUE) on `token_hash`. + +**Purpose:** Web session management. Tokens are SHA-256 hashed before storage. Sessions expire and are cleaned up by `session_cleanup` task. See `auth_service.py`. + +--- + +### 1.3 `blocklist_sources` + +Blocklist source definitions for the import pipeline. + +| Column | Type | Constraints | +|---|---|---| +| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT | +| `name` | TEXT | NOT NULL | +| `url` | TEXT | NOT NULL UNIQUE | +| `enabled` | INTEGER | NOT NULL DEFAULT 1 (boolean) | +| `created_at` | TEXT | NOT NULL DEFAULT ISO 8601 | +| `updated_at` | TEXT | NOT NULL DEFAULT ISO 8601 | + +**Indexes:** PK only. + +**Purpose:** Defines sources for blocklist imports. See `blocklist_repo`, `blocklist_service`, `blocklist_import_workflow`. + +--- + +### 1.4 `import_log` + +Audit log of individual blocklist import operations. + +| Column | Type | Constraints | +|---|---|---| +| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT | +| `source_id` | INTEGER | REFERENCES `blocklist_sources(id)` ON DELETE RESTRICT | +| `source_url` | TEXT | NOT NULL | +| `timestamp` | INTEGER | NOT NULL (UNIX epoch) | +| `ips_imported` | INTEGER | NOT NULL DEFAULT 0 | +| `ips_skipped` | INTEGER | NOT NULL DEFAULT 0 | +| `errors` | TEXT | | + +**Indexes:** +- `idx_import_log_id_desc` on `(id DESC)` — cursor pagination +- `idx_import_log_source_id_desc` on `(source_id, id DESC)` — filtered pagination + +**Purpose:** Audit trail for imports. `source_id` RESTRICT prevents source deletion when logs exist. See migration 9. + +**Migration 8:** `timestamp` migrated from TEXT ISO 8601 to INTEGER UNIX epoch. + +--- + +### 1.5 `geo_cache` + +Geo-IP lookup cache for ban IP metadata. + +| Column | Type | Constraints | +|---|---|---| +| `ip` | TEXT | PRIMARY KEY | +| `country_code` | TEXT | | +| `country_name` | TEXT | | +| `asn` | TEXT | | +| `org` | TEXT | | +| `cached_at` | TEXT | NOT NULL DEFAULT ISO 8601 | + +**Additional (migration 3):** +| Column | Type | Constraints | +|---|---|---| +| `last_seen` | TEXT | NOT NULL DEFAULT ISO 8601 | + +**Indexes:** PK only. + +**Purpose:** Caches GeoIP results to reduce third-party API calls. TTL managed by `geo_cache_cleanup` task. See `geo_cache_repo`, `geo_service`. + +--- + +### 1.6 `history_archive` + +Archived ban/unban history mirrored from fail2ban DB. + +| Column | Type | Constraints | +|---|---|---| +| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT | +| `jail` | TEXT | NOT NULL | +| `ip` | TEXT | NOT NULL | +| `timeofban` | INTEGER | NOT NULL (UNIX epoch) | +| `bancount` | INTEGER | NOT NULL | +| `data` | TEXT | NOT NULL (JSON) | +| `action` | TEXT | NOT NULL CHECK IN ('ban', 'unban') | +| `created_at` | TEXT | NOT NULL DEFAULT ISO 8601 | + +**Constraints:** `UNIQUE(ip, jail, action, timeofban)` prevents duplicate archive rows. + +**Indexes:** +- `idx_history_archive_jail_timeofban` on `(jail, timeofban DESC)` — dashboard filter by jail + time ordering +- `idx_history_archive_timeofban_jail_action` on `(timeofban DESC, jail, action)` — timeline filters +- `idx_history_archive_ip` on `(ip)` — IP prefix/exact searches +- `idx_history_archive_action` on `(action)` — ban/unban filtering + +**Purpose:** Long-term ban history. Synced from fail2ban DB by `history_sync` task. See `history_archive_repo`, `history_service`. + +--- + +### 1.7 `scheduler_lock` + +Database-backed mutex for multi-worker scheduler safety. + +| Column | Type | Constraints | +|---|---|---| +| `id` | INTEGER | PRIMARY KEY CHECK (id = 1) — singleton row | +| `pid` | INTEGER | NOT NULL | +| `hostname` | TEXT | NOT NULL | +| `created_at` | REAL | NOT NULL (UNIX epoch) | +| `heartbeat_at` | REAL | NOT NULL (UNIX epoch) | + +**Indexes:** PK only (singleton constraint). + +**Purpose:** Only one worker process holds the scheduler lock at a time. Lock is heartbeat-renewed by `scheduler_lock_heartbeat` task. Uses `BEGIN IMMEDIATE` transaction to acquire atomically. See `scheduler_lock.py`. + +--- + +### 1.8 `import_runs` + +Tracks unique blocklist imports for idempotent retries. + +| Column | Type | Constraints | +|---|---|---| +| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT | +| `source_id` | INTEGER | NOT NULL REFERENCES `blocklist_sources(id)` ON DELETE CASCADE | +| `content_hash` | TEXT | NOT NULL | +| `status` | TEXT | NOT NULL CHECK IN ('pending', 'completed', 'failed') | +| `imported_count` | INTEGER | NOT NULL DEFAULT 0 | +| `skipped_count` | INTEGER | NOT NULL DEFAULT 0 | +| `error_message` | TEXT | | +| `created_at` | TEXT | NOT NULL DEFAULT ISO 8601 | +| `updated_at` | TEXT | NOT NULL DEFAULT ISO 8601 | + +**Constraints:** `UNIQUE(source_id, content_hash)` — same source + content = same import run. + +**Indexes:** `idx_import_runs_source_status` on `(source_id, status)` — lookup completed imports by source. + +**Purpose:** Prevents duplicate IP bans on import crash/retry. See migration 6 and `blocklist_import_workflow`. + +--- + +### 1.9 `schema_migrations` + +Tracks applied schema versions. + +| Column | Type | Constraints | +|---|---|---| +| `version` | INTEGER | PRIMARY KEY | +| `migrated_at` | TEXT | NOT NULL DEFAULT ISO 8601 | + +**Indexes:** PK only. + +**Purpose:** Idempotent schema migration tracker. Records each applied version number. See `init_db()` and `_migrate_schema()`. + +--- + +## 2. Fail2ban Database Schema + +Read-only access via `fail2ban_db_repo`. Fail2ban manages this DB; BanGUI mirrors data into `history_archive`. + +### 2.1 `fail2banDb` + +| Column | Type | Constraints | +|---|---|---| +| `version` | INTEGER | | + +Single row tracking DB schema version. + +--- + +### 2.2 `jails` + +| Column | Type | Constraints | +|---|---|---| +| `name` | TEXT | NOT NULL UNIQUE | +| `enabled` | INTEGER | NOT NULL DEFAULT 1 | + +**Indexes:** `jails_name` on `(name)`. + +--- + +### 2.3 `logs` + +| Column | Type | Constraints | +|---|---|---| +| `jail` | TEXT | NOT NULL FK → `jails(name)` ON DELETE CASCADE | +| `path` | TEXT | | +| `firstlinemd5` | TEXT | | +| `lastfilepos` | INTEGER | DEFAULT 0 | +| `UNIQUE(jail, path)` | | | +| `UNIQUE(jail, path, firstlinemd5)` | | | + +**Indexes:** `logs_path` on `(path)`, `logs_jail_path` on `(jail, path)`. + +--- + +### 2.4 `bans` + +| Column | Type | Constraints | +|---|---|---| +| `jail` | TEXT | NOT NULL FK → `jails(name)` | +| `ip` | TEXT | | +| `timeofban` | INTEGER | NOT NULL | +| `bantime` | INTEGER | NOT NULL | +| `bancount` | INTEGER | NOT NULL DEFAULT 1 | +| `data` | JSON | | + +**Indexes:** +- `bans_jail_timeofban_ip` on `(jail, timeofban)` +- `bans_jail_ip` on `(jail, ip)` +- `bans_ip` on `(ip)` + +--- + +### 2.5 `bips` + +Backup IPs table (ban backup). + +| Column | Type | Constraints | +|---|---|---| +| `ip` | TEXT | NOT NULL | +| `jail` | TEXT | NOT NULL FK → `jails(name)` | +| `timeofban` | INTEGER | NOT NULL | +| `bantime` | INTEGER | NOT NULL | +| `bancount` | INTEGER | NOT NULL DEFAULT 1 | +| `data` | JSON | | +| PRIMARY KEY | `(ip, jail)` | | + +**Indexes:** `bips_timeofban` on `(timeofban)`, `bips_ip` on `(ip)`. + +--- + +## 3. Relationships and Constraints + +``` +blocklist_sources (1) ──(id)──→ import_log.source_id [RESTRICT on delete] + └──→ import_runs.source_id [CASCADE on delete] + +settings: standalone (key-value, no FK) +sessions: standalone (token hash, no FK) +geo_cache: standalone (IP → geo data, no FK) +history_archive: standalone (archived ban history, no FK) +scheduler_lock: singleton row (id=1), no FK +schema_migrations: standalone (migration tracking, no FK) +``` + +Fail2ban tables are separate and read-only from BanGUI's perspective. + +--- + +## 4. Indexes Summary + +| Table | Index | Columns | +|---|---|---| +| `sessions` | `idx_sessions_token_hash` | `token_hash` UNIQUE | +| `import_log` | `idx_import_log_id_desc` | `id DESC` | +| `import_log` | `idx_import_log_source_id_desc` | `source_id, id DESC` | +| `import_runs` | `idx_import_runs_source_status` | `source_id, status` | +| `history_archive` | `idx_history_archive_jail_timeofban` | `jail, timeofban DESC` | +| `history_archive` | `idx_history_archive_timeofban_jail_action` | `timeofban DESC, jail, action` | +| `history_archive` | `idx_history_archive_ip` | `ip` | +| `history_archive` | `idx_history_archive_action` | `action` | +| `jails` | `jails_name` | `name` | +| `logs` | `logs_path` | `path` | +| `logs` | `logs_jail_path` | `jail, path` | +| `bans` | `bans_jail_timeofban_ip` | `jail, timeofban` | +| `bans` | `bans_jail_ip` | `jail, ip` | +| `bans` | `bans_ip` | `ip` | +| `bips` | `bips_timeofban` | `timeofban` | +| `bips` | `bips_ip` | `ip` | + +--- + +## 5. Migration History + +| Version | Description | +|---|---| +| 1 | Initial schema: `settings`, `sessions`, `blocklist_sources`, `import_log`, `geo_cache`, `history_archive`, `schema_migrations` | +| 2 | Hash session tokens (`token_hash` column). Invalidates all existing sessions. | +| 3 | Add `last_seen` to `geo_cache` for retention policy. | +| 4 | Add `scheduler_lock` table for multi-worker scheduler mutex. | +| 5 | Add indexes to `history_archive` for query performance (4 indexes). | +| 6 | Add `import_runs` table for idempotent import tracking. | +| 7 | Add indexes to `import_log` for cursor-based pagination. | +| 8 | Migrate `import_log.timestamp` from TEXT ISO 8601 → INTEGER UNIX epoch. | +| 9 | Change `import_log.source_id` FK to `ON DELETE RESTRICT` (prevents orphaned logs). Recreate table with new FK semantics. | + +**Current schema version:** 9 (`_CURRENT_SCHEMA_VERSION` in `db.py`). + +--- + +## 6. Performance Notes + +- **WAL mode** (`PRAGMA journal_mode=WAL`) — concurrent reads allowed, better write performance under concurrency. +- **Foreign keys enforced** (`PRAGMA foreign_keys=ON`) — data integrity at DB level. +- **Busy timeout** 5000 ms — prevents "database is locked" errors under contention. +- **`history_archive` indexes** — tuned for dashboard filter + time ordering + pagination. See migration 5 and `PERFORMANCE.md`. +- **`import_log` indexes** — tuned for cursor-based pagination (newest-first by id). See migration 7. +- **`geo_cache` PK on `ip`** — O(1) lookup for geo enrichment on ban events. +- **`scheduler_lock` singleton** (`CHECK (id = 1)`) — trivial lock existence check. + +For detailed query patterns and benchmarks, see `Docs/PERFORMANCE.md`. diff --git a/Docs/Tasks.md b/Docs/Tasks.md index b20f337..e8824ab 100644 --- a/Docs/Tasks.md +++ b/Docs/Tasks.md @@ -1,18 +1,3 @@ -### Issue #40: DOCUMENTATION - Missing Deployment Best Practices - -**Files affected**: `Docs/Deployment.md`, Docker configuration - -**Create/Update**: -- Security best practices -- Performance tuning -- Monitoring setup -- Scaling guidelines - -**References**: -- DATABASE_API_DEPLOYMENT_ISSUES.md - Issue "6 Build & Deployment" - ---- - ### Issue #41: DOCUMENTATION - Missing Database Schema Documentation **Create**: Document: