Refactor config loading and add status code docs

- Move config loading to dedicated ConfigLoader class with validation
- Add DATABASE_MIGRATIONS.md content to TROUBLESHOOTING.md
- Add API_STATUS_CODES.md documenting all API response codes
- Update runner.csx to use new config structure
- Add check_responses.py validation script
- Update config tests for new structure

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Docs/API_STATUS_CODES.md

@@ -0,0 +1,730 @@
+# API Status Codes Reference
+
+Complete reference of all HTTP status codes returned by the BanGUI API v1.
+Use this document to handle every possible response from every endpoint.
+
+---
+
+## Status Code Taxonomy
+
+| Code | Meaning | When Used |
+|------|---------|-----------|
+| **200** | OK | Successful GET, PUT, POST (no creation) |
+| **201** | Created | Successful POST that created a resource |
+| **204** | No Content | Successful DELETE or PUT with no response body |
+| **400** | Bad Request | Invalid input, validation failure, bad IP, URL validation |
+| **401** | Unauthorized | Missing, expired, or invalid session |
+| **404** | Not Found | Entity does not exist |
+| **409** | Conflict | State conflict (already exists, already done, operation failed) |
+| **422** | Unprocessable Entity | Request body validation failed (Pydantic) |
+| **429** | Too Many Requests | Rate limit exceeded |
+| **500** | Internal Server Error | Unexpected server failure |
+| **502** | Bad Gateway | fail2ban socket unreachable |
+| **503** | Service Unavailable | Setup incomplete or component degraded |
+
+---
+
+## /api/v1/auth
+
+### POST /api/v1/auth/login
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Login successful | `LoginResponse` |
+| 401 | Invalid password | Error body |
+| 422 | Validation error — invalid request body | Error body |
+| 429 | Too many login attempts, retry after delay | Error body |
+| 503 | Setup not complete | Error body |
+
+### GET /api/v1/auth/session
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Session valid | `SessionValidResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+
+### POST /api/v1/auth/logout
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Logout successful | `LogoutResponse` |
+| 401 | Session missing or invalid (silently successful) | Error body |
+
+---
+
+## /api/v1/setup
+
+### GET /api/v1/setup
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Setup status returned | `SetupStatusResponse` |
+
+### POST /api/v1/setup
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 201 | Setup completed successfully | `SetupResponse` |
+| 400 | Validation error in request body | Error body |
+| 409 | Setup already completed | Error body |
+
+### GET /api/v1/setup/timezone
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Timezone returned | `SetupTimezoneResponse` |
+
+---
+
+## /api/v1/health
+
+### GET /api/v1/health
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | All components healthy | `HealthResponse` |
+| 503 | fail2ban offline or component degraded | `HealthResponse` |
+
+---
+
+## /api/v1/dashboard
+
+### GET /api/v1/dashboard/status
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Server status returned | `ServerStatusResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### GET /api/v1/dashboard/bans
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Ban list returned | `DashboardBanListResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### GET /api/v1/dashboard/bans/by-country
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Ban counts by country returned | `BansByCountryResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### GET /api/v1/dashboard/bans/trend
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Ban trend data returned | `BanTrendResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### GET /api/v1/dashboard/bans/by-jail
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Ban counts by jail returned | `BansByJailResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+---
+
+## /api/v1/bans
+
+### GET /api/v1/bans/active
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Active ban list returned | `ActiveBanListResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### POST /api/v1/bans
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 201 | IP banned successfully | `JailCommandResponse` |
+| 400 | Invalid IP address | Error body |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | Jail not found | Error body |
+| 409 | Ban command failed in fail2ban | Error body |
+| 429 | Rate limit exceeded for ban operations | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### DELETE /api/v1/bans
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | IP unbanned successfully | `JailCommandResponse` |
+| 400 | Invalid IP address | Error body |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | Jail not found | Error body |
+| 409 | Unban command failed in fail2ban | Error body |
+| 429 | Rate limit exceeded for unban operations | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### DELETE /api/v1/bans/all
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | All bans cleared | `UnbanAllResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+---
+
+## /api/v1/jails
+
+### GET /api/v1/jails
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Jails list returned | `JailListResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### GET /api/v1/jails/{name}
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Jail detail returned | `JailDetailResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | Jail not found | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### POST /api/v1/jails/reload-all
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | All jails reloaded | `JailCommandResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 409 | fail2ban reports operation failed | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### POST /api/v1/jails/{name}/start
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Jail started | `JailCommandResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | Jail not found | Error body |
+| 409 | fail2ban reports operation failed | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### POST /api/v1/jails/{name}/stop
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Jail stopped | `JailCommandResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 409 | fail2ban reports operation failed | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### POST /api/v1/jails/{name}/idle
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Idle mode toggled | `JailCommandResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | Jail not found | Error body |
+| 409 | fail2ban reports operation failed | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### POST /api/v1/jails/{name}/reload
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Jail reloaded | `JailCommandResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | Jail not found | Error body |
+| 409 | fail2ban reports operation failed | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### GET /api/v1/jails/{name}/ignoreip
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Ignore list returned | `IgnoreListResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | Jail not found | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### POST /api/v1/jails/{name}/ignoreip
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 201 | IP added to ignore list | `JailCommandResponse` |
+| 400 | IP or network invalid | Error body |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | Jail not found | Error body |
+| 409 | fail2ban reports operation failed | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### DELETE /api/v1/jails/{name}/ignoreip
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | IP removed from ignore list | `JailCommandResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | Jail not found | Error body |
+| 409 | fail2ban reports operation failed | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### POST /api/v1/jails/{name}/ignoreself
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | ignoreself toggled | `JailCommandResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | Jail not found | Error body |
+| 409 | fail2ban reports operation failed | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### GET /api/v1/jails/{name}/banned
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Banned IPs returned | `JailBannedIpsResponse` |
+| 400 | page or page_size out of range | Error body |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | Jail not found | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+---
+
+## /api/v1/history
+
+### GET /api/v1/history
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | History list returned | `HistoryListResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### GET /api/v1/history/archive
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Archived history list returned | `HistoryListResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### GET /api/v1/history/{ip}
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | IP history detail returned | `IpDetailResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | No history found for this IP | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+---
+
+## /api/v1/geo
+
+### GET /api/v1/geo/lookup/{ip}
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | IP lookup result returned | `IpLookupResponse` |
+| 400 | Invalid IP address | Error body |
+| 401 | Session missing, expired, or invalid | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### GET /api/v1/geo/stats
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Geo cache stats returned | `GeoCacheStatsResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+
+### POST /api/v1/geo/re-resolve
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Re-resolve result | `GeoReResolveResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+
+---
+
+## /api/v1/server
+
+### GET /api/v1/server/settings
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Server settings returned | `ServerSettingsResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### PUT /api/v1/server/settings
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 204 | Settings updated successfully | No body |
+| 400 | Set command rejected by fail2ban | Error body |
+| 401 | Session missing, expired, or invalid | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### POST /api/v1/server/flush-logs
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Logs flushed successfully | `FlushLogsResponse` |
+| 400 | Command rejected by fail2ban | Error body |
+| 401 | Session missing, expired, or invalid | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+---
+
+## /api/v1/config
+
+### GET /api/v1/config/global
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Global config returned | `GlobalConfigResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### PUT /api/v1/config/global
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 204 | Global config updated successfully | No body |
+| 400 | Set command rejected or log_target invalid | Error body |
+| 401 | Session missing, expired, or invalid | Error body |
+| 429 | Rate limit exceeded for config update operations | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### POST /api/v1/config/reload
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 204 | Fail2ban reloaded successfully | No body |
+| 401 | Session missing, expired, or invalid | Error body |
+| 409 | Reload command failed in fail2ban | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### POST /api/v1/config/restart
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 204 | Fail2ban restarted successfully | No body |
+| 401 | Session missing, expired, or invalid | Error body |
+| 409 | Stop command failed in fail2ban | Error body |
+| 502 | fail2ban unreachable for stop command | Error body |
+| 503 | fail2ban did not come back online within 10s | Error body |
+
+### POST /api/v1/config/regex-test
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Regex test result | `RegexTestResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 422 | Invalid regex pattern | Error body |
+
+### POST /api/v1/config/preview-log
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Log preview result | `LogPreviewResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 422 | Invalid regex pattern | Error body |
+
+### GET /api/v1/config/map-color-thresholds
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Color thresholds returned | `MapColorThresholdsResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+
+### PUT /api/v1/config/map-color-thresholds
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Color thresholds updated | `MapColorThresholdsResponse` |
+| 400 | Validation error (thresholds not properly ordered) | Error body |
+| 401 | Session missing, expired, or invalid | Error body |
+| 429 | Rate limit exceeded for config update operations | Error body |
+
+### GET /api/v1/config/fail2ban-log
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Log file lines returned | `Fail2BanLogResponse` |
+| 400 | Log target not a file or path outside allowed directory | Error body |
+| 401 | Session missing, expired, or invalid | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### GET /api/v1/config/service-status
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Service status returned | `ServiceStatusResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+---
+
+## /api/v1/config/jails (jail_config router)
+
+### GET /api/v1/config/jails
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Jails config list returned | `JailConfigListResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### GET /api/v1/config/jails/{name}
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Jail config detail returned | `JailConfigDetailResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | Jail not found in config | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### PUT /api/v1/config/jails/{name}
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Jail config updated | `JailConfigDetailResponse` |
+| 400 | Invalid value for a property | Error body |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | Jail not found in config | Error body |
+| 422 | Validation error | Error body |
+| 429 | Rate limit exceeded for jail config operations | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### POST /api/v1/config/jails/{name}/commit
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Changes committed successfully | `JailConfigDetailResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | Jail not found in config | Error body |
+| 409 | Commit failed (fail2ban rejected the new config) | Error body |
+| 429 | Rate limit exceeded for jail config operations | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### GET /api/v1/config/jails/{name}/rollback
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Rollback successful | `JailConfigDetailResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | Jail not found in config | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### DELETE /api/v1/config/jails/{name}
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 204 | Jail deleted successfully | No body |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | Jail not found in config | Error body |
+| 409 | Jail is a shipped default (conf-only) | Error body |
+| 429 | Rate limit exceeded for jail config operations | Error body |
+
+### POST /api/v1/config/jails
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 201 | Jail created | `JailConfigDetailResponse` |
+| 400 | Invalid jail name | Error body |
+| 401 | Session missing, expired, or invalid | Error body |
+| 409 | Jail already exists | Error body |
+| 429 | Rate limit exceeded for jail config operations | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### GET /api/v1/config/jails/{name}/files
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Config files returned | `ConfigFileListResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | Jail not found in config | Error body |
+
+---
+
+## /api/v1/config/filters (filter_config router)
+
+### GET /api/v1/config/filters
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Filter list returned | `FilterListResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### GET /api/v1/config/filters/{name}
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Filter config returned | `FilterConfig` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | Filter not found in filter.d/ | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### PUT /api/v1/config/filters/{name}
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Filter updated | `FilterConfig` |
+| 400 | Invalid filter name | Error body |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | Filter not found | Error body |
+| 422 | Regex pattern failed to compile | Error body |
+| 429 | Rate limit exceeded for filter update operations | Error body |
+| 500 | Failed to write .local file | Error body |
+
+### POST /api/v1/config/filters
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 201 | Filter created | `FilterConfig` |
+| 400 | Invalid filter name or regex too long | Error body |
+| 401 | Session missing, expired, or invalid | Error body |
+| 409 | Filter already exists | Error body |
+| 422 | Regex pattern failed to compile | Error body |
+| 429 | Rate limit exceeded for filter create operations | Error body |
+| 500 | Failed to write .local file | Error body |
+
+### DELETE /api/v1/config/filters/{name}
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 204 | Filter deleted successfully | No body |
+| 400 | Invalid filter name | Error body |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | Filter not found | Error body |
+| 409 | Filter is a shipped default (conf-only) | Error body |
+| 429 | Rate limit exceeded for filter delete operations | Error body |
+| 500 | Failed to delete .local file | Error body |
+
+---
+
+## /api/v1/config/actions (action_config router)
+
+### GET /api/v1/config/actions
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Action list returned | `ActionListResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### GET /api/v1/config/actions/{name}
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Action config returned | `ActionConfig` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | Action not found in action.d/ | Error body |
+| 502 | fail2ban unreachable | Error body |
+
+### PUT /api/v1/config/actions/{name}
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Action updated | `ActionConfig` |
+| 400 | Invalid action name | Error body |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | Action not found | Error body |
+| 429 | Rate limit exceeded for action update operations | Error body |
+| 500 | Failed to write .local file | Error body |
+
+### POST /api/v1/config/actions
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 201 | Action created | `ActionConfig` |
+| 400 | Invalid action name | Error body |
+| 401 | Session missing, expired, or invalid | Error body |
+| 409 | Action already exists | Error body |
+| 429 | Rate limit exceeded for action create operations | Error body |
+| 500 | Failed to write .local file | Error body |
+
+### DELETE /api/v1/config/actions/{name}
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 204 | Action deleted successfully | No body |
+| 400 | Invalid action name | Error body |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | Action not found | Error body |
+| 409 | Action is a shipped default (conf-only) | Error body |
+| 429 | Rate limit exceeded for action delete operations | Error body |
+| 500 | Failed to delete .local file | Error body |
+
+---
+
+## /api/v1/blocklists
+
+### GET /api/v1/blocklists
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Blocklist sources returned | `BlocklistListResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+
+### POST /api/v1/blocklists
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 201 | Blocklist source created | `BlocklistSource` |
+| 400 | URL validation failed | Error body |
+| 401 | Session missing, expired, or invalid | Error body |
+
+### POST /api/v1/blocklists/import
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Import completed | `ImportRunResult` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 429 | Rate limit exceeded for blocklist import | Error body |
+
+### GET /api/v1/blocklists/schedule
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Schedule info returned | `ScheduleInfo` |
+| 401 | Session missing, expired, or invalid | Error body |
+
+### PUT /api/v1/blocklists/schedule
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Schedule updated | `ScheduleInfo` |
+| 401 | Session missing, expired, or invalid | Error body |
+
+### GET /api/v1/blocklists/log
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Import log returned | `ImportLogListResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+
+### GET /api/v1/blocklists/{source_id}
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Blocklist source returned | `BlocklistSource` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | Blocklist source not found | Error body |
+
+### PUT /api/v1/blocklists/{source_id}
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Blocklist source updated | `BlocklistSource` |
+| 400 | URL validation failed | Error body |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | Blocklist source not found | Error body |
+
+### DELETE /api/v1/blocklists/{source_id}
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 204 | Blocklist source deleted successfully | No body |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | Blocklist source not found | Error body |
+
+### GET /api/v1/blocklists/{source_id}/preview
+| Status | Description | Response Model |
+|--------|-------------|----------------|
+| 200 | Blocklist preview returned | `PreviewResponse` |
+| 401 | Session missing, expired, or invalid | Error body |
+| 404 | Blocklist source not found | Error body |
+| 502 | URL could not be reached | Error body |
+
+---
+
+## Error Response Format
+
+All error responses follow this structure:
+
+```json
+{
+  "code": "error_code_string",
+  "detail": "Human-readable error message",
+  "metadata": {
+    "key": "value"
+  }
+}
+```
+
+### Common error_code values
+
+| code | Meaning |
+|------|---------|
+| `not_found` | Requested entity does not exist |
+| `invalid_input` | Validation failure or bad parameters |
+| `conflict` | State conflict (already exists, already done) |
+| `authentication_required` | Session missing or invalid |
+| `rate_limit_exceeded` | Rate limit hit — check `retry_after_seconds` in metadata |
+| `fail2ban_unreachable` | fail2ban socket cannot be reached |
+| `config_validation_failed` | Config value rejected |
+| `config_file_not_found` | Config file does not exist |
+| `jail_not_found` | Jail does not exist |
+| `filter_not_found` | Filter does not exist |
+| `action_not_found` | Action does not exist |
+| `blocklist_source_not_found` | Blocklist source does not exist |
+| `setup_already_complete` | Setup has already been run |
+
+---
+
+## Status Code Decision Guide
+
+**Frontend gets 400 — what's wrong?**
+- Has `code: "invalid_input"` → validation failure, check `detail`
+- Has `code: "jail_not_found"` → jail doesn't exist
+- Has `code: "config_validation_failed"` → config value rejected
+
+**Frontend gets 502 — what's wrong?**
+- fail2ban is down or socket path wrong
+- Check `code: "fail2ban_unreachable"`
+
+**Frontend gets 503 — what's wrong?**
+- Setup not complete (`code: "setup_already_complete"`)
+- Health check: fail2ban offline or component degraded
+
+**Frontend gets 409 — what's wrong?**
+- Already done: jail already active/inactive, setup already complete
+- Operation failed: fail2ban rejected the command
+- Conflict: resource already exists
+
+**Frontend gets 429 — what's wrong?**
+- Rate limit exceeded
+- `metadata.retry_after_seconds` tells you how long to wait

Docs/DATABASE_MIGRATIONS.md

@@ -1,152 +0,0 @@
-# Database Migrations
-
-BanGUI uses SQLite with a versioned migration system. Migrations are applied automatically on startup.
-
-## Schema Version Table
-
-The `schema_migrations` table tracks applied migrations:
-
-```sql
-CREATE TABLE IF NOT EXISTS schema_migrations (
-    version     INTEGER PRIMARY KEY,
-    migrated_at TEXT    NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
-);
-```
-
-## How Migrations Work
-
-On startup (`init_db()`):
-
-1. Current schema version is read from `schema_migrations`
-2. If version < latest, each missing migration is applied in order
-3. Each migration runs inside a `BEGIN IMMEDIATE ... COMMIT` transaction
-4. On failure, `ROLLBACK` restores database to pre-migration state
-
-## Transactional Guarantees
-
-Every migration is **atomic**. If any statement fails:
-
-- All DDL changes are rolled back
-- `schema_migrations` table is NOT updated
-- Next startup re-applies the same migration from scratch
-
-```python
-try:
-    await db.execute("BEGIN IMMEDIATE;")
-    for statement in statements:
-        await db.execute(statement)
-    await db.execute("INSERT INTO schema_migrations (version) VALUES (?);", (version,))
-    await db.commit()
-except Exception:
-    await db.rollback()
-    raise
-```
-
-## Idempotency
-
-Migrations use `CREATE TABLE IF NOT EXISTS` and `CREATE INDEX IF NOT EXISTS` where possible. Re-running a failed or partial migration is safe.
-
-## WAL Mode and Crash Safety
-
-BanGUI uses SQLite WAL mode (`PRAGMA journal_mode=WAL`). After a crash:
-
-- SQLite auto-recovers using the WAL file
-- `.wal` file may contain uncommitted changes that are rolled back
-- Orphaned `.wal` files from previous crashes are detected and cleaned up on startup
-
-### Detecting Orphaned WAL Files
-
-On startup, if the database is in WAL mode but no WAL file exists:
-
-```python
-async def _cleanup_orphaned_wal_files(db: aiosqlite.Connection, db_path: Path) -> None:
-    """Remove orphaned WAL files after crashes."""
-    wal_path = Path(str(db_path) + "-wal")
-    if wal_path.exists() and db_path.exists():
-        # Check if WAL file is stale (database was opened since)
-        pass  # SQLite handles this automatically
-```
-
-## Migration Failure Recovery
-
-If a migration fails mid-way:
-
-1. **Startup fails** — application refuses to start
-2. **Rollback occurs** — database returns to pre-migration state
-3. **Logs show error** — exception with full traceback
-
-### Manual Recovery Steps
-
-1. **Check current schema version:**
-   ```bash
-   sqlite3 bangui.db "SELECT MAX(version) FROM schema_migrations;"
-   ```
-
-2. **Check which tables exist:**
-   ```bash
-   sqlite3 bangui.db "SELECT name FROM sqlite_master WHERE type='table';"
-   ```
-
-3. **Manually apply the failed migration:**
-   ```bash
-   sqlite3 bangui.db "BEGIN IMMEDIATE;"
-   # Run your migration SQL here
-   sqlite3 bangui.db "INSERT INTO schema_migrations (version) VALUES (?);"
-   sqlite3 bangui.db "COMMIT;"
-   ```
-
-4. **Or roll back to a known state:**
-   ```bash
-   sqlite3 bangui.db "DELETE FROM schema_migrations WHERE version > ?;"
-   ```
-
-### Complete Database Reset (Development Only)
-
-If the database is unrecoverable:
-
-```bash
-rm bangui.db bangui.db-wal bangui.db-shm
-# Restart application - schema will be recreated from migration 1
-```
-
-## Migration Version History
-
-| Version | Description |
-|---------|-------------|
-| 1 | Initial schema (settings, sessions, blocklist_sources, import_log, geo_cache, history_archive) |
-| 2 | Hash session tokens (DROP + recreate sessions) |
-| 3 | Add last_seen to geo_cache |
-| 4 | Add scheduler_lock table |
-| 5 | Add indexes to history_archive |
-| 6 | Add import_runs table for idempotent imports |
-| 7 | Add indexes to import_log |
-| 8 | Migrate import_log.timestamp TEXT→INTEGER UNIX |
-| 9 | Change import_log.source_id FK to ON DELETE RESTRICT |
-
-## Adding New Migrations
-
-1. Increment `_CURRENT_SCHEMA_VERSION` in `backend/app/db.py`
-2. Add migration script to `_MIGRATIONS` dict with new version key
-3. Write migration as `CREATE IF NOT EXISTS` or `ALTER TABLE ADD COLUMN` to ensure idempotency
-4. Test with `test_apply_migration_is_atomic_rollback` pattern
-5. Update this document with migration description
-
-## Long-Running Migrations
-
-For migrations that modify large tables:
-
-- Use `ALTER TABLE ADD COLUMN` (instant on SQLite)
-- Avoid `CREATE INDEX CONCURRENTLY` (SQLite does not support this)
-- For table rebuilds, split into phases with explicit progress tracking
-
-## Disaster Recovery Checklist
-
-If database is corrupted after migration failure:
-
-- [ ] Stop all BanGUI instances
-- [ ] Backup `bangui.db`, `bangui.db-wal`, `bangui.db-shm`
-- [ ] Run `PRAGMA integrity_check;`
-- [ ] Identify last successful migration version
-- [ ] Delete `schema_migrations` rows for failed migrations
-- [ ] Either: manually fix migration, or restore from backup
-- [ ] Restart application

Docs/TROUBLESHOOTING.md

@@ -331,6 +331,93 @@ sqlite3 /var/lib/bangui/bangui.db "PRAGMA integrity_check;"
 
 ---
 
+## Configuration Validation at Startup
+
+BanGUI validates configuration at startup. Errors raised here indicate misconfiguration that must be fixed before the application can start.
+
+### Database Parent Directory Does Not Exist
+
+**Symptom:** Application fails to start with: `Database parent directory does not exist: /path/to/parent`
+
+**Cause:** The parent directory of `BANGUI_DATABASE_PATH` does not exist.
+
+**Solution:**
+```bash
+mkdir -p /path/to/parent
+# Then restart BanGUI
+```
+
+---
+
+### Database Parent Directory Not Writable
+
+**Symptom:** Application fails to start with: `Database parent directory not writable: /path/to/parent`
+
+**Cause:** The process cannot write to the database parent directory.
+
+**Solution:**
+```bash
+chmod 755 /path/to/parent
+# Verify the user running BanGUI owns the directory or has write access
+```
+
+---
+
+### fail2ban Socket Not Readable
+
+**Symptom:** Application fails to start with: `fail2ban socket not readable: /path/to/socket`
+
+**Cause:** The socket file exists but is not readable by the BanGUI process.
+
+**Solution:**
+```bash
+chmod 644 /path/to/socket
+ls -la /path/to/socket
+```
+
+---
+
+### fail2ban Config Directory Does Not Exist
+
+**Symptom:** Application fails to start with: `fail2ban config directory does not exist: /path/to/config`
+
+**Cause:** `BANGUI_FAIL2BAN_CONFIG_DIR` points to a directory that does not exist.
+
+**Solution:**
+- Mount the fail2ban configuration directory at the expected path
+- Or adjust `BANGUI_FAIL2BAN_CONFIG_DIR` to point to the correct location
+- In Docker: add a volume mount for the fail2ban config directory
+
+---
+
+### GeoIP Database File Does Not Exist
+
+**Symptom:** Application fails to start with: `GeoIP database file does not exist: /path/to/GeoLite2-Country.mmdb`
+
+**Cause:** `BANGUI_GEOIP_DB_PATH` points to a file that does not exist.
+
+**Solution:**
+1. Download the MaxMind GeoLite2-Country database from https://dev.maxmind.com/geoip/geolite2-country
+2. Place it at the configured path, or update `BANGUI_GEOIP_DB_PATH` to the correct location
+3. Alternatively, set `BANGUI_GEOIP_DB_PATH` to `null` to disable GeoIP lookups
+
+---
+
+### session_secret Too Short or Weak
+
+**Symptom:** Application fails to start with: `session_secret must be at least 32 characters` or `session_secret is too weak`
+
+**Cause:** `BANGUI_SESSION_SECRET` is missing, too short, or contains common weak words.
+
+**Solution:**
+```bash
+# Generate a new secret
+python -c "import secrets; print(secrets.token_hex(32))"
+```
+Then set it in your `.env` file or environment variables.
+
+---
+
 ## Getting Help
 
 If issues persist after following this guide:

Docs/Tasks.md

@@ -1,94 +1,3 @@
-### Issue #16: MEDIUM - Silent Failures in Error Handling (Broad Exception Handlers)
-
-**Where found**: 
-- `backend/app/routers/config_misc.py` (line 54) - `except Exception:`
-- `backend/app/ban_service.py` - Multiple broad exception handlers
-- Silent failures hide programming errors
-
-**Why this is needed**: 
-Broad `except Exception:` catches programming errors (AttributeError, KeyError) alongside legitimate errors, hiding bugs in logs.
-
-**Goal**: 
-Catch only specific exceptions; let programming errors bubble up to global error handler.
-
-**What to do**:
-1. Replace broad handlers with specific exceptions:
-   ```python
-   try:
-       config = parse_config(raw_text)
-   except ConfigParseError as e:  # Specific
-       logger.error(f"Config parse failed: {e}")
-   except FileNotFoundError as e:
-       logger.error(f"Config file not found: {e}")
-   except Exception as e:
-       logger.exception("Unexpected error parsing config")
-       raise  # Re-raise to global handler
-   ```
-2. Create domain-specific exception classes
-3. Document what exceptions each function can raise
-4. Update tests to verify exception handling
-
-**Possible traps and issues**:
-- Missing exception types will let errors bubble up unexpectedly
-- Catching too few exceptions leads to uncaught errors
-- Global exception handler needed to catch unhandled exceptions
-
-**Docs changes needed**:
-- Add exception handling guidelines to dev docs
-- Create exception taxonomy document
-
-**Doc references**:
-- DETAILED_FINDINGS.md - Issue #16 "Broad Exception Handlers"
-
----
-
-### Issue #17: MEDIUM - No API Response Status Code Documentation
-
-**Where found**: 
-- All routers lack OpenAPI `responses={}` documentation
-- Status codes for success/failure not documented
-- Frontend must infer from response body
-
-**Why this is needed**: 
-Frontend doesn't know:
-- Is 400 a validation error or configuration error?
-- Is 502 from backend or fail2ban?
-- Which 503 status means setup incomplete vs fail2ban down?
-
-**Goal**: 
-Document all possible status codes and response formats for each endpoint.
-
-**What to do**:
-1. Add to each router endpoint:
-   ```python
-   @router.post(
-       "/login",
-       responses={
-           200: {"description": "Login successful", "model": LoginResponse},
-           400: {"description": "Invalid request format"},
-           401: {"description": "Invalid password"},
-           429: {"description": "Too many attempts, retry after 60s"},
-           503: {"description": "Setup not complete"},
-       }
-   )
-   ```
-2. Generate OpenAPI schema with descriptions
-3. Update API docs with status code reference table
-4. Validate in CI that all endpoints documented
-
-**Possible traps and issues**:
-- Documentation might become stale as code changes
-- Multiple response types for single status code (must document each)
-
-**Docs changes needed**:
-- Create API reference documenting all status codes
-- Add endpoint documentation template
-
-**Doc references**:
-- DATABASE_API_DEPLOYMENT_ISSUES.md - Issue "2.3 Missing Status Code Documentation"
-
----
-
 ### Issue #18: MEDIUM - Configuration Validation Missing at Startup
 
 **Where found**: 

Docs/runner.csx

@@ -113,10 +113,28 @@ for (int i = 0; i < items.Count; i++)
     );
     if (cts.IsCancellationRequested) break;
 
-    // Step 3 — check for "yes" in the reply
-    if (!confirmation.Contains("yes", StringComparison.OrdinalIgnoreCase))
+    // Step 3 — check for "yes" in the reply, with retry logic for issue resolution
+    int maxRetries = 3;
+    int retryCount = 0;
+    bool taskConfirmed = confirmation.Contains("yes", StringComparison.OrdinalIgnoreCase);
+
+    while (!taskConfirmed && retryCount < maxRetries)
     {
-        Console.WriteLine("\n[runner] Task not confirmed as done. Stopping.");
+        retryCount++;
+        Console.WriteLine($"\n[runner] Attempt {retryCount}/{maxRetries}: Resolving remaining issues and running tests...\n");
+        
+        confirmation = await RunCopilot(
+            new[] { "--continue" },
+            "resolve any remaining issues, make sure all tests are running and pass. then confirm with yes if done"
+        );
+        if (cts.IsCancellationRequested) break;
+        
+        taskConfirmed = confirmation.Contains("yes", StringComparison.OrdinalIgnoreCase);
+    }
+
+    if (!taskConfirmed)
+    {
+        Console.WriteLine($"\n[runner] Task not confirmed as done after {maxRetries} attempts. Stopping.");
         break;
     }