BanGUI/Docs/Tasks.md

BanGUI — Task List

This document breaks the entire BanGUI project into development stages, ordered so that each stage builds on the previous one. Every task is described in prose with enough detail for a developer to begin work. References point to the relevant documentation.

---

Task 1 — Make Geo-Cache Persistent ✅ DONE

Goal: Minimise calls to the external geo-IP lookup service by caching results in the database.

Details:

• Currently geo-IP results may only live in memory and are lost on restart. Persist every successful geo-lookup result into the database so the external service is called as rarely as possible.
• On each geo-lookup request, first query the database for a cached entry for that IP. Only call the external service if no cached entry exists (or the entry has expired, if a TTL policy is desired).
• After a successful external lookup, write the result back to the database immediately.
• Review the existing implementation in app/services/geo_service.py and the related repository/model code. Verify that:
  • The DB table/model for geo-cache entries exists and has the correct schema (IP, country, city, latitude, longitude, looked-up timestamp, etc.).
  • The repository layer exposes get_by_ip and upsert (or equivalent) methods.
  • The service checks the cache before calling the external API.
  • Bulk inserts are used where multiple IPs need to be resolved at once (see Task 3).

---

Task 2 — Fix geo_lookup_request_failed Warnings ✅ DONE

Goal: Investigate and fix the frequent geo_lookup_request_failed log warnings that occur with an empty error field.

Resolution: The root cause was str(exc) returning "" for aiohttp exceptions with no message (e.g. ServerDisconnectedError). Fixed by:

• Replacing error=str(exc) with error=repr(exc) in both lookup() and _batch_api_call() so the exception class name is always present in the log.
• Adding exc_type=type(exc).__name__ field to every network-error log event for easy filtering.
• Moving import aiohttp from the TYPE_CHECKING block to a regular runtime import and replacing the raw-float timeout arguments with aiohttp.ClientTimeout(total=...), removing the # type: ignore[arg-type] workarounds.
• Three new tests in TestErrorLogging verify empty-message exceptions are correctly captured.

Observed behaviour (from container logs):

{"ip": "197.221.98.153", "error": "", "event": "geo_lookup_request_failed", ...}
{"ip": "197.231.178.38", "error": "", "event": "geo_lookup_request_failed", ...}
{"ip": "197.234.201.154", "error": "", "event": "geo_lookup_request_failed", ...}
{"ip": "197.234.206.108", "error": "", "event": "geo_lookup_request_failed", ...}

Details:

• Open app/services/geo_service.py and trace the code path that emits the geo_lookup_request_failed event.
• The error field is empty, which suggests the request may silently fail (e.g. the external service returns a non-200 status, an empty body, or the response parsing swallows the real error).
• Ensure the actual HTTP status code and response body (or exception message) are captured and logged in the error field so failures are diagnosable.
• Check whether the external geo-IP service has rate-limiting or IP-range restrictions that could explain the failures.
• Add proper error handling: distinguish between transient errors (timeout, 429, 5xx) and permanent ones (invalid IP, 404) so retries can be applied only when appropriate.

---

Task 3 — Non-Blocking Web Requests & Bulk DB Operations ✅ DONE

Goal: Ensure the web UI remains responsive while geo-IP lookups and database writes are in progress.

Resolution:

• Bulk DB writes: geo_service.lookup_batch now collects resolved IPs into pos_rows / neg_ips lists across the chunk loop and flushes them with two executemany calls per chunk instead of one execute per IP.
• lookup_cached_only: New function that returns (geo_map, uncached) immediately from the in-memory + SQLite cache with no API calls. Used by bans_by_country for its hot path.
• Background geo resolution: bans_by_country calls lookup_cached_only for an instant response, then fires asyncio.create_task(geo_service.lookup_batch(uncached, …)) to populate the cache in the background for subsequent requests.
• Batch enrichment for get_active_bans: jail_service.get_active_bans now accepts http_session / app_db and resolves all banned IPs in a single lookup_batch call (chunked 100-IP batches) instead of firing one coroutine per IP through asyncio.gather.
• 12 new tests across test_geo_service.py, test_jail_service.py, and test_ban_service.py; ruff and mypy --strict clean; 145 tests pass.

Details:

• After the geo-IP service was integrated, web UI requests became slow or appeared to hang because geo lookups and individual DB writes block the async event loop.
• Bulk DB operations: When multiple IPs need geo data at once (e.g. loading the ban list), collect all uncached IPs and resolve them in a single batch. Use bulk INSERT … ON CONFLICT (or equivalent) to write results to the DB in one round-trip instead of one query per IP.
• Non-blocking external calls: Make sure all HTTP calls to the external geo-IP service use an async HTTP client (httpx.AsyncClient or similar) so the event loop is never blocked by network I/O.
• Non-blocking DB access: Ensure all database operations use the async SQLAlchemy session (or are off-loaded to a thread) so they do not block request handling.
• Background processing: Consider moving bulk geo-lookups into a background task (e.g. the existing task infrastructure in app/tasks/) so the API endpoint returns immediately and the UI is updated once results are ready.

---

Task 4 — Better Jail Configuration ✅ DONE

Goal: Expose the full fail2ban configuration surface (jails, filters, actions) in the web UI.

Reference config directory: /home/lukas/Volume/repo/BanGUI/Docker/fail2ban-dev-config/fail2ban/

Implementation summary:

• Backend: New app/models/file_config.py, app/services/file_config_service.py, and app/routers/file_config.py with full CRUD for jail.d/, filter.d/, action.d/ files. Path-traversal prevention via _assert_within() + _validate_new_name(). app/config.py extended with fail2ban_config_dir setting.
• Backend (socket): Added delete_log_path() to config_service.py + DELETE /api/config/jails/{name}/logpath endpoint.
• Docker: Both compose files updated with BANGUI_FAIL2BAN_CONFIG_DIR env var; volume mount changed :ro → :rw.
• Frontend: New Jail Files, Filters, Actions tabs in ConfigPage.tsx. Delete buttons for log paths in jail accordion. Full API call layer in api/config.ts + new types in types/config.ts.
• Tests: 44 service unit tests + 19 router integration tests; all pass; ruff clean.

Task 4c audit findings — options not yet exposed in the UI:

• Per-jail: ignoreip, bantime.increment, bantime.rndtime, bantime.maxtime, bantime.factor, bantime.formula, bantime.multipliers, bantime.overalljails, ignorecommand, prefregex, timezone, journalmatch, usedns, backend (read-only shown), destemail, sender, action override
• Global: allowipv6, before includes

4a — Activate / Deactivate Jail Configs ✅ DONE

• Listed all .conf and .local files in jail.d/ via GET /api/config/jail-files.
• Toggle enabled/disabled via PUT /api/config/jail-files/{filename}/enabled which patches the enabled = true/false line in the config file, preserving all comments.
• Frontend: Jail Files tab with enabled Switch per file and read-only content viewer.

4b — Editable Log Paths ✅ DONE

• Added DELETE /api/config/jails/{name}/logpath?log_path=… endpoint (uses fail2ban socket set <jail> dellogpath).
• Frontend: each log path in the Jails accordion now has a dismiss button to remove it.

4c — Audit Missing Config Options ✅ DONE

• Audit findings documented above.

4d — Filter Configuration (filter.d) ✅ DONE

• Listed all filter files via GET /api/config/filters.
• View and edit individual filters via GET/PUT /api/config/filters/{name}.
• Create new filter via POST /api/config/filters.
• Frontend: Filters tab with accordion-per-file, editable textarea, save button, and create-new form.

4e — Action Configuration (action.d) ✅ DONE

• Listed all action files via GET /api/config/actions.
• View and edit individual actions via GET/PUT /api/config/actions/{name}.
• Create new action via POST /api/config/actions.
• Frontend: Actions tab with identical structure to Filters tab.

4f — Create New Configuration Files ✅ DONE

• Create filter and action files via POST /api/config/filters and POST /api/config/actions with name validation (_SAFE_NAME_RE) and 512 KB content size limit.
• Frontend: "New Filter/Action File" section at the bottom of each tab with name input, content textarea, and create button.

---

Task 5 — Add Log Path to Jail (Config UI) ✅ DONE

Goal: Allow users to add new log file paths to an existing fail2ban jail directly from the Configuration → Jails tab, completing the "Add Log Observation" feature from Features.md § 6.3.

Implementation summary:

• ConfigPage.tsx JailAccordionPanel:
  • Added addLogPath and AddLogPathRequest imports.
  • Added state: newLogPath, newLogPathTail (default true), addingLogPath.
  • Added handleAddLogPath callback: calls addLogPath(jail.name, { log_path, tail }), appends path to logPaths state, clears input, shows success/error feedback.
  • Added inline "Add Log Path" form below the existing log-path list — an Input for the file path, a Switch for tail/head selection, and an "Add" button with aria-label="Add log path".
• 6 new frontend tests in src/components/__tests__/ConfigPageLogPath.test.tsx covering: rendering, disabled state, enabled state, successful add, success message, and API error surfacing.
• tsc --noEmit, eslint: zero errors.