BanGUI/Docs/Testing-Requirements.md

Testing Requirements

Coverage Threshold

• Minimum: 80% line coverage for all backend code
• Critical paths (auth, banning, scheduling, API endpoints): 100%

CI Enforcement

.github/workflows/ci.yml runs pytest with --cov-fail-under=80. Build fails if coverage drops below threshold.

Running Tests Locally

cd backend
pytest --cov=app --cov-report=term-missing

Coverage Reports

• Terminal: --cov-report=term-missing
• HTML: --cov-report=html (output in htmlcov/)

Coverage Badge

Add to README once CI runs successfully:

[![Coverage](https://codecov.io/gh/<owner>/BanGUI/branch/main/graph/badge.svg)](https://codecov.io/gh/<owner>/BanGUI)

Requires codecov.io integration with repository.

Writing Tests

• Follow pattern: test_<unit>_<scenario>_<expected>
• Mock external dependencies (fail2ban socket, aiohttp calls)
• Test happy path AND error/edge cases
• See Docs/Backend-Development.md §9 for detailed testing guide

E2E Testing

An end-to-end test suite using Robot Framework with the Browser library (Playwright-backed) exercises the full running stack: frontend → backend → fail2ban → database.

Running E2E Tests

make e2e

Requires:

• BANGUI_SESSION_SECRET env var must be set (see Backend-Development.md for setup)
• Stack must be startable via make up (Docker/Podman + compose installed)
• rfbrowser init is run automatically by the e2e target (Playwright browsers downloaded on first run; re-run after robotframework-browser version changes)

HTML Report

After a run, open e2e/results/report.html in a browser to view the detailed HTML report with screenshots on failure.

Writing New E2E Tests

Place new .robot files in e2e/tests/. Use e2e/resources/common.resource for shared variables and setup/teardown, and e2e/resources/auth.resource for the Login As Admin keyword.

E2E-3 — Ban Pipeline Timing

Test E2E-3 (e2e/tests/02_ban_records.robot: Simulated Failed Logins Appear As Ban Records) exercises the full ban pipeline:

simulate_failed_logins.sh → fail2ban log scan → ban recorded in fail2ban DB
     → backend polls socket (on-demand, no push) → /api/bans/active
     → history_sync archive (every 300 s) → /api/history

Key timing facts:

• fail2ban (manual-Jail, backend=polling) re-reads auth.log on its own interval, not event-driven.
• maxretry=3 means a ban triggers after the 3rd matching line. simulate_failed_logins.sh writes 5 lines to ensure the threshold is crossed.
• 15 s sleep in the test gives fail2ban time to detect and record the ban before the first assertion. This is a heuristic — the actual polling interval depends on fail2ban's internal cycle.
• history_sync runs every 300 s (HISTORY_SYNC_INTERVAL in backend/app/tasks/history_sync.py). The History page reads from the archive DB, so it may lag up to 300 s behind real-time. The E2E test uses GET /api/bans/active (direct socket query) for the API assertion to avoid this lag.
• Pagination: the History page paginates results. Use ?page_size=500 to push the test IP onto the first page, or assert via the API.

If the test fails at Step 2 (no ban detected via API) but check_ban_status.sh shows the IP is banned inside the container, the backend-to-fail2ban socket path is broken. If check_ban_status.sh also shows no ban, the log volume mapping is wrong (fail2ban is not reading the file simulate_failed_logins.sh writes to).

E2E-4 — Blocklist Import

Test E2E-4 (e2e/tests/03_blocklist_import.robot: Manual Blocklist Import Completes Without Error) exercises the full import pipeline:

UI button click → POST /api/v1/blocklists/import → async background task
    → DNS validation → HTTP fetch (external or local mock)
    → IP parsing → fail2ban ban_ip call → DB write → import log entry

Key facts:

• Rate limit: BANGUI_RATE_LIMIT_BLOCKLIST_IMPORT_PER_HOUR = 10 per client IP. E2E tests bypass this by sending a unique X-Forwarded-For header (e.g., 10.0.0.99). The header is only honoured when the client IP is in BANGUI_TRUSTED_PROXIES.
• Network dependency: The import fetches the blocklist URL over HTTP. In CI environments without internet access the test starts a local Python http.server (port 8765) serving e2e/test_blocklist.txt. The Ensure Blocklist Source Exists keyword points the source URL at http://localhost:8765/test.txt when no internet is detected.
• "Import ran" vs "bans added": These are separate outcomes. The test asserts that the log entry count increases — confirming the import ran to completion — regardless of whether any IPs were actually banned.
• Timeout: Large lists may exceed the 45 s button-wait timeout. Increase as needed.
• Selector: The import button is selected via css=[data-testid="blocklist-import-button"],button. The data-testid attribute must be added to the frontend component (see [E2E-6] in Tasks.md). If the attribute is absent, the fallback button selector is used.

Teardown: Cleanup Mock Server stops the local HTTP server started in the test.

E2E-5 — Config Field Edit Persistence

Test E2E-5 (e2e/tests/04_config_edit.robot: Config Field Edit Persists After Reload) exercises the auto-save round-trip:

UI edit → useAutoSave debounce (500 ms) → PATCH /api/config/jails/:name
        → fail2ban config write → GET /api/jails rehydration on reload

Key facts:

• Debounce: useAutoSave fires no HTTP request until 500 ms of inactivity after the last keystroke. The test waits for the "Saved" indicator ([role="status"]:has-text("Saved")) rather than a fixed Sleep, ensuring the PATCH actually fired before the reload.
• Selector: input[aria-label="Ban Time"] is used to locate the bantime field — no data-* attribute required. The aria-label is stable across refactors.
• Teardown: Restore Original Ban Time is set as [Teardown] so it runs even when the test fails mid-way. Config edits restart fail2ban internally; restoring state prevents subsequent tests from reading modified values.
• Run order: E2E-5 should run last in the suite to avoid destabilising fail2ban health for other tests.