118 lines
6.2 KiB
Markdown
118 lines
6.2 KiB
Markdown
# Testing Requirements
|
|
|
|
## Coverage Threshold
|
|
|
|
- **Minimum: 80% line coverage** for all backend code
|
|
- Critical paths (auth, banning, scheduling, API endpoints): **100%**
|
|
|
|
## CI Enforcement
|
|
|
|
`.github/workflows/ci.yml` runs pytest with `--cov-fail-under=80`. Build fails if coverage drops below threshold.
|
|
|
|
## Running Tests Locally
|
|
|
|
```bash
|
|
cd backend
|
|
pytest --cov=app --cov-report=term-missing
|
|
```
|
|
|
|
## Coverage Reports
|
|
|
|
- Terminal: `--cov-report=term-missing`
|
|
- HTML: `--cov-report=html` (output in `htmlcov/`)
|
|
|
|
## Coverage Badge
|
|
|
|
Add to README once CI runs successfully:
|
|
|
|
```md
|
|
[](https://codecov.io/gh/<owner>/BanGUI)
|
|
```
|
|
|
|
Requires codecov.io integration with repository.
|
|
|
|
## Writing Tests
|
|
|
|
- Follow pattern: `test_<unit>_<scenario>_<expected>`
|
|
- Mock external dependencies (fail2ban socket, aiohttp calls)
|
|
- Test happy path AND error/edge cases
|
|
- See `Docs/Backend-Development.md §9` for detailed testing guide
|
|
|
|
## E2E Testing
|
|
|
|
An end-to-end test suite using **Robot Framework** with the Browser library (Playwright-backed) exercises the full running stack: frontend → backend → fail2ban → database.
|
|
|
|
### Running E2E Tests
|
|
|
|
```bash
|
|
make e2e
|
|
```
|
|
|
|
Requires:
|
|
- `BANGUI_SESSION_SECRET` env var must be set (see [Backend-Development.md](Backend-Development.md) for setup)
|
|
- Stack must be startable via `make up` (Docker/Podman + compose installed)
|
|
- `rfbrowser init` is run automatically by the `e2e` target (Playwright browsers downloaded on first run; re-run after `robotframework-browser` version changes)
|
|
|
|
### HTML Report
|
|
|
|
After a run, open `e2e/results/report.html` in a browser to view the detailed HTML report with screenshots on failure.
|
|
|
|
### Writing New E2E Tests
|
|
|
|
Place new `.robot` files in `e2e/tests/`. Use `e2e/resources/common.resource` for shared variables and setup/teardown, and `e2e/resources/auth.resource` for the `Login As Admin` keyword.
|
|
|
|
### E2E-3 — Ban Pipeline Timing
|
|
|
|
Test **E2E-3** (`e2e/tests/02_ban_records.robot`: *Simulated Failed Logins Appear As Ban Records*) exercises the full ban pipeline:
|
|
|
|
```
|
|
simulate_failed_logins.sh → fail2ban log scan → ban recorded in fail2ban DB
|
|
→ backend polls socket (on-demand, no push) → /api/bans/active
|
|
→ history_sync archive (every 300 s) → /api/history
|
|
```
|
|
|
|
Key timing facts:
|
|
|
|
- **fail2ban** (`manual-Jail`, `backend=polling`) re-reads `auth.log` on its own interval, not event-driven.
|
|
- **maxretry=3** means a ban triggers after the 3rd matching line. `simulate_failed_logins.sh` writes 5 lines to ensure the threshold is crossed.
|
|
- **15 s sleep** in the test gives fail2ban time to detect and record the ban before the first assertion. This is a heuristic — the actual polling interval depends on fail2ban's internal cycle.
|
|
- **history_sync** runs every 300 s (`HISTORY_SYNC_INTERVAL` in `backend/app/tasks/history_sync.py`). The History page reads from the archive DB, so it may lag up to 300 s behind real-time. The E2E test uses `GET /api/bans/active` (direct socket query) for the API assertion to avoid this lag.
|
|
- **Pagination**: the History page paginates results. Use `?page_size=500` to push the test IP onto the first page, or assert via the API.
|
|
|
|
If the test fails at Step 2 (no ban detected via API) but `check_ban_status.sh` shows the IP is banned inside the container, the backend-to-fail2ban socket path is broken. If `check_ban_status.sh` also shows no ban, the log volume mapping is wrong (fail2ban is not reading the file `simulate_failed_logins.sh` writes to).
|
|
|
|
### E2E-4 — Blocklist Import
|
|
|
|
Test **E2E-4** (`e2e/tests/03_blocklist_import.robot`: *Manual Blocklist Import Completes Without Error*) exercises the full import pipeline:
|
|
|
|
```
|
|
UI button click → POST /api/v1/blocklists/import → async background task
|
|
→ DNS validation → HTTP fetch (external or local mock)
|
|
→ IP parsing → fail2ban ban_ip call → DB write → import log entry
|
|
```
|
|
|
|
Key facts:
|
|
|
|
- **Rate limit**: `BANGUI_RATE_LIMIT_BLOCKLIST_IMPORT_PER_HOUR = 10` per client IP. E2E tests bypass this by sending a unique `X-Forwarded-For` header (e.g., `10.0.0.99`). The header is only honoured when the client IP is in `BANGUI_TRUSTED_PROXIES`.
|
|
- **Network dependency**: The import fetches the blocklist URL over HTTP. In CI environments without internet access the test starts a local Python `http.server` (port 8765) serving `e2e/test_blocklist.txt`. The `Ensure Blocklist Source Exists` keyword points the source URL at `http://localhost:8765/test.txt` when no internet is detected.
|
|
- **"Import ran" vs "bans added"**: These are separate outcomes. The test asserts that the log entry count increases — confirming the import ran to completion — regardless of whether any IPs were actually banned.
|
|
- **Timeout**: Large lists may exceed the 45 s button-wait timeout. Increase as needed.
|
|
- **Selector**: The import button is selected via `css=[data-testid="blocklist-import-button"],button`. The `data-testid` attribute must be added to the frontend component (see [E2E-6] in [Tasks.md](Tasks.md)). If the attribute is absent, the fallback `button` selector is used.
|
|
|
|
**Teardown**: `Cleanup Mock Server` stops the local HTTP server started in the test.
|
|
|
|
### E2E-5 — Config Field Edit Persistence
|
|
|
|
Test **E2E-5** (`e2e/tests/04_config_edit.robot`: *Config Field Edit Persists After Reload*) exercises the auto-save round-trip:
|
|
|
|
```
|
|
UI edit → useAutoSave debounce (500 ms) → PATCH /api/config/jails/:name
|
|
→ fail2ban config write → GET /api/jails rehydration on reload
|
|
```
|
|
|
|
Key facts:
|
|
|
|
- **Debounce**: `useAutoSave` fires no HTTP request until 500 ms of inactivity after the last keystroke. The test waits for the "Saved" indicator (`[role="status"]:has-text("Saved")`) rather than a fixed `Sleep`, ensuring the PATCH actually fired before the reload.
|
|
- **Selector**: `input[aria-label="Ban Time"]` is used to locate the bantime field — no `data-*` attribute required. The `aria-label` is stable across refactors.
|
|
- **Teardown**: `Restore Original Ban Time` is set as `[Teardown]` so it runs even when the test fails mid-way. Config edits restart fail2ban internally; restoring state prevents subsequent tests from reading modified values.
|
|
- **Run order**: E2E-5 should run last in the suite to avoid destabilising fail2ban health for other tests. |