docs: update documentation and e2e tests

- Add configuration docs for database and rate limiting
- Remove completed tasks from tracking list
- Update testing requirements with new test patterns
- Enhance web development docs with frontend guidelines
- Expand page loading and ban records e2e test coverage

Docs/Testing-Requirements.md

@@ -59,4 +59,24 @@ After a run, open `e2e/results/report.html` in a browser to view the detailed HT
 
 ### Writing New E2E Tests
 
-Place new `.robot` files in `e2e/tests/`. Use `e2e/resources/common.resource` for shared variables and setup/teardown, and `e2e/resources/auth.resource` for the `Login As Admin` keyword.
+Place new `.robot` files in `e2e/tests/`. Use `e2e/resources/common.resource` for shared variables and setup/teardown, and `e2e/resources/auth.resource` for the `Login As Admin` keyword.
+
+### E2E-3 — Ban Pipeline Timing
+
+Test **E2E-3** (`e2e/tests/02_ban_records.robot`: *Simulated Failed Logins Appear As Ban Records*) exercises the full ban pipeline:
+
+```
+simulate_failed_logins.sh → fail2ban log scan → ban recorded in fail2ban DB
+     → backend polls socket (on-demand, no push) → /api/bans/active
+     → history_sync archive (every 300 s) → /api/history
+```
+
+Key timing facts:
+
+- **fail2ban** (`manual-Jail`, `backend=polling`) re-reads `auth.log` on its own interval, not event-driven.
+- **maxretry=3** means a ban triggers after the 3rd matching line. `simulate_failed_logins.sh` writes 5 lines to ensure the threshold is crossed.
+- **15 s sleep** in the test gives fail2ban time to detect and record the ban before the first assertion. This is a heuristic — the actual polling interval depends on fail2ban's internal cycle.
+- **history_sync** runs every 300 s (`HISTORY_SYNC_INTERVAL` in `backend/app/tasks/history_sync.py`). The History page reads from the archive DB, so it may lag up to 300 s behind real-time. The E2E test uses `GET /api/bans/active` (direct socket query) for the API assertion to avoid this lag.
+- **Pagination**: the History page paginates results. Use `?page_size=500` to push the test IP onto the first page, or assert via the API.
+
+If the test fails at Step 2 (no ban detected via API) but `check_ban_status.sh` shows the IP is banned inside the container, the backend-to-fail2ban socket path is broken. If `check_ban_status.sh` also shows no ban, the log volume mapping is wrong (fail2ban is not reading the file `simulate_failed_logins.sh` writes to).