feat: add e2e test suite with Robot Framework
Add e2e/ dir with Robot Framework tests for page loading, ban records, blocklist import, config edit. Add requirements.txt. Update Makefile with test commands. Update .gitignore, backend docs, testing requirements docs.
This commit is contained in:
3
.gitignore
vendored
3
.gitignore
vendored
@@ -112,3 +112,6 @@ Docker/fail2ban-dev-config/**
|
||||
*.tmp
|
||||
*.bak
|
||||
*.orig
|
||||
|
||||
# ── E2E test results ───────────────────────────
|
||||
e2e/results/
|
||||
|
||||
@@ -3010,6 +3010,7 @@ atomic_write(path, updated_content) # Atomic write, auto-cleanup on error
|
||||
- Every merge request must pass: ruff, type checker, all tests.
|
||||
- Do not merge with failing CI.
|
||||
- Keep pull requests small and focused — one feature or fix per PR.
|
||||
- **E2E test results** (`e2e/results/`) are gitignored — never commit test outputs or HTML reports.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -37,3 +37,26 @@ Requires codecov.io integration with repository.
|
||||
- Mock external dependencies (fail2ban socket, aiohttp calls)
|
||||
- Test happy path AND error/edge cases
|
||||
- See `Docs/Backend-Development.md §9` for detailed testing guide
|
||||
|
||||
## E2E Testing
|
||||
|
||||
An end-to-end test suite using **Robot Framework** with the Browser library (Playwright-backed) exercises the full running stack: frontend → backend → fail2ban → database.
|
||||
|
||||
### Running E2E Tests
|
||||
|
||||
```bash
|
||||
make e2e
|
||||
```
|
||||
|
||||
Requires:
|
||||
- `BANGUI_SESSION_SECRET` env var must be set (see [Backend-Development.md](Backend-Development.md) for setup)
|
||||
- Stack must be startable via `make up` (Docker/Podman + compose installed)
|
||||
- `rfbrowser init` is run automatically by the `e2e` target (Playwright browsers downloaded on first run; re-run after `robotframework-browser` version changes)
|
||||
|
||||
### HTML Report
|
||||
|
||||
After a run, open `e2e/results/report.html` in a browser to view the detailed HTML report with screenshots on failure.
|
||||
|
||||
### Writing New E2E Tests
|
||||
|
||||
Place new `.robot` files in `e2e/tests/`. Use `e2e/resources/common.resource` for shared variables and setup/teardown, and `e2e/resources/auth.resource` for the `Login As Admin` keyword.
|
||||
17
Makefile
17
Makefile
@@ -12,6 +12,7 @@
|
||||
# make logs — tail logs for all debug services
|
||||
# make restart — restart the debug stack
|
||||
# make dev-ban-test — one-command smoke test of the ban pipeline
|
||||
# make e2e — run the Robot Framework E2E test suite
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
|
||||
COMPOSE_FILE := Docker/compose.debug.yml
|
||||
@@ -39,7 +40,7 @@ COMPOSE := $(shell command -v podman-compose 2>/dev/null \
|
||||
# Detect available container runtime (podman or docker).
|
||||
RUNTIME := $(shell command -v podman 2>/dev/null || echo "docker")
|
||||
|
||||
.PHONY: up down build restart logs clean dev-ban-test
|
||||
.PHONY: up down build restart logs clean dev-ban-test e2e
|
||||
|
||||
## Start the debug stack (detached).
|
||||
## Ensures log stub files exist so fail2ban can open them on first start.
|
||||
@@ -71,6 +72,20 @@ clean:
|
||||
$(RUNTIME) rmi $(DEV_IMAGES) 2>/dev/null || true
|
||||
@echo "All debug volumes and local images removed. Run 'make up' to rebuild and start fresh."
|
||||
|
||||
## Run the Robot Framework E2E test suite.
|
||||
## Requires: stack up (make up), BANGUI_SESSION_SECRET env var set.
|
||||
## Installs: pip install -r e2e/requirements.txt && rfbrowser init
|
||||
e2e: up
|
||||
@echo "Waiting for stack to be healthy..."
|
||||
@timeout=120; \
|
||||
until curl -sf http://localhost:8000/api/health > /dev/null 2>&1; do \
|
||||
sleep 5; timeout=$$((timeout-5)); \
|
||||
if [ $$timeout -le 0 ]; then echo "Backend not healthy after 120s"; exit 1; fi; \
|
||||
done
|
||||
pip install -r e2e/requirements.txt -q
|
||||
rfbrowser init --quiet
|
||||
robot --outputdir e2e/results e2e/tests/
|
||||
|
||||
## One-command smoke test for the ban pipeline:
|
||||
## 1. Start fail2ban, 2. write failure lines, 3. check ban status.
|
||||
dev-ban-test:
|
||||
|
||||
2
e2e/requirements.txt
Normal file
2
e2e/requirements.txt
Normal file
@@ -0,0 +1,2 @@
|
||||
robotframework>=7
|
||||
robotframework-browser>=18
|
||||
31
e2e/resources/auth.resource
Normal file
31
e2e/resources/auth.resource
Normal file
@@ -0,0 +1,31 @@
|
||||
*** Settings ***
|
||||
Resource ${CURDIR}/common.resource
|
||||
|
||||
*** Keywords ***
|
||||
Login As Admin
|
||||
# Check setup status.
|
||||
${response}= GET ${BACKEND_URL}/api/setup/status
|
||||
${body}= Set Variable ${response.json()}
|
||||
|
||||
IF ${body}[setup_complete] == ${false}
|
||||
# Complete the setup wizard with the dev master password ("Hallo123!").
|
||||
${password}= Set Variable Hallo123!
|
||||
${hashed}= Evaluate "sha256('${password}'.encode()).hexdigest()" modules=hashlib
|
||||
${setup_payload}= Create Dictionary password=${hashed}
|
||||
POST ${BACKEND_URL}/api/setup/complete json=${setup_payload}
|
||||
|
||||
# Retry login after setup.
|
||||
${response}= GET ${BACKEND_URL}/api/auth/login
|
||||
END
|
||||
|
||||
# Perform login.
|
||||
${password}= Set Variable Hallo123!
|
||||
${hashed}= Evaluate "sha256('${password}'.encode()).hexdigest()" modules=hashlib
|
||||
${login_payload}= Create Dictionary password=${hashed}
|
||||
${response}= POST ${BACKEND_URL}/api/auth/login json=${login_payload}
|
||||
|
||||
# Store session cookie for subsequent requests.
|
||||
${cookies}= Get Cookies
|
||||
${session_cookie}= Get Cookie Value bangui_session
|
||||
Set Browser Variables session_cookie=${session_cookie}
|
||||
Log Logged in as admin.
|
||||
25
e2e/resources/common.resource
Normal file
25
e2e/resources/common.resource
Normal file
@@ -0,0 +1,25 @@
|
||||
*** Settings ***
|
||||
Library Browser
|
||||
Library HttpLibrary
|
||||
|
||||
Variables ${CURDIR}/../../.env
|
||||
|
||||
# Health check timeout for suite setup (120 s poll interval).
|
||||
Suite Setup Wait For Backend Health timeout=120 interval=5
|
||||
|
||||
*** Variables ***
|
||||
${FRONTEND_URL} http://localhost:5173
|
||||
${BACKEND_URL} http://localhost:8000
|
||||
|
||||
*** Keywords ***
|
||||
Wait For Backend Health
|
||||
[Arguments] ${timeout}=120 ${interval}=5
|
||||
${deadline}= Evaluate time.time() + ${timeout}
|
||||
WHILE True
|
||||
${now}= Evaluate time.time()
|
||||
IF ${now} >= ${deadline} FAIL Backend did not become healthy within ${timeout} seconds
|
||||
${response}= GET ${BACKEND_URL}/api/health expected_status=200
|
||||
IF ${response.status} == 200 BREAK
|
||||
Sleep ${interval}
|
||||
END
|
||||
Log Backend is healthy.
|
||||
13
e2e/tests/01_page_loading.robot
Normal file
13
e2e/tests/01_page_loading.robot
Normal file
@@ -0,0 +1,13 @@
|
||||
*** Settings ***
|
||||
Resource ${CURDIR}/../../resources/common.resource
|
||||
|
||||
*** Test Cases ***
|
||||
Page Loads And Shows Navigation
|
||||
New Browser chromium headless=${TRUE}
|
||||
New Page ${FRONTEND_URL}
|
||||
|
||||
# Confirm the page title or root element is present.
|
||||
${title}= Get Title
|
||||
Should Not Be Empty ${title}
|
||||
|
||||
Close Browser
|
||||
16
e2e/tests/02_ban_records.robot
Normal file
16
e2e/tests/02_ban_records.robot
Normal file
@@ -0,0 +1,16 @@
|
||||
*** Settings ***
|
||||
Resource ${CURDIR}/../../resources/common.resource
|
||||
Resource ${CURDIR}/../../resources/auth.resource
|
||||
|
||||
*** Test Cases ***
|
||||
Ban Records Are Visible
|
||||
New Browser chromium headless=${TRUE}
|
||||
Login As Admin
|
||||
|
||||
Go To ${FRONTEND_URL}/bans
|
||||
|
||||
# Basic presence check — the ban table or empty state should be present.
|
||||
${content}= Get Page Source
|
||||
Should Not Be Empty ${content}
|
||||
|
||||
Close Browser
|
||||
15
e2e/tests/03_blocklist_import.robot
Normal file
15
e2e/tests/03_blocklist_import.robot
Normal file
@@ -0,0 +1,15 @@
|
||||
*** Settings ***
|
||||
Resource ${CURDIR}/../../resources/common.resource
|
||||
Resource ${CURDIR}/../../resources/auth.resource
|
||||
|
||||
*** Test Cases ***
|
||||
Blocklist Import Page Opens
|
||||
New Browser chromium headless=${TRUE}
|
||||
Login As Admin
|
||||
|
||||
Go To ${FRONTEND_URL}/blocklists
|
||||
|
||||
${content}= Get Page Source
|
||||
Should Not Be Empty ${content}
|
||||
|
||||
Close Browser
|
||||
15
e2e/tests/04_config_edit.robot
Normal file
15
e2e/tests/04_config_edit.robot
Normal file
@@ -0,0 +1,15 @@
|
||||
*** Settings ***
|
||||
Resource ${CURDIR}/../../resources/common.resource
|
||||
Resource ${CURDIR}/../../resources/auth.resource
|
||||
|
||||
*** Test Cases ***
|
||||
Config Edit Page Opens
|
||||
New Browser chromium headless=${TRUE}
|
||||
Login As Admin
|
||||
|
||||
Go To ${FRONTEND_URL}/config
|
||||
|
||||
${content}= Get Page Source
|
||||
Should Not Be Empty ${content}
|
||||
|
||||
Close Browser
|
||||
Reference in New Issue
Block a user