BanGUI/e2e/Instructions.md

# E2E Tests — Running Robot Framework Tests

## Setup

Install dependencies:
```bash
pip install -r requirements.txt
rfbrowser init
```

## Run All Tests

```bash
robot --outputdir results --log log.html --report report.html tests/
```

## Run Specific Test File

```bash
robot --outputdir results tests/01_page_loading.robot
```

## Run with Browser Visible

```bash
robot --outputdir results --variable BROWSER:chromium tests/
```

## View Results

Open `results/log.html` or `results/report.html` in a browser.

---

# AI Agent — General Instructions

You are an autonomous coding agent working on **BanGUI**, a web application for monitoring, managing, and configuring fail2ban through a clean web interface. This document defines how you operate, what rules you follow, and which workflow you repeat for every task.

Read this document completely before starting any work.

---

## 1. Project Context

BanGUI consists of two main parts:

| Layer | Stack | Docs |
|---|---|---|
| **Backend** | Python 3.12+, FastAPI, Pydantic v2, aiosqlite, structlog | [Backend-Development.md](Backend-Development.md) |
| **Frontend** | TypeScript, React, Fluent UI v9, Vite | [Web-Development.md](Web-Development.md) |

Supporting documentation you must know and respect:

| Document | Purpose |
|---|---|
| [Features.md](Features.md) | Complete feature list and expected behaviour |
| [Architekture.md](Architekture.md) | System architecture, component relationships, data flow |
| [Web-Design.md](Web-Design.md) | Visual design rules, theming, layout, spacing, motion |
| [Backend-Development.md](Backend-Development.md) | Backend coding rules, project structure, conventions |
| [Web-Development.md](Web-Development.md) | Frontend coding rules, project structure, conventions |

**Always** consult the relevant document before writing code. If your planned change contradicts any rule defined in those documents, the document wins — adjust your approach.

---

## 2. General Rules

### 2.1 Follow the Docs

- Every coding convention, naming rule, project structure decision, and library choice is defined in the development docs. Do not deviate.
- Backend code follows [Backend-Development.md](Backend-Development.md) — strict typing, async only, structlog, Pydantic models, layered architecture (routers → services → repositories).
- Frontend code follows [Web-Development.md](Web-Development.md) — strict TypeScript, Fluent UI v9 only, `makeStyles` for styling, typed API calls, hooks for state.
- Visual decisions follow [Web-Design.md](Web-Design.md) — Fluent design tokens, semantic colour slots, 4 px spacing grid, correct elevation and motion.

### 2.2 Write Production-Quality Code

- Write code as if it ships today. No TODOs, no placeholders, no half-implementations.
- Every function has explicit type annotations (Python) or type signatures (TypeScript).
- Every public function has a docstring (Python — Google style) or JSDoc comment (TypeScript).
- No `any` in TypeScript. No `Any` in Python (unless justified with a comment).
- No magic numbers or strings — use named constants.
- No dead code, no commented-out blocks, no unused imports.

### 2.3 Keep It Small and Focused

- One function does one thing.
- One component per file.
- One service per domain.
- If a file grows beyond ~150 lines (components) or ~200 lines (services), split it.

### 2.4 Never Break Existing Code

- Before changing any file, understand what it does and who depends on it.
- Run the existing test suite before and after your changes. If tests fail after your change, fix them before moving on.
- Do not remove or rename public APIs without updating all callers.

### 2.5 Think Before You Code

- Read the task description carefully. If it is ambiguous, check [Features.md](Features.md) and [Architekture.md](Architekture.md) for clarification.
- Plan your changes before writing code. Identify which files are affected, which layers are involved, and what tests are needed.
- Prefer the simplest correct solution. Do not over-engineer.

---

## 3. Task Workflow

Repeat the following cycle for every task. Do not skip steps.


### Step 1 — Plan Your Steps

- Break the task into concrete implementation steps.
- Identify which files need to be created, modified, or deleted.
- Identify which layers are affected (router, service, repository, model, component, hook, page, type, etc.).
- Identify edge cases and error scenarios.
- Write down your plan before touching any code.

### Step 2 — Write Code

- Implement the feature or fix following the plan.
- Follow all rules from the relevant development docs:
  - Backend → [Backend-Development.md](Backend-Development.md)
  - Frontend → [Web-Development.md](Web-Development.md)
  - Design → [Web-Design.md](Web-Design.md)
  - Architecture → [Architekture.md](Architekture.md)
- Write clean, well-structured, fully typed code.
- Keep commits atomic — one logical change per commit.

### Step 3 — Add Logging

- Add structured log statements at key points in new or modified code.
- Backend: use **structlog** with contextual key-value pairs — never `print()`.
- Log at appropriate levels: `info` for operational events, `warning` for recoverable issues, `error` for failures.
- Never log sensitive data (passwords, tokens, session IDs).

### Step 4 — Write Tests

- Write tests for every new or changed piece of functionality.
- Backend: use `pytest` + `pytest-asyncio` + `httpx.AsyncClient`. See [Backend-Development.md § 9](Backend-Development.md).
- Frontend: test components and hooks according to the frontend test setup.
- Test the happy path **and** error/edge cases.
- Mock external dependencies — tests must never touch real infrastructure.
- Follow the naming pattern: `test_<unit>_<scenario>_<expected>`.

### Step 5 — Review Your Code

Run a thorough self-review before considering the task done. Check **all** of the following:

#### 5.1 — Warnings and Errors

- Backend: run `ruff check` and `mypy --strict` (or `pyright --strict`). Fix every warning and error.
- Frontend: run `tsc --noEmit` and `eslint`. Fix every warning and error.
- Zero warnings, zero errors — no exceptions.

#### 5.2 — Test Coverage

- Run the test suite with coverage enabled.
- Aim for **>80 % line coverage** overall.
- Critical paths (auth, banning, scheduling, API endpoints) must be **100 %** covered.
- If coverage is below the threshold, write additional tests before proceeding.

#### 5.3 — Coding Principles

Verify your code against the coding principles defined in [Backend-Development.md § 13](Backend-Development.md) and [Web-Development.md](Web-Development.md):

- [ ] **Clean Code** — Meaningful names, small functions, no magic values, guard clauses over deep nesting.
- [ ] **Separation of Concerns** — Each module has a single, well-defined responsibility. Layers are not mixed.
- [ ] **Single Responsibility Principle** — Every class and function has one reason to change.
- [ ] **DRY** — No duplicated logic. Shared behaviour is extracted.
- [ ] **KISS** — The simplest correct solution is used. No over-engineering.
- [ ] **Type Safety** — All types are explicit. No `any` / `Any`. No `# type: ignore` without justification.

#### 5.4 — Architecture Compliance

Verify against [Architekture.md](Architekture.md) and the project structure rules:

- [ ] Files are in the correct directories (routers in `routers/`, services in `services/`, components in `components/`, etc.).
- [ ] Dependencies flow in the right direction (routers → services → repositories; pages → components → hooks).
- [ ] No circular imports.
- [ ] No business logic in routers or components.
- [ ] No HTTP/framework concerns in services or repositories.
- [ ] Pydantic models separate request, response, and domain shapes.
- [ ] Frontend types live in `types/`, not scattered across components.

### Step 6 — Update Documentation

- If your change introduces new features, new endpoints, new components, or changes existing behaviour, update the relevant docs:
  - [Features.md](Features.md) — if feature behaviour changed.
  - [Architekture.md](Architekture.md) — if new modules, services, or data flows were added.
  - [Backend-Development.md](Backend-Development.md) or [Web-Development.md](Web-Development.md) — if new conventions were established.
- Keep documentation accurate and in sync with the code. Outdated docs are worse than no docs.


## 5. When You Are Stuck

- Re-read the task description and the relevant docs.
- Search the existing codebase for similar patterns — follow established conventions.
- Check the fail2ban source code in `fail2ban-master/` if you need to understand how fail2ban works internally.
- If a decision is genuinely ambiguous and no document covers it, choose the simplest option that is consistent with existing code and document your reasoning in a code comment.

---

## 6. What You Must Never Do

- **Never** commit code that does not compile or has type errors.
- **Never** commit code without tests.
- **Never** use libraries that are explicitly forbidden in the development docs.
- **Never** bypass the linter or type checker with blanket ignores.
- **Never** hard-code secrets, passwords, or tokens.
- **Never** push directly to `main` — always use feature branches.
- **Never** skip the review step — sloppy code compounds over time.
- **Never** leave a task half-done — finish it or revert it.

---

## 7. First-Run Setup

### Initialize the Development Environment

Before starting the stack for the first time, set up the required environment variables:

1. **Copy the example environment file:**
   ```bash
   cp .env.example .env
   ```

2. **Generate a session secret:**
   ```bash
   python -c 'import secrets; print(secrets.token_hex(32))'
   ```
   Copy the output and paste it as the value for `BANGUI_SESSION_SECRET` in your `.env` file.

3. **Optional: Customize other settings**
   - Edit `.env` to adjust timezone, port numbers, or other settings
   - Default values are sensible for development (UTC, ports 8000/5173)

4. **Start the stack:**
   ```bash
   make up
   ```

**Note:** The session secret is critical for security. Do not commit `.env` to version control — it is already in `.gitignore`. Each environment (dev, staging, production) must have its own unique secret.

---

## 8. Dev Quick-Reference

### Start / stop the stack

```bash
make up      # start all containers (from repo root)
make down    # stop all containers
make logs    # tail logs
```

Backend: `http://127.0.0.1:8000` · Frontend (Vite proxy): `http://127.0.0.1:5173`

### API login (dev)

The backend accepts **plaintext** passwords (no frontend hashing).  
The session cookie is named `bangui_session`.

```bash
# Dev master password: Hallo123!
TOKEN=$(curl -s -X POST http://127.0.0.1:8000/api/v1/auth/login \
  -H 'Content-Type: application/json' \
  -d '{"password":"Hallo123!"}' \
  | python3 -c 'import sys,json; print(json.load(sys.stdin)["expires_at"])')
# Login sets bangui_session cookie automatically.
# Use the cookie in subsequent requests:
curl -b "bangui_session=$(cat ~/.bangui_session)" http://127.0.0.1:8000/api/v1/dashboard/status
```

---

## E2E Testing Notes

Debugging failures: open `results/log.html` (not output.xml) for full request/response traces.

### Auth (RequestsLibrary vs Browser)

`Login As Admin` uses browser JavaScript (`Evaluate JavaScript`) — session cookie lives in the **browser context only**.
Any `RequestsLibrary` keyword (GET/POST to API) in the same test needs its own auth session.
Use `Login Via HTTP` from `auth.resource` to get a session cookie for RequestsLibrary calls.
RequestsLibrary and Browser library share no state.

### CSRF Protection

Backend enforces CSRF on all POST/PUT/DELETE via `X-BanGUI-Request: 1` header.
RequestsLibrary sessions must include this header on creation:
```
Create Session    bangsess    ${BACKEND_URL}    headers=${headers}    # headers = {X-BanGUI-Request: 1}
```
Then all requests on that session inherit it.

### Robot Variable Type Rules

Bare values in `Create Dictionary` are **strings**, not Python types:
- `enabled=true` → `"true"` (string) — backend Pydantic expects boolean → validation fails
- Fix: `enabled=${TRUE}` for booleans, `${60}` for integers
- `${NONE}` is Robot's null/None equivalent
- `${len(sources)}` is invalid — use `Get Length    ${sources}` keyword instead

### Network Mode (podman-compose)

With `network_mode: host`, containers share the host network namespace.
- Backend can reach host's `127.0.0.1:PORT` — needed for mock HTTP servers
- Frontend must set `VITE_BACKEND_URL=http://localhost:8000` since `localhost` now resolves to host
- Vite proxy works when frontend also uses `network_mode: host` (same namespace)

### SSRF Protection

Blocklist source URLs are validated: hostname must resolve to a **public** IP.
- `127.0.0.1` and other loopback addresses are rejected
- In dev mode (`BANGUI_LOG_LEVEL=debug`), loopback is allowed for e2e testing
- For real e2e tests, use a public blocklist URL or ensure mock server is reachable

### API Response Keys

Always verify response shape in `results/log.html` — field names differ from expectations:
- Blocklist import log: `items` not `entries`
- Error responses: `{"code": "...", "detail": "..."}`

### Process Library Teardown

`Terminate Process` fails if the process alias doesn't exist (mock server not started).
Wrap in `Run Keyword And Return Status` to avoid teardown cascading failures.