diff --git a/README.md b/README.md new file mode 100644 index 0000000..a78b12c --- /dev/null +++ b/README.md @@ -0,0 +1,140 @@ +# Aniworld Download Manager + +A web-based anime download manager with REST API, WebSocket real-time updates, and a modern web interface. + +## Features + +- Web interface for managing anime library +- REST API for programmatic access +- WebSocket real-time progress updates +- Download queue with priority management +- Automatic library scanning for missing episodes +- JWT-based authentication +- SQLite database for persistence + +## Quick Start + +### Prerequisites + +- Python 3.10+ +- Conda (recommended) or virtualenv + +### Installation + +1. Clone the repository: + +```bash +git clone https://github.com/your-repo/aniworld.git +cd aniworld +``` + +2. Create and activate conda environment: + +```bash +conda create -n AniWorld python=3.10 +conda activate AniWorld +``` + +3. Install dependencies: + +```bash +pip install -r requirements.txt +``` + +4. Start the server: + +```bash +python -m uvicorn src.server.fastapi_app:app --host 127.0.0.1 --port 8000 +``` + +5. Open http://127.0.0.1:8000 in your browser + +### First-Time Setup + +1. Navigate to http://127.0.0.1:8000/setup +2. Set a master password (minimum 8 characters, mixed case, number, special character) +3. Configure your anime directory path +4. Login with your master password + +## Documentation + +| Document | Description | +| ---------------------------------------------- | -------------------------------- | +| [docs/API.md](docs/API.md) | REST API and WebSocket reference | +| [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) | System architecture and design | +| [docs/CONFIGURATION.md](docs/CONFIGURATION.md) | Configuration options | +| [docs/DATABASE.md](docs/DATABASE.md) | Database schema | +| [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) | Developer setup guide | +| [docs/TESTING.md](docs/TESTING.md) | Testing guidelines | + +## Project Structure + +``` +src/ ++-- cli/ # CLI interface (legacy) ++-- config/ # Application settings ++-- core/ # Domain logic +| +-- SeriesApp.py # Main application facade +| +-- SerieScanner.py # Directory scanning +| +-- entities/ # Domain entities +| +-- providers/ # External provider adapters ++-- server/ # FastAPI web server + +-- api/ # REST API endpoints + +-- services/ # Business logic + +-- models/ # Pydantic models + +-- database/ # SQLAlchemy ORM + +-- middleware/ # Auth, rate limiting +``` + +## API Endpoints + +| Endpoint | Description | +| ------------------------------ | -------------------------------- | +| `POST /api/auth/login` | Authenticate and get JWT token | +| `GET /api/anime` | List anime with missing episodes | +| `GET /api/anime/search?query=` | Search for anime | +| `POST /api/queue/add` | Add episodes to download queue | +| `POST /api/queue/start` | Start queue processing | +| `GET /api/queue/status` | Get queue status | +| `WS /ws/connect` | WebSocket for real-time updates | + +See [docs/API.md](docs/API.md) for complete API reference. + +## Configuration + +Environment variables (via `.env` file): + +| Variable | Default | Description | +| ----------------- | ------------------------------ | ---------------------- | +| `JWT_SECRET_KEY` | (random) | Secret for JWT signing | +| `DATABASE_URL` | `sqlite:///./data/aniworld.db` | Database connection | +| `ANIME_DIRECTORY` | (empty) | Path to anime library | +| `LOG_LEVEL` | `INFO` | Logging level | + +See [docs/CONFIGURATION.md](docs/CONFIGURATION.md) for all options. + +## Running Tests + +```bash +# Run all tests +conda run -n AniWorld python -m pytest tests/ -v + +# Run unit tests only +conda run -n AniWorld python -m pytest tests/unit/ -v + +# Run integration tests +conda run -n AniWorld python -m pytest tests/integration/ -v +``` + +## Technology Stack + +- **Web Framework**: FastAPI 0.104.1 +- **Database**: SQLite + SQLAlchemy 2.0 +- **Auth**: JWT (python-jose) + passlib +- **Validation**: Pydantic 2.5 +- **Logging**: structlog +- **Testing**: pytest + pytest-asyncio + +## License + +MIT License diff --git a/SERVER_COMMANDS.md b/SERVER_COMMANDS.md deleted file mode 100644 index d0a0c7c..0000000 --- a/SERVER_COMMANDS.md +++ /dev/null @@ -1,215 +0,0 @@ -# Server Management Commands - -Quick reference for starting, stopping, and managing the Aniworld server. - -## Start Server - -### Using the start script (Recommended) - -```bash -./start_server.sh -``` - -### Using conda directly - -```bash -conda run -n AniWorld python run_server.py -``` - -### Using uvicorn directly - -```bash -conda run -n AniWorld python -m uvicorn src.server.fastapi_app:app --host 127.0.0.1 --port 8000 --reload -``` - -## Stop Server - -### Using the stop script (Recommended) - -```bash -./stop_server.sh -``` - -### Manual commands - -**Kill uvicorn processes:** - -```bash -pkill -f "uvicorn.*fastapi_app:app" -``` - -**Kill process on port 8000:** - -```bash -lsof -ti:8000 | xargs kill -9 -``` - -**Kill run_server.py processes:** - -```bash -pkill -f "run_server.py" -``` - -## Check Server Status - -**Check if port 8000 is in use:** - -```bash -lsof -i:8000 -``` - -**Check for running uvicorn processes:** - -```bash -ps aux | grep uvicorn -``` - -**Check server is responding:** - -```bash -curl http://127.0.0.1:8000/api/health -``` - -## Restart Server - -```bash -./stop_server.sh && ./start_server.sh -``` - -## Common Issues - -### "Address already in use" Error - -**Problem:** Port 8000 is already occupied - -**Solution:** - -```bash -./stop_server.sh -# or -lsof -ti:8000 | xargs kill -9 -``` - -### Server not responding - -**Check logs:** - -```bash -tail -f logs/app.log -``` - -**Check if process is running:** - -```bash -ps aux | grep uvicorn -``` - -### Cannot connect to server - -**Verify server is running:** - -```bash -curl http://127.0.0.1:8000/api/health -``` - -**Check firewall:** - -```bash -sudo ufw status -``` - -## Development Mode - -**Run with auto-reload:** - -```bash -./start_server.sh # Already includes --reload -``` - -**Run with custom port:** - -```bash -conda run -n AniWorld python -m uvicorn src.server.fastapi_app:app --host 127.0.0.1 --port 8080 --reload -``` - -**Run with debug logging:** - -```bash -export LOG_LEVEL=DEBUG -./start_server.sh -``` - -## Production Mode - -**Run without auto-reload:** - -```bash -conda run -n AniWorld python -m uvicorn src.server.fastapi_app:app --host 0.0.0.0 --port 8000 --workers 4 -``` - -**Run with systemd (Linux):** - -```bash -sudo systemctl start aniworld -sudo systemctl stop aniworld -sudo systemctl restart aniworld -sudo systemctl status aniworld -``` - -## URLs - -- **Web Interface:** http://127.0.0.1:8000 -- **API Documentation:** http://127.0.0.1:8000/api/docs -- **Login Page:** http://127.0.0.1:8000/login -- **Queue Management:** http://127.0.0.1:8000/queue -- **Health Check:** http://127.0.0.1:8000/api/health - -## Default Credentials - -- **Password:** `Hallo123!` - -## Log Files - -- **Application logs:** `logs/app.log` -- **Download logs:** `logs/downloads/` -- **Error logs:** Check console output or systemd journal - -## Quick Troubleshooting - -| Symptom | Solution | -| ------------------------ | ------------------------------------ | -| Port already in use | `./stop_server.sh` | -| Server won't start | Check `logs/app.log` | -| 404 errors | Verify URL and check routing | -| WebSocket not connecting | Check server is running and firewall | -| Slow responses | Check system resources (`htop`) | -| Database errors | Check `data/` directory permissions | - -## Environment Variables - -```bash -# Set log level -export LOG_LEVEL=DEBUG|INFO|WARNING|ERROR - -# Set server port -export PORT=8000 - -# Set host -export HOST=127.0.0.1 - -# Set workers (production) -export WORKERS=4 -``` - -## Related Scripts - -- `start_server.sh` - Start the server -- `stop_server.sh` - Stop the server -- `run_server.py` - Python server runner -- `scripts/setup.py` - Initial setup - -## More Information - -- [User Guide](docs/user_guide.md) -- [API Reference](docs/api_reference.md) -- [Deployment Guide](docs/deployment.md) diff --git a/diagrams/README.md b/diagrams/README.md new file mode 100644 index 0000000..1e6144d --- /dev/null +++ b/diagrams/README.md @@ -0,0 +1,23 @@ +# Architecture Diagrams + +This directory contains architecture diagram source files for the Aniworld documentation. + +## Diagrams + +### System Architecture (Mermaid) + +See [system-architecture.mmd](system-architecture.mmd) for the system overview diagram. + +### Rendering + +Diagrams can be rendered using: + +- Mermaid Live Editor: https://mermaid.live/ +- VS Code Mermaid extension +- GitHub/GitLab native Mermaid support + +## Formats + +- `.mmd` - Mermaid diagram source files +- `.svg` - Exported vector graphics (add when needed) +- `.png` - Exported raster graphics (add when needed) diff --git a/diagrams/download-flow.mmd b/diagrams/download-flow.mmd new file mode 100644 index 0000000..66800ea --- /dev/null +++ b/diagrams/download-flow.mmd @@ -0,0 +1,44 @@ +%%{init: {'theme': 'base'}}%% +sequenceDiagram + participant Client + participant FastAPI + participant AuthMiddleware + participant DownloadService + participant ProgressService + participant WebSocketService + participant SeriesApp + participant Database + + Note over Client,Database: Download Flow + + %% Add to queue + Client->>FastAPI: POST /api/queue/add + FastAPI->>AuthMiddleware: Validate JWT + AuthMiddleware-->>FastAPI: OK + FastAPI->>DownloadService: add_to_queue() + DownloadService->>Database: save_item() + Database-->>DownloadService: item_id + DownloadService-->>FastAPI: [item_ids] + FastAPI-->>Client: 201 Created + + %% Start queue + Client->>FastAPI: POST /api/queue/start + FastAPI->>AuthMiddleware: Validate JWT + AuthMiddleware-->>FastAPI: OK + FastAPI->>DownloadService: start_queue_processing() + + loop For each pending item + DownloadService->>SeriesApp: download_episode() + + loop Progress updates + SeriesApp->>ProgressService: emit("progress_updated") + ProgressService->>WebSocketService: broadcast_to_room() + WebSocketService-->>Client: WebSocket message + end + + SeriesApp-->>DownloadService: completed + DownloadService->>Database: update_status() + end + + DownloadService-->>FastAPI: OK + FastAPI-->>Client: 200 OK diff --git a/diagrams/system-architecture.mmd b/diagrams/system-architecture.mmd new file mode 100644 index 0000000..6445d57 --- /dev/null +++ b/diagrams/system-architecture.mmd @@ -0,0 +1,82 @@ +%%{init: {'theme': 'base', 'themeVariables': { 'primaryColor': '#4a90d9'}}}%% +flowchart TB + subgraph Clients["Client Layer"] + Browser["Web Browser
(HTML/CSS/JS)"] + CLI["CLI Client
(Main.py)"] + end + + subgraph Server["Server Layer (FastAPI)"] + direction TB + Middleware["Middleware
Auth, Rate Limit, Error Handler"] + + subgraph API["API Routers"] + AuthAPI["/api/auth"] + AnimeAPI["/api/anime"] + QueueAPI["/api/queue"] + ConfigAPI["/api/config"] + SchedulerAPI["/api/scheduler"] + HealthAPI["/health"] + WebSocketAPI["/ws"] + end + + subgraph Services["Services"] + AuthService["AuthService"] + AnimeService["AnimeService"] + DownloadService["DownloadService"] + ConfigService["ConfigService"] + ProgressService["ProgressService"] + WebSocketService["WebSocketService"] + end + end + + subgraph Core["Core Layer"] + SeriesApp["SeriesApp"] + SerieScanner["SerieScanner"] + SerieList["SerieList"] + end + + subgraph Data["Data Layer"] + SQLite[(SQLite
aniworld.db)] + ConfigJSON[(config.json)] + FileSystem[(File System
Anime Directory)] + end + + subgraph External["External"] + Provider["Anime Provider
(aniworld.to)"] + end + + %% Client connections + Browser -->|HTTP/WebSocket| Middleware + CLI -->|Direct| SeriesApp + + %% Middleware to API + Middleware --> API + + %% API to Services + AuthAPI --> AuthService + AnimeAPI --> AnimeService + QueueAPI --> DownloadService + ConfigAPI --> ConfigService + SchedulerAPI --> AnimeService + WebSocketAPI --> WebSocketService + + %% Services to Core + AnimeService --> SeriesApp + DownloadService --> SeriesApp + + %% Services to Data + AuthService --> ConfigJSON + ConfigService --> ConfigJSON + DownloadService --> SQLite + AnimeService --> SQLite + + %% Core to Data + SeriesApp --> SerieScanner + SeriesApp --> SerieList + SerieScanner --> FileSystem + SerieScanner --> Provider + + %% Event flow + ProgressService -.->|Events| WebSocketService + DownloadService -.->|Progress| ProgressService + WebSocketService -.->|Broadcast| Browser diff --git a/docs/API.md b/docs/API.md new file mode 100644 index 0000000..5ad0b20 --- /dev/null +++ b/docs/API.md @@ -0,0 +1,1176 @@ +# API Documentation + +## Document Purpose + +This document provides comprehensive REST API and WebSocket reference for the Aniworld application. + +Source: [src/server/fastapi_app.py](../src/server/fastapi_app.py#L1-L252) + +--- + +## 1. API Overview + +### Base URL and Versioning + +| Environment | Base URL | +| ------------------- | --------------------------------- | +| Local Development | `http://127.0.0.1:8000` | +| API Documentation | `http://127.0.0.1:8000/api/docs` | +| ReDoc Documentation | `http://127.0.0.1:8000/api/redoc` | + +The API does not use versioning prefixes. All endpoints are available under `/api/*`. + +Source: [src/server/fastapi_app.py](../src/server/fastapi_app.py#L177-L184) + +### Authentication + +The API uses JWT Bearer Token authentication. + +**Header Format:** + +``` +Authorization: Bearer +``` + +**Public Endpoints (no authentication required):** + +- `/api/auth/*` - Authentication endpoints +- `/api/health` - Health check endpoints +- `/api/docs`, `/api/redoc` - API documentation +- `/static/*` - Static files +- `/`, `/login`, `/setup`, `/queue` - UI pages + +Source: [src/server/middleware/auth.py](../src/server/middleware/auth.py#L39-L52) + +### Content Types + +| Direction | Content-Type | +| --------- | ----------------------------- | +| Request | `application/json` | +| Response | `application/json` | +| WebSocket | `application/json` (messages) | + +### Common Headers + +| Header | Required | Description | +| --------------- | -------- | ------------------------------------ | +| `Authorization` | Yes\* | Bearer token for protected endpoints | +| `Content-Type` | Yes | `application/json` for POST/PUT | +| `Origin` | No | Required for CORS preflight | + +\*Not required for public endpoints listed above. + +--- + +## 2. Authentication Endpoints + +Prefix: `/api/auth` + +Source: [src/server/api/auth.py](../src/server/api/auth.py#L1-L180) + +### POST /api/auth/setup + +Initial setup endpoint to configure the master password. Can only be called once. + +**Request Body:** + +```json +{ + "master_password": "string (min 8 chars, mixed case, number, special char)", + "anime_directory": "string (optional, path to anime folder)" +} +``` + +**Response (201 Created):** + +```json +{ + "status": "ok" +} +``` + +**Errors:** + +- `400 Bad Request` - Master password already configured or invalid password + +Source: [src/server/api/auth.py](../src/server/api/auth.py#L28-L90) + +### POST /api/auth/login + +Validate master password and return JWT token. + +**Request Body:** + +```json +{ + "password": "string", + "remember": false +} +``` + +**Response (200 OK):** + +```json +{ + "access_token": "eyJ...", + "token_type": "bearer", + "expires_at": "2025-12-14T10:30:00Z" +} +``` + +**Errors:** + +- `401 Unauthorized` - Invalid credentials +- `429 Too Many Requests` - Account locked due to failed attempts + +Source: [src/server/api/auth.py](../src/server/api/auth.py#L93-L124) + +### POST /api/auth/logout + +Logout by revoking token. + +**Response (200 OK):** + +```json +{ + "status": "ok", + "message": "Logged out successfully" +} +``` + +Source: [src/server/api/auth.py](../src/server/api/auth.py#L127-L140) + +### GET /api/auth/status + +Return whether master password is configured and if caller is authenticated. + +**Response (200 OK):** + +```json +{ + "configured": true, + "authenticated": true +} +``` + +Source: [src/server/api/auth.py](../src/server/api/auth.py#L157-L162) + +--- + +## 3. Anime Endpoints + +Prefix: `/api/anime` + +Source: [src/server/api/anime.py](../src/server/api/anime.py#L1-L812) + +### Series Identifier Convention + +The API uses two identifier fields: + +| Field | Purpose | Example | +| -------- | ---------------------------------------------------- | -------------------------- | +| `key` | **Primary identifier** - provider-assigned, URL-safe | `"attack-on-titan"` | +| `folder` | Metadata only - filesystem folder name | `"Attack on Titan (2013)"` | + +Use `key` for all API operations. The `folder` field is for display purposes only. + +### GET /api/anime/status + +Get anime library status information. + +**Authentication:** Required + +**Response (200 OK):** + +```json +{ + "directory": "/path/to/anime", + "series_count": 42 +} +``` + +Source: [src/server/api/anime.py](../src/server/api/anime.py#L28-L58) + +### GET /api/anime + +List library series that have missing episodes. + +**Authentication:** Required + +**Query Parameters:** +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `page` | int | 1 | Page number (must be positive) | +| `per_page` | int | 20 | Items per page (max 1000) | +| `sort_by` | string | null | Sort field: `title`, `id`, `name`, `missing_episodes` | +| `filter` | string | null | Filter term | + +**Response (200 OK):** + +```json +[ + { + "key": "beheneko-the-elf-girls-cat", + "name": "Beheneko", + "site": "aniworld.to", + "folder": "beheneko the elf girls cat (2025)", + "missing_episodes": { "1": [1, 2, 3, 4] }, + "link": "" + } +] +``` + +Source: [src/server/api/anime.py](../src/server/api/anime.py#L155-L303) + +### GET /api/anime/search + +Search the provider for anime series matching a query. + +**Authentication:** Not required + +**Query Parameters:** +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `query` | string | Yes | Search term (max 200 chars) | + +**Response (200 OK):** + +```json +[ + { + "key": "attack-on-titan", + "name": "Attack on Titan", + "site": "aniworld.to", + "folder": "Attack on Titan (2013)", + "missing_episodes": {}, + "link": "https://aniworld.to/anime/stream/attack-on-titan" + } +] +``` + +Source: [src/server/api/anime.py](../src/server/api/anime.py#L431-L474) + +### POST /api/anime/search + +Search via POST body. + +**Request Body:** + +```json +{ + "query": "attack on titan" +} +``` + +**Response:** Same as GET /api/anime/search + +Source: [src/server/api/anime.py](../src/server/api/anime.py#L477-L495) + +### POST /api/anime/add + +Add a new series to the library. + +**Authentication:** Required + +**Request Body:** + +```json +{ + "link": "https://aniworld.to/anime/stream/attack-on-titan", + "name": "Attack on Titan" +} +``` + +**Response (200 OK):** + +```json +{ + "status": "success", + "message": "Successfully added series: Attack on Titan", + "key": "attack-on-titan", + "folder": "Attack on Titan", + "db_id": 1 +} +``` + +Source: [src/server/api/anime.py](../src/server/api/anime.py#L604-L710) + +### POST /api/anime/rescan + +Trigger a rescan of the local library. + +**Authentication:** Required + +**Response (200 OK):** + +```json +{ + "success": true, + "message": "Rescan started successfully" +} +``` + +Source: [src/server/api/anime.py](../src/server/api/anime.py#L306-L337) + +### GET /api/anime/{anime_id} + +Return detailed information about a specific series. + +**Authentication:** Not required + +**Path Parameters:** +| Parameter | Description | +|-----------|-------------| +| `anime_id` | Series `key` (primary) or `folder` (deprecated fallback) | + +**Response (200 OK):** + +```json +{ + "key": "attack-on-titan", + "title": "Attack on Titan", + "folder": "Attack on Titan (2013)", + "episodes": ["1-1", "1-2", "1-3"], + "description": null +} +``` + +Source: [src/server/api/anime.py](../src/server/api/anime.py#L713-L793) + +--- + +## 4. Download Queue Endpoints + +Prefix: `/api/queue` + +Source: [src/server/api/download.py](../src/server/api/download.py#L1-L529) + +### GET /api/queue/status + +Get current download queue status and statistics. + +**Authentication:** Required + +**Response (200 OK):** + +```json +{ + "status": { + "is_running": false, + "is_paused": false, + "active_downloads": [], + "pending_queue": [], + "completed_downloads": [], + "failed_downloads": [] + }, + "statistics": { + "total_items": 5, + "pending_count": 3, + "active_count": 1, + "completed_count": 1, + "failed_count": 0, + "total_downloaded_mb": 1024.5, + "average_speed_mbps": 2.5, + "estimated_time_remaining": 3600 + } +} +``` + +Source: [src/server/api/download.py](../src/server/api/download.py#L21-L56) + +### POST /api/queue/add + +Add episodes to the download queue. + +**Authentication:** Required + +**Request Body:** + +```json +{ + "serie_id": "attack-on-titan", + "serie_folder": "Attack on Titan (2013)", + "serie_name": "Attack on Titan", + "episodes": [ + { "season": 1, "episode": 1, "title": "Episode 1" }, + { "season": 1, "episode": 2, "title": "Episode 2" } + ], + "priority": "NORMAL" +} +``` + +**Priority Values:** `LOW`, `NORMAL`, `HIGH` + +**Response (201 Created):** + +```json +{ + "status": "success", + "message": "Added 2 episode(s) to download queue", + "added_items": ["uuid1", "uuid2"], + "item_ids": ["uuid1", "uuid2"], + "failed_items": [] +} +``` + +Source: [src/server/api/download.py](../src/server/api/download.py#L59-L120) + +### POST /api/queue/start + +Start automatic queue processing. + +**Authentication:** Required + +**Response (200 OK):** + +```json +{ + "status": "success", + "message": "Queue processing started" +} +``` + +Source: [src/server/api/download.py](../src/server/api/download.py#L293-L331) + +### POST /api/queue/stop + +Stop processing new downloads from queue. + +**Authentication:** Required + +**Response (200 OK):** + +```json +{ + "status": "success", + "message": "Queue processing stopped (current download will continue)" +} +``` + +Source: [src/server/api/download.py](../src/server/api/download.py#L334-L387) + +### POST /api/queue/pause + +Pause queue processing (alias for stop). + +**Authentication:** Required + +**Response (200 OK):** + +```json +{ + "status": "success", + "message": "Queue processing paused" +} +``` + +Source: [src/server/api/download.py](../src/server/api/download.py#L416-L445) + +### DELETE /api/queue/{item_id} + +Remove a specific item from the download queue. + +**Authentication:** Required + +**Path Parameters:** +| Parameter | Description | +|-----------|-------------| +| `item_id` | Download item UUID | + +**Response (204 No Content)** + +Source: [src/server/api/download.py](../src/server/api/download.py#L225-L256) + +### DELETE /api/queue + +Remove multiple items from the download queue. + +**Authentication:** Required + +**Request Body:** + +```json +{ + "item_ids": ["uuid1", "uuid2"] +} +``` + +**Response (204 No Content)** + +Source: [src/server/api/download.py](../src/server/api/download.py#L259-L290) + +### DELETE /api/queue/completed + +Clear completed downloads from history. + +**Authentication:** Required + +**Response (200 OK):** + +```json +{ + "status": "success", + "message": "Cleared 5 completed item(s)", + "count": 5 +} +``` + +Source: [src/server/api/download.py](../src/server/api/download.py#L123-L149) + +### DELETE /api/queue/failed + +Clear failed downloads from history. + +**Authentication:** Required + +**Response (200 OK):** + +```json +{ + "status": "success", + "message": "Cleared 2 failed item(s)", + "count": 2 +} +``` + +Source: [src/server/api/download.py](../src/server/api/download.py#L152-L178) + +### DELETE /api/queue/pending + +Clear all pending downloads from the queue. + +**Authentication:** Required + +**Response (200 OK):** + +```json +{ + "status": "success", + "message": "Removed 10 pending item(s)", + "count": 10 +} +``` + +Source: [src/server/api/download.py](../src/server/api/download.py#L181-L207) + +### POST /api/queue/reorder + +Reorder items in the pending queue. + +**Authentication:** Required + +**Request Body:** + +```json +{ + "item_ids": ["uuid3", "uuid1", "uuid2"] +} +``` + +**Response (200 OK):** + +```json +{ + "status": "success", + "message": "Queue reordered with 3 items" +} +``` + +Source: [src/server/api/download.py](../src/server/api/download.py#L448-L477) + +### POST /api/queue/retry + +Retry failed downloads. + +**Authentication:** Required + +**Request Body:** + +```json +{ + "item_ids": ["uuid1", "uuid2"] +} +``` + +Pass empty `item_ids` array to retry all failed items. + +**Response (200 OK):** + +```json +{ + "status": "success", + "message": "Retrying 2 failed item(s)", + "retried_count": 2, + "retried_ids": ["uuid1", "uuid2"] +} +``` + +Source: [src/server/api/download.py](../src/server/api/download.py#L480-L514) + +--- + +## 5. Configuration Endpoints + +Prefix: `/api/config` + +Source: [src/server/api/config.py](../src/server/api/config.py#L1-L374) + +### GET /api/config + +Return current application configuration. + +**Authentication:** Required + +**Response (200 OK):** + +```json +{ + "name": "Aniworld", + "data_dir": "data", + "scheduler": { + "enabled": true, + "interval_minutes": 60 + }, + "logging": { + "level": "INFO", + "file": null, + "max_bytes": null, + "backup_count": 3 + }, + "backup": { + "enabled": false, + "path": "data/backups", + "keep_days": 30 + }, + "other": {} +} +``` + +Source: [src/server/api/config.py](../src/server/api/config.py#L16-L27) + +### PUT /api/config + +Apply an update to the configuration. + +**Authentication:** Required + +**Request Body:** + +```json +{ + "scheduler": { + "enabled": true, + "interval_minutes": 30 + }, + "logging": { + "level": "DEBUG" + } +} +``` + +**Response (200 OK):** Updated configuration object + +Source: [src/server/api/config.py](../src/server/api/config.py#L30-L47) + +### POST /api/config/validate + +Validate a configuration without applying it. + +**Authentication:** Required + +**Request Body:** Full `AppConfig` object + +**Response (200 OK):** + +```json +{ + "valid": true, + "errors": [] +} +``` + +Source: [src/server/api/config.py](../src/server/api/config.py#L50-L64) + +### GET /api/config/backups + +List all available configuration backups. + +**Authentication:** Required + +**Response (200 OK):** + +```json +[ + { + "name": "config_backup_20251213_090130.json", + "size": 1024, + "created": "2025-12-13T09:01:30Z" + } +] +``` + +Source: [src/server/api/config.py](../src/server/api/config.py#L67-L81) + +### POST /api/config/backups + +Create a backup of the current configuration. + +**Authentication:** Required + +**Query Parameters:** +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `name` | string | No | Custom backup name | + +**Response (200 OK):** + +```json +{ + "name": "config_backup_20251213_090130.json", + "message": "Backup created successfully" +} +``` + +Source: [src/server/api/config.py](../src/server/api/config.py#L84-L102) + +### POST /api/config/backups/{backup_name}/restore + +Restore configuration from a backup. + +**Authentication:** Required + +**Response (200 OK):** Restored configuration object + +Source: [src/server/api/config.py](../src/server/api/config.py#L105-L123) + +### DELETE /api/config/backups/{backup_name} + +Delete a configuration backup. + +**Authentication:** Required + +**Response (200 OK):** + +```json +{ + "message": "Backup 'config_backup_20251213.json' deleted successfully" +} +``` + +Source: [src/server/api/config.py](../src/server/api/config.py#L126-L142) + +### POST /api/config/directory + +Update anime directory configuration. + +**Authentication:** Required + +**Request Body:** + +```json +{ + "directory": "/path/to/anime" +} +``` + +**Response (200 OK):** + +```json +{ + "message": "Anime directory updated successfully", + "synced_series": 15 +} +``` + +Source: [src/server/api/config.py](../src/server/api/config.py#L189-L247) + +--- + +## 6. Scheduler Endpoints + +Prefix: `/api/scheduler` + +Source: [src/server/api/scheduler.py](../src/server/api/scheduler.py#L1-L122) + +### GET /api/scheduler/config + +Get current scheduler configuration. + +**Authentication:** Required + +**Response (200 OK):** + +```json +{ + "enabled": true, + "interval_minutes": 60 +} +``` + +Source: [src/server/api/scheduler.py](../src/server/api/scheduler.py#L22-L42) + +### POST /api/scheduler/config + +Update scheduler configuration. + +**Authentication:** Required + +**Request Body:** + +```json +{ + "enabled": true, + "interval_minutes": 30 +} +``` + +**Response (200 OK):** Updated scheduler configuration + +Source: [src/server/api/scheduler.py](../src/server/api/scheduler.py#L45-L75) + +### POST /api/scheduler/trigger-rescan + +Manually trigger a library rescan. + +**Authentication:** Required + +**Response (200 OK):** + +```json +{ + "success": true, + "message": "Rescan started successfully" +} +``` + +Source: [src/server/api/scheduler.py](../src/server/api/scheduler.py#L78-L122) + +--- + +## 7. Health Check Endpoints + +Prefix: `/health` + +Source: [src/server/api/health.py](../src/server/api/health.py#L1-L267) + +### GET /health + +Basic health check endpoint. + +**Authentication:** Not required + +**Response (200 OK):** + +```json +{ + "status": "healthy", + "timestamp": "2025-12-13T10:30:00.000Z", + "version": "1.0.0" +} +``` + +Source: [src/server/api/health.py](../src/server/api/health.py#L151-L161) + +### GET /health/detailed + +Comprehensive health check with database, filesystem, and system metrics. + +**Authentication:** Not required + +**Response (200 OK):** + +```json +{ + "status": "healthy", + "timestamp": "2025-12-13T10:30:00.000Z", + "version": "1.0.0", + "dependencies": { + "database": { + "status": "healthy", + "connection_time_ms": 1.5, + "message": "Database connection successful" + }, + "filesystem": { + "status": "healthy", + "data_dir_writable": true, + "logs_dir_writable": true + }, + "system": { + "cpu_percent": 25.0, + "memory_percent": 45.0, + "memory_available_mb": 8192.0, + "disk_percent": 60.0, + "disk_free_mb": 102400.0, + "uptime_seconds": 86400.0 + } + }, + "startup_time": "2025-12-13T08:00:00.000Z" +} +``` + +Source: [src/server/api/health.py](../src/server/api/health.py#L164-L200) + +--- + +## 8. WebSocket Protocol + +Endpoint: `/ws/connect` + +Source: [src/server/api/websocket.py](../src/server/api/websocket.py#L1-L260) + +### Connection + +**URL:** `ws://127.0.0.1:8000/ws/connect` + +**Query Parameters:** +| Parameter | Required | Description | +|-----------|----------|-------------| +| `token` | No | JWT token for authenticated access | + +### Message Types + +| Type | Direction | Description | +| ------------------- | ---------------- | -------------------------- | +| `connected` | Server -> Client | Connection confirmation | +| `ping` | Client -> Server | Keepalive request | +| `pong` | Server -> Client | Keepalive response | +| `download_progress` | Server -> Client | Download progress update | +| `download_complete` | Server -> Client | Download completed | +| `download_failed` | Server -> Client | Download failed | +| `download_added` | Server -> Client | Item added to queue | +| `download_removed` | Server -> Client | Item removed from queue | +| `queue_status` | Server -> Client | Queue status update | +| `queue_started` | Server -> Client | Queue processing started | +| `queue_stopped` | Server -> Client | Queue processing stopped | +| `scan_progress` | Server -> Client | Library scan progress | +| `scan_complete` | Server -> Client | Library scan completed | +| `system_info` | Server -> Client | System information message | +| `error` | Server -> Client | Error message | + +Source: [src/server/models/websocket.py](../src/server/models/websocket.py#L25-L57) + +### Room Subscriptions + +Clients can join/leave rooms to receive specific updates. + +**Join Room:** + +```json +{ + "action": "join", + "data": { "room": "downloads" } +} +``` + +**Leave Room:** + +```json +{ + "action": "leave", + "data": { "room": "downloads" } +} +``` + +**Available Rooms:** + +- `downloads` - Download progress and status updates + +### Server Message Format + +```json +{ + "type": "download_progress", + "timestamp": "2025-12-13T10:30:00.000Z", + "data": { + "download_id": "uuid-here", + "key": "attack-on-titan", + "folder": "Attack on Titan (2013)", + "percent": 45.2, + "speed_mbps": 2.5, + "eta_seconds": 180 + } +} +``` + +### WebSocket Status Endpoint + +**GET /ws/status** + +Returns WebSocket service status. + +**Response (200 OK):** + +```json +{ + "status": "operational", + "active_connections": 5, + "supported_message_types": [ + "download_progress", + "download_complete", + "download_failed", + "queue_status", + "connected", + "ping", + "pong", + "error" + ] +} +``` + +Source: [src/server/api/websocket.py](../src/server/api/websocket.py#L238-L260) + +--- + +## 9. Data Models + +### Download Item + +```json +{ + "id": "uuid-string", + "serie_id": "attack-on-titan", + "serie_folder": "Attack on Titan (2013)", + "serie_name": "Attack on Titan", + "episode": { + "season": 1, + "episode": 1, + "title": "To You, in 2000 Years" + }, + "status": "pending", + "priority": "NORMAL", + "added_at": "2025-12-13T10:00:00Z", + "started_at": null, + "completed_at": null, + "progress": null, + "error": null, + "retry_count": 0, + "source_url": null +} +``` + +**Status Values:** `pending`, `downloading`, `paused`, `completed`, `failed`, `cancelled` + +**Priority Values:** `LOW`, `NORMAL`, `HIGH` + +Source: [src/server/models/download.py](../src/server/models/download.py#L63-L118) + +### Episode Identifier + +```json +{ + "season": 1, + "episode": 1, + "title": "Episode Title" +} +``` + +Source: [src/server/models/download.py](../src/server/models/download.py#L36-L41) + +### Download Progress + +```json +{ + "percent": 45.2, + "downloaded_mb": 256.0, + "total_mb": 512.0, + "speed_mbps": 2.5, + "eta_seconds": 180 +} +``` + +Source: [src/server/models/download.py](../src/server/models/download.py#L44-L60) + +--- + +## 10. Error Handling + +### HTTP Status Codes + +| Code | Meaning | When Used | +| ---- | --------------------- | --------------------------------- | +| 200 | OK | Successful request | +| 201 | Created | Resource created | +| 204 | No Content | Successful deletion | +| 400 | Bad Request | Invalid request body/parameters | +| 401 | Unauthorized | Missing or invalid authentication | +| 403 | Forbidden | Insufficient permissions | +| 404 | Not Found | Resource does not exist | +| 422 | Unprocessable Entity | Validation error | +| 429 | Too Many Requests | Rate limit exceeded | +| 500 | Internal Server Error | Server-side error | + +### Error Response Format + +```json +{ + "success": false, + "error": "VALIDATION_ERROR", + "message": "Human-readable error message", + "details": { + "field": "Additional context" + }, + "request_id": "uuid-for-tracking" +} +``` + +Source: [src/server/middleware/error_handler.py](../src/server/middleware/error_handler.py#L26-L56) + +### Common Error Codes + +| Error Code | HTTP Status | Description | +| ---------------------- | ----------- | ------------------------------ | +| `AUTHENTICATION_ERROR` | 401 | Invalid or missing credentials | +| `AUTHORIZATION_ERROR` | 403 | Insufficient permissions | +| `VALIDATION_ERROR` | 422 | Request validation failed | +| `NOT_FOUND_ERROR` | 404 | Resource not found | +| `CONFLICT_ERROR` | 409 | Resource conflict | +| `RATE_LIMIT_ERROR` | 429 | Rate limit exceeded | + +--- + +## 11. Rate Limiting + +### Authentication Endpoints + +| Endpoint | Limit | Window | +| ---------------------- | ---------- | ---------- | +| `POST /api/auth/login` | 5 requests | 60 seconds | +| `POST /api/auth/setup` | 5 requests | 60 seconds | + +Source: [src/server/middleware/auth.py](../src/server/middleware/auth.py#L143-L162) + +### Origin-Based Limiting + +All endpoints from the same origin are limited to 60 requests per minute per origin. + +Source: [src/server/middleware/auth.py](../src/server/middleware/auth.py#L115-L133) + +### Rate Limit Response + +```json +{ + "detail": "Too many authentication attempts, try again later" +} +``` + +HTTP Status: 429 Too Many Requests + +--- + +## 12. Pagination + +The anime list endpoint supports pagination. + +**Query Parameters:** +| Parameter | Default | Max | Description | +|-----------|---------|-----|-------------| +| `page` | 1 | - | Page number (1-indexed) | +| `per_page` | 20 | 1000 | Items per page | + +**Example:** + +``` +GET /api/anime?page=2&per_page=50 +``` + +Source: [src/server/api/anime.py](../src/server/api/anime.py#L180-L220) diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md new file mode 100644 index 0000000..6bb6b7e --- /dev/null +++ b/docs/ARCHITECTURE.md @@ -0,0 +1,452 @@ +# Architecture Documentation + +## Document Purpose + +This document describes the system architecture of the Aniworld anime download manager. + +--- + +## 1. System Overview + +Aniworld is a web-based anime download manager built with Python, FastAPI, and SQLite. It provides a REST API and WebSocket interface for managing anime libraries, downloading episodes, and tracking progress. + +### High-Level Architecture + +``` ++------------------+ +------------------+ +------------------+ +| Web Browser | | CLI Client | | External | +| (Frontend) | | (Main.py) | | Providers | ++--------+---------+ +--------+---------+ +--------+---------+ + | | | + | HTTP/WebSocket | Direct | HTTP + | | | ++--------v---------+ +--------v---------+ +--------v---------+ +| | | | | | +| FastAPI <-----> Core Layer <-----> Provider | +| Server Layer | | (SeriesApp) | | Adapters | +| | | | | | ++--------+---------+ +--------+---------+ +------------------+ + | | + | | ++--------v---------+ +--------v---------+ +| | | | +| SQLite DB | | File System | +| (aniworld.db) | | (data/*.json) | +| | | | ++------------------+ +------------------+ +``` + +Source: [src/server/fastapi_app.py](../src/server/fastapi_app.py#L1-L252) + +--- + +## 2. Architectural Layers + +### 2.1 CLI Layer (`src/cli/`) + +Legacy command-line interface for direct interaction with the core layer. + +| Component | File | Purpose | +| --------- | ----------------------------- | --------------- | +| Main | [Main.py](../src/cli/Main.py) | CLI entry point | + +### 2.2 Server Layer (`src/server/`) + +FastAPI-based REST API and WebSocket server. + +``` +src/server/ ++-- fastapi_app.py # Application entry point, lifespan management ++-- api/ # API route handlers +| +-- anime.py # /api/anime/* endpoints +| +-- auth.py # /api/auth/* endpoints +| +-- config.py # /api/config/* endpoints +| +-- download.py # /api/queue/* endpoints +| +-- scheduler.py # /api/scheduler/* endpoints +| +-- websocket.py # /ws/* WebSocket handlers +| +-- health.py # /health/* endpoints ++-- controllers/ # Page controllers for HTML rendering +| +-- page_controller.py # UI page routes +| +-- health_controller.py# Health check route +| +-- error_controller.py # Error pages (404, 500) ++-- services/ # Business logic +| +-- anime_service.py # Anime operations +| +-- auth_service.py # Authentication +| +-- config_service.py # Configuration management +| +-- download_service.py # Download queue management +| +-- progress_service.py # Progress tracking +| +-- websocket_service.py# WebSocket broadcasting +| +-- queue_repository.py # Database persistence ++-- models/ # Pydantic models +| +-- auth.py # Auth request/response models +| +-- config.py # Configuration models +| +-- download.py # Download queue models +| +-- websocket.py # WebSocket message models ++-- middleware/ # Request processing +| +-- auth.py # JWT validation, rate limiting +| +-- error_handler.py # Exception handlers +| +-- setup_redirect.py # Setup flow redirect ++-- database/ # SQLAlchemy ORM +| +-- connection.py # Database connection +| +-- models.py # ORM models +| +-- service.py # Database service ++-- web/ # Static files and templates + +-- static/ # CSS, JS, images + +-- templates/ # Jinja2 templates +``` + +Source: [src/server/](../src/server/) + +### 2.3 Core Layer (`src/core/`) + +Domain logic for anime series management. + +``` +src/core/ ++-- SeriesApp.py # Main application facade ++-- SerieScanner.py # Directory scanning ++-- entities/ # Domain entities +| +-- series.py # Serie class +| +-- SerieList.py # SerieList collection ++-- providers/ # External provider adapters +| +-- base_provider.py # Loader interface +| +-- provider_factory.py # Provider registry ++-- interfaces/ # Abstract interfaces +| +-- callbacks.py # Progress callback system ++-- exceptions/ # Domain exceptions + +-- Exceptions.py # Custom exceptions +``` + +Source: [src/core/](../src/core/) + +### 2.4 Infrastructure Layer (`src/infrastructure/`) + +Cross-cutting concerns. + +``` +src/infrastructure/ ++-- logging/ # Structured logging setup ++-- security/ # Security utilities +``` + +### 2.5 Configuration Layer (`src/config/`) + +Application settings management. + +| Component | File | Purpose | +| --------- | ---------------------------------------- | ------------------------------- | +| Settings | [settings.py](../src/config/settings.py) | Environment-based configuration | + +Source: [src/config/settings.py](../src/config/settings.py#L1-L96) + +--- + +## 3. Component Interactions + +### 3.1 Request Flow (REST API) + +``` +1. Client sends HTTP request +2. AuthMiddleware validates JWT token (if required) +3. Rate limiter checks request frequency +4. FastAPI router dispatches to endpoint handler +5. Endpoint calls service layer +6. Service layer uses core layer or database +7. Response returned as JSON +``` + +Source: [src/server/middleware/auth.py](../src/server/middleware/auth.py#L1-L209) + +### 3.2 Download Flow + +``` +1. POST /api/queue/add + +-- DownloadService.add_to_queue() + +-- QueueRepository.save_item() -> SQLite + +2. POST /api/queue/start + +-- DownloadService.start_queue_processing() + +-- Process pending items sequentially + +-- ProgressService emits events + +-- WebSocketService broadcasts to clients + +3. During download: + +-- ProgressService.emit("progress_updated") + +-- WebSocketService.broadcast_to_room() + +-- Client receives WebSocket message +``` + +Source: [src/server/services/download_service.py](../src/server/services/download_service.py#L1-L150) + +### 3.3 WebSocket Event Flow + +``` +1. Client connects to /ws/connect +2. Server sends "connected" message +3. Client joins room: {"action": "join", "data": {"room": "downloads"}} +4. ProgressService emits events +5. WebSocketService broadcasts to room subscribers +6. Client receives real-time updates +``` + +Source: [src/server/api/websocket.py](../src/server/api/websocket.py#L1-L260) + +--- + +## 4. Design Patterns + +### 4.1 Repository Pattern + +Database access is abstracted through repository classes. + +```python +# QueueRepository provides CRUD for download items +class QueueRepository: + async def save_item(self, item: DownloadItem) -> None: ... + async def get_all_items(self) -> List[DownloadItem]: ... + async def delete_item(self, item_id: str) -> bool: ... +``` + +Source: [src/server/services/queue_repository.py](../src/server/services/queue_repository.py) + +### 4.2 Dependency Injection + +FastAPI's `Depends()` provides constructor injection. + +```python +@router.get("/status") +async def get_status( + download_service: DownloadService = Depends(get_download_service), +): + ... +``` + +Source: [src/server/utils/dependencies.py](../src/server/utils/dependencies.py) + +### 4.3 Event-Driven Architecture + +Progress updates use an event subscription model. + +```python +# ProgressService publishes events +progress_service.emit("progress_updated", event) + +# WebSocketService subscribes +progress_service.subscribe("progress_updated", ws_handler) +``` + +Source: [src/server/fastapi_app.py](../src/server/fastapi_app.py#L98-L108) + +### 4.4 Singleton Pattern + +Services use module-level singletons for shared state. + +```python +# In download_service.py +_download_service_instance: Optional[DownloadService] = None + +def get_download_service() -> DownloadService: + global _download_service_instance + if _download_service_instance is None: + _download_service_instance = DownloadService(...) + return _download_service_instance +``` + +Source: [src/server/services/download_service.py](../src/server/services/download_service.py) + +--- + +## 5. Data Flow + +### 5.1 Series Identifier Convention + +The system uses two identifier fields: + +| Field | Type | Purpose | Example | +| -------- | -------- | -------------------------------------- | -------------------------- | +| `key` | Primary | Provider-assigned, URL-safe identifier | `"attack-on-titan"` | +| `folder` | Metadata | Filesystem folder name (display only) | `"Attack on Titan (2013)"` | + +All API operations use `key`. The `folder` is for filesystem operations only. + +Source: [src/server/database/models.py](../src/server/database/models.py#L26-L50) + +### 5.2 Database Schema + +``` ++----------------+ +----------------+ +--------------------+ +| anime_series | | episodes | | download_queue_item| ++----------------+ +----------------+ +--------------------+ +| id (PK) |<--+ | id (PK) | +-->| id (PK) | +| key (unique) | | | series_id (FK) |---+ | series_id (FK) | +| name | +---| season | | status | +| site | | episode_number | | priority | +| folder | | title | | progress_percent | +| created_at | | is_downloaded | | added_at | +| updated_at | | file_path | | started_at | ++----------------+ +----------------+ +--------------------+ +``` + +Source: [src/server/database/models.py](../src/server/database/models.py#L1-L200) + +### 5.3 Configuration Storage + +Configuration is stored in `data/config.json`: + +```json +{ + "name": "Aniworld", + "data_dir": "data", + "scheduler": { "enabled": true, "interval_minutes": 60 }, + "logging": { "level": "INFO" }, + "backup": { "enabled": false, "path": "data/backups" }, + "other": { + "master_password_hash": "$pbkdf2-sha256$...", + "anime_directory": "/path/to/anime" + } +} +``` + +Source: [data/config.json](../data/config.json) + +--- + +## 6. Technology Stack + +| Layer | Technology | Version | Purpose | +| ------------- | ------------------- | ------- | ---------------------- | +| Web Framework | FastAPI | 0.104.1 | REST API, WebSocket | +| ASGI Server | Uvicorn | 0.24.0 | HTTP server | +| Database | SQLite + SQLAlchemy | 2.0.35 | Persistence | +| Auth | python-jose | 3.3.0 | JWT tokens | +| Password | passlib | 1.7.4 | bcrypt hashing | +| Validation | Pydantic | 2.5.0 | Data models | +| Templates | Jinja2 | 3.1.2 | HTML rendering | +| Logging | structlog | 24.1.0 | Structured logging | +| Testing | pytest | 7.4.3 | Unit/integration tests | + +Source: [requirements.txt](../requirements.txt) + +--- + +## 7. Scalability Considerations + +### Current Limitations + +1. **Single-process deployment**: In-memory rate limiting and session state are not shared across processes. + +2. **SQLite database**: Not suitable for high concurrency. Consider PostgreSQL for production. + +3. **Sequential downloads**: Only one download processes at a time by design. + +### Recommended Improvements for Scale + +| Concern | Current | Recommended | +| -------------- | --------------- | ----------------- | +| Rate limiting | In-memory dict | Redis | +| Session store | In-memory | Redis or database | +| Database | SQLite | PostgreSQL | +| Task queue | In-memory deque | Celery + Redis | +| Load balancing | None | Nginx/HAProxy | + +--- + +## 8. Integration Points + +### 8.1 External Providers + +The system integrates with anime streaming providers via the Loader interface. + +```python +class Loader(ABC): + @abstractmethod + def search(self, query: str) -> List[Serie]: ... + + @abstractmethod + def get_episodes(self, serie: Serie) -> Dict[int, List[int]]: ... +``` + +Source: [src/core/providers/base_provider.py](../src/core/providers/base_provider.py) + +### 8.2 Filesystem Integration + +The scanner reads anime directories to detect downloaded episodes. + +```python +SerieScanner( + basePath="/path/to/anime", # Anime library directory + loader=provider, # Provider for metadata + db_session=session # Optional database +) +``` + +Source: [src/core/SerieScanner.py](../src/core/SerieScanner.py#L59-L96) + +--- + +## 9. Security Architecture + +### 9.1 Authentication Flow + +``` +1. User sets master password via POST /api/auth/setup +2. Password hashed with pbkdf2_sha256 (via passlib) +3. Hash stored in config.json +4. Login validates password, returns JWT token +5. JWT contains: session_id, user, created_at, expires_at +6. Subsequent requests include: Authorization: Bearer +``` + +Source: [src/server/services/auth_service.py](../src/server/services/auth_service.py#L1-L150) + +### 9.2 Password Requirements + +- Minimum 8 characters +- Mixed case (upper and lower) +- At least one number +- At least one special character + +Source: [src/server/services/auth_service.py](../src/server/services/auth_service.py#L97-L125) + +### 9.3 Rate Limiting + +| Endpoint | Limit | Window | +| ----------------- | ----------- | ---------- | +| `/api/auth/login` | 5 requests | 60 seconds | +| `/api/auth/setup` | 5 requests | 60 seconds | +| All origins | 60 requests | 60 seconds | + +Source: [src/server/middleware/auth.py](../src/server/middleware/auth.py#L54-L68) + +--- + +## 10. Deployment Modes + +### 10.1 Development + +```bash +# Run with hot reload +python -m uvicorn src.server.fastapi_app:app --reload +``` + +### 10.2 Production + +```bash +# Via conda environment +conda run -n AniWorld python -m uvicorn src.server.fastapi_app:app \ + --host 127.0.0.1 --port 8000 +``` + +### 10.3 Configuration + +Environment variables (via `.env` or shell): + +| Variable | Default | Description | +| ----------------- | ------------------------------ | ---------------------- | +| `JWT_SECRET_KEY` | Random | Secret for JWT signing | +| `DATABASE_URL` | `sqlite:///./data/aniworld.db` | Database connection | +| `ANIME_DIRECTORY` | (empty) | Path to anime library | +| `LOG_LEVEL` | `INFO` | Logging level | +| `CORS_ORIGINS` | `localhost:3000,8000` | Allowed CORS origins | + +Source: [src/config/settings.py](../src/config/settings.py#L1-L96) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md new file mode 100644 index 0000000..480e03a --- /dev/null +++ b/docs/CHANGELOG.md @@ -0,0 +1,78 @@ +# Changelog + +## Document Purpose + +This document tracks all notable changes to the Aniworld project. + +### What This Document Contains + +- **Version History**: All released versions with dates +- **Added Features**: New functionality in each release +- **Changed Features**: Modifications to existing features +- **Deprecated Features**: Features marked for removal +- **Removed Features**: Features removed from the codebase +- **Fixed Bugs**: Bug fixes with issue references +- **Security Fixes**: Security-related changes +- **Breaking Changes**: Changes requiring user action + +### What This Document Does NOT Contain + +- Internal refactoring details (unless user-facing) +- Commit-level changes +- Work-in-progress features +- Roadmap or planned features + +### Target Audience + +- All users and stakeholders +- Operators planning upgrades +- Developers tracking changes +- Support personnel + +--- + +## Format + +This changelog follows [Keep a Changelog](https://keepachangelog.com/) principles and adheres to [Semantic Versioning](https://semver.org/). + +## Sections for Each Release + +```markdown +## [Version] - YYYY-MM-DD + +### Added + +- New features + +### Changed + +- Changes to existing functionality + +### Deprecated + +- Features that will be removed in future versions + +### Removed + +- Features removed in this release + +### Fixed + +- Bug fixes + +### Security + +- Security-related fixes +``` + +--- + +## Unreleased + +_Changes that are in development but not yet released._ + +--- + +## Version History + +_To be documented as versions are released._ diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md new file mode 100644 index 0000000..3b341c8 --- /dev/null +++ b/docs/CONFIGURATION.md @@ -0,0 +1,298 @@ +# Configuration Reference + +## Document Purpose + +This document provides a comprehensive reference for all configuration options in the Aniworld application. + +--- + +## 1. Configuration Overview + +### Configuration Sources + +Aniworld uses a layered configuration system: + +1. **Environment Variables** (highest priority) +2. **`.env` file** in project root +3. **`data/config.json`** file +4. **Default values** (lowest priority) + +### Loading Mechanism + +Configuration is loaded at application startup via Pydantic Settings. + +```python +# src/config/settings.py +class Settings(BaseSettings): + model_config = SettingsConfigDict(env_file=".env", extra="ignore") +``` + +Source: [src/config/settings.py](../src/config/settings.py#L1-L96) + +--- + +## 2. Environment Variables + +### Authentication Settings + +| Variable | Type | Default | Description | +| ----------------------- | ------ | ---------------- | ------------------------------------------------------------------- | +| `JWT_SECRET_KEY` | string | (random) | Secret key for JWT token signing. Auto-generated if not set. | +| `PASSWORD_SALT` | string | `"default-salt"` | Salt for password hashing. | +| `MASTER_PASSWORD_HASH` | string | (none) | Pre-hashed master password. Loaded from config.json if not set. | +| `MASTER_PASSWORD` | string | (none) | **DEVELOPMENT ONLY** - Plaintext password. Never use in production. | +| `SESSION_TIMEOUT_HOURS` | int | `24` | JWT token expiry time in hours. | + +Source: [src/config/settings.py](../src/config/settings.py#L13-L42) + +### Server Settings + +| Variable | Type | Default | Description | +| ----------------- | ------ | -------------------------------- | --------------------------------------------------------------------- | +| `ANIME_DIRECTORY` | string | `""` | Path to anime library directory. | +| `LOG_LEVEL` | string | `"INFO"` | Logging level: DEBUG, INFO, WARNING, ERROR, CRITICAL. | +| `DATABASE_URL` | string | `"sqlite:///./data/aniworld.db"` | Database connection string. | +| `CORS_ORIGINS` | string | `"http://localhost:3000"` | Comma-separated allowed CORS origins. Use `*` for localhost defaults. | +| `API_RATE_LIMIT` | int | `100` | Maximum API requests per minute. | + +Source: [src/config/settings.py](../src/config/settings.py#L43-L68) + +### Provider Settings + +| Variable | Type | Default | Description | +| ------------------ | ------ | --------------- | --------------------------------------------- | +| `DEFAULT_PROVIDER` | string | `"aniworld.to"` | Default anime provider. | +| `PROVIDER_TIMEOUT` | int | `30` | HTTP timeout for provider requests (seconds). | +| `RETRY_ATTEMPTS` | int | `3` | Number of retry attempts for failed requests. | + +Source: [src/config/settings.py](../src/config/settings.py#L69-L79) + +--- + +## 3. Configuration File (config.json) + +Location: `data/config.json` + +### File Structure + +```json +{ + "name": "Aniworld", + "data_dir": "data", + "scheduler": { + "enabled": true, + "interval_minutes": 60 + }, + "logging": { + "level": "INFO", + "file": null, + "max_bytes": null, + "backup_count": 3 + }, + "backup": { + "enabled": false, + "path": "data/backups", + "keep_days": 30 + }, + "other": { + "master_password_hash": "$pbkdf2-sha256$...", + "anime_directory": "/path/to/anime" + }, + "version": "1.0.0" +} +``` + +Source: [data/config.json](../data/config.json) + +--- + +## 4. Configuration Sections + +### 4.1 General Settings + +| Field | Type | Default | Description | +| ---------- | ------ | ------------ | ------------------------------ | +| `name` | string | `"Aniworld"` | Application name. | +| `data_dir` | string | `"data"` | Base directory for data files. | + +Source: [src/server/models/config.py](../src/server/models/config.py#L62-L66) + +### 4.2 Scheduler Settings + +Controls automatic library rescanning. + +| Field | Type | Default | Description | +| ---------------------------- | ---- | ------- | -------------------------------------------- | +| `scheduler.enabled` | bool | `true` | Enable/disable automatic scans. | +| `scheduler.interval_minutes` | int | `60` | Minutes between automatic scans. Minimum: 1. | + +Source: [src/server/models/config.py](../src/server/models/config.py#L5-L12) + +### 4.3 Logging Settings + +| Field | Type | Default | Description | +| ---------------------- | ------ | -------- | ------------------------------------------------- | +| `logging.level` | string | `"INFO"` | Log level: DEBUG, INFO, WARNING, ERROR, CRITICAL. | +| `logging.file` | string | `null` | Optional log file path. | +| `logging.max_bytes` | int | `null` | Maximum log file size for rotation. | +| `logging.backup_count` | int | `3` | Number of rotated log files to keep. | + +Source: [src/server/models/config.py](../src/server/models/config.py#L27-L46) + +### 4.4 Backup Settings + +| Field | Type | Default | Description | +| ------------------ | ------ | ---------------- | -------------------------------- | +| `backup.enabled` | bool | `false` | Enable automatic config backups. | +| `backup.path` | string | `"data/backups"` | Directory for backup files. | +| `backup.keep_days` | int | `30` | Days to retain backups. | + +Source: [src/server/models/config.py](../src/server/models/config.py#L15-L24) + +### 4.5 Other Settings (Dynamic) + +The `other` field stores arbitrary settings. + +| Key | Type | Description | +| ---------------------- | ------ | --------------------------------------- | +| `master_password_hash` | string | Hashed master password (pbkdf2-sha256). | +| `anime_directory` | string | Path to anime library. | +| `advanced` | object | Advanced configuration options. | + +--- + +## 5. Configuration Precedence + +Settings are resolved in this order (first match wins): + +1. Environment variable (e.g., `ANIME_DIRECTORY`) +2. `.env` file in project root +3. `data/config.json` (for dynamic settings) +4. Code defaults in `Settings` class + +--- + +## 6. Validation Rules + +### Password Requirements + +Master password must meet all criteria: + +- Minimum 8 characters +- At least one uppercase letter +- At least one lowercase letter +- At least one digit +- At least one special character + +Source: [src/server/services/auth_service.py](../src/server/services/auth_service.py#L97-L125) + +### Logging Level Validation + +Must be one of: `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL` + +Source: [src/server/models/config.py](../src/server/models/config.py#L43-L47) + +### Backup Path Validation + +If `backup.enabled` is `true`, `backup.path` must be set. + +Source: [src/server/models/config.py](../src/server/models/config.py#L87-L91) + +--- + +## 7. Example Configurations + +### Minimal Development Setup + +**.env file:** + +``` +LOG_LEVEL=DEBUG +ANIME_DIRECTORY=/home/user/anime +``` + +### Production Setup + +**.env file:** + +``` +JWT_SECRET_KEY=your-secure-random-key-here +DATABASE_URL=postgresql+asyncpg://user:pass@localhost/aniworld +LOG_LEVEL=WARNING +CORS_ORIGINS=https://your-domain.com +API_RATE_LIMIT=60 +``` + +### Docker Setup + +```yaml +# docker-compose.yml +environment: + - JWT_SECRET_KEY=${JWT_SECRET_KEY} + - DATABASE_URL=sqlite:///./data/aniworld.db + - ANIME_DIRECTORY=/media/anime + - LOG_LEVEL=INFO +volumes: + - ./data:/app/data + - /media/anime:/media/anime:ro +``` + +--- + +## 8. Configuration Backup Management + +### Automatic Backups + +Backups are created automatically before config changes when `backup.enabled` is `true`. + +Location: `data/config_backups/` + +Naming: `config_backup_YYYYMMDD_HHMMSS.json` + +### Manual Backup via API + +```bash +# Create backup +curl -X POST http://localhost:8000/api/config/backups \ + -H "Authorization: Bearer $TOKEN" + +# List backups +curl http://localhost:8000/api/config/backups \ + -H "Authorization: Bearer $TOKEN" + +# Restore backup +curl -X POST http://localhost:8000/api/config/backups/config_backup_20251213.json/restore \ + -H "Authorization: Bearer $TOKEN" +``` + +Source: [src/server/api/config.py](../src/server/api/config.py#L67-L142) + +--- + +## 9. Troubleshooting + +### Configuration Not Loading + +1. Check file permissions on `data/config.json` +2. Verify JSON syntax with a validator +3. Check logs for Pydantic validation errors + +### Environment Variable Not Working + +1. Ensure variable name matches exactly (case-sensitive) +2. Check `.env` file location (project root) +3. Restart application after changes + +### Master Password Issues + +1. Password hash is stored in `config.json` under `other.master_password_hash` +2. Delete this field to reset (requires re-setup) +3. Check hash format starts with `$pbkdf2-sha256$` + +--- + +## 10. Related Documentation + +- [API.md](API.md) - Configuration API endpoints +- [DEVELOPMENT.md](DEVELOPMENT.md) - Development environment setup +- [ARCHITECTURE.md](ARCHITECTURE.md) - Configuration service architecture diff --git a/docs/DATABASE.md b/docs/DATABASE.md new file mode 100644 index 0000000..3d3a2bb --- /dev/null +++ b/docs/DATABASE.md @@ -0,0 +1,326 @@ +# Database Documentation + +## Document Purpose + +This document describes the database schema, models, and data layer of the Aniworld application. + +--- + +## 1. Database Overview + +### Technology + +- **Database Engine**: SQLite 3 (default), PostgreSQL supported +- **ORM**: SQLAlchemy 2.0 with async support (aiosqlite) +- **Location**: `data/aniworld.db` (configurable via `DATABASE_URL`) + +Source: [src/config/settings.py](../src/config/settings.py#L53-L55) + +### Connection Configuration + +```python +# Default connection string +DATABASE_URL = "sqlite+aiosqlite:///./data/aniworld.db" + +# PostgreSQL alternative +DATABASE_URL = "postgresql+asyncpg://user:pass@localhost/aniworld" +``` + +Source: [src/server/database/connection.py](../src/server/database/connection.py) + +--- + +## 2. Entity Relationship Diagram + +``` ++-------------------+ +-------------------+ +------------------------+ +| anime_series | | episodes | | download_queue_item | ++-------------------+ +-------------------+ +------------------------+ +| id (PK) |<--+ | id (PK) | +-->| id (PK, VARCHAR) | +| key (UNIQUE) | | | series_id (FK)----+---+ | series_id (FK)---------+ +| name | +---| | | status | +| site | | season | | priority | +| folder | | episode_number | | season | +| created_at | | title | | episode | +| updated_at | | file_path | | progress_percent | ++-------------------+ | is_downloaded | | error_message | + | created_at | | retry_count | + | updated_at | | added_at | + +-------------------+ | started_at | + | completed_at | + | created_at | + | updated_at | + +------------------------+ +``` + +--- + +## 3. Table Schemas + +### 3.1 anime_series + +Stores anime series metadata. + +| Column | Type | Constraints | Description | +| ------------ | ------------- | -------------------------- | ------------------------------------------------------- | +| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Internal database ID | +| `key` | VARCHAR(255) | UNIQUE, NOT NULL, INDEX | **Primary identifier** - provider-assigned URL-safe key | +| `name` | VARCHAR(500) | NOT NULL, INDEX | Display name of the series | +| `site` | VARCHAR(500) | NOT NULL | Provider site URL | +| `folder` | VARCHAR(1000) | NOT NULL | Filesystem folder name (metadata only) | +| `created_at` | DATETIME | NOT NULL, DEFAULT NOW | Record creation timestamp | +| `updated_at` | DATETIME | NOT NULL, ON UPDATE NOW | Last update timestamp | + +**Identifier Convention:** + +- `key` is the **primary identifier** for all operations (e.g., `"attack-on-titan"`) +- `folder` is **metadata only** for filesystem operations (e.g., `"Attack on Titan (2013)"`) +- `id` is used only for database relationships + +Source: [src/server/database/models.py](../src/server/database/models.py#L23-L87) + +### 3.2 episodes + +Stores individual episode information. + +| Column | Type | Constraints | Description | +| ---------------- | ------------- | ---------------------------- | ----------------------------- | +| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Internal database ID | +| `series_id` | INTEGER | FOREIGN KEY, NOT NULL, INDEX | Reference to anime_series.id | +| `season` | INTEGER | NOT NULL | Season number (1-based) | +| `episode_number` | INTEGER | NOT NULL | Episode number within season | +| `title` | VARCHAR(500) | NULLABLE | Episode title if known | +| `file_path` | VARCHAR(1000) | NULLABLE | Local file path if downloaded | +| `is_downloaded` | BOOLEAN | NOT NULL, DEFAULT FALSE | Download status flag | +| `created_at` | DATETIME | NOT NULL, DEFAULT NOW | Record creation timestamp | +| `updated_at` | DATETIME | NOT NULL, ON UPDATE NOW | Last update timestamp | + +**Foreign Key:** + +- `series_id` -> `anime_series.id` (ON DELETE CASCADE) + +Source: [src/server/database/models.py](../src/server/database/models.py#L122-L181) + +### 3.3 download_queue_item + +Stores download queue items with status tracking. + +| Column | Type | Constraints | Description | +| ------------------ | ------------- | --------------------------- | ------------------------------ | +| `id` | VARCHAR(36) | PRIMARY KEY | UUID identifier | +| `series_id` | INTEGER | FOREIGN KEY, NOT NULL | Reference to anime_series.id | +| `season` | INTEGER | NOT NULL | Season number | +| `episode` | INTEGER | NOT NULL | Episode number | +| `status` | VARCHAR(20) | NOT NULL, DEFAULT 'pending' | Download status | +| `priority` | VARCHAR(10) | NOT NULL, DEFAULT 'NORMAL' | Queue priority | +| `progress_percent` | FLOAT | NULLABLE | Download progress (0-100) | +| `error_message` | TEXT | NULLABLE | Error description if failed | +| `retry_count` | INTEGER | NOT NULL, DEFAULT 0 | Number of retry attempts | +| `source_url` | VARCHAR(2000) | NULLABLE | Download source URL | +| `added_at` | DATETIME | NOT NULL, DEFAULT NOW | When added to queue | +| `started_at` | DATETIME | NULLABLE | When download started | +| `completed_at` | DATETIME | NULLABLE | When download completed/failed | +| `created_at` | DATETIME | NOT NULL, DEFAULT NOW | Record creation timestamp | +| `updated_at` | DATETIME | NOT NULL, ON UPDATE NOW | Last update timestamp | + +**Status Values:** `pending`, `downloading`, `paused`, `completed`, `failed`, `cancelled` + +**Priority Values:** `LOW`, `NORMAL`, `HIGH` + +**Foreign Key:** + +- `series_id` -> `anime_series.id` (ON DELETE CASCADE) + +Source: [src/server/database/models.py](../src/server/database/models.py#L200-L300) + +--- + +## 4. Indexes + +| Table | Index Name | Columns | Purpose | +| --------------------- | ----------------------- | ----------- | --------------------------------- | +| `anime_series` | `ix_anime_series_key` | `key` | Fast lookup by primary identifier | +| `anime_series` | `ix_anime_series_name` | `name` | Search by name | +| `episodes` | `ix_episodes_series_id` | `series_id` | Join with series | +| `download_queue_item` | `ix_download_series_id` | `series_id` | Filter by series | +| `download_queue_item` | `ix_download_status` | `status` | Filter by status | + +--- + +## 5. Model Layer + +### 5.1 SQLAlchemy ORM Models + +```python +# src/server/database/models.py + +class AnimeSeries(Base, TimestampMixin): + __tablename__ = "anime_series" + + id: Mapped[int] = mapped_column(Integer, primary_key=True) + key: Mapped[str] = mapped_column(String(255), unique=True, index=True) + name: Mapped[str] = mapped_column(String(500), index=True) + site: Mapped[str] = mapped_column(String(500)) + folder: Mapped[str] = mapped_column(String(1000)) + + episodes: Mapped[List["Episode"]] = relationship( + "Episode", back_populates="series", cascade="all, delete-orphan" + ) +``` + +Source: [src/server/database/models.py](../src/server/database/models.py#L23-L87) + +### 5.2 Pydantic API Models + +```python +# src/server/models/download.py + +class DownloadItem(BaseModel): + id: str + serie_id: str # Maps to anime_series.key + serie_folder: str # Metadata only + serie_name: str + episode: EpisodeIdentifier + status: DownloadStatus + priority: DownloadPriority +``` + +Source: [src/server/models/download.py](../src/server/models/download.py#L63-L118) + +### 5.3 Model Mapping + +| API Field | Database Column | Notes | +| -------------- | --------------------- | ------------------ | +| `serie_id` | `anime_series.key` | Primary identifier | +| `serie_folder` | `anime_series.folder` | Metadata only | +| `serie_name` | `anime_series.name` | Display name | + +--- + +## 6. Repository Pattern + +The `QueueRepository` class provides data access abstraction. + +```python +class QueueRepository: + async def save_item(self, item: DownloadItem) -> None: + """Save or update a download item.""" + + async def get_all_items(self) -> List[DownloadItem]: + """Get all items from database.""" + + async def delete_item(self, item_id: str) -> bool: + """Delete item by ID.""" + + async def get_items_by_status( + self, status: DownloadStatus + ) -> List[DownloadItem]: + """Get items filtered by status.""" +``` + +Source: [src/server/services/queue_repository.py](../src/server/services/queue_repository.py) + +--- + +## 7. Database Service + +The `AnimeSeriesService` provides async CRUD operations. + +```python +class AnimeSeriesService: + @staticmethod + async def create( + db: AsyncSession, + key: str, + name: str, + site: str, + folder: str + ) -> AnimeSeries: + """Create a new anime series.""" + + @staticmethod + async def get_by_key( + db: AsyncSession, + key: str + ) -> Optional[AnimeSeries]: + """Get series by primary key identifier.""" +``` + +Source: [src/server/database/service.py](../src/server/database/service.py) + +--- + +## 8. Data Integrity Rules + +### Validation Constraints + +| Field | Rule | Error Message | +| ------------------------- | ------------------------ | ------------------------------------- | +| `anime_series.key` | Non-empty, max 255 chars | "Series key cannot be empty" | +| `anime_series.name` | Non-empty, max 500 chars | "Series name cannot be empty" | +| `episodes.season` | 0-1000 | "Season number must be non-negative" | +| `episodes.episode_number` | 0-10000 | "Episode number must be non-negative" | + +Source: [src/server/database/models.py](../src/server/database/models.py#L89-L119) + +### Cascade Rules + +- Deleting `anime_series` deletes all related `episodes` and `download_queue_item` + +--- + +## 9. Migration Strategy + +Currently, SQLAlchemy's `create_all()` is used for schema creation. + +```python +# src/server/database/connection.py +async def init_db(): + async with engine.begin() as conn: + await conn.run_sync(Base.metadata.create_all) +``` + +For production migrations, Alembic is recommended but not yet implemented. + +Source: [src/server/database/connection.py](../src/server/database/connection.py) + +--- + +## 10. Common Query Patterns + +### Get all series with missing episodes + +```python +series = await db.execute( + select(AnimeSeries).options(selectinload(AnimeSeries.episodes)) +) +for serie in series.scalars(): + downloaded = [e for e in serie.episodes if e.is_downloaded] +``` + +### Get pending downloads ordered by priority + +```python +items = await db.execute( + select(DownloadQueueItem) + .where(DownloadQueueItem.status == "pending") + .order_by( + case( + (DownloadQueueItem.priority == "HIGH", 1), + (DownloadQueueItem.priority == "NORMAL", 2), + (DownloadQueueItem.priority == "LOW", 3), + ), + DownloadQueueItem.added_at + ) +) +``` + +--- + +## 11. Database Location + +| Environment | Default Location | +| ----------- | ------------------------------------------------- | +| Development | `./data/aniworld.db` | +| Production | Via `DATABASE_URL` environment variable | +| Testing | In-memory SQLite (`sqlite+aiosqlite:///:memory:`) | diff --git a/docs/DEVELOPMENT.md b/docs/DEVELOPMENT.md new file mode 100644 index 0000000..dc18406 --- /dev/null +++ b/docs/DEVELOPMENT.md @@ -0,0 +1,64 @@ +# Development Guide + +## Document Purpose + +This document provides guidance for developers working on the Aniworld project. + +### What This Document Contains + +- **Prerequisites**: Required software and tools +- **Environment Setup**: Step-by-step local development setup +- **Project Structure**: Source code organization explanation +- **Development Workflow**: Branch strategy, commit conventions +- **Coding Standards**: Style guide, linting, formatting +- **Running the Application**: Development server, CLI usage +- **Debugging Tips**: Common debugging approaches +- **IDE Configuration**: VS Code settings, recommended extensions +- **Contributing Guidelines**: How to submit changes +- **Code Review Process**: Review checklist and expectations + +### What This Document Does NOT Contain + +- Production deployment (see [DEPLOYMENT.md](DEPLOYMENT.md)) +- API reference (see [API.md](API.md)) +- Architecture decisions (see [ARCHITECTURE.md](ARCHITECTURE.md)) +- Test writing guides (see [TESTING.md](TESTING.md)) +- Security guidelines (see [SECURITY.md](SECURITY.md)) + +### Target Audience + +- New Developers joining the project +- Contributors (internal and external) +- Anyone setting up a development environment + +--- + +## Sections to Document + +1. Prerequisites + - Python version + - Conda environment + - Node.js (if applicable) + - Git +2. Getting Started + - Clone repository + - Setup conda environment + - Install dependencies + - Configuration setup +3. Project Structure Overview +4. Development Server + - Starting FastAPI server + - Hot reload configuration + - Debug mode +5. CLI Development +6. Code Style + - PEP 8 compliance + - Type hints requirements + - Docstring format + - Import organization +7. Git Workflow + - Branch naming + - Commit message format + - Pull request process +8. Common Development Tasks +9. Troubleshooting Development Issues diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..5c57b87 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,39 @@ +# Aniworld Documentation + +## Overview + +This directory contains all documentation for the Aniworld anime download manager project. + +## Documentation Structure + +| Document | Purpose | Target Audience | +| ---------------------------------------- | ---------------------------------------------- | ---------------------------------- | +| [ARCHITECTURE.md](ARCHITECTURE.md) | System architecture and design decisions | Architects, Senior Developers | +| [API.md](API.md) | REST API reference and WebSocket documentation | Frontend Developers, API Consumers | +| [DEVELOPMENT.md](DEVELOPMENT.md) | Developer setup and contribution guide | All Developers | +| [DEPLOYMENT.md](DEPLOYMENT.md) | Deployment and operations guide | DevOps, System Administrators | +| [DATABASE.md](DATABASE.md) | Database schema and data models | Backend Developers | +| [TESTING.md](TESTING.md) | Testing strategy and guidelines | QA Engineers, Developers | +| [SECURITY.md](SECURITY.md) | Security considerations and guidelines | Security Engineers, All Developers | +| [CONFIGURATION.md](CONFIGURATION.md) | Configuration options reference | Operators, Developers | +| [CHANGELOG.md](CHANGELOG.md) | Version history and changes | All Stakeholders | +| [TROUBLESHOOTING.md](TROUBLESHOOTING.md) | Common issues and solutions | Support, Operators | +| [features.md](features.md) | Feature list and capabilities | Product Owners, Users | +| [instructions.md](instructions.md) | AI agent development instructions | AI Agents, Developers | + +## Documentation Standards + +- All documentation uses Markdown format +- Keep documentation up-to-date with code changes +- Include code examples where applicable +- Use clear, concise language +- Include diagrams for complex concepts (use Mermaid syntax) + +## Contributing to Documentation + +When adding or updating documentation: + +1. Follow the established format in each document +2. Update the README.md if adding new documents +3. Ensure cross-references are valid +4. Review for spelling and grammar diff --git a/docs/TESTING.md b/docs/TESTING.md new file mode 100644 index 0000000..4cf745b --- /dev/null +++ b/docs/TESTING.md @@ -0,0 +1,71 @@ +# Testing Documentation + +## Document Purpose + +This document describes the testing strategy, guidelines, and practices for the Aniworld project. + +### What This Document Contains + +- **Testing Strategy**: Overall approach to quality assurance +- **Test Categories**: Unit, integration, API, performance, security tests +- **Test Structure**: Organization of test files and directories +- **Writing Tests**: Guidelines for writing effective tests +- **Fixtures and Mocking**: Shared test utilities and mock patterns +- **Running Tests**: Commands and configurations +- **Coverage Requirements**: Minimum coverage thresholds +- **CI/CD Integration**: How tests run in automation +- **Test Data Management**: Managing test fixtures and data +- **Best Practices**: Do's and don'ts for testing + +### What This Document Does NOT Contain + +- Production deployment (see [DEPLOYMENT.md](DEPLOYMENT.md)) +- Security audit procedures (see [SECURITY.md](SECURITY.md)) +- Bug tracking and issue management +- Performance benchmarking results + +### Target Audience + +- Developers writing tests +- QA Engineers +- CI/CD Engineers +- Code reviewers + +--- + +## Sections to Document + +1. Testing Philosophy + - Test pyramid approach + - Quality gates +2. Test Categories + - Unit Tests (`tests/unit/`) + - Integration Tests (`tests/integration/`) + - API Tests (`tests/api/`) + - Frontend Tests (`tests/frontend/`) + - Performance Tests (`tests/performance/`) + - Security Tests (`tests/security/`) +3. Test Structure and Naming + - File naming conventions + - Test function naming + - Test class organization +4. Running Tests + - pytest commands + - Running specific tests + - Verbose output + - Coverage reports +5. Fixtures and Conftest + - Shared fixtures + - Database fixtures + - Mock services +6. Mocking Guidelines + - What to mock + - Mock patterns + - External service mocks +7. Coverage Requirements +8. CI/CD Integration +9. Writing Good Tests + - Arrange-Act-Assert pattern + - Test isolation + - Edge cases +10. Common Pitfalls to Avoid diff --git a/features.md b/docs/features.md similarity index 100% rename from features.md rename to docs/features.md diff --git a/docs/identifier_standardization_validation.md b/docs/identifier_standardization_validation.md deleted file mode 100644 index efb1e3f..0000000 --- a/docs/identifier_standardization_validation.md +++ /dev/null @@ -1,422 +0,0 @@ -# Series Identifier Standardization - Validation Instructions - -## Overview - -This document provides comprehensive instructions for AI agents to validate the **Series Identifier Standardization** change across the Aniworld codebase. The change standardizes `key` as the primary identifier for series and relegates `folder` to metadata-only status. - -## Summary of the Change - -| Field | Purpose | Usage | -| -------- | ------------------------------------------------------------------------------ | --------------------------------------------------------------- | -| `key` | **Primary Identifier** - Provider-assigned, URL-safe (e.g., `attack-on-titan`) | All lookups, API operations, database queries, WebSocket events | -| `folder` | **Metadata Only** - Filesystem folder name (e.g., `Attack on Titan (2013)`) | Display purposes, filesystem operations only | -| `id` | **Database Primary Key** - Internal auto-increment integer | Database relationships only | - ---- - -## Validation Checklist - -### Phase 2: Application Layer Services - -**Files to validate:** - -1. **`src/server/services/anime_service.py`** - - - [ ] Class docstring explains `key` vs `folder` convention - - [ ] All public methods accept `key` parameter for series identification - - [ ] No methods accept `folder` as an identifier parameter - - [ ] Event handler methods document key/folder convention - - [ ] Progress tracking uses `key` in progress IDs where possible - -2. **`src/server/services/download_service.py`** - - - [ ] `DownloadItem` uses `serie_id` (which should be the `key`) - - [ ] `serie_folder` is documented as metadata only - - [ ] Queue operations look up series by `key` not `folder` - - [ ] Persistence format includes `serie_id` as the key identifier - -3. **`src/server/services/websocket_service.py`** - - - [ ] Module docstring explains key/folder convention - - [ ] Broadcast methods include `key` in message payloads - - [ ] `folder` is documented as optional/display only - - [ ] Event broadcasts use `key` as primary identifier - -4. **`src/server/services/scan_service.py`** - - - [ ] Scan operations use `key` for identification - - [ ] Progress events include `key` field - -5. **`src/server/services/progress_service.py`** - - [ ] Progress tracking includes `key` in metadata where applicable - -**Validation Commands:** - -```bash -# Check service layer for folder-based lookups -grep -rn "by_folder\|folder.*=.*identifier\|folder.*lookup" src/server/services/ --include="*.py" - -# Verify key is used in services -grep -rn "serie_id\|series_key\|key.*identifier" src/server/services/ --include="*.py" -``` - ---- - -### Phase 3: API Endpoints and Responses - -**Files to validate:** - -1. **`src/server/api/anime.py`** - - - [ ] `AnimeSummary` model has `key` field with proper description - - [ ] `AnimeDetail` model has `key` field with proper description - - [ ] API docstrings explain `key` is the primary identifier - - [ ] `folder` field descriptions state "metadata only" - - [ ] Endpoint paths use `key` parameter (e.g., `/api/anime/{key}`) - - [ ] No endpoints use `folder` as path parameter for lookups - -2. **`src/server/api/download.py`** - - - [ ] Download endpoints use `serie_id` (key) for operations - - [ ] Request models document key/folder convention - - [ ] Response models include `key` as primary identifier - -3. **`src/server/models/anime.py`** - - - [ ] Module docstring explains identifier convention - - [ ] `AnimeSeriesResponse` has `key` field properly documented - - [ ] `SearchResult` has `key` field properly documented - - [ ] Field validators normalize `key` to lowercase - - [ ] `folder` fields document metadata-only purpose - -4. **`src/server/models/download.py`** - - - [ ] `DownloadItem` has `serie_id` documented as the key - - [ ] `serie_folder` documented as metadata only - - [ ] Field descriptions are clear about primary vs metadata - -5. **`src/server/models/websocket.py`** - - [ ] Module docstring explains key/folder convention - - [ ] Message models document `key` as primary identifier - - [ ] `folder` documented as optional display metadata - -**Validation Commands:** - -```bash -# Check API endpoints for folder-based paths -grep -rn "folder.*Path\|/{folder}" src/server/api/ --include="*.py" - -# Verify key is used in endpoints -grep -rn "/{key}\|series_key\|serie_id" src/server/api/ --include="*.py" - -# Check model field descriptions -grep -rn "Field.*description.*identifier\|Field.*description.*key\|Field.*description.*folder" src/server/models/ --include="*.py" -``` - ---- - -### Phase 4: Frontend Integration - -**Files to validate:** - -1. **`src/server/web/static/js/app.js`** - - - [ ] `selectedSeries` Set uses `key` values, not `folder` - - [ ] `seriesData` array comments indicate `key` as primary identifier - - [ ] Selection operations use `key` property - - [ ] API calls pass `key` for series identification - - [ ] WebSocket message handlers extract `key` from data - - [ ] No code uses `folder` for series lookups - -2. **`src/server/web/static/js/queue.js`** - - - [ ] Queue items reference series by `key` or `serie_id` - - [ ] WebSocket handlers extract `key` from messages - - [ ] UI operations use `key` for identification - - [ ] `serie_folder` used only for display - -3. **`src/server/web/static/js/websocket_client.js`** - - - [ ] Message handling preserves `key` field - - [ ] No transformation that loses `key` information - -4. **HTML Templates** (`src/server/web/templates/`) - - [ ] Data attributes use `key` for identification (e.g., `data-key`) - - [ ] No `data-folder` used for identification purposes - - [ ] Display uses `folder` or `name` appropriately - -**Validation Commands:** - -```bash -# Check JavaScript for folder-based lookups -grep -rn "\.folder\s*==\|folder.*identifier\|getByFolder" src/server/web/static/js/ --include="*.js" - -# Check data attributes in templates -grep -rn "data-key\|data-folder\|data-series" src/server/web/templates/ --include="*.html" -``` - ---- - -### Phase 5: Database Operations - -**Files to validate:** - -1. **`src/server/database/models.py`** - - - [ ] `AnimeSeries` model has `key` column with unique constraint - - [ ] `key` column is indexed - - [ ] Model docstring explains identifier convention - - [ ] `folder` column docstring states "metadata only" - - [ ] Validators check `key` is not empty - - [ ] No `folder` uniqueness constraint (unless intentional) - -2. **`src/server/database/service.py`** - - - [ ] `AnimeSeriesService` has `get_by_key()` method - - [ ] Class docstring explains lookup convention - - [ ] No `get_by_folder()` without deprecation - - [ ] All CRUD operations use `key` for identification - - [ ] Logging uses `key` in messages - -**Validation Commands:** - -```bash -# Check database models -grep -rn "unique=True\|index=True" src/server/database/models.py - -# Check service lookups -grep -rn "get_by_key\|get_by_folder\|filter.*key\|filter.*folder" src/server/database/service.py -``` - ---- - -### Phase 6: WebSocket Events - -**Files to validate:** - -1. **All WebSocket broadcast calls** should include `key` in payload: - - - `download_progress` → includes `key` - - `download_complete` → includes `key` - - `download_failed` → includes `key` - - `scan_progress` → includes `key` (where applicable) - - `queue_status` → items include `key` - -2. **Message format validation:** - ```json - { - "type": "download_progress", - "data": { - "key": "attack-on-titan", // PRIMARY - always present - "folder": "Attack on Titan (2013)", // OPTIONAL - display only - "progress": 45.5, - ... - } - } - ``` - -**Validation Commands:** - -```bash -# Check WebSocket broadcast calls -grep -rn "broadcast.*key\|send_json.*key" src/server/services/ --include="*.py" - -# Check message construction -grep -rn '"key":\|"folder":' src/server/services/ --include="*.py" -``` - ---- - -### Phase 7: Test Coverage - -**Test files to validate:** - -1. **`tests/unit/test_serie_class.py`** - - - [ ] Tests for key validation (empty, whitespace, None) - - [ ] Tests for key as primary identifier - - [ ] Tests for folder as metadata only - -2. **`tests/unit/test_anime_service.py`** - - - [ ] Service tests use `key` for operations - - [ ] Mock objects have proper `key` attributes - -3. **`tests/unit/test_database_models.py`** - - - [ ] Tests for `key` uniqueness constraint - - [ ] Tests for `key` validation - -4. **`tests/unit/test_database_service.py`** - - - [ ] Tests for `get_by_key()` method - - [ ] No tests for deprecated folder lookups - -5. **`tests/api/test_anime_endpoints.py`** - - - [ ] API tests use `key` in requests - - [ ] Mock `FakeSerie` has proper `key` attribute - - [ ] Comments explain key/folder convention - -6. **`tests/unit/test_websocket_service.py`** - - [ ] WebSocket tests verify `key` in messages - - [ ] Broadcast tests include `key` in payload - -**Validation Commands:** - -```bash -# Run all tests -conda run -n AniWorld python -m pytest tests/ -v --tb=short - -# Run specific test files -conda run -n AniWorld python -m pytest tests/unit/test_serie_class.py -v -conda run -n AniWorld python -m pytest tests/unit/test_database_models.py -v -conda run -n AniWorld python -m pytest tests/api/test_anime_endpoints.py -v - -# Search tests for identifier usage -grep -rn "key.*identifier\|folder.*metadata" tests/ --include="*.py" -``` - ---- - -## Common Issues to Check - -### 1. Inconsistent Naming - -Look for inconsistent parameter names: - -- `serie_key` vs `series_key` vs `key` -- `serie_id` should refer to `key`, not database `id` -- `serie_folder` vs `folder` - -### 2. Missing Documentation - -Check that ALL models, services, and APIs document: - -- What `key` is and how to use it -- That `folder` is metadata only - -### 3. Legacy Code Patterns - -Search for deprecated patterns: - -```python -# Bad - using folder for lookup -series = get_by_folder(folder_name) - -# Good - using key for lookup -series = get_by_key(series_key) -``` - -### 4. API Response Consistency - -Verify all API responses include: - -- `key` field (primary identifier) -- `folder` field (optional, for display) - -### 5. Frontend Data Flow - -Verify the frontend: - -- Stores `key` in selection sets -- Passes `key` to API calls -- Uses `folder` only for display - ---- - -## Deprecation Warnings - -The following should have deprecation warnings (for removal in v3.0.0): - -1. Any `get_by_folder()` or `GetByFolder()` methods -2. Any API endpoints that accept `folder` as a lookup parameter -3. Any frontend code that uses `folder` for identification - -**Example deprecation:** - -```python -import warnings - -def get_by_folder(self, folder: str): - """DEPRECATED: Use get_by_key() instead.""" - warnings.warn( - "get_by_folder() is deprecated, use get_by_key(). " - "Will be removed in v3.0.0", - DeprecationWarning, - stacklevel=2 - ) - # ... implementation -``` - ---- - -## Automated Validation Script - -Run this script to perform automated checks: - -```bash -#!/bin/bash -# identifier_validation.sh - -echo "=== Series Identifier Standardization Validation ===" -echo "" - -echo "1. Checking core entities..." -grep -rn "PRIMARY IDENTIFIER\|metadata only" src/core/entities/ --include="*.py" | head -20 - -echo "" -echo "2. Checking for deprecated folder lookups..." -grep -rn "get_by_folder\|GetByFolder" src/ --include="*.py" - -echo "" -echo "3. Checking API models for key field..." -grep -rn 'key.*Field\|Field.*key' src/server/models/ --include="*.py" | head -20 - -echo "" -echo "4. Checking database models..." -grep -rn "key.*unique\|key.*index" src/server/database/models.py - -echo "" -echo "5. Checking frontend key usage..." -grep -rn "selectedSeries\|\.key\|data-key" src/server/web/static/js/ --include="*.js" | head -20 - -echo "" -echo "6. Running tests..." -conda run -n AniWorld python -m pytest tests/unit/test_serie_class.py -v --tb=short - -echo "" -echo "=== Validation Complete ===" -``` - ---- - -## Expected Results - -After validation, you should confirm: - -1. ✅ All core entities use `key` as primary identifier -2. ✅ All services look up series by `key` -3. ✅ All API endpoints use `key` for operations -4. ✅ All database queries use `key` for lookups -5. ✅ Frontend uses `key` for selection and API calls -6. ✅ WebSocket events include `key` in payload -7. ✅ All tests pass -8. ✅ Documentation clearly explains the convention -9. ✅ Deprecation warnings exist for legacy patterns - ---- - -## Sign-off - -Once validation is complete, update this section: - -- [x] Phase 1: Core Entities - Validated by: **AI Agent** Date: **28 Nov 2025** -- [x] Phase 2: Services - Validated by: **AI Agent** Date: **28 Nov 2025** -- [ ] Phase 3: API - Validated by: **\_\_\_** Date: **\_\_\_** -- [ ] Phase 4: Frontend - Validated by: **\_\_\_** Date: **\_\_\_** -- [ ] Phase 5: Database - Validated by: **\_\_\_** Date: **\_\_\_** -- [ ] Phase 6: WebSocket - Validated by: **\_\_\_** Date: **\_\_\_** -- [ ] Phase 7: Tests - Validated by: **\_\_\_** Date: **\_\_\_** - -**Final Approval:** \***\*\*\*\*\***\_\_\_\***\*\*\*\*\*** Date: **\*\***\_**\*\*** diff --git a/docs/infrastructure.md b/docs/infrastructure.md deleted file mode 100644 index 3675889..0000000 --- a/docs/infrastructure.md +++ /dev/null @@ -1,440 +0,0 @@ -# Aniworld Web Application Infrastructure - -```bash -conda activate AniWorld -``` - -## Project Structure - -``` -src/ -├── core/ # Core application logic -│ ├── SeriesApp.py # Main application class -│ ├── SerieScanner.py # Directory scanner -│ ├── entities/ # Domain entities (series.py, SerieList.py) -│ ├── interfaces/ # Abstract interfaces (providers.py, callbacks.py) -│ ├── providers/ # Content providers (aniworld, streaming) -│ └── exceptions/ # Custom exceptions -├── server/ # FastAPI web application -│ ├── fastapi_app.py # Main FastAPI application -│ ├── controllers/ # Route controllers (health, page, error) -│ ├── api/ # API routes (auth, config, anime, download, websocket) -│ ├── models/ # Pydantic models -│ ├── services/ # Business logic services -│ ├── database/ # SQLAlchemy ORM layer -│ ├── utils/ # Utilities (dependencies, templates, security) -│ └── web/ # Frontend (templates, static assets) -├── cli/ # CLI application -data/ # Config, database, queue state -logs/ # Application logs -tests/ # Test suites -``` - -## Technology Stack - -| Layer | Technology | -| --------- | ---------------------------------------------- | -| Backend | FastAPI, Uvicorn, SQLAlchemy, SQLite, Pydantic | -| Frontend | HTML5, CSS3, Vanilla JS, Bootstrap 5, HTMX | -| Security | JWT (python-jose), bcrypt (passlib) | -| Real-time | Native WebSocket | - -## Series Identifier Convention - -Throughout the codebase, three identifiers are used for anime series: - -| Identifier | Type | Purpose | Example | -| ---------- | --------------- | ----------------------------------------------------------- | -------------------------- | -| `key` | Unique, Indexed | **PRIMARY** - All lookups, API operations, WebSocket events | `"attack-on-titan"` | -| `folder` | String | Display/filesystem metadata only (never for lookups) | `"Attack on Titan (2013)"` | -| `id` | Primary Key | Internal database key for relationships | `1`, `42` | - -### Key Format Requirements - -- **Lowercase only**: No uppercase letters allowed -- **URL-safe**: Only alphanumeric characters and hyphens -- **Hyphen-separated**: Words separated by single hyphens -- **No leading/trailing hyphens**: Must start and end with alphanumeric -- **No consecutive hyphens**: `attack--titan` is invalid - -**Valid examples**: `"attack-on-titan"`, `"one-piece"`, `"86-eighty-six"`, `"re-zero"` -**Invalid examples**: `"Attack On Titan"`, `"attack_on_titan"`, `"attack on titan"` - -### Notes - -- **Backward Compatibility**: API endpoints accepting `anime_id` will check `key` first, then fall back to `folder` lookup -- **New Code**: Always use `key` for identification; `folder` is metadata only - -## API Endpoints - -### Authentication (`/api/auth`) - -- `POST /login` - Master password authentication (returns JWT) -- `POST /logout` - Invalidate session -- `GET /status` - Check authentication status - -### Configuration (`/api/config`) - -- `GET /` - Get configuration -- `PUT /` - Update configuration -- `POST /validate` - Validate without applying -- `GET /backups` - List backups -- `POST /backups/{name}/restore` - Restore backup - -### Anime (`/api/anime`) - -- `GET /` - List anime with missing episodes (returns `key` as identifier) -- `GET /{anime_id}` - Get anime details (accepts `key` or `folder` for backward compatibility) -- `POST /search` - Search for anime (returns `key` as identifier) -- `POST /add` - Add new series (extracts `key` from link URL) -- `POST /rescan` - Trigger library rescan - -**Response Models:** - -- `AnimeSummary`: `key` (primary identifier), `name`, `site`, `folder` (metadata), `missing_episodes`, `link` -- `AnimeDetail`: `key` (primary identifier), `title`, `folder` (metadata), `episodes`, `description` - -### Download Queue (`/api/queue`) - -- `GET /status` - Queue status and statistics -- `POST /add` - Add episodes to queue -- `DELETE /{item_id}` - Remove item -- `POST /start` | `/stop` | `/pause` | `/resume` - Queue control -- `POST /retry` - Retry failed downloads -- `DELETE /completed` - Clear completed items - -**Request Models:** - -- `DownloadRequest`: `serie_id` (key, primary identifier), `serie_folder` (filesystem path), `serie_name` (display), `episodes`, `priority` - -**Response Models:** - -- `DownloadItem`: `id`, `serie_id` (key), `serie_folder` (metadata), `serie_name`, `episode`, `status`, `progress` -- `QueueStatus`: `is_running`, `is_paused`, `active_downloads`, `pending_queue`, `completed_downloads`, `failed_downloads` - -### WebSocket (`/ws/connect`) - -Real-time updates for downloads, scans, and queue operations. - -**Rooms**: `downloads`, `download_progress`, `scan_progress` - -**Message Types**: `download_progress`, `download_complete`, `download_failed`, `queue_status`, `scan_progress`, `scan_complete`, `scan_failed` - -**Series Identifier in Messages:** -All series-related WebSocket events include `key` as the primary identifier in their data payload: - -```json -{ - "type": "download_progress", - "timestamp": "2025-10-17T10:30:00.000Z", - "data": { - "download_id": "abc123", - "key": "attack-on-titan", - "folder": "Attack on Titan (2013)", - "percent": 45.2, - "speed_mbps": 2.5, - "eta_seconds": 180 - } -} -``` - -## Database Models - -| Model | Purpose | -| ----------------- | ---------------------------------------- | -| AnimeSeries | Series metadata (key, name, folder, etc) | -| Episode | Episodes linked to series | -| DownloadQueueItem | Queue items with status and progress | -| UserSession | JWT sessions with expiry | - -**Mixins**: `TimestampMixin` (created_at, updated_at), `SoftDeleteMixin` - -### AnimeSeries Identifier Fields - -| Field | Type | Purpose | -| -------- | --------------- | ------------------------------------------------- | -| `id` | Primary Key | Internal database key for relationships | -| `key` | Unique, Indexed | **PRIMARY IDENTIFIER** for all lookups | -| `folder` | String | Filesystem metadata only (not for identification) | - -**Database Service Methods:** - -- `AnimeSeriesService.get_by_key(key)` - **Primary lookup method** -- `AnimeSeriesService.get_by_id(id)` - Internal lookup by database ID -- No `get_by_folder()` method exists - folder is never used for lookups - -### DownloadQueueItem Fields - -| Field | Type | Purpose | -| -------------- | ----------- | ----------------------------------------- | -| `id` | String (PK) | UUID for the queue item | -| `serie_id` | String | Series key for identification | -| `serie_folder` | String | Filesystem folder path | -| `serie_name` | String | Display name for the series | -| `season` | Integer | Season number | -| `episode` | Integer | Episode number | -| `status` | Enum | pending, downloading, completed, failed | -| `priority` | Enum | low, normal, high | -| `progress` | Float | Download progress percentage (0.0-100.0) | -| `error` | String | Error message if failed | -| `retry_count` | Integer | Number of retry attempts | -| `added_at` | DateTime | When item was added to queue | -| `started_at` | DateTime | When download started (nullable) | -| `completed_at` | DateTime | When download completed/failed (nullable) | - -## Data Storage - -### Storage Architecture - -The application uses **SQLite database** as the primary storage for all application data. - -| Data Type | Storage Location | Service | -| -------------- | ------------------ | --------------------------------------- | -| Anime Series | `data/aniworld.db` | `AnimeSeriesService` | -| Episodes | `data/aniworld.db` | `AnimeSeriesService` | -| Download Queue | `data/aniworld.db` | `DownloadService` via `QueueRepository` | -| User Sessions | `data/aniworld.db` | `AuthService` | -| Configuration | `data/config.json` | `ConfigService` | - -### Download Queue Storage - -The download queue is stored in SQLite via `QueueRepository`, which wraps `DownloadQueueService`: - -```python -# QueueRepository provides async operations for queue items -repository = QueueRepository(session_factory) - -# Save item to database -saved_item = await repository.save_item(download_item) - -# Get pending items (ordered by priority and add time) -pending = await repository.get_pending_items() - -# Update item status -await repository.update_status(item_id, DownloadStatus.COMPLETED) - -# Update download progress -await repository.update_progress(item_id, progress=45.5, downloaded=450, total=1000, speed=2.5) -``` - -**Queue Persistence Features:** - -- Queue state survives server restarts -- Items in `downloading` status are reset to `pending` on startup -- Failed items within retry limit are automatically re-queued -- Completed and failed history is preserved (with limits) -- Real-time progress updates are persisted to database - -### Anime Series Database Storage - -```python -# Add series to database -await AnimeSeriesService.create(db_session, series_data) - -# Query series by key -series = await AnimeSeriesService.get_by_key(db_session, "attack-on-titan") - -# Update series -await AnimeSeriesService.update(db_session, series_id, update_data) -``` - -### Legacy File Storage (Deprecated) - -The legacy file-based storage is **deprecated** and will be removed in v3.0.0: - -- `Serie.save_to_file()` - Deprecated, use `AnimeSeriesService.create()` -- `Serie.load_from_file()` - Deprecated, use `AnimeSeriesService.get_by_key()` -- `SerieList.add()` - Deprecated, use `SerieList.add_to_db()` - -Deprecation warnings are raised when using these methods. - -## Core Services - -### SeriesApp (`src/core/SeriesApp.py`) - -Main engine for anime series management with async support, progress callbacks, and cancellation. - -**Key Methods:** - -- `search(words)` - Search for anime series -- `download(serie_folder, season, episode, key, language)` - Download an episode -- `rescan()` - Rescan directory for missing episodes -- `get_all_series_from_data_files()` - Load all series from data files in the anime directory (used for database sync on startup) - -### Data File to Database Sync - -On application startup, the system automatically syncs series from data files to the database: - -1. After `download_service.initialize()` succeeds -2. `SeriesApp.get_all_series_from_data_files()` loads all series from `data` files -3. Each series is added to the database via `SerieList.add_to_db()` -4. Existing series are skipped (no duplicates) -5. Sync continues silently even if individual series fail - -This ensures that series metadata stored in filesystem data files is available in the database for the web application. - -### Callback System (`src/core/interfaces/callbacks.py`) - -- `ProgressCallback`, `ErrorCallback`, `CompletionCallback` -- Context classes include `key` + optional `folder` fields -- Thread-safe `CallbackManager` for multiple callback registration - -### Services (`src/server/services/`) - -| Service | Purpose | -| ---------------- | ----------------------------------------- | -| AnimeService | Series management, scans (uses SeriesApp) | -| DownloadService | Queue management, download execution | -| ScanService | Library scan operations with callbacks | -| ProgressService | Centralized progress tracking + WebSocket | -| WebSocketService | Real-time connection management | -| AuthService | JWT authentication, rate limiting | -| ConfigService | Configuration persistence with backups | - -## Validation Utilities (`src/server/utils/validators.py`) - -Provides data validation functions for ensuring data integrity across the application. - -### Series Key Validation - -- **`validate_series_key(key)`**: Validates key format (URL-safe, lowercase, hyphens only) - - Valid: `"attack-on-titan"`, `"one-piece"`, `"86-eighty-six"` - - Invalid: `"Attack On Titan"`, `"attack_on_titan"`, `"attack on titan"` -- **`validate_series_key_or_folder(identifier, allow_folder=True)`**: Backward-compatible validation - - Returns tuple `(identifier, is_key)` where `is_key` indicates if it's a valid key format - - Set `allow_folder=False` to require strict key format - -### Other Validators - -| Function | Purpose | -| --------------------------- | ------------------------------------------ | -| `validate_series_name` | Series display name validation | -| `validate_episode_range` | Episode range validation (1-1000) | -| `validate_download_quality` | Quality setting (360p-1080p, best, worst) | -| `validate_language` | Language codes (ger-sub, ger-dub, etc.) | -| `validate_anime_url` | Aniworld.to/s.to URL validation | -| `validate_backup_name` | Backup filename validation | -| `validate_config_data` | Configuration data structure validation | -| `sanitize_filename` | Sanitize filenames for safe filesystem use | - -## Template Helpers (`src/server/utils/template_helpers.py`) - -Provides utilities for template rendering and series data preparation. - -### Core Functions - -| Function | Purpose | -| -------------------------- | --------------------------------- | -| `get_base_context` | Base context for all templates | -| `render_template` | Render template with context | -| `validate_template_exists` | Check if template file exists | -| `list_available_templates` | List all available template files | - -### Series Context Helpers - -All series helpers use `key` as the primary identifier: - -| Function | Purpose | -| ----------------------------------- | ---------------------------------------------- | -| `prepare_series_context` | Prepare series data for templates (uses `key`) | -| `get_series_by_key` | Find series by `key` (not `folder`) | -| `filter_series_by_missing_episodes` | Filter series with missing episodes | - -**Example Usage:** - -```python -from src.server.utils.template_helpers import prepare_series_context - -series_data = [ - {"key": "attack-on-titan", "name": "Attack on Titan", "folder": "Attack on Titan (2013)"}, - {"key": "one-piece", "name": "One Piece", "folder": "One Piece (1999)"} -] -prepared = prepare_series_context(series_data, sort_by="name") -# Returns sorted list using 'key' as identifier -``` - -## Frontend - -### Static Files - -- CSS: `styles.css` (Fluent UI design), `ux_features.css` (accessibility) -- JS: `app.js`, `queue.js`, `websocket_client.js`, accessibility modules - -### WebSocket Client - -Native WebSocket wrapper with Socket.IO-compatible API: - -```javascript -const socket = io(); -socket.join("download_progress"); -socket.on("download_progress", (data) => { - /* ... */ -}); -``` - -### Authentication - -JWT tokens stored in localStorage, included as `Authorization: Bearer `. - -## Testing - -```bash -# All tests -conda run -n AniWorld python -m pytest tests/ -v - -# Unit tests only -conda run -n AniWorld python -m pytest tests/unit/ -v - -# API tests -conda run -n AniWorld python -m pytest tests/api/ -v -``` - -## Production Notes - -### Current (Single-Process) - -- SQLite with WAL mode -- In-memory WebSocket connections -- File-based config and queue persistence - -### Multi-Process Deployment - -- Switch to PostgreSQL/MySQL -- Move WebSocket registry to Redis -- Use distributed locking for queue operations -- Consider Redis for session/cache storage - -## Code Examples - -### API Usage with Key Identifier - -```python -# Fetching anime list - response includes 'key' as identifier -response = requests.get("/api/anime", headers={"Authorization": f"Bearer {token}"}) -anime_list = response.json() -# Each item has: key="attack-on-titan", folder="Attack on Titan (2013)", ... - -# Fetching specific anime by key (preferred) -response = requests.get("/api/anime/attack-on-titan", headers={"Authorization": f"Bearer {token}"}) - -# Adding to download queue using key -download_request = { - "serie_id": "attack-on-titan", # Use key, not folder - "serie_folder": "Attack on Titan (2013)", # Metadata for filesystem - "serie_name": "Attack on Titan", - "episodes": ["S01E01", "S01E02"], - "priority": 1 -} -response = requests.post("/api/queue/add", json=download_request, headers=headers) -``` - -### WebSocket Event Handling - -```javascript -// WebSocket events always include 'key' as identifier -socket.on("download_progress", (data) => { - const key = data.key; // Primary identifier: "attack-on-titan" - const folder = data.folder; // Metadata: "Attack on Titan (2013)" - updateProgressBar(key, data.percent); -}); -``` diff --git a/instructions.md b/docs/instructions.md similarity index 100% rename from instructions.md rename to docs/instructions.md diff --git a/todolist.md b/todolist.md new file mode 100644 index 0000000..8006861 --- /dev/null +++ b/todolist.md @@ -0,0 +1,190 @@ +# Todolist - Architecture and Design Issues + +This document tracks design and architecture issues discovered during documentation review. + +--- + +## Issues + +### 1. In-Memory Rate Limiting Not Persistent + +**Title:** In-memory rate limiting resets on process restart + +**Severity:** medium + +**Location:** [src/server/middleware/auth.py](src/server/middleware/auth.py#L54-L68) + +**Description:** Rate limiting state is stored in memory dictionaries (`_rate`, `_origin_rate`) which reset when the process restarts, allowing attackers to bypass lockouts. + +**Suggested action:** Implement Redis-backed rate limiting for production deployments; add documentation warning about single-process limitation. + +--- + +### 2. Failed Login Attempts Not Persisted + +**Title:** Failed login attempts stored in-memory only + +**Severity:** medium + +**Location:** [src/server/services/auth_service.py](src/server/services/auth_service.py#L62-L74) + +**Description:** The `_failed` dictionary tracking failed login attempts resets on process restart, allowing brute-force bypass via service restart. + +**Suggested action:** Store failed attempts in database or Redis; add configurable lockout policy. + +--- + +### 3. Duplicate Health Endpoints + +**Title:** Health endpoints defined in two locations + +**Severity:** low + +**Location:** [src/server/api/health.py](src/server/api/health.py) and [src/server/controllers/health_controller.py](src/server/controllers/health_controller.py) + +**Description:** Health check functionality is split between `api/health.py` (detailed checks) and `controllers/health_controller.py` (basic check). Both are registered, potentially causing confusion. + +**Suggested action:** Consolidate health endpoints into a single module; remove duplicate controller. + +--- + +### 4. Deprecation Warnings in Production Code + +**Title:** Deprecated file-based scan method still in use + +**Severity:** low + +**Location:** [src/core/SerieScanner.py](src/core/SerieScanner.py#L129-L145) + +**Description:** The `scan()` method emits deprecation warnings but is still callable. CLI may still use this method. + +**Suggested action:** Complete migration to `scan_async()` with database storage; remove deprecated method after CLI update. + +--- + +### 5. SQLite Concurrency Limitations + +**Title:** SQLite not suitable for high concurrency + +**Severity:** medium + +**Location:** [src/config/settings.py](src/config/settings.py#L53-L55) + +**Description:** Default database is SQLite (`sqlite:///./data/aniworld.db`) which has limited concurrent write support. May cause issues under load. + +**Suggested action:** Document PostgreSQL migration path; add connection pooling configuration for production. + +--- + +### 6. Master Password Hash in Config File + +**Title:** Password hash stored in plaintext config file + +**Severity:** medium + +**Location:** [data/config.json](data/config.json) (other.master_password_hash) + +**Description:** The bcrypt password hash is stored in `config.json` which may be world-readable depending on deployment. + +**Suggested action:** Ensure config file has restricted permissions (600); consider environment variable for hash in production. + +--- + +### 7. Module-Level Singleton Pattern + +**Title:** Singleton services using module-level globals + +**Severity:** low + +**Location:** [src/server/services/download_service.py](src/server/services/download_service.py), [src/server/utils/dependencies.py](src/server/utils/dependencies.py) + +**Description:** Services use module-level `_instance` variables for singletons, making testing harder and preventing multi-instance hosting. + +**Suggested action:** Migrate to FastAPI app.state for service storage; document testing patterns for singleton cleanup. + +--- + +### 8. Hardcoded Provider + +**Title:** Default provider hardcoded + +**Severity:** low + +**Location:** [src/config/settings.py](src/config/settings.py#L66-L68) + +**Description:** The `default_provider` setting defaults to `"aniworld.to"` but provider switching is not fully implemented in the API. + +**Suggested action:** Implement provider selection endpoint; document available providers. + +--- + +### 9. Inconsistent Error Response Format + +**Title:** Some endpoints return different error formats + +**Severity:** low + +**Location:** [src/server/api/download.py](src/server/api/download.py), [src/server/api/anime.py](src/server/api/anime.py) + +**Description:** Most endpoints use the standard error response format from `error_handler.py`, but some handlers return raw `{"detail": "..."}` responses. + +**Suggested action:** Audit all endpoints for consistent error response structure; use custom exception classes uniformly. + +--- + +### 10. Missing Input Validation on WebSocket + +**Title:** WebSocket messages lack comprehensive validation + +**Severity:** low + +**Location:** [src/server/api/websocket.py](src/server/api/websocket.py#L120-L145) + +**Description:** Client messages are parsed with basic Pydantic validation, but room names and action types are not strictly validated against an allow-list. + +**Suggested action:** Add explicit room name validation; rate-limit WebSocket message frequency. + +--- + +### 11. No Token Revocation Storage + +**Title:** JWT token revocation is a no-op + +**Severity:** medium + +**Location:** [src/server/services/auth_service.py](src/server/services/auth_service.py) + +**Description:** The `revoke_token()` method exists but does not persist revocations. Logged-out tokens remain valid until expiry. + +**Suggested action:** Implement token blacklist in database or Redis; check blacklist in `create_session_model()`. + +--- + +### 12. Anime Directory Validation + +**Title:** Anime directory path not validated on startup + +**Severity:** low + +**Location:** [src/server/fastapi_app.py](src/server/fastapi_app.py#L107-L125) + +**Description:** The configured anime directory is used without validation that it exists and is readable. Errors only appear when scanning. + +**Suggested action:** Add directory validation in lifespan startup; return clear error if path invalid. + +--- + +## Summary + +| Severity | Count | +| --------- | ------ | +| High | 0 | +| Medium | 5 | +| Low | 7 | +| **Total** | **12** | + +--- + +## Changelog Note + +**2025-12-13**: Initial documentation review completed. Created comprehensive API.md with all REST and WebSocket endpoints documented with source references. Updated ARCHITECTURE.md with system overview, layer descriptions, design patterns, and data flow diagrams. Created README.md with quick start guide. Identified 12 design/architecture issues requiring attention.