diff --git a/docs/API.md b/docs/API.md deleted file mode 100644 index cbbe532..0000000 --- a/docs/API.md +++ /dev/null @@ -1,1596 +0,0 @@ -# API Documentation - -## Document Purpose - -This document provides comprehensive REST API and WebSocket reference for the Aniworld application. - -Source: [src/server/fastapi_app.py](../src/server/fastapi_app.py#L1-L252) - ---- - -## 1. API Overview - -### Base URL and Versioning - -| Environment | Base URL | -| ------------------- | --------------------------------- | -| Local Development | `http://127.0.0.1:8000` | -| API Documentation | `http://127.0.0.1:8000/api/docs` | -| ReDoc Documentation | `http://127.0.0.1:8000/api/redoc` | - -The API does not use versioning prefixes. All endpoints are available under `/api/*`. - -Source: [src/server/fastapi_app.py](../src/server/fastapi_app.py#L177-L184) - -### Authentication - -The API uses JWT Bearer Token authentication. - -**Header Format:** - -``` -Authorization: Bearer -``` - -**Public Endpoints (no authentication required):** - -- `/api/auth/*` - Authentication endpoints -- `/api/health` - Health check endpoints -- `/api/docs`, `/api/redoc` - API documentation -- `/static/*` - Static files -- `/`, `/login`, `/setup`, `/queue` - UI pages - -Source: [src/server/middleware/auth.py](../src/server/middleware/auth.py#L39-L52) - -### Content Types - -| Direction | Content-Type | -| --------- | ----------------------------- | -| Request | `application/json` | -| Response | `application/json` | -| WebSocket | `application/json` (messages) | - -### Common Headers - -| Header | Required | Description | -| --------------- | -------- | ------------------------------------ | -| `Authorization` | Yes\* | Bearer token for protected endpoints | -| `Content-Type` | Yes | `application/json` for POST/PUT | -| `Origin` | No | Required for CORS preflight | - -\*Not required for public endpoints listed above. - ---- - -## 2. Authentication Endpoints - -Prefix: `/api/auth` - -Source: [src/server/api/auth.py](../src/server/api/auth.py#L1-L180) - -### POST /api/auth/setup - -Initial setup endpoint to configure the master password. Can only be called once. - -**Request Body:** - -```json -{ - "master_password": "string (min 8 chars, mixed case, number, special char)", - "anime_directory": "string (optional, path to anime folder)" -} -``` - -**Response (201 Created):** - -```json -{ - "status": "ok" -} -``` - -**Errors:** - -- `400 Bad Request` - Master password already configured or invalid password - -Source: [src/server/api/auth.py](../src/server/api/auth.py#L28-L90) - -### POST /api/auth/login - -Validate master password and return JWT token. - -**Request Body:** - -```json -{ - "password": "string", - "remember": false -} -``` - -**Response (200 OK):** - -```json -{ - "access_token": "eyJ...", - "token_type": "bearer", - "expires_at": "2025-12-14T10:30:00Z" -} -``` - -**Errors:** - -- `401 Unauthorized` - Invalid credentials -- `429 Too Many Requests` - Account locked due to failed attempts - -Source: [src/server/api/auth.py](../src/server/api/auth.py#L93-L124) - -### POST /api/auth/logout - -Logout by revoking token. - -**Response (200 OK):** - -```json -{ - "status": "ok", - "message": "Logged out successfully" -} -``` - -Source: [src/server/api/auth.py](../src/server/api/auth.py#L127-L140) - -### GET /api/auth/status - -Return whether master password is configured and if caller is authenticated. - -**Response (200 OK):** - -```json -{ - "configured": true, - "authenticated": true -} -``` - -Source: [src/server/api/auth.py](../src/server/api/auth.py#L157-L162) - ---- - -## 3. Anime Endpoints - -Prefix: `/api/anime` - -Source: [src/server/api/anime.py](../src/server/api/anime.py#L1-L812) - -### Series Identifier Convention - -The API uses two identifier fields: - -| Field | Purpose | Example | -| -------- | ---------------------------------------------------- | -------------------------- | -| `key` | **Primary identifier** - provider-assigned, URL-safe | `"attack-on-titan"` | -| `folder` | Metadata only - filesystem folder name | `"Attack on Titan (2013)"` | - -Use `key` for all API operations. The `folder` field is for display purposes only. - -### GET /api/anime/status - -Get anime library status information. - -**Authentication:** Required - -**Response (200 OK):** - -```json -{ - "directory": "/path/to/anime", - "series_count": 42 -} -``` - -Source: [src/server/api/anime.py](../src/server/api/anime.py#L28-L58) - -### GET /api/anime - -List library series that have missing episodes. - -**Authentication:** Required - -**Query Parameters:** -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `page` | int | 1 | Page number (must be positive) | -| `per_page` | int | 20 | Items per page (max 1000) | -| `sort_by` | string | null | Sort field: `title`, `id`, `name`, `missing_episodes` | -| `filter` | string | null | Filter: `missing_episodes` (shows series with any missing episodes), `no_episodes` (shows series with zero downloaded episodes) | - -**Filter Details:** - -- `missing_episodes`: Returns series that have at least one missing episode recorded in the database (`is_downloaded=False`) -- `no_episodes`: Returns series that have missing episodes and no downloaded episodes (i.e., only missing episodes exist in the database) -- Episodes in the database represent MISSING episodes (from episodeDict during scanning) -- `is_downloaded=False` means the episode file was not found in the folder - -**Response (200 OK):** - -```json -[ - { - "key": "beheneko-the-elf-girls-cat", - "name": "Beheneko", - "site": "aniworld.to", - "folder": "beheneko the elf girls cat (2025)", - "missing_episodes": { "1": [1, 2, 3, 4] }, - "link": "" - } -] -``` - -**Example with filter:** - -```bash -GET /api/anime?filter=no_episodes -``` - -Source: [src/server/api/anime.py](../src/server/api/anime.py#L155-L303) - -### GET /api/anime/search - -Search the provider for anime series matching a query. - -**Authentication:** Not required - -**Query Parameters:** -| Parameter | Type | Required | Description | -|-----------|------|----------|-------------| -| `query` | string | Yes | Search term (max 200 chars) | - -**Response (200 OK):** - -```json -[ - { - "key": "attack-on-titan", - "name": "Attack on Titan", - "site": "aniworld.to", - "folder": "Attack on Titan (2013)", - "missing_episodes": {}, - "link": "https://aniworld.to/anime/stream/attack-on-titan" - } -] -``` - -Source: [src/server/api/anime.py](../src/server/api/anime.py#L431-L474) - -### POST /api/anime/search - -Search via POST body. - -**Request Body:** - -```json -{ - "query": "attack on titan" -} -``` - -**Response:** Same as GET /api/anime/search - -Source: [src/server/api/anime.py](../src/server/api/anime.py#L477-L495) - -### POST /api/anime/add - -Add a new series to the library with automatic database persistence, folder creation, and episode scanning. - -**Authentication:** Required - -**Request Body:** - -```json -{ - "link": "https://aniworld.to/anime/stream/attack-on-titan", - "name": "Attack on Titan" -} -``` - -**Response (200 OK):** - -```json -{ - "status": "success", - "message": "Successfully added series: Attack on Titan", - "key": "attack-on-titan", - "folder": "Attack on Titan", - "db_id": 1, - "missing_episodes": ["1-1", "1-2", "1-3"], - "total_missing": 3 -} -``` - -**Enhanced Flow:** - -1. Validates the request (link format, name) -2. Creates Serie object with sanitized folder name -3. Saves to database via AnimeDBService -4. Creates folder using sanitized display name (not internal key) -5. Performs targeted episode scan for this anime only -6. Returns response with missing episodes count - -**Folder Name Sanitization:** - -- Removes invalid filesystem characters: `< > : " / \ | ? *` -- Trims leading/trailing whitespace and dots -- Preserves Unicode characters (for Japanese titles) -- Example: `"Attack on Titan: Final Season"` → `"Attack on Titan Final Season"` - -Source: [src/server/api/anime.py](../src/server/api/anime.py#L604-L710) - -### POST /api/anime/rescan - -Trigger a rescan of the local library. - -**Authentication:** Required - -**Response (200 OK):** - -```json -{ - "success": true, - "message": "Rescan started successfully" -} -``` - -Source: [src/server/api/anime.py](../src/server/api/anime.py#L306-L337) - -### GET /api/anime/{anime_id} - -Return detailed information about a specific series. - -**Authentication:** Not required - -**Path Parameters:** -| Parameter | Description | -|-----------|-------------| -| `anime_id` | Series `key` (primary) or `folder` (deprecated fallback) | - -**Response (200 OK):** - -```json -{ - "key": "attack-on-titan", - "title": "Attack on Titan", - "folder": "Attack on Titan (2013)", - "episodes": ["1-1", "1-2", "1-3"], - "description": null -} -``` - -Source: [src/server/api/anime.py](../src/server/api/anime.py#L713-L793) - ---- - -## 4. Download Queue Endpoints - -Prefix: `/api/queue` - -Source: [src/server/api/download.py](../src/server/api/download.py#L1-L529) - -### GET /api/queue/status - -Get current download queue status and statistics. - -**Authentication:** Required - -**Response (200 OK):** - -```json -{ - "status": { - "is_running": false, - "is_paused": false, - "active_downloads": [], - "pending_queue": [], - "completed_downloads": [], - "failed_downloads": [] - }, - "statistics": { - "total_items": 5, - "pending_count": 3, - "active_count": 1, - "completed_count": 1, - "failed_count": 0, - "total_downloaded_mb": 1024.5, - "average_speed_mbps": 2.5, - "estimated_time_remaining": 3600 - } -} -``` - -Source: [src/server/api/download.py](../src/server/api/download.py#L21-L56) - -### POST /api/queue/add - -Add episodes to the download queue. - -**Authentication:** Required - -**Request Body:** - -```json -{ - "serie_id": "attack-on-titan", - "serie_folder": "Attack on Titan (2013)", - "serie_name": "Attack on Titan", - "episodes": [ - { "season": 1, "episode": 1, "title": "Episode 1" }, - { "season": 1, "episode": 2, "title": "Episode 2" } - ], - "priority": "NORMAL" -} -``` - -**Priority Values:** `LOW`, `NORMAL`, `HIGH` - -**Response (201 Created):** - -```json -{ - "status": "success", - "message": "Added 2 episode(s) to download queue", - "added_items": ["uuid1", "uuid2"], - "item_ids": ["uuid1", "uuid2"], - "failed_items": [] -} -``` - -Source: [src/server/api/download.py](../src/server/api/download.py#L59-L120) - -### POST /api/queue/start - -Start automatic queue processing. - -**Authentication:** Required - -**Response (200 OK):** - -```json -{ - "status": "success", - "message": "Queue processing started" -} -``` - -Source: [src/server/api/download.py](../src/server/api/download.py#L293-L331) - -### POST /api/queue/stop - -Stop processing new downloads from queue. - -**Authentication:** Required - -**Response (200 OK):** - -```json -{ - "status": "success", - "message": "Queue processing stopped (current download will continue)" -} -``` - -Source: [src/server/api/download.py](../src/server/api/download.py#L334-L387) - -### POST /api/queue/pause - -Pause queue processing (alias for stop). - -**Authentication:** Required - -**Response (200 OK):** - -```json -{ - "status": "success", - "message": "Queue processing paused" -} -``` - -Source: [src/server/api/download.py](../src/server/api/download.py#L416-L445) - -### DELETE /api/queue/{item_id} - -Remove a specific item from the download queue. - -**Authentication:** Required - -**Path Parameters:** -| Parameter | Description | -|-----------|-------------| -| `item_id` | Download item UUID | - -**Response (204 No Content)** - -Source: [src/server/api/download.py](../src/server/api/download.py#L225-L256) - -### DELETE /api/queue - -Remove multiple items from the download queue. - -**Authentication:** Required - -**Request Body:** - -```json -{ - "item_ids": ["uuid1", "uuid2"] -} -``` - -**Response (204 No Content)** - -Source: [src/server/api/download.py](../src/server/api/download.py#L259-L290) - -### DELETE /api/queue/completed - -Clear completed downloads from history. - -**Authentication:** Required - -**Response (200 OK):** - -```json -{ - "status": "success", - "message": "Cleared 5 completed item(s)", - "count": 5 -} -``` - -Source: [src/server/api/download.py](../src/server/api/download.py#L123-L149) - -### DELETE /api/queue/failed - -Clear failed downloads from history. - -**Authentication:** Required - -**Response (200 OK):** - -```json -{ - "status": "success", - "message": "Cleared 2 failed item(s)", - "count": 2 -} -``` - -Source: [src/server/api/download.py](../src/server/api/download.py#L152-L178) - -### DELETE /api/queue/pending - -Clear all pending downloads from the queue. - -**Authentication:** Required - -**Response (200 OK):** - -```json -{ - "status": "success", - "message": "Removed 10 pending item(s)", - "count": 10 -} -``` - -Source: [src/server/api/download.py](../src/server/api/download.py#L181-L207) - -### POST /api/queue/reorder - -Reorder items in the pending queue. - -**Authentication:** Required - -**Request Body:** - -```json -{ - "item_ids": ["uuid3", "uuid1", "uuid2"] -} -``` - -**Response (200 OK):** - -```json -{ - "status": "success", - "message": "Queue reordered with 3 items" -} -``` - -Source: [src/server/api/download.py](../src/server/api/download.py#L448-L477) - -### POST /api/queue/retry - -Retry failed downloads. - -**Authentication:** Required - -**Request Body:** - -```json -{ - "item_ids": ["uuid1", "uuid2"] -} -``` - -Pass empty `item_ids` array to retry all failed items. - -**Response (200 OK):** - -```json -{ - "status": "success", - "message": "Retrying 2 failed item(s)", - "retried_count": 2, - "retried_ids": ["uuid1", "uuid2"] -} -``` - -Source: [src/server/api/download.py](../src/server/api/download.py#L480-L514) - ---- - -## 5. Configuration Endpoints - -Prefix: `/api/config` - -Source: [src/server/api/config.py](../src/server/api/config.py#L1-L374) - -### GET /api/config - -Return current application configuration. - -**Authentication:** Required - -**Response (200 OK):** - -```json -{ - "name": "Aniworld", - "data_dir": "data", - "scheduler": { - "enabled": true, - "interval_minutes": 60, - "schedule_time": "03:00", - "schedule_days": ["mon", "tue", "wed", "thu", "fri", "sat", "sun"], - "auto_download_after_rescan": false - }, - "logging": { - "level": "INFO", - "file": null, - "max_bytes": null, - "backup_count": 3 - }, - "backup": { - "enabled": false, - "path": "data/backups", - "keep_days": 30 - }, - "other": {} -} -``` - -Source: [src/server/api/config.py](../src/server/api/config.py#L16-L27) - -### PUT /api/config - -Apply an update to the configuration. - -**Authentication:** Required - -**Request Body:** - -```json -{ - "scheduler": { - "enabled": true, - "interval_minutes": 60, - "schedule_time": "06:30", - "schedule_days": ["mon", "wed", "fri"] - }, - "logging": { - "level": "DEBUG" - } -} -``` - -**Response (200 OK):** Updated configuration object - -Source: [src/server/api/config.py](../src/server/api/config.py#L30-L47) - -### POST /api/config/validate - -Validate a configuration without applying it. - -**Authentication:** Required - -**Request Body:** Full `AppConfig` object - -**Response (200 OK):** - -```json -{ - "valid": true, - "errors": [] -} -``` - -Source: [src/server/api/config.py](../src/server/api/config.py#L50-L64) - -### GET /api/config/backups - -List all available configuration backups. - -**Authentication:** Required - -**Response (200 OK):** - -```json -[ - { - "name": "config_backup_20251213_090130.json", - "size": 1024, - "created": "2025-12-13T09:01:30Z" - } -] -``` - -Source: [src/server/api/config.py](../src/server/api/config.py#L67-L81) - -### POST /api/config/backups - -Create a backup of the current configuration. - -**Authentication:** Required - -**Query Parameters:** -| Parameter | Type | Required | Description | -|-----------|------|----------|-------------| -| `name` | string | No | Custom backup name | - -**Response (200 OK):** - -```json -{ - "name": "config_backup_20251213_090130.json", - "message": "Backup created successfully" -} -``` - -Source: [src/server/api/config.py](../src/server/api/config.py#L84-L102) - -### POST /api/config/backups/{backup_name}/restore - -Restore configuration from a backup. - -**Authentication:** Required - -**Response (200 OK):** Restored configuration object - -Source: [src/server/api/config.py](../src/server/api/config.py#L105-L123) - -### DELETE /api/config/backups/{backup_name} - -Delete a configuration backup. - -**Authentication:** Required - -**Response (200 OK):** - -```json -{ - "message": "Backup 'config_backup_20251213.json' deleted successfully" -} -``` - -Source: [src/server/api/config.py](../src/server/api/config.py#L126-L142) - -### POST /api/config/directory - -Update anime directory configuration. - -**Authentication:** Required - -**Request Body:** - -```json -{ - "directory": "/path/to/anime" -} -``` - -**Response (200 OK):** - -```json -{ - "message": "Anime directory updated successfully", - "synced_series": 15 -} -``` - -Source: [src/server/api/config.py](../src/server/api/config.py#L189-L247) - ---- - -## 6. NFO Management Endpoints - -Prefix: `/api/nfo` - -Source: [src/server/api/nfo.py](../src/server/api/nfo.py#L1-L684) - -These endpoints manage tvshow.nfo metadata files and associated media (poster, logo, fanart) for anime series. NFO files use Kodi/XBMC format and are scraped from TMDB API. - -**Prerequisites:** - -- TMDB API key must be configured in settings -- NFO service returns 503 if API key not configured - -### GET /api/nfo/{serie_id}/check - -Check if NFO file and media files exist for a series. - -**Authentication:** Required - -**Path Parameters:** - -- `serie_id` (string): Series identifier - -**Response (200 OK):** - -```json -{ - "serie_id": "one-piece", - "serie_folder": "One Piece (1999)", - "has_nfo": true, - "nfo_path": "/path/to/anime/One Piece (1999)/tvshow.nfo", - "media_files": { - "has_poster": true, - "has_logo": false, - "has_fanart": true, - "poster_path": "/path/to/anime/One Piece (1999)/poster.jpg", - "logo_path": null, - "fanart_path": "/path/to/anime/One Piece (1999)/fanart.jpg" - } -} -``` - -**Errors:** - -- `401 Unauthorized` - Not authenticated -- `404 Not Found` - Series not found -- `503 Service Unavailable` - TMDB API key not configured - -Source: [src/server/api/nfo.py](../src/server/api/nfo.py#L90-L147) - -### POST /api/nfo/{serie_id}/create - -Create NFO file and download media for a series. - -**Authentication:** Required - -**Path Parameters:** - -- `serie_id` (string): Series identifier - -**Request Body:** - -```json -{ - "serie_name": "One Piece", - "year": 1999, - "download_poster": true, - "download_logo": true, - "download_fanart": true, - "overwrite_existing": false -} -``` - -**Fields:** - -- `serie_name` (string, optional): Series name for TMDB search (defaults to folder name) -- `year` (integer, optional): Series year to help narrow TMDB search -- `download_poster` (boolean, default: true): Download poster.jpg -- `download_logo` (boolean, default: true): Download logo.png -- `download_fanart` (boolean, default: true): Download fanart.jpg -- `overwrite_existing` (boolean, default: false): Overwrite existing NFO - -**Response (200 OK):** - -```json -{ - "serie_id": "one-piece", - "serie_folder": "One Piece (1999)", - "nfo_path": "/path/to/anime/One Piece (1999)/tvshow.nfo", - "media_files": { - "has_poster": true, - "has_logo": true, - "has_fanart": true, - "poster_path": "/path/to/anime/One Piece (1999)/poster.jpg", - "logo_path": "/path/to/anime/One Piece (1999)/logo.png", - "fanart_path": "/path/to/anime/One Piece (1999)/fanart.jpg" - }, - "message": "NFO and media files created successfully" -} -``` - -**Errors:** - -- `401 Unauthorized` - Not authenticated -- `404 Not Found` - Series not found -- `409 Conflict` - NFO already exists (use `overwrite_existing: true`) -- `503 Service Unavailable` - TMDB API error or key not configured - -Source: [src/server/api/nfo.py](../src/server/api/nfo.py#L150-L240) - -### PUT /api/nfo/{serie_id}/update - -Update existing NFO file with fresh TMDB data. - -**Authentication:** Required - -**Path Parameters:** - -- `serie_id` (string): Series identifier - -**Query Parameters:** - -- `download_media` (boolean, default: true): Re-download media files - -**Response (200 OK):** - -```json -{ - "serie_id": "one-piece", - "serie_folder": "One Piece (1999)", - "nfo_path": "/path/to/anime/One Piece (1999)/tvshow.nfo", - "media_files": { - "has_poster": true, - "has_logo": true, - "has_fanart": true, - "poster_path": "/path/to/anime/One Piece (1999)/poster.jpg", - "logo_path": "/path/to/anime/One Piece (1999)/logo.png", - "fanart_path": "/path/to/anime/One Piece (1999)/fanart.jpg" - }, - "message": "NFO updated successfully" -} -``` - -**Errors:** - -- `401 Unauthorized` - Not authenticated -- `404 Not Found` - Series or NFO not found (use create endpoint) -- `503 Service Unavailable` - TMDB API error - -Source: [src/server/api/nfo.py](../src/server/api/nfo.py#L243-L325) - -### GET /api/nfo/{serie_id}/content - -Get NFO file XML content for a series. - -**Authentication:** Required - -**Path Parameters:** - -- `serie_id` (string): Series identifier - -**Response (200 OK):** - -```json -{ - "serie_id": "one-piece", - "serie_folder": "One Piece (1999)", - "content": "\n...", - "file_size": 2048, - "last_modified": "2026-01-15T10:30:00" -} -``` - -**Errors:** - -- `401 Unauthorized` - Not authenticated -- `404 Not Found` - Series or NFO not found - -Source: [src/server/api/nfo.py](../src/server/api/nfo.py#L328-L397) - -### GET /api/nfo/{serie_id}/media/status - -Get media files status for a series. - -**Authentication:** Required - -**Path Parameters:** - -- `serie_id` (string): Series identifier - -**Response (200 OK):** - -```json -{ - "has_poster": true, - "has_logo": false, - "has_fanart": true, - "poster_path": "/path/to/anime/One Piece (1999)/poster.jpg", - "logo_path": null, - "fanart_path": "/path/to/anime/One Piece (1999)/fanart.jpg" -} -``` - -**Errors:** - -- `401 Unauthorized` - Not authenticated -- `404 Not Found` - Series not found - -Source: [src/server/api/nfo.py](../src/server/api/nfo.py#L400-L447) - -### POST /api/nfo/{serie_id}/media/download - -Download missing media files for a series. - -**Authentication:** Required - -**Path Parameters:** - -- `serie_id` (string): Series identifier - -**Request Body:** - -```json -{ - "download_poster": true, - "download_logo": true, - "download_fanart": true -} -``` - -**Response (200 OK):** - -```json -{ - "has_poster": true, - "has_logo": true, - "has_fanart": true, - "poster_path": "/path/to/anime/One Piece (1999)/poster.jpg", - "logo_path": "/path/to/anime/One Piece (1999)/logo.png", - "fanart_path": "/path/to/anime/One Piece (1999)/fanart.jpg" -} -``` - -**Errors:** - -- `401 Unauthorized` - Not authenticated -- `404 Not Found` - Series or NFO not found (NFO required for TMDB ID) -- `503 Service Unavailable` - TMDB API error - -Source: [src/server/api/nfo.py](../src/server/api/nfo.py#L450-L519) - -### POST /api/nfo/batch/create - -Batch create NFO files for multiple series. - -**Authentication:** Required - -**Request Body:** - -```json -{ - "serie_ids": ["one-piece", "naruto", "bleach"], - "download_media": true, - "skip_existing": true, - "max_concurrent": 3 -} -``` - -**Fields:** - -- `serie_ids` (array of strings): Series identifiers to process -- `download_media` (boolean, default: true): Download media files -- `skip_existing` (boolean, default: true): Skip series with existing NFOs -- `max_concurrent` (integer, 1-10, default: 3): Number of concurrent operations - -**Response (200 OK):** - -```json -{ - "total": 3, - "successful": 2, - "failed": 0, - "skipped": 1, - "results": [ - { - "serie_id": "one-piece", - "serie_folder": "One Piece (1999)", - "success": true, - "message": "NFO created successfully", - "nfo_path": "/path/to/anime/One Piece (1999)/tvshow.nfo" - }, - { - "serie_id": "naruto", - "serie_folder": "Naruto (2002)", - "success": false, - "message": "Skipped - NFO already exists", - "nfo_path": null - }, - { - "serie_id": "bleach", - "serie_folder": "Bleach (2004)", - "success": true, - "message": "NFO created successfully", - "nfo_path": "/path/to/anime/Bleach (2004)/tvshow.nfo" - } - ] -} -``` - -**Errors:** - -- `401 Unauthorized` - Not authenticated -- `503 Service Unavailable` - TMDB API key not configured - -Source: [src/server/api/nfo.py](../src/server/api/nfo.py#L522-L634) - -### GET /api/nfo/missing - -Get list of series without NFO files. - -**Authentication:** Required - -**Response (200 OK):** - -```json -{ - "total_series": 150, - "missing_nfo_count": 23, - "series": [ - { - "serie_id": "dragon-ball", - "serie_folder": "Dragon Ball (1986)", - "serie_name": "Dragon Ball", - "has_media": false, - "media_files": { - "has_poster": false, - "has_logo": false, - "has_fanart": false, - "poster_path": null, - "logo_path": null, - "fanart_path": null - } - } - ] -} -``` - -**Errors:** - -- `401 Unauthorized` - Not authenticated -- `503 Service Unavailable` - TMDB API key not configured - -Source: [src/server/api/nfo.py](../src/server/api/nfo.py#L637-L684) - ---- - -## 7. Scheduler Endpoints - -Prefix: `/api/scheduler` - -All GET/POST config responses share the same envelope: - -```json -{ - "success": true, - "config": { ... }, - "status": { ... } -} -``` - -Source: [src/server/api/scheduler.py](../src/server/api/scheduler.py) - -### GET /api/scheduler/config - -Get current scheduler configuration and runtime status. - -**Authentication:** Required - -**Response (200 OK):** - -```json -{ - "success": true, - "config": { - "enabled": true, - "interval_minutes": 60, - "schedule_time": "03:00", - "schedule_days": ["mon", "tue", "wed", "thu", "fri", "sat", "sun"], - "auto_download_after_rescan": false - }, - "status": { - "is_running": true, - "next_run": "2025-07-15T03:00:00+00:00", - "last_run": null, - "scan_in_progress": false - } -} -``` - -### POST /api/scheduler/config - -Update scheduler configuration and apply changes immediately. - -**Authentication:** Required - -**Request Body (all fields optional, uses model defaults):** - -```json -{ - "enabled": true, - "schedule_time": "06:30", - "schedule_days": ["mon", "wed", "fri"], - "auto_download_after_rescan": true -} -``` - -**Response (200 OK):** Same envelope as GET, reflecting saved values. - -**Validation errors (422):** - -- `schedule_time` must match `HH:MM` (00:00–23:59) -- `schedule_days` entries must be one of `mon tue wed thu fri sat sun` -- `interval_minutes` must be ≥ 1 - -### POST /api/scheduler/trigger-rescan - -Manually trigger a library rescan (and auto-download if configured). - -**Authentication:** Required - -**Response (200 OK):** - -```json -{ - "message": "Rescan started successfully" -} -``` - -**Error responses:** - -- `503` — SeriesApp not yet initialised -- `500` — Rescan failed unexpectedly - ---- - -## 8. Health Check Endpoints - -Prefix: `/health` - -Source: [src/server/api/health.py](../src/server/api/health.py#L1-L267) - -### GET /health - -Basic health check endpoint. - -**Authentication:** Not required - -**Response (200 OK):** - -```json -{ - "status": "healthy", - "timestamp": "2025-12-13T10:30:00.000Z", - "version": "1.0.1" -} -``` - -Source: [src/server/api/health.py](../src/server/api/health.py#L151-L161) - -### GET /health/detailed - -Comprehensive health check with database, filesystem, and system metrics. - -**Authentication:** Not required - -**Response (200 OK):** - -```json -{ - "status": "healthy", - "timestamp": "2025-12-13T10:30:00.000Z", - "version": "1.0.1", - "dependencies": { - "database": { - "status": "healthy", - "connection_time_ms": 1.5, - "message": "Database connection successful" - }, - "filesystem": { - "status": "healthy", - "data_dir_writable": true, - "logs_dir_writable": true - }, - "system": { - "cpu_percent": 25.0, - "memory_percent": 45.0, - "memory_available_mb": 8192.0, - "disk_percent": 60.0, - "disk_free_mb": 102400.0, - "uptime_seconds": 86400.0 - } - }, - "startup_time": "2025-12-13T08:00:00.000Z" -} -``` - -Source: [src/server/api/health.py](../src/server/api/health.py#L164-L200) - ---- - -## 9. WebSocket Protocol - -Endpoint: `/ws/connect` - -Source: [src/server/api/websocket.py](../src/server/api/websocket.py#L1-L260) - -### Connection - -**URL:** `ws://127.0.0.1:8000/ws/connect` - -**Query Parameters:** -| Parameter | Required | Description | -|-----------|----------|-------------| -| `token` | No | JWT token for authenticated access | - -### Message Types - -| Type | Direction | Description | -| ------------------- | ---------------- | -------------------------- | -| `connected` | Server -> Client | Connection confirmation | -| `ping` | Client -> Server | Keepalive request | -| `pong` | Server -> Client | Keepalive response | -| `download_progress` | Server -> Client | Download progress update | -| `download_complete` | Server -> Client | Download completed | -| `download_failed` | Server -> Client | Download failed | -| `download_added` | Server -> Client | Item added to queue | -| `download_removed` | Server -> Client | Item removed from queue | -| `queue_status` | Server -> Client | Queue status update | -| `queue_started` | Server -> Client | Queue processing started | -| `queue_stopped` | Server -> Client | Queue processing stopped | -| `scan_progress` | Server -> Client | Library scan progress | -| `scan_complete` | Server -> Client | Library scan completed | -| `system_info` | Server -> Client | System information message | -| `error` | Server -> Client | Error message | - -Source: [src/server/models/websocket.py](../src/server/models/websocket.py#L25-L57) - -### Room Subscriptions - -Clients can join/leave rooms to receive specific updates. - -**Join Room:** - -```json -{ - "action": "join", - "data": { "room": "downloads" } -} -``` - -**Leave Room:** - -```json -{ - "action": "leave", - "data": { "room": "downloads" } -} -``` - -**Available Rooms:** - -- `downloads` - Download progress and status updates - -### Server Message Format - -```json -{ - "type": "download_progress", - "timestamp": "2025-12-13T10:30:00.000Z", - "data": { - "download_id": "uuid-here", - "key": "attack-on-titan", - "folder": "Attack on Titan (2013)", - "percent": 45.2, - "speed_mbps": 2.5, - "eta_seconds": 180 - } -} -``` - -### WebSocket Status Endpoint - -**GET /ws/status** - -Returns WebSocket service status. - -**Response (200 OK):** - -```json -{ - "status": "operational", - "active_connections": 5, - "supported_message_types": [ - "download_progress", - "download_complete", - "download_failed", - "queue_status", - "connected", - "ping", - "pong", - "error" - ] -} -``` - -Source: [src/server/api/websocket.py](../src/server/api/websocket.py#L238-L260) - ---- - -## 10. Data Models - -### Download Item - -```json -{ - "id": "uuid-string", - "serie_id": "attack-on-titan", - "serie_folder": "Attack on Titan (2013)", - "serie_name": "Attack on Titan", - "episode": { - "season": 1, - "episode": 1, - "title": "To You, in 2000 Years" - }, - "status": "pending", - "priority": "NORMAL", - "added_at": "2025-12-13T10:00:00Z", - "started_at": null, - "completed_at": null, - "progress": null, - "error": null, - "retry_count": 0, - "source_url": null -} -``` - -**Status Values:** `pending`, `downloading`, `paused`, `completed`, `failed`, `cancelled` - -**Priority Values:** `LOW`, `NORMAL`, `HIGH` - -Source: [src/server/models/download.py](../src/server/models/download.py#L63-L118) - -### Episode Identifier - -```json -{ - "season": 1, - "episode": 1, - "title": "Episode Title" -} -``` - -Source: [src/server/models/download.py](../src/server/models/download.py#L36-L41) - -### Download Progress - -```json -{ - "percent": 45.2, - "downloaded_mb": 256.0, - "total_mb": 512.0, - "speed_mbps": 2.5, - "eta_seconds": 180 -} -``` - -Source: [src/server/models/download.py](../src/server/models/download.py#L44-L60) - ---- - -## 11. Error Handling - -### HTTP Status Codes - -| Code | Meaning | When Used | -| ---- | --------------------- | --------------------------------- | -| 200 | OK | Successful request | -| 201 | Created | Resource created | -| 204 | No Content | Successful deletion | -| 400 | Bad Request | Invalid request body/parameters | -| 401 | Unauthorized | Missing or invalid authentication | -| 403 | Forbidden | Insufficient permissions | -| 404 | Not Found | Resource does not exist | -| 422 | Unprocessable Entity | Validation error | -| 429 | Too Many Requests | Rate limit exceeded | -| 500 | Internal Server Error | Server-side error | - -### Error Response Format - -```json -{ - "success": false, - "error": "VALIDATION_ERROR", - "message": "Human-readable error message", - "details": { - "field": "Additional context" - }, - "request_id": "uuid-for-tracking" -} -``` - -Source: [src/server/middleware/error_handler.py](../src/server/middleware/error_handler.py#L26-L56) - -### Common Error Codes - -| Error Code | HTTP Status | Description | -| ---------------------- | ----------- | ------------------------------ | -| `AUTHENTICATION_ERROR` | 401 | Invalid or missing credentials | -| `AUTHORIZATION_ERROR` | 403 | Insufficient permissions | -| `VALIDATION_ERROR` | 422 | Request validation failed | -| `NOT_FOUND_ERROR` | 404 | Resource not found | -| `CONFLICT_ERROR` | 409 | Resource conflict | -| `RATE_LIMIT_ERROR` | 429 | Rate limit exceeded | - ---- - -## 12. Rate Limiting - -### Authentication Endpoints - -| Endpoint | Limit | Window | -| ---------------------- | ---------- | ---------- | -| `POST /api/auth/login` | 5 requests | 60 seconds | -| `POST /api/auth/setup` | 5 requests | 60 seconds | - -Source: [src/server/middleware/auth.py](../src/server/middleware/auth.py#L143-L162) - -### Origin-Based Limiting - -All endpoints from the same origin are limited to 60 requests per minute per origin. - -Source: [src/server/middleware/auth.py](../src/server/middleware/auth.py#L115-L133) - -### Rate Limit Response - -```json -{ - "detail": "Too many authentication attempts, try again later" -} -``` - -HTTP Status: 429 Too Many Requests - ---- - -## 13. Pagination - -The anime list endpoint supports pagination. - -**Query Parameters:** -| Parameter | Default | Max | Description | -|-----------|---------|-----|-------------| -| `page` | 1 | - | Page number (1-indexed) | -| `per_page` | 20 | 1000 | Items per page | - -**Example:** - -``` -GET /api/anime?page=2&per_page=50 -``` - -Source: [src/server/api/anime.py](../src/server/api/anime.py#L180-L220) diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md deleted file mode 100644 index 4891ea5..0000000 --- a/docs/ARCHITECTURE.md +++ /dev/null @@ -1,817 +0,0 @@ -# Architecture Documentation - -## Document Purpose - -This document describes the system architecture of the Aniworld anime download manager. - ---- - -## 1. System Overview - -Aniworld is a web-based anime download manager built with Python, FastAPI, and SQLite. It provides a REST API and WebSocket interface for managing anime libraries, downloading episodes, and tracking progress. - -### High-Level Architecture - -``` -+------------------+ +------------------+ +------------------+ -| Web Browser | | CLI Client | | External | -| (Frontend) | | (Main.py) | | Providers | -+--------+---------+ +--------+---------+ +--------+---------+ - | | | - | HTTP/WebSocket | Direct | HTTP - | | | -+--------v---------+ +--------v---------+ +--------v---------+ -| | | | | | -| FastAPI <-----> Core Layer <-----> Provider | -| Server Layer | | (SeriesApp) | | Adapters | -| | | | | | -+--------+---------+ +--------+---------+ +------------------+ - | | - | | -+--------v---------+ +--------v---------+ -| | | | -| SQLite DB | | File System | -| (aniworld.db) | | (anime/*/) | -| - Series data | | - Video files | -| - Episodes | | - NFO files | -| - Queue state | | - Media files | -+------------------+ +------------------+ -``` - -Source: [src/server/fastapi_app.py](../src/server/fastapi_app.py#L1-L252) - ---- - -## 2. Architectural Layers - -### 2.1 CLI Layer (`src/cli/`) - -Legacy command-line interface for direct interaction with the core layer. - -| Component | File | Purpose | -| --------- | ----------------------------- | --------------- | -| Main | [Main.py](../src/cli/Main.py) | CLI entry point | - -### 2.2 Server Layer (`src/server/`) - -FastAPI-based REST API and WebSocket server. - -``` -src/server/ -+-- fastapi_app.py # Application entry point, lifespan management -+-- api/ # API route handlers -| +-- anime.py # /api/anime/* endpoints -| +-- auth.py # /api/auth/* endpoints -| +-- config.py # /api/config/* endpoints -| +-- download.py # /api/queue/* endpoints -| +-- scheduler.py # /api/scheduler/* endpoints -| +-- nfo.py # /api/nfo/* endpoints -| +-- websocket.py # /ws/* WebSocket handlers -| +-- health.py # /health/* endpoints -+-- controllers/ # Page controllers for HTML rendering -| +-- page_controller.py # UI page routes -| +-- health_controller.py# Health check route -| +-- error_controller.py # Error pages (404, 500) -+-- services/ # Business logic -| +-- anime_service.py # Anime operations -| +-- auth_service.py # Authentication -| +-- config_service.py # Configuration management -| +-- download_service.py # Download queue management -| +-- progress_service.py # Progress tracking -| +-- websocket_service.py# WebSocket broadcasting -| +-- queue_repository.py # Database persistence -| +-- nfo_service.py # NFO metadata management -| +-- setup_service.py # Series key resolution from folder names -| +-- folder_scan_service.py # Daily folder maintenance scan -+-- models/ # Pydantic models -| +-- auth.py # Auth request/response models -| +-- config.py # Configuration models -| +-- download.py # Download queue models -| +-- websocket.py # WebSocket message models -+-- middleware/ # Request processing -| +-- auth.py # JWT validation, rate limiting -| +-- error_handler.py # Exception handlers -| +-- setup_redirect.py # Setup flow redirect -+-- database/ # SQLAlchemy ORM -| +-- connection.py # Database connection -| +-- models.py # ORM models -| +-- service.py # Database service -+-- utils/ # Utility modules -| +-- filesystem.py # Folder sanitization, path safety -| +-- validators.py # Input validation utilities -| +-- dependencies.py # FastAPI dependency injection -+-- web/ # Static files and templates - +-- static/ # CSS, JS, images - +-- templates/ # Jinja2 templates -``` - -Source: [src/server/](../src/server/) - -### 2.2.1 Frontend Architecture (`src/server/web/static/`) - -The frontend uses a modular architecture with no build step required. CSS and JavaScript files are organized by responsibility. - -#### CSS Structure - -``` -src/server/web/static/css/ -+-- styles.css # Entry point with @import statements -+-- base/ -| +-- variables.css # CSS custom properties (colors, fonts, spacing) -| +-- reset.css # CSS reset and normalize styles -| +-- typography.css # Font styles, headings, text utilities -+-- components/ -| +-- buttons.css # All button styles -| +-- cards.css # Card and panel components -| +-- forms.css # Form inputs, labels, validation styles -| +-- modals.css # Modal and overlay styles -| +-- navigation.css # Header, nav, sidebar styles -| +-- progress.css # Progress bars, loading indicators -| +-- notifications.css # Toast, alerts, messages -| +-- tables.css # Table and list styles -| +-- status.css # Status badges and indicators -+-- pages/ -| +-- login.css # Login page specific styles -| +-- index.css # Index/library page specific styles -| +-- queue.css # Queue page specific styles -+-- utilities/ - +-- animations.css # Keyframes and animation classes - +-- responsive.css # Media queries and breakpoints - +-- helpers.css # Utility classes (hidden, flex, spacing) -``` - -#### JavaScript Structure - -JavaScript uses the IIFE pattern with a shared `AniWorld` namespace for browser compatibility without build tools. - -``` -src/server/web/static/js/ -+-- shared/ # Shared utilities used by all pages -| +-- constants.js # API endpoints, localStorage keys, defaults -| +-- auth.js # Token management (getToken, setToken, checkAuth) -| +-- api-client.js # Fetch wrapper with auto-auth headers -| +-- theme.js # Dark/light theme toggle -| +-- ui-utils.js # Toast notifications, format helpers -| +-- websocket-client.js # Socket.IO wrapper -+-- index/ # Index page modules -| +-- series-manager.js # Series list rendering and filtering -| +-- selection-manager.js# Multi-select and bulk download -| +-- search.js # Series search functionality -| +-- scan-manager.js # Library rescan operations -| +-- scheduler-config.js # Scheduler configuration -| +-- logging-config.js # Logging configuration -| +-- advanced-config.js # Advanced settings -| +-- main-config.js # Main configuration and backup -| +-- config-manager.js # Config modal orchestrator -| +-- socket-handler.js # WebSocket event handlers -| +-- app-init.js # Application initialization -+-- queue/ # Queue page modules - +-- queue-api.js # Queue API interactions - +-- queue-renderer.js # Queue list rendering - +-- progress-handler.js # Download progress updates - +-- queue-socket-handler.js # WebSocket events for queue - +-- queue-init.js # Queue page initialization -``` - -#### Module Pattern - -All JavaScript modules follow the IIFE pattern with namespace: - -```javascript -var AniWorld = window.AniWorld || {}; - -AniWorld.ModuleName = (function () { - "use strict"; - - // Private variables and functions - - // Public API - return { - init: init, - publicMethod: publicMethod, - }; -})(); -``` - -Source: [src/server/web/static/](../src/server/web/static/) - -### 2.3 Core Layer (`src/core/`) - -Domain logic for anime series management. - -``` -src/core/ -+-- SeriesApp.py # Main application facade -+-- SerieScanner.py # Directory scanning, targeted single-series scan -+-- entities/ # Domain entities -| +-- series.py # Serie class with sanitized_folder property -| +-- SerieList.py # SerieList collection with sanitized folder support -| +-- nfo_models.py # Pydantic models for tvshow.nfo (TVShowNFO, ActorInfo…) -+-- services/ # Domain services -| +-- nfo_service.py # NFO lifecycle: create / update tvshow.nfo -| +-- nfo_repair_service.py # Detect & repair incomplete tvshow.nfo files -| | # (parse_nfo_tags, find_missing_tags, NfoRepairService) -| +-- tmdb_client.py # Async TMDB API client -+-- utils/ # Utility helpers (no side-effects) -| +-- nfo_generator.py # TVShowNFO → XML serialiser -| +-- nfo_mapper.py # TMDB API dict → TVShowNFO (tmdb_to_nfo_model, -| | # _extract_rating_by_country, _extract_fsk_rating) -| +-- image_downloader.py # TMDB image downloader -+-- providers/ # External provider adapters -| +-- base_provider.py # Loader interface -| +-- provider_factory.py # Provider registry -+-- interfaces/ # Abstract interfaces -| +-- callbacks.py # Progress callback system -+-- exceptions/ # Domain exceptions - +-- Exceptions.py # Custom exceptions -``` - -**Key Components:** - -| Component | Purpose | -| -------------- | -------------------------------------------------------------------------- | -| `SeriesApp` | Main application facade for anime operations | -| `SerieScanner` | Scans directories for anime; `scan_single_series()` for targeted scans | -| `Serie` | Domain entity with `sanitized_folder` property for filesystem-safe names | -| `SerieList` | Collection management with automatic folder creation using sanitized names | - -**Initialization:** - -`SeriesApp` is initialized with `skip_load=True` passed to `SerieList`, preventing automatic loading of series from data files on every instantiation. Series data is loaded once during application setup via `sync_series_from_data_files()` in the FastAPI lifespan, which reads data files and syncs them to the database. Subsequent operations load series from the database through the service layer. - -Source: [src/core/](../src/core/) - -### 2.4 Infrastructure Layer (`src/infrastructure/`) - -Cross-cutting concerns. - -``` -src/infrastructure/ -+-- logging/ # Structured logging setup -+-- security/ # Security utilities -``` - -### 2.5 Configuration Layer (`src/config/`) - -Application settings management. - -| Component | File | Purpose | -| --------- | ---------------------------------------- | ------------------------------- | -| Settings | [settings.py](../src/config/settings.py) | Environment-based configuration | - -Source: [src/config/settings.py](../src/config/settings.py#L1-L96) - ---- - -## 12. Startup Sequence - -The FastAPI lifespan function (`src/server/fastapi_app.py`) runs the following steps on every server start. - -### 12.1 Startup Order - -``` -1. Logging configured - -2. Temp folder purged ← cleans leftover partial download files - +-- Iterate ./Temp/ and delete every file and sub-directory - +-- Create ./Temp/ if it does not exist - +-- Errors are logged as warnings; startup continues regardless - -3. Database initialised (required – abort on failure) - +-- SQLite file created / migrated via init_db() - -4. Configuration loaded from data/config.json - +-- Synced to settings (ENV vars take precedence) - -5. Progress & WebSocket services wired up - -6. Series loaded from database into memory - -7. Download service initialised (queue restored from DB) - -8. Background loader service started - -9. Scheduler service started - +-- Cron-based library rescans configured - +-- Optional: auto-download missing episodes after rescan - +-- Optional: folder maintenance (NFO repair, key resolution, renaming, poster checks) during scheduled runs -``` - -### 12.2 Temp Folder Guarantee - -Every server start begins with a clean `./Temp/` directory. This ensures that partial `.part` files or stale temp videos from a crashed or force-killed previous session are never left behind before new downloads start. - -Source: [src/server/fastapi_app.py](../src/server/fastapi_app.py) - ---- - -## 11. Graceful Shutdown - -The application implements a comprehensive graceful shutdown mechanism that ensures data integrity and proper cleanup when the server is stopped via Ctrl+C (SIGINT) or SIGTERM. - -### 11.1 Shutdown Sequence - -``` -1. SIGINT/SIGTERM received - +-- Uvicorn catches signal - +-- Stops accepting new requests - -2. FastAPI lifespan shutdown triggered - +-- 30 second total timeout - -3. WebSocket shutdown (5s timeout) - +-- Broadcast {"type": "server_shutdown"} to all clients - +-- Close each connection with code 1001 (Going Away) - +-- Clear connection tracking data - -4. Download service stop (10s timeout) - +-- Set shutdown flag - +-- Persist active download as "pending" in database - +-- Cancel active download task - +-- Shutdown ThreadPoolExecutor with wait - -5. Progress service cleanup - +-- Clear event subscribers - +-- Clear active progress tracking - -6. Database cleanup (10s timeout) - +-- SQLite: Run PRAGMA wal_checkpoint(TRUNCATE) - +-- Dispose async engine - +-- Dispose sync engine - -7. Process exits cleanly -``` - -Source: [src/server/fastapi_app.py](../src/server/fastapi_app.py#L142-L210) - -### 11.2 Key Components - -| Component | File | Shutdown Method | -| ------------------- | ------------------------------------------------------------------- | ------------------------------ | -| WebSocket Service | [websocket_service.py](../src/server/services/websocket_service.py) | `shutdown(timeout=5.0)` | -| Download Service | [download_service.py](../src/server/services/download_service.py) | `stop(timeout=10.0)` | -| Database Connection | [connection.py](../src/server/database/connection.py) | `close_db()` | -| Uvicorn Config | [run_server.py](../run_server.py) | `timeout_graceful_shutdown=30` | -| Stop Script | [stop_server.sh](../stop_server.sh) | SIGTERM with fallback | - -### 11.3 Data Integrity Guarantees - -1. **Active downloads preserved**: In-progress downloads are saved as "pending" and can resume on restart. - -2. **Database WAL flushed**: SQLite WAL checkpoint ensures all writes are in the main database file. - -3. **WebSocket clients notified**: Clients receive shutdown message before connection closes. - -4. **Thread pool cleanup**: Background threads complete or are gracefully cancelled. - -### 11.4 Manual Stop - -```bash -# Graceful stop via script (sends SIGTERM, waits up to 30s) -./stop_server.sh - -# Or press Ctrl+C in terminal running the server -``` - -Source: [stop_server.sh](../stop_server.sh#L1-L80) - ---- - -## 3. Component Interactions - -### 3.1 Request Flow (REST API) - -``` -1. Client sends HTTP request -2. AuthMiddleware validates JWT token (if required) -3. Rate limiter checks request frequency -4. FastAPI router dispatches to endpoint handler -5. Endpoint calls service layer -6. Service layer uses core layer or database -7. Response returned as JSON -``` - -Source: [src/server/middleware/auth.py](../src/server/middleware/auth.py#L1-L209) - -### 3.2 Download Flow - -``` -1. POST /api/queue/add - +-- DownloadService.add_to_queue() - +-- QueueRepository.save_item() -> SQLite - -2. POST /api/queue/start - +-- DownloadService.start_queue_processing() - +-- Process pending items sequentially - +-- ProgressService emits events - +-- WebSocketService broadcasts to clients - -3. During download: - +-- Provider writes to ./Temp/ (+ ./Temp/.part fragments) - +-- ProgressService.emit("progress_updated") - +-- WebSocketService.broadcast_to_room() - +-- Client receives WebSocket message - -4. After download attempt (success OR failure): - +-- _cleanup_temp_file() removes ./Temp/ and all .part fragments - +-- On success: file was already moved to final destination before cleanup - +-- On failure / exception: no partial files remain in ./Temp/ -``` - -#### Temp Directory Contract - -| Situation | Outcome | -| -------------------------------- | ------------------------------------------------------------------- | -| Server start | Entire `./Temp/` directory is purged before any service initialises | -| Successful download | Temp file moved to destination, then removed from `./Temp/` | -| Failed download (provider error) | Temp + `.part` fragments removed by `_cleanup_temp_file()` | -| Exception / cancellation | Temp + `.part` fragments removed in `except` block | - -Source: [src/server/services/download_service.py](../src/server/services/download_service.py#L1-L150), -[src/core/providers/aniworld_provider.py](../src/core/providers/aniworld_provider.py), -[src/core/providers/enhanced_provider.py](../src/core/providers/enhanced_provider.py) - -### 3.3 WebSocket Event Flow - -``` -1. Client connects to /ws/connect -2. Server sends "connected" message -3. Client joins room: {"action": "join", "data": {"room": "downloads"}} -4. ProgressService emits events -5. WebSocketService broadcasts to room subscribers -6. Client receives real-time updates -``` - -Source: [src/server/api/websocket.py](../src/server/api/websocket.py#L1-L260) - ---- - -## 4. Design Patterns - -### 4.1 Repository Pattern (Service Layer as Repository) - -**Architecture Decision**: The Service Layer serves as the Repository layer for database access. - -Database access is abstracted through service classes in `src/server/database/service.py` that provide CRUD operations and act as the repository layer. This eliminates the need for a separate repository layer while maintaining clean separation of concerns. - -**Service Layer Classes** (acting as repositories): - -- `AnimeSeriesService` - CRUD operations for anime series -- `EpisodeService` - CRUD operations for episodes -- `DownloadQueueService` - CRUD operations for download queue -- `UserSessionService` - CRUD operations for user sessions -- `SystemSettingsService` - CRUD operations for system settings - -**Key Principles**: - -1. **No Direct Database Queries**: Controllers and business logic services MUST use service layer methods -2. **Service Layer Encapsulation**: All SQLAlchemy queries are encapsulated in service methods -3. **Consistent Interface**: Services provide consistent async methods for all database operations -4. **Single Responsibility**: Each service manages one entity type - -**Example Usage**: - -```python -# CORRECT: Use service layer -from src.server.database.service import AnimeSeriesService - -async with get_db_session() as db: - series = await AnimeSeriesService.get_by_key(db, "attack-on-titan") - await AnimeSeriesService.update(db, series.id, has_nfo=True) - -# INCORRECT: Direct database query -result = await db.execute(select(AnimeSeries).filter(...)) # ❌ Never do this -``` - -**Special Case - Queue Repository Adapter**: - -The `QueueRepository` in `src/server/services/queue_repository.py` is an adapter that wraps `DownloadQueueService` to provide domain model conversion between Pydantic models and SQLAlchemy models: - -```python -# QueueRepository provides CRUD with model conversion -class QueueRepository: - async def save_item(self, item: DownloadItem) -> None: ... # Converts Pydantic → SQLAlchemy - async def get_all_items(self) -> List[DownloadItem]: ... # Converts SQLAlchemy → Pydantic - async def delete_item(self, item_id: str) -> bool: ... -``` - -Source: [src/server/database/service.py](../src/server/database/service.py), [src/server/services/queue_repository.py](../src/server/services/queue_repository.py) - -### 4.2 Dependency Injection - -FastAPI's `Depends()` provides constructor injection. - -```python -@router.get("/status") -async def get_status( - download_service: DownloadService = Depends(get_download_service), -): - ... -``` - -Source: [src/server/utils/dependencies.py](../src/server/utils/dependencies.py) - -### 4.3 Event-Driven Architecture - -Progress updates use an event subscription model. - -```python -# ProgressService publishes events -progress_service.emit("progress_updated", event) - -# WebSocketService subscribes -progress_service.subscribe("progress_updated", ws_handler) -``` - -Source: [src/server/fastapi_app.py](../src/server/fastapi_app.py#L98-L108) - -### 4.4 Singleton Pattern - -Services use module-level singletons for shared state. - -```python -# In download_service.py -_download_service_instance: Optional[DownloadService] = None - -def get_download_service() -> DownloadService: - global _download_service_instance - if _download_service_instance is None: - _download_service_instance = DownloadService(...) - return _download_service_instance -``` - -### 4.5 Error Handling Pattern - -**Architecture Decision**: Dual error handling approach based on exception source. - -The application uses two complementary error handling mechanisms: - -1. **FastAPI HTTPException** - For simple validation and HTTP-level errors -2. **Custom Exception Hierarchy** - For business logic and service-level errors with rich context - -#### Exception Hierarchy - -```python -# Base exception with HTTP status mapping -AniWorldAPIException(message, status_code, error_code, details) -├── AuthenticationError (401) -├── AuthorizationError (403) -├── ValidationError (422) -├── NotFoundError (404) -├── ConflictError (409) -├── BadRequestError (400) -├── RateLimitError (429) -└── ServerError (500) - ├── DownloadError - ├── ConfigurationError - ├── ProviderError - └── DatabaseError -``` - -#### When to Use Each - -**Use HTTPException for:** - -- Simple parameter validation (missing fields, wrong type) -- Direct HTTP-level errors (401, 403, 404 without business context) -- Quick endpoint-specific failures - -**Use Custom Exceptions for:** - -- Service-layer business logic errors (AnimeServiceError, ConfigServiceError) -- Errors needing rich context (details dict, error codes) -- Errors that should be logged with specific categorization -- Cross-cutting concerns (authentication, authorization, rate limiting) - -**Example:** - -```python -# Simple validation - Use HTTPException -if not series_key: - raise HTTPException(status_code=400, detail="series_key required") - -# Business logic error - Use custom exception -try: - await anime_service.add_series(series_key) -except AnimeServiceError as e: - raise ServerError( - message=f"Failed to add series: {e}", - error_code="ANIME_ADD_FAILED", - details={"series_key": series_key} - ) -``` - -#### Global Exception Handlers - -All custom exceptions are automatically handled by global middleware that: - -- Converts exceptions to structured JSON responses -- Logs errors with appropriate severity -- Includes request ID for tracking -- Provides consistent error format - -**Source**: [src/server/exceptions/\_\_init\_\_.py](../src/server/exceptions/__init__.py), [src/server/middleware/error_handler.py](../src/server/middleware/error_handler.py) - -Source: [src/server/services/download_service.py](../src/server/services/download_service.py) - ---- - -## 5. Data Flow - -### 5.1 Series Identifier Convention - -The system uses two identifier fields: - -| Field | Type | Purpose | Example | -| -------- | -------- | -------------------------------------- | -------------------------- | -| `key` | Primary | Provider-assigned, URL-safe identifier | `"attack-on-titan"` | -| `folder` | Metadata | Filesystem folder name (display only) | `"Attack on Titan (2013)"` | - -All API operations use `key`. The `folder` is for filesystem operations only. - -Source: [src/server/database/models.py](../src/server/database/models.py#L26-L50) - -### 5.2 Database Schema - -``` -+----------------+ +----------------+ +--------------------+ -| anime_series | | episodes | | download_queue_item| -+----------------+ +----------------+ +--------------------+ -| id (PK) |<--+ | id (PK) | +-->| id (PK) | -| key (unique) | | | series_id (FK) |---+ | series_id (FK) | -| name | +---| season | | status | -| site | | episode_number | | priority | -| folder | | title | | progress_percent | -| created_at | | is_downloaded | | added_at | -| updated_at | | file_path | | started_at | -+----------------+ +----------------+ +--------------------+ -``` - -Source: [src/server/database/models.py](../src/server/database/models.py#L1-L200) - -### 5.3 Configuration Storage - -Configuration is stored in `data/config.json`: - -```json -{ - "name": "Aniworld", - "data_dir": "data", - "scheduler": { - "enabled": true, - "schedule_time": "03:00", - "schedule_days": ["mon", "tue", "wed", "thu", "fri", "sat", "sun"], - "auto_download_after_rescan": false - }, - "logging": { "level": "INFO" }, - "backup": { "enabled": false, "path": "data/backups" }, - "other": { - "master_password_hash": "$pbkdf2-sha256$...", - "anime_directory": "/path/to/anime" - } -} -``` - -Source: [data/config.json](../data/config.json) - ---- - -## 6. Technology Stack - -| Layer | Technology | Version | Purpose | -| ------------- | ------------------- | ------- | ---------------------- | -| Web Framework | FastAPI | 0.104.1 | REST API, WebSocket | -| ASGI Server | Uvicorn | 0.24.0 | HTTP server | -| Database | SQLite + SQLAlchemy | 2.0.35 | Persistence | -| Auth | python-jose | 3.3.0 | JWT tokens | -| Password | passlib | 1.7.4 | bcrypt hashing | -| Validation | Pydantic | 2.5.0 | Data models | -| Templates | Jinja2 | 3.1.2 | HTML rendering | -| Logging | structlog | 24.1.0 | Structured logging | -| Testing | pytest | 7.4.3 | Unit/integration tests | - -Source: [requirements.txt](../requirements.txt) - ---- - -## 7. Scalability Considerations - -### Current Limitations - -1. **Single-process deployment**: In-memory rate limiting and session state are not shared across processes. - -2. **SQLite database**: Not suitable for high concurrency. Consider PostgreSQL for production. - -3. **Sequential downloads**: Only one download processes at a time by design. - -### Recommended Improvements for Scale - -| Concern | Current | Recommended | -| -------------- | --------------- | ----------------- | -| Rate limiting | In-memory dict | Redis | -| Session store | In-memory | Redis or database | -| Database | SQLite | PostgreSQL | -| Task queue | In-memory deque | Celery + Redis | -| Load balancing | None | Nginx/HAProxy | - ---- - -## 8. Integration Points - -### 8.1 External Providers - -The system integrates with anime streaming providers via the Loader interface. - -```python -class Loader(ABC): - @abstractmethod - def search(self, query: str) -> List[Serie]: ... - - @abstractmethod - def get_episodes(self, serie: Serie) -> Dict[int, List[int]]: ... -``` - -Source: [src/core/providers/base_provider.py](../src/core/providers/base_provider.py) - -### 8.2 Filesystem Integration - -The scanner reads anime directories to detect downloaded episodes. - -```python -SerieScanner( - basePath="/path/to/anime", # Anime library directory - loader=provider, # Provider for metadata - db_session=session # Optional database -) -``` - -Source: [src/core/SerieScanner.py](../src/core/SerieScanner.py#L59-L96) - ---- - -## 9. Security Architecture - -### 9.1 Authentication Flow - -``` -1. User sets master password via POST /api/auth/setup -2. Password hashed with pbkdf2_sha256 (via passlib) -3. Hash stored in config.json -4. Login validates password, returns JWT token -5. JWT contains: session_id, user, created_at, expires_at -6. Subsequent requests include: Authorization: Bearer -``` - -Source: [src/server/services/auth_service.py](../src/server/services/auth_service.py#L1-L150) - -### 9.2 Password Requirements - -- Minimum 8 characters -- Mixed case (upper and lower) -- At least one number -- At least one special character - -Source: [src/server/services/auth_service.py](../src/server/services/auth_service.py#L97-L125) - -### 9.3 Rate Limiting - -| Endpoint | Limit | Window | -| ----------------- | ----------- | ---------- | -| `/api/auth/login` | 5 requests | 60 seconds | -| `/api/auth/setup` | 5 requests | 60 seconds | -| All origins | 60 requests | 60 seconds | - -Source: [src/server/middleware/auth.py](../src/server/middleware/auth.py#L54-L68) - ---- - -## 10. Deployment Modes - -### 10.1 Development - -```bash -# Run with hot reload -python -m uvicorn src.server.fastapi_app:app --reload -``` - -### 10.2 Production - -```bash -# Via conda environment -conda run -n AniWorld python -m uvicorn src.server.fastapi_app:app \ - --host 127.0.0.1 --port 8000 -``` - -### 10.3 Configuration - -Environment variables (via `.env` or shell): - -| Variable | Default | Description | -| ----------------- | ------------------------------ | ---------------------- | -| `JWT_SECRET_KEY` | Random | Secret for JWT signing | -| `DATABASE_URL` | `sqlite:///./data/aniworld.db` | Database connection | -| `ANIME_DIRECTORY` | (empty) | Path to anime library | -| `LOG_LEVEL` | `INFO` | Logging level | -| `CORS_ORIGINS` | `localhost:3000,8000` | Allowed CORS origins | - -Source: [src/config/settings.py](../src/config/settings.py#L1-L96) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md deleted file mode 100644 index 5b77386..0000000 --- a/docs/CHANGELOG.md +++ /dev/null @@ -1,243 +0,0 @@ -# Changelog - -## Document Purpose - -This document tracks all notable changes to the Aniworld project. - -### What This Document Contains - -- **Version History**: All released versions with dates -- **Added Features**: New functionality in each release -- **Changed Features**: Modifications to existing features -- **Deprecated Features**: Features marked for removal -- **Removed Features**: Features removed from the codebase -- **Fixed Bugs**: Bug fixes with issue references -- **Security Fixes**: Security-related changes -- **Breaking Changes**: Changes requiring user action - -### What This Document Does NOT Contain - -- Internal refactoring details (unless user-facing) -- Commit-level changes -- Work-in-progress features -- Roadmap or planned features - -### Target Audience - -- All users and stakeholders -- Operators planning upgrades -- Developers tracking changes -- Support personnel - ---- - -## Format - -This changelog follows [Keep a Changelog](https://keepachangelog.com/) principles and adheres to [Semantic Versioning](https://semver.org/). - ---- - -## [Unreleased] - 2026-06-05 - -### Fixed - -- **Folder scan series key resolution**: Fixed "Could not resolve series key for folder, skipping" warnings during library setup. `_resolve_key_via_search()` now uses fuzzy title matching instead of exact string comparison. - - Added `_normalize_title()` to strip anime suffixes: `(TV)`, `(Anime)`, `(OAD)`, `(OVA)`, `(Special)`, `(Movie)`, `(Spin-Off)` - - Added `_titles_match()` using `difflib.SequenceMatcher` with 0.85 similarity threshold for tolerance of minor title variations - - Added debug logging for title mismatches and multiple search results - ---- - -## [1.3.1] - 2026-02-22 - -### Added - -- **Encoding detection for HTML parsing** (`src/core/providers/aniworld_provider.py`): - Added `_decode_html_content()` function that uses `chardet` to detect the actual - encoding of HTML content before parsing. Falls back to UTF-8 with `errors='replace'` - to handle pages with mismatched encoding declarations. Applied to all BeautifulSoup - parsing calls to prevent "Some characters could not be decoded" warnings. -- **chardet dependency**: Added `chardet>=5.2.0` to `requirements.txt` for encoding detection. - -### Added - -- **Temp file cleanup after every download** (`src/core/providers/aniworld_provider.py`, - `src/core/providers/enhanced_provider.py`): Module-level helper - `_cleanup_temp_file()` removes the working temp file and any yt-dlp `.part` - fragments after each download attempt — on success, on failure, and on - exceptions (including `BrokenPipeError` and cancellation). Ensures that no - partial files accumulate in `./Temp/` across multiple runs. -- **Temp folder purge on server start** (`src/server/fastapi_app.py`): The - FastAPI lifespan startup now iterates `./Temp/` and deletes every file and - sub-directory before the rest of the initialisation sequence runs. If the - folder does not exist it is created. Errors are caught and logged as warnings - so that they never abort startup. - ---- - -## [1.3.0] - 2026-02-22 - -### Added - -- **NFO tag completeness (`nfo_mapper.py`)**: All 17 required NFO tags are now - explicitly populated during creation: `originaltitle`, `sorttitle`, `year`, - `plot`, `outline`, `tagline`, `runtime`, `premiered`, `status`, `imdbid`, - `genre`, `studio`, `country`, `actor`, `watched`, `dateadded`, `mpaa`. -- **`src/core/utils/nfo_mapper.py`**: New module containing - `tmdb_to_nfo_model()`, `_extract_rating_by_country()`, and - `_extract_fsk_rating()`. Extracted from `NFOService` to keep files under - 500 lines and isolate pure mapping logic. -- **US MPAA rating**: `_extract_rating_by_country(ratings, "US")` now maps the - US TMDB content rating to the `` NFO tag. -- **`NfoRepairService` (`src/core/services/nfo_repair_service.py`)**: New service - that detects incomplete `tvshow.nfo` files and triggers TMDB re-fetch. - Provides `parse_nfo_tags()`, `find_missing_tags()`, `nfo_needs_repair()`, and - `NfoRepairService.repair_series()`. 13 required tags are checked. -- **`perform_nfo_repair_scan()` - (`src/server/services/folder_scan_service.py`)**: New async function - that iterates every series directory, checks whether `tvshow.nfo` is missing - required tags using `nfo_needs_repair()`, and queues the series for background - reload via `asyncio.create_task`. Skips gracefully when `tmdb_api_key` or - `anime_directory` is not configured. -- **NFO repair wired into scheduled folder scan (`src/server/services/folder_scan_service.py`)**: - `perform_nfo_repair_scan(background_loader=None)` is called during the - scheduled daily folder scan, keeping startup fast while ensuring regular - maintenance. - -### Changed - -- `NFOService._tmdb_to_nfo_model()` and `NFOService._extract_fsk_rating()` moved - to `src/core/utils/nfo_mapper.py` as module-level functions - `tmdb_to_nfo_model()` and `_extract_fsk_rating()`. -- `src/core/services/nfo_service.py` reduced from 640 → 471 lines. - ---- - -## [Unreleased] - 2026-01-18 - -### Added - -- **Cron-based Scheduler**: Replaced the asyncio sleep-loop with APScheduler's `AsyncIOScheduler + CronTrigger` - - Schedule rescans at a specific **time of day** (`HH:MM`) on selected **days of the week** - - New `SchedulerConfig` fields: `schedule_time` (default `"03:00"`), `schedule_days` (default all 7), `auto_download_after_rescan` (default `false`) - - Old `interval_minutes` field retained for backward compatibility -- **Auto-download after rescan**: When `auto_download_after_rescan` is enabled, missing episodes are automatically queued for download after each scheduled rescan -- **Day-of-week UI**: New day-of-week pill toggles (Mon–Sun) in the Settings → Scheduler section -- **Live config reload**: POST `/api/scheduler/config` reschedules the APScheduler job without restarting the application -- **Enriched API response**: GET/POST `/api/scheduler/config` now returns `{"success", "config", "status"}` envelope including `next_run`, `last_run`, and `scan_in_progress` - -### Changed - -- Scheduler API response format: previously returned flat config; now returns `{"success": true, "config": {...}, "status": {...}}` -- `reload_config()` is now a synchronous method accepting a `SchedulerConfig` argument (previously async, no arguments) -- Dependencies: added `APScheduler>=3.10.4` to `requirements.txt` - -### Fixed - -- **Series Visibility**: Fixed issue where series added to the database weren't appearing in the API/UI - - Series are now loaded from database into SeriesApp's in-memory cache on startup - - Added `_load_series_from_db()` call after initial database sync in FastAPI lifespan -- **Episode Tracking**: Fixed missing episodes not being saved to database when adding new series - - Missing episodes are now persisted to the `episodes` table after the targeted scan - - Episodes are properly synced during rescan operations (added/removed based on filesystem state) -- **Database Synchronization**: Improved data consistency between database and in-memory cache - - Rescan process properly updates episodes: adds new missing episodes, removes downloaded ones - - All series operations now maintain database and cache synchronization - -### Technical Details - -- Modified `src/server/fastapi_app.py` to load series from database after sync -- Modified `src/server/api/anime.py` to save scanned episodes to database -- Episodes table properly tracks missing episodes with automatic cleanup - -### Deprecated - -- **Legacy Series Files (key/data)**: File-based series storage is deprecated. `key` and `data` files in anime folders will be removed in v3.0.0. Database storage is now the primary method. See [docs/MIGRATION_GUIDE.md](docs/MIGRATION_GUIDE.md) for details. - ---- - -## Sections for Each Release - -```markdown -## [Version] - YYYY-MM-DD - -### Added - -- New features - -### Changed - -- Changes to existing functionality - -### Deprecated - -- Features that will be removed in future versions - -### Removed - -- Features removed in this release - -### Fixed - -- Bug fixes - -### Security - -- Security-related fixes -``` - ---- - -## Unreleased - -_Changes that are in development but not yet released._ - -### Added - -- **Comprehensive Test Suite**: Created 1,070+ tests across 4 priority tiers - - **TIER 1 (Critical)**: 159 tests - Scheduler, NFO batch operations, download queue, persistence - - **TIER 2 (High Priority)**: 390 tests - JavaScript framework, dark mode, setup page, settings modal, WebSocket, queue UI - - **TIER 3 (Medium Priority)**: 156 tests - WebSocket load, concurrent operations, retry logic, NFO performance, series parsing, TMDB integration - - **TIER 4 (Polish)**: 426 tests - Internationalization (89), user preferences (68), accessibility (250+), media server compatibility (19) -- **Frontend Testing Infrastructure**: Vitest for unit tests, Playwright for E2E tests -- **Security Test Coverage**: Complete testing for authentication, authorization, CSRF, XSS, SQL injection -- **Performance Validation**: WebSocket load (200+ concurrent clients), batch operations, concurrent access -- **Accessibility Tests**: WCAG 2.1 AA compliance testing (keyboard navigation, ARIA labels, screen readers) -- **Media Server Compatibility**: NFO format validation for Kodi, Plex, Jellyfin, and Emby - -### Changed - -- Updated testing documentation (TESTING_COMPLETE.md, instructions.md) to reflect 100% completion of all test tiers - -### Fixed - -- **Enhanced Anime Add Flow**: Automatic database persistence, targeted episode scanning, and folder creation with sanitized names -- Filesystem utility module (`src/server/utils/filesystem.py`) with `sanitize_folder_name()`, `is_safe_path()`, and `create_safe_folder()` functions -- `Serie.sanitized_folder` property for generating filesystem-safe folder names from display names -- `SerieScanner.scan_single_series()` method for targeted scanning of individual anime without full library rescan -- Add series API response now includes `missing_episodes` list and `total_missing` count -- Database transaction support with `@transactional` decorator and `atomic()` context manager -- Transaction propagation modes (REQUIRED, REQUIRES_NEW, NESTED) for fine-grained control -- Savepoint support for nested transactions with partial rollback capability -- `TransactionManager` helper class for manual transaction control -- Bulk operations: `bulk_mark_downloaded`, `bulk_delete`, `clear_all` for batch processing -- `rotate_session` atomic operation for secure session rotation -- Transaction utilities: `is_session_in_transaction`, `get_session_transaction_depth` -- `get_transactional_session` for sessions without auto-commit - -### Changed - -- `QueueRepository.save_item()` now uses atomic transactions for data consistency -- `QueueRepository.clear_all()` now uses atomic transactions for all-or-nothing behavior -- Service layer documentation updated to reflect transaction-aware design - -### Fixed - -- Scan status indicator now correctly shows running state after page reload during active scan -- Improved reliability of process status updates in the UI header - ---- - -## Version History - -_To be documented as versions are released._ diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md deleted file mode 100644 index 6df784e..0000000 --- a/docs/CONFIGURATION.md +++ /dev/null @@ -1,374 +0,0 @@ -# Configuration Reference - -## Document Purpose - -This document provides a comprehensive reference for all configuration options in the Aniworld application. - ---- - -## 1. Configuration Overview - -### Configuration Sources - -Aniworld uses a layered configuration system with **explicit precedence rules**: - -1. **Environment Variables** (highest priority) - Takes precedence over all other sources -2. **`.env` file** in project root - Loaded as environment variables -3. **`data/config.json`** file - Persistent file-based configuration -4. **Default values** (lowest priority) - Built-in fallback values - -### Precedence Rules - -**Critical Principle**: `ENV VARS > config.json > defaults` - -- **Environment variables always win**: If a value is set via environment variable, it will NOT be overridden by config.json -- **config.json as fallback**: If an ENV var is not set (or is empty/default), the value from config.json is used -- **Defaults as last resort**: Built-in default values are used only if neither ENV var nor config.json provide a value - -### Loading Mechanism - -Configuration is loaded at application startup in `src/server/fastapi_app.py`: - -1. **Pydantic Settings** loads ENV vars and .env file with defaults -2. **config.json** is loaded via `ConfigService` -3. **Selective sync**: config.json values sync to settings **only if** ENV var not set -4. **Runtime access**: Code uses `settings` object (which has final merged values) - -**Example**: - -```bash -# If ENV var is set: -ANIME_DIRECTORY=/env/path # This takes precedence - -# config.json has: -{"other": {"anime_directory": "/config/path"}} # This is ignored - -# Result: settings.anime_directory = "/env/path" -``` - -**Source**: [src/config/settings.py](../src/config/settings.py#L1-L96), [src/server/fastapi_app.py](../src/server/fastapi_app.py#L139-L185) - ---- - -## 2. Environment Variables - -### Authentication Settings - -| Variable | Type | Default | Description | -| ----------------------- | ------ | ---------------- | ------------------------------------------------------------------- | -| `JWT_SECRET_KEY` | string | (random) | Secret key for JWT token signing. Auto-generated if not set. | -| `PASSWORD_SALT` | string | `"default-salt"` | Salt for password hashing. | -| `MASTER_PASSWORD_HASH` | string | (none) | Pre-hashed master password. Loaded from config.json if not set. | -| `MASTER_PASSWORD` | string | (none) | **DEVELOPMENT ONLY** - Plaintext password. Never use in production. | -| `SESSION_TIMEOUT_HOURS` | int | `24` | JWT token expiry time in hours. | - -Source: [src/config/settings.py](../src/config/settings.py#L13-L42) - -### Server Settings - -| Variable | Type | Default | Description | -| ----------------- | ------ | -------------------------------- | --------------------------------------------------------------------- | -| `ANIME_DIRECTORY` | string | `""` | Path to anime library directory. | -| `LOG_LEVEL` | string | `"INFO"` | Logging level: DEBUG, INFO, WARNING, ERROR, CRITICAL. | -| `DATABASE_URL` | string | `"sqlite:///./data/aniworld.db"` | Database connection string. | -| `CORS_ORIGINS` | string | `"http://localhost:3000"` | Comma-separated allowed CORS origins. Use `*` for localhost defaults. | -| `API_RATE_LIMIT` | int | `100` | Maximum API requests per minute. | - -Source: [src/config/settings.py](../src/config/settings.py#L43-L68) - -### Provider Settings - -| Variable | Type | Default | Description | -| ------------------ | ------ | --------------- | --------------------------------------------- | -| `DEFAULT_PROVIDER` | string | `"aniworld.to"` | Default anime provider. | -| `PROVIDER_TIMEOUT` | int | `30` | HTTP timeout for provider requests (seconds). | -| `RETRY_ATTEMPTS` | int | `3` | Number of retry attempts for failed requests. | - -Source: [src/config/settings.py](../src/config/settings.py#L69-L79) - -### NFO Settings - -| Variable | Type | Default | Description | -| --------------------- | ------ | -------- | -------------------------------------------------- | -| `TMDB_API_KEY` | string | `""` | The Movie Database (TMDB) API key for metadata. | -| `NFO_AUTO_CREATE` | bool | `true` | Automatically create NFO files during downloads. | -| `NFO_UPDATE_ON_SCAN` | bool | `false` | Update existing NFO files when scanning library. | -| `NFO_DOWNLOAD_POSTER` | bool | `true` | Download poster images along with NFO files. | -| `NFO_DOWNLOAD_LOGO` | bool | `false` | Download logo images along with NFO files. | -| `NFO_DOWNLOAD_FANART` | bool | `false` | Download fanart images along with NFO files. | -| `NFO_IMAGE_SIZE` | string | `"w500"` | Image size for TMDB images (w500, w780, original). | - -Source: [src/server/models/config.py](../src/server/models/config.py#L109-L132) - ---- - -## 3. Configuration File (config.json) - -Location: `data/config.json` - -### File Structure - -```json -{ - "name": "Aniworld", - "data_dir": "data", - "scheduler": { - "enabled": true, - "interval_minutes": 60, - "schedule_time": "03:00", - "schedule_days": ["mon", "tue", "wed", "thu", "fri", "sat", "sun"], - "auto_download_after_rescan": false, - "folder_scan_enabled": false - }, - "logging": { - "level": "INFO", - "file": null, - "max_bytes": null, - "backup_count": 3 - }, - "backup": { - "enabled": false, - "path": "data/backups", - "keep_days": 30 - }, - "nfo": { - "tmdb_api_key": "", - "auto_create": true, - "update_on_scan": false, - "download_poster": true, - "download_logo": false, - "download_fanart": false, - "image_size": "w500" - }, - "other": { - "master_password_hash": "$pbkdf2-sha256$...", - "anime_directory": "/path/to/anime" - }, - "version": "1.0.1" -} -``` - -Source: [data/config.json](../data/config.json) - ---- - -## 4. Configuration Sections - -### 4.1 General Settings - -| Field | Type | Default | Description | -| ---------- | ------ | ------------ | ------------------------------ | -| `name` | string | `"Aniworld"` | Application name. | -| `data_dir` | string | `"data"` | Base directory for data files. | - -Source: [src/server/models/config.py](../src/server/models/config.py#L62-L66) - -### 4.2 Scheduler Settings - -Controls automatic cron-based library rescanning (powered by APScheduler). - -| Field | Type | Default | Description | -| -------------------------------------- | ------------ | --------------------------------------------- | -------------------------------------------------------------------- | -| `scheduler.enabled` | bool | `true` | Enable/disable automatic scans. | -| `scheduler.interval_minutes` | int | `60` | Legacy field kept for backward compatibility. Minimum: 1. | -| `scheduler.schedule_time` | string | `"03:00"` | Daily run time in 24-h `HH:MM` format. | -| `scheduler.schedule_days` | list[string] | `["mon","tue","wed","thu","fri","sat","sun"]` | Days of the week to run the scan. Empty list disables the cron job. | -| `scheduler.auto_download_after_rescan` | bool | `false` | Automatically queue missing episodes for download after each rescan. | -| `scheduler.folder_scan_enabled` | bool | `false` | Run folder maintenance (NFO repair, folder renaming, poster checks) during scheduled runs. **When enabled, series folders are automatically renamed to match the ` (<year>)` convention derived from their `tvshow.nfo` files.** | - -Valid day abbreviations: `mon`, `tue`, `wed`, `thu`, `fri`, `sat`, `sun`. - -Source: [src/server/models/config.py](../src/server/models/config.py#L5-L12) - -### 4.3 Logging Settings - -| Field | Type | Default | Description | -| ---------------------- | ------ | -------- | ------------------------------------------------- | -| `logging.level` | string | `"INFO"` | Log level: DEBUG, INFO, WARNING, ERROR, CRITICAL. | -| `logging.file` | string | `null` | Optional log file path. | -| `logging.max_bytes` | int | `null` | Maximum log file size for rotation. | -| `logging.backup_count` | int | `3` | Number of rotated log files to keep. | - -Source: [src/server/models/config.py](../src/server/models/config.py#L27-L46) - -### 4.4 Backup Settings - -| Field | Type | Default | Description | -| ------------------ | ------ | ---------------- | -------------------------------- | -| `backup.enabled` | bool | `false` | Enable automatic config backups. | -| `backup.path` | string | `"data/backups"` | Directory for backup files. | -| `backup.keep_days` | int | `30` | Days to retain backups. | - -Source: [src/server/models/config.py](../src/server/models/config.py#L15-L24) - -### 4.5 NFO Settings - -| Field | Type | Default | Description | -| --------------------- | ------ | -------- | ------------------------------------------------------------- | -| `nfo.tmdb_api_key` | string | `""` | The Movie Database (TMDB) API key for fetching metadata. | -| `nfo.auto_create` | bool | `true` | Automatically create NFO files when downloading episodes. | -| `nfo.update_on_scan` | bool | `false` | Update existing NFO files during library scan operations. | -| `nfo.download_poster` | bool | `true` | Download poster images (poster.jpg) along with NFO files. | -| `nfo.download_logo` | bool | `false` | Download logo images (logo.png) along with NFO files. | -| `nfo.download_fanart` | bool | `false` | Download fanart images (fanart.jpg) along with NFO files. | -| `nfo.image_size` | string | `"w500"` | TMDB image size: `w500` (recommended), `w780`, or `original`. | - -**Notes:** - -- Obtain a TMDB API key from https://www.themoviedb.org/settings/api -- `auto_create` creates NFO files during the download process -- `update_on_scan` refreshes metadata when scanning existing anime -- `download_poster` also controls whether the scheduled folder scan checks for and re-downloads missing or corrupted `poster.jpg` files (see [NFO_GUIDE.md](NFO_GUIDE.md#6-poster-check)) -- Image downloads require valid `tmdb_api_key` -- `TMDB_API_KEY` environment variable is optional when `nfo.tmdb_api_key` is configured in `data/config.json` -- Larger image sizes (`w780`, `original`) consume more storage space - -Source: [src/server/models/config.py](../src/server/models/config.py#L109-L132) - -### 4.6 Other Settings (Dynamic) - -The `other` field stores arbitrary settings. - -| Key | Type | Description | -| ---------------------- | ------ | --------------------------------------- | -| `master_password_hash` | string | Hashed master password (pbkdf2-sha256). | -| `anime_directory` | string | Path to anime library. | -| `advanced` | object | Advanced configuration options. | - ---- - -## 5. Configuration Precedence - -Settings are resolved in this order (first match wins): - -1. Environment variable (e.g., `ANIME_DIRECTORY`) -2. `.env` file in project root -3. `data/config.json` (for dynamic settings) -4. Code defaults in `Settings` class - ---- - -## 6. Validation Rules - -### Password Requirements - -Master password must meet all criteria: - -- Minimum 8 characters -- At least one uppercase letter -- At least one lowercase letter -- At least one digit -- At least one special character - -Source: [src/server/services/auth_service.py](../src/server/services/auth_service.py#L97-L125) - -### Logging Level Validation - -Must be one of: `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL` - -Source: [src/server/models/config.py](../src/server/models/config.py#L43-L47) - -### Backup Path Validation - -If `backup.enabled` is `true`, `backup.path` must be set. - -Source: [src/server/models/config.py](../src/server/models/config.py#L87-L91) - ---- - -## 7. Example Configurations - -### Minimal Development Setup - -**.env file:** - -``` -LOG_LEVEL=DEBUG -ANIME_DIRECTORY=/home/user/anime -``` - -### Production Setup - -**.env file:** - -``` -JWT_SECRET_KEY=your-secure-random-key-here -DATABASE_URL=postgresql+asyncpg://user:pass@localhost/aniworld -LOG_LEVEL=WARNING -CORS_ORIGINS=https://your-domain.com -API_RATE_LIMIT=60 -``` - -### Docker Setup - -```yaml -# docker-compose.yml -environment: - - JWT_SECRET_KEY=${JWT_SECRET_KEY} - - DATABASE_URL=sqlite:///./data/aniworld.db - - ANIME_DIRECTORY=/media/anime - - LOG_LEVEL=INFO -volumes: - - ./data:/app/data - - /media/anime:/media/anime:ro -``` - ---- - -## 8. Configuration Backup Management - -### Automatic Backups - -Backups are created automatically before config changes when `backup.enabled` is `true`. - -Location: `data/config_backups/` - -Naming: `config_backup_YYYYMMDD_HHMMSS.json` - -### Manual Backup via API - -```bash -# Create backup -curl -X POST http://localhost:8000/api/config/backups \ - -H "Authorization: Bearer $TOKEN" - -# List backups -curl http://localhost:8000/api/config/backups \ - -H "Authorization: Bearer $TOKEN" - -# Restore backup -curl -X POST http://localhost:8000/api/config/backups/config_backup_20251213.json/restore \ - -H "Authorization: Bearer $TOKEN" -``` - -Source: [src/server/api/config.py](../src/server/api/config.py#L67-L142) - ---- - -## 9. Troubleshooting - -### Configuration Not Loading - -1. Check file permissions on `data/config.json` -2. Verify JSON syntax with a validator -3. Check logs for Pydantic validation errors - -### Environment Variable Not Working - -1. Ensure variable name matches exactly (case-sensitive) -2. Check `.env` file location (project root) -3. Restart application after changes - -### Master Password Issues - -1. Password hash is stored in `config.json` under `other.master_password_hash` -2. Delete this field to reset (requires re-setup) -3. Check hash format starts with `$pbkdf2-sha256$` - ---- - -## 10. Related Documentation - -- [API.md](API.md) - Configuration API endpoints -- [DEVELOPMENT.md](DEVELOPMENT.md) - Development environment setup -- [ARCHITECTURE.md](ARCHITECTURE.md) - Configuration service architecture diff --git a/docs/DATABASE.md b/docs/DATABASE.md deleted file mode 100644 index 9b2b9c6..0000000 --- a/docs/DATABASE.md +++ /dev/null @@ -1,642 +0,0 @@ -# Database Documentation - -## Document Purpose - -This document describes the database schema, models, and data layer of the Aniworld application. - ---- - -## 1. Database Overview - -### Technology - -- **Database Engine**: SQLite 3 (default), PostgreSQL supported -- **ORM**: SQLAlchemy 2.0 with async support (aiosqlite) -- **Location**: `data/aniworld.db` (configurable via `DATABASE_URL`) - -Source: [src/config/settings.py](../src/config/settings.py#L53-L55) - -### Connection Configuration - -```python -# Default connection string -DATABASE_URL = "sqlite+aiosqlite:///./data/aniworld.db" - -# PostgreSQL alternative -DATABASE_URL = "postgresql+asyncpg://user:pass@localhost/aniworld" -``` - -Source: [src/server/database/connection.py](../src/server/database/connection.py) - ---- - -## 2. Entity Relationship Diagram - -``` -+---------------------+ +-------------------+ +-------------------+ +------------------------+ -| system_settings | | anime_series | | episodes | | download_queue_item | -+---------------------+ +-------------------+ +-------------------+ +------------------------+ -| id (PK) | | id (PK) |<--+ | id (PK) | +-->| id (PK, VARCHAR) | -| initial_scan_... | | key (UNIQUE) | | | series_id (FK)----+---+ | series_id (FK)---------+ -| initial_nfo_scan... | | name | +---| | | status | -| initial_media_... | | site | | season | | priority | -| last_scan_timestamp | | folder | | episode_number | | season | -| created_at | | created_at | | title | | episode | -| updated_at | | updated_at | | file_path | | progress_percent | -+---------------------+ +-------------------+ | is_downloaded | | error_message | - | created_at | | retry_count | - | updated_at | | added_at | - +-------------------+ | started_at | - | completed_at | - | created_at | - | updated_at | - +------------------------+ -``` - ---- - -## 3. Table Schemas - -### 3.1 system_settings - -Stores application-wide system settings and initialization state. - -| Column | Type | Constraints | Description | -| ------------------------------ | -------- | -------------------------- | --------------------------------------------- | -| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Internal database ID (only one row) | -| `initial_scan_completed` | BOOLEAN | NOT NULL, DEFAULT FALSE | Whether initial anime folder scan is complete | -| `initial_nfo_scan_completed` | BOOLEAN | NOT NULL, DEFAULT FALSE | Whether initial NFO scan is complete | -| `initial_media_scan_completed` | BOOLEAN | NOT NULL, DEFAULT FALSE | Whether initial media scan is complete | -| `last_scan_timestamp` | DATETIME | NULLABLE | Timestamp of last completed scan | -| `created_at` | DATETIME | NOT NULL, DEFAULT NOW | Record creation timestamp | -| `updated_at` | DATETIME | NOT NULL, ON UPDATE NOW | Last update timestamp | - -**Purpose:** - -This table tracks the initialization status of the application to ensure that expensive one-time setup operations (like scanning the entire anime directory) only run on the first startup, not on every restart. - -- Only one row exists in this table -- The `initial_scan_completed` flag prevents redundant full directory scans on each startup -- The NFO and media scan flags similarly track completion of those setup tasks - -Source: [src/server/database/models.py](../src/server/database/models.py), [src/server/database/system_settings_service.py](../src/server/database/system_settings_service.py) - -### 3.2 anime_series - -Stores anime series metadata. Corresponds to the core `Serie` class. - -| Column | Type | Constraints | Description | -| ---------------- | ------------- | -------------------------- | ------------------------------------------------------- | -| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Internal database ID | -| `key` | VARCHAR(255) | UNIQUE, NOT NULL, INDEX | **Primary identifier** - provider-assigned URL-safe key | -| `name` | VARCHAR(500) | NOT NULL, INDEX | Display name of the series | -| `site` | VARCHAR(500) | NOT NULL | Provider site URL | -| `folder` | VARCHAR(1000) | NOT NULL | Filesystem folder name (metadata only) | -| `year` | INTEGER | NULLABLE | Release year of the series | -| `nfo_path` | VARCHAR(1000) | NULLABLE | Path to tvshow.nfo metadata file | -| `tmdb_id` | INTEGER | NULLABLE, INDEX | TMDB (The Movie Database) ID for metadata | -| `tvdb_id` | INTEGER | NULLABLE, INDEX | TVDB (TheTVDB) ID for metadata | -| `has_nfo` | BOOLEAN | NOT NULL, DEFAULT FALSE | Whether tvshow.nfo exists | -| `loading_status` | VARCHAR(50) | NOT NULL, DEFAULT 'completed' | Status: pending, loading_episodes, loading_nfo, completed, failed | -| `created_at` | DATETIME | NOT NULL, DEFAULT NOW | Record creation timestamp | -| `updated_at` | DATETIME | NOT NULL, ON UPDATE NOW | Last update timestamp | - -**Identifier Convention:** - -- `key` is the **primary identifier** for all operations (e.g., `"attack-on-titan"`) -- `folder` is **metadata only** for filesystem operations (e.g., `"Attack on Titan (2013)"`) -- `id` is used only for database relationships - -**EpisodeDict Mapping:** - -The `episodeDict` (season → episode numbers mapping) is stored as individual `Episode` records: -- Each `Episode` has `season` and `episode_number` columns -- Relationship: `AnimeSeries.episodes` returns all Episode records for that series - -Source: [src/server/database/models.py](../src/server/database/models.py#L23-L150) - -### 3.3 episodes - -Stores **missing episodes** that need to be downloaded. Episodes are automatically managed during scans: - -- New missing episodes are added to the database -- Episodes that are no longer missing (files now exist) are removed from the database -- When an episode is downloaded, it can be marked with `is_downloaded=True` or removed from tracking - -| Column | Type | Constraints | Description | -| ---------------- | ------------- | ---------------------------- | ----------------------------- | -| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Internal database ID | -| `series_id` | INTEGER | FOREIGN KEY, NOT NULL, INDEX | Reference to anime_series.id | -| `season` | INTEGER | NOT NULL | Season number (1-based) | -| `episode_number` | INTEGER | NOT NULL | Episode number within season | -| `title` | VARCHAR(500) | NULLABLE | Episode title if known | -| `file_path` | VARCHAR(1000) | NULLABLE | Local file path if downloaded | -| `is_downloaded` | BOOLEAN | NOT NULL, DEFAULT FALSE | Download status flag | -| `created_at` | DATETIME | NOT NULL, DEFAULT NOW | Record creation timestamp | -| `updated_at` | DATETIME | NOT NULL, ON UPDATE NOW | Last update timestamp | - -**Foreign Key:** - -- `series_id` -> `anime_series.id` (ON DELETE CASCADE) - -Source: [src/server/database/models.py](../src/server/database/models.py#L122-L181) - -### 3.4 download_queue_item - -Stores download queue items with status tracking. - -| Column | Type | Constraints | Description | -| ------------------ | ------------- | --------------------------- | ------------------------------ | -| `id` | VARCHAR(36) | PRIMARY KEY | UUID identifier | -| `series_id` | INTEGER | FOREIGN KEY, NOT NULL | Reference to anime_series.id | -| `season` | INTEGER | NOT NULL | Season number | -| `episode` | INTEGER | NOT NULL | Episode number | -| `status` | VARCHAR(20) | NOT NULL, DEFAULT 'pending' | Download status | -| `priority` | VARCHAR(10) | NOT NULL, DEFAULT 'NORMAL' | Queue priority | -| `progress_percent` | FLOAT | NULLABLE | Download progress (0-100) | -| `error_message` | TEXT | NULLABLE | Error description if failed | -| `retry_count` | INTEGER | NOT NULL, DEFAULT 0 | Number of retry attempts | -| `source_url` | VARCHAR(2000) | NULLABLE | Download source URL | -| `added_at` | DATETIME | NOT NULL, DEFAULT NOW | When added to queue | -| `started_at` | DATETIME | NULLABLE | When download started | -| `completed_at` | DATETIME | NULLABLE | When download completed/failed | -| `created_at` | DATETIME | NOT NULL, DEFAULT NOW | Record creation timestamp | -| `updated_at` | DATETIME | NOT NULL, ON UPDATE NOW | Last update timestamp | - -**Status Values:** `pending`, `downloading`, `paused`, `completed`, `failed`, `cancelled` - -**Priority Values:** `LOW`, `NORMAL`, `HIGH` - -**Foreign Key:** - -- `series_id` -> `anime_series.id` (ON DELETE CASCADE) - -Source: [src/server/database/models.py](../src/server/database/models.py#L200-L300) - ---- - -## 4. Indexes - -| Table | Index Name | Columns | Purpose | -| --------------------- | ----------------------- | ----------- | --------------------------------- | -| `system_settings` | N/A (single row) | N/A | Only one row, no indexes needed | -| `anime_series` | `ix_anime_series_key` | `key` | Fast lookup by primary identifier | -| `anime_series` | `ix_anime_series_name` | `name` | Search by name | -| `episodes` | `ix_episodes_series_id` | `series_id` | Join with series | -| `download_queue_item` | `ix_download_series_id` | `series_id` | Filter by series | -| `download_queue_item` | `ix_download_status` | `status` | Filter by status | - ---- - -## 5. Model Layer - -### 5.1 SQLAlchemy ORM Models - -```python -# src/server/database/models.py - -class AnimeSeries(Base, TimestampMixin): - __tablename__ = "anime_series" - - id: Mapped[int] = mapped_column(Integer, primary_key=True) - key: Mapped[str] = mapped_column(String(255), unique=True, index=True) - name: Mapped[str] = mapped_column(String(500), index=True) - site: Mapped[str] = mapped_column(String(500)) - folder: Mapped[str] = mapped_column(String(1000)) - - episodes: Mapped[List["Episode"]] = relationship( - "Episode", back_populates="series", cascade="all, delete-orphan" - ) -``` - -Source: [src/server/database/models.py](../src/server/database/models.py#L23-L87) - -### 5.2 Pydantic API Models - -```python -# src/server/models/download.py - -class DownloadItem(BaseModel): - id: str - serie_id: str # Maps to anime_series.key - serie_folder: str # Metadata only - serie_name: str - episode: EpisodeIdentifier - status: DownloadStatus - priority: DownloadPriority -``` - -Source: [src/server/models/download.py](../src/server/models/download.py#L63-L118) - -### 5.3 Model Mapping - -| API Field | Database Column | Notes | -| -------------- | --------------------- | ------------------ | -| `serie_id` | `anime_series.key` | Primary identifier | -| `serie_folder` | `anime_series.folder` | Metadata only | -| `serie_name` | `anime_series.name` | Display name | - ---- - -## 6. Transaction Support - -### 6.1 Overview - -The database layer provides comprehensive transaction support to ensure data consistency across compound operations. All write operations can be wrapped in explicit transactions. - -Source: [src/server/database/transaction.py](../src/server/database/transaction.py) - -### 6.2 Transaction Utilities - -| Component | Type | Description | -| ------------------------- | ----------------- | ---------------------------------------- | -| `@transactional` | Decorator | Wraps function in transaction boundary | -| `atomic()` | Async context mgr | Provides atomic operation block | -| `atomic_sync()` | Sync context mgr | Sync version of atomic() | -| `TransactionContext` | Class | Explicit sync transaction control | -| `AsyncTransactionContext` | Class | Explicit async transaction control | -| `TransactionManager` | Class | Helper for manual transaction management | - -### 6.3 Transaction Propagation Modes - -| Mode | Behavior | -| -------------- | ------------------------------------------------ | -| `REQUIRED` | Use existing transaction or create new (default) | -| `REQUIRES_NEW` | Always create new transaction | -| `NESTED` | Create savepoint within existing transaction | - -### 6.4 Usage Examples - -**Using @transactional decorator:** - -```python -from src.server.database.transaction import transactional - -@transactional() -async def compound_operation(db: AsyncSession, data: dict): - # All operations commit together or rollback on error - series = await AnimeSeriesService.create(db, ...) - episode = await EpisodeService.create(db, series_id=series.id, ...) - return series, episode -``` - -**Using atomic() context manager:** - -```python -from src.server.database.transaction import atomic - -async def some_function(db: AsyncSession): - async with atomic(db) as tx: - await operation1(db) - await operation2(db) - # Auto-commits on success, rolls back on exception -``` - -**Using savepoints for partial rollback:** - -```python -async with atomic(db) as tx: - await outer_operation(db) - - async with tx.savepoint() as sp: - await risky_operation(db) - if error_condition: - await sp.rollback() # Only rollback nested ops - - await final_operation(db) # Still executes -``` - -Source: [src/server/database/transaction.py](../src/server/database/transaction.py) - -### 6.5 Connection Module Additions - -| Function | Description | -| ------------------------------- | -------------------------------------------- | -| `get_transactional_session` | Session without auto-commit for transactions | -| `TransactionManager` | Helper class for manual transaction control | -| `is_session_in_transaction` | Check if session is in active transaction | -| `get_session_transaction_depth` | Get nesting depth of transactions | - -Source: [src/server/database/connection.py](../src/server/database/connection.py) - ---- - -## 7. Repository Pattern - -The `QueueRepository` class provides data access abstraction. - -```python -class QueueRepository: - async def save_item(self, item: DownloadItem) -> None: - """Save or update a download item (atomic operation).""" - - async def get_all_items(self) -> List[DownloadItem]: - """Get all items from database.""" - - async def delete_item(self, item_id: str) -> bool: - """Delete item by ID.""" - - async def clear_all(self) -> int: - """Clear all items (atomic operation).""" -``` - -Note: Compound operations (`save_item`, `clear_all`) are wrapped in `atomic()` transactions. - -Source: [src/server/services/queue_repository.py](../src/server/services/queue_repository.py) - ---- - -## 8. Database Service - -The `AnimeSeriesService` provides async CRUD operations. - -```python -class AnimeSeriesService: - @staticmethod - async def create( - db: AsyncSession, - key: str, - name: str, - site: str, - folder: str - ) -> AnimeSeries: - """Create a new anime series.""" - - @staticmethod - async def get_by_key( - db: AsyncSession, - key: str - ) -> Optional[AnimeSeries]: - """Get series by primary key identifier.""" -``` - -### Bulk Operations - -Services provide bulk operations for transaction-safe batch processing: - -| Service | Method | Description | -| ---------------------- | ---------------------- | ------------------------------ | -| `EpisodeService` | `bulk_mark_downloaded` | Mark multiple episodes at once | -| `DownloadQueueService` | `bulk_delete` | Delete multiple queue items | -| `DownloadQueueService` | `clear_all` | Clear entire queue | -| `UserSessionService` | `rotate_session` | Revoke old + create new atomic | -| `UserSessionService` | `cleanup_expired` | Bulk delete expired sessions | - -Source: [src/server/database/service.py](../src/server/database/service.py) - ---- - -## 9. Data Integrity Rules - -### Validation Constraints - -| Field | Rule | Error Message | -| ------------------------- | ------------------------ | ------------------------------------- | -| `anime_series.key` | Non-empty, max 255 chars | "Series key cannot be empty" | -| `anime_series.name` | Non-empty, max 500 chars | "Series name cannot be empty" | -| `episodes.season` | 0-1000 | "Season number must be non-negative" | -| `episodes.episode_number` | 0-10000 | "Episode number must be non-negative" | - -Source: [src/server/database/models.py](../src/server/database/models.py#L89-L119) - -### Cascade Rules - -- Deleting `anime_series` deletes all related `episodes` and `download_queue_item` - ---- - -## 10. Migration Strategy - -Currently, SQLAlchemy's `create_all()` is used for schema creation. - -```python -# src/server/database/connection.py -async def init_db(): - async with engine.begin() as conn: - await conn.run_sync(Base.metadata.create_all) -``` - -For production migrations, Alembic is recommended but not yet implemented. - -Source: [src/server/database/connection.py](../src/server/database/connection.py) - ---- - -## 11. Common Query Patterns - -### Get all series with missing episodes - -```python -series = await db.execute( - select(AnimeSeries).options(selectinload(AnimeSeries.episodes)) -) -for serie in series.scalars(): - downloaded = [e for e in serie.episodes if e.is_downloaded] -``` - -### Get pending downloads ordered by priority - -```python -items = await db.execute( - select(DownloadQueueItem) - .where(DownloadQueueItem.status == "pending") - .order_by( - case( - (DownloadQueueItem.priority == "HIGH", 1), - (DownloadQueueItem.priority == "NORMAL", 2), - (DownloadQueueItem.priority == "LOW", 3), - ), - DownloadQueueItem.added_at - ) -) -``` - ---- - -## 12. Series Storage: Database vs Files (Deprecated) - -### File-Based Storage (Removed in v2.0) - -Prior to v2.0, series metadata was stored in two files per anime folder: - -| File | Contents | -| -------- | ------------------------------------------------------- | -| `key` | Series provider key (e.g., `"attack-on-titan"`) | -| `data` | JSON serialization of `Serie` object | - -File structure example: -``` -/anime/Attack on Titan (2013)/ -├── key # Contains: attack-on-titan -├── data # Contains: {"key": "...", "name": "...", "episodeDict": {...}} -├── Season 1/ -│ └── ... -``` - -### Database Storage (Current) - -Since v2.0, all series metadata is stored in the `anime_series` table with `Episode` records for episode tracking. This provides: - -- **ACID transactions** for data consistency -- **Foreign key constraints** (cascade delete) -- **Indexed queries** for fast lookups -- **No filesystem dependency** for metadata - -### Migration from Files to Database - -The `Serie.save_to_file()` and `Serie.load_from_file()` methods are deprecated but still functional for backward compatibility during migration: - -```python -from src.core.entities.series import Serie - -# Old file-based loading (deprecated) -serie = Serie.load_from_file("/anime/Attack on Titan (2013)/data") - -# New database-based loading -from src.server.database.service import AnimeSeriesService -serie = await AnimeSeriesService.get_by_key(db, "attack-on-titan") -``` - -### Removing File Dependencies - -After verifying database schema supports all fields, file-based storage can be removed: - -1. ✅ Schema verified: All `Serie` fields have corresponding DB columns -2. ✅ Migration complete: All existing series migrated to database -3. ❌ File cleanup: Remove `key` and `data` files (pending) - -**Note:** The `save_to_file()` and `load_from_file()` methods will be removed in v3.0.0. - ---- - -## 12. Series Persistence Flow - -When a directory scan discovers or updates series, the scanner persists data to the database instead of writing to disk files. - -### Scan Flow - -``` -Scan Directory - │ - ▼ -Find MP4 Files → Extract Serie Key - │ - ▼ -Check DB for Existing Series (by key) - │ - ├─── EXISTS ──────────────────────► Update Series Metadata - │ │ - │ ▼ - │ Sync Episodes to DB - │ │ - │◄──────────────────────────────────────┘ - │ - └─── NEW ───────────────────────────► Create New Series Record - │ - ▼ - Create Episode Records - │ - ▼ - Return to Scan Loop -``` - -### Key Methods - -**SerieScanner._persist_serie_to_db()** -- Called after `get_missing_episodes_and_season()` computes episodeDict -- Uses `AnimeSeriesService.get_by_key()` to check if series exists -- If exists: calls `AnimeSeriesService.update()` + `_sync_episodes_to_db()` -- If new: calls `AnimeSeriesService.create()` + creates episodes - -**SerieScanner._sync_episodes_to_db()** -- Gets existing episodes from DB via `EpisodeService.get_by_series()` -- Compares with new episodeDict -- Removes episodes no longer missing (unless `is_downloaded=True`) -- Adds new missing episodes -- Preserves `is_downloaded=True` episodes when removing missing ones - -**SerieList.add_to_db()** -- Used when adding a new discovered series via API -- Creates filesystem folder + database record + episode records - -### Episode Sync Logic - -```python -# For each episode in DB but not in new episodeDict: -if episode.is_downloaded: - # Keep - file exists, don't remove - pass -else: - # Remove - no longer missing - EpisodeService.delete() - -# For each episode in new episodeDict but not in DB: -# Add as new missing episode -EpisodeService.create(is_downloaded=False) -``` - -### Transaction Handling - -- DB operations use their own session with commit/rollback -- If DB write fails, error is logged and scan continues -- File-based `save_to_file()` no longer called during scan - -### Migration Path - -1. v2.x: Scanner writes to both DB (primary) and files (fallback) -2. v3.0: Scanner writes only to DB, file methods removed - ---- - -## 13. Series Persistence - -### Schema - -**AnimeSeries Table**: Stores series metadata (key, name, site, folder, year) - -| Column | Type | Constraints | Description | -|-----------|--------------|---------------------------|----------------------| -| `id` | INTEGER | PRIMARY KEY | Auto-increment | -| `key` | VARCHAR(255) | UNIQUE, NOT NULL | Series provider key | -| `name` | VARCHAR(500) | NOT NULL | Display name | -| `site` | VARCHAR(500) | | Provider site URL | -| `folder` | VARCHAR(1000)| | Filesystem folder | - -**Episode Table**: Stores per-episode metadata (season, episode_number, is_downloaded) - -| Column | Type | Constraints | Description | -|-----------------|--------------|---------------------------|----------------------| -| `id` | INTEGER | PRIMARY KEY | Auto-increment | -| `series_id` | INTEGER | FOREIGN KEY → anime_series| Parent series | -| `season` | INTEGER | NOT NULL | Season number | -| `episode_number`| INTEGER | NOT NULL | Episode number | -| `is_downloaded` | BOOLEAN | DEFAULT FALSE | Download status | - -### Relationships - -- `AnimeSeries.episodes` → List of Episode objects (one-to-many) -- `Episode.series` → Parent AnimeSeries (many-to-one) -- Cascade delete: Deleting a series removes all its episodes - -### Queries - -```python -# Get all series with episodes -AnimeSeriesService.get_all(db, with_episodes=True) - -# Get by provider key -AnimeSeriesService.get_by_key(db, key) - -# Get by folder path -AnimeSeriesService.get_by_folder(db, folder) -``` - ---- - -## 14. Database Location - -| Environment | Default Location | -| ----------- | ------------------------------------------------- | -| Development | `./data/aniworld.db` | -| Production | Via `DATABASE_URL` environment variable | -| Testing | In-memory SQLite (`sqlite+aiosqlite:///:memory:`) | diff --git a/docs/DEVELOPMENT.md b/docs/DEVELOPMENT.md deleted file mode 100644 index a96648e..0000000 --- a/docs/DEVELOPMENT.md +++ /dev/null @@ -1,436 +0,0 @@ -# Development Guide - -## Document Purpose - -This document provides guidance for developers working on the Aniworld project. - -### What This Document Contains - -- **Prerequisites**: Required software and tools -- **Environment Setup**: Step-by-step local development setup -- **Project Structure**: Source code organization explanation -- **Development Workflow**: Branch strategy, commit conventions -- **Coding Standards**: Style guide, linting, formatting -- **Running the Application**: Development server, CLI usage -- **Debugging Tips**: Common debugging approaches -- **IDE Configuration**: VS Code settings, recommended extensions -- **Contributing Guidelines**: How to submit changes -- **Code Review Process**: Review checklist and expectations - -### What This Document Does NOT Contain - -- Production deployment (see [DEPLOYMENT.md](DEPLOYMENT.md)) -- API reference (see [API.md](API.md)) -- Architecture decisions (see [ARCHITECTURE.md](ARCHITECTURE.md)) -- Test writing guides (see [TESTING.md](TESTING.md)) -- Security guidelines (see [SECURITY.md](SECURITY.md)) - -### Target Audience - -- New Developers joining the project -- Contributors (internal and external) -- Anyone setting up a development environment - ---- - -## Sections to Document - -1. Prerequisites - - Python version - - Conda environment - - Node.js (if applicable) - - Git -2. Getting Started - - Clone repository - - Setup conda environment - - Install dependencies - - Configuration setup -3. Project Structure Overview -4. Development Server - - Starting FastAPI server - - Hot reload configuration - - Debug mode -5. CLI Development -6. Code Style - - PEP 8 compliance - - Type hints requirements - - Docstring format - - Import organization -7. Git Workflow - - Branch naming - - Commit message format - - Pull request process -8. Common Development Tasks - -### Adding Queue Deduplication - -The download queue prevents duplicate entries at two levels: - -**In-Memory Deduplication** (`src/server/services/download_service.py`): -- `_pending_by_episode` dict tracks pending episodes: key = `(serie_id, season, episode)` -- `_add_to_pending_queue()` updates the dict when adding items -- `add_to_queue()` checks this dict before adding episodes (includes batch-local dedup) -- `_remove_from_pending_queue()` cleans up the dict when items are removed - -**Database Constraint** (`src/server/models.py`): -- `DownloadQueueItem` has a unique index on `episode_id` via `__table_args__` -- Prevents duplicate queue entries at the database level -- Unique constraint: `Index("ix_download_queue_episode_pending", "episode_id", unique=True)` - -**Scheduler Cooldown** (`src/server/services/scheduler_service.py`): -- `_last_auto_download_time` tracks when auto-download last ran -- 5-minute cooldown prevents rapid re-triggers -- Checked at start of `_auto_download_missing()` - -### Episode Lifecycle - -Episodes transition through states stored in the `episodes` table: - -| State | `is_downloaded` | `file_path` | Description | -|-------|----------------|-------------|-------------| -| Missing | `False` | `NULL` | Episode not yet downloaded | -| Downloaded | `True` | Set | Episode exists on disk | - -**State Transitions:** -1. **Missing → Downloaded**: When download completes, `_remove_episode_from_missing_list()` calls `EpisodeService.mark_downloaded()` to set `is_downloaded=True` and populate `file_path`. The episode record is NOT deleted. - -**Query Implications:** -- `get_series_with_missing_episodes()`: Filters for `is_downloaded=False` to find series with undownloaded episodes -- `get_series_with_no_episodes()`: Finds series with `is_downloaded=False` episodes but NO `is_downloaded=True` episodes (completely unwatched series) - -### Mocking the Download Queue - -When testing components that use the download queue: - -```python -# Mock repository for unit tests -class MockQueueRepository: - def __init__(self): - self._items: Dict[str, DownloadItem] = {} - - async def save_item(self, item: DownloadItem) -> DownloadItem: - self._items[item.id] = item - return item - - async def get_all_items(self) -> List[DownloadItem]: - return list(self._items.values()) - -# Use in fixture -@pytest.fixture -def mock_queue_repository(): - return MockQueueRepository() - -@pytest.fixture -def download_service(mock_anime_service, mock_queue_repository): - return DownloadService( - anime_service=mock_anime_service, - queue_repository=mock_queue_repository, - max_retries=3, - ) -``` - -9. Troubleshooting Development Issues - -### Async Context Managers for aiohttp - -All `aiohttp.ClientSession` usages must be wrapped in `async with`: - -```python -# Correct — session properly closed on exit -async with TMDBClient(api_key="key") as client: - result = await client.search_tv_show("Show") - -# Wrong — session may leak if exception occurs -client = TMDBClient(api_key="key") -result = await client.search_tv_show("Show") -await client.close() # May not be called if exception raised earlier -``` - -**Why:** -- `aiohttp.ClientSession` holds TCP connections that must be explicitly closed -- If exception occurs before `close()`, session leaks -- Context manager guarantees `__aexit__` runs even on exceptions - -**Services that use aiohttp:** -- `TMDBClient` — has `__aenter__`/`__aexit__`, use `async with` -- `ImageDownloader` — has `__aenter__`/`__aexit__`, use `async with` -- `NFOService` — wraps both above, use `async with` - -**Verification:** -- Missing context manager usage triggers `__del__` warning on garbage collection -- Integration tests verify no "Unclosed client session" errors in logs - -### Scheduler Persistence and Recovery - -The scheduler uses APScheduler's in-memory job store. Jobs are reconstructed from `config.json` on every startup — no separate database is needed. - -```python -# Jobs are built from config on startup — no persistence DB required -scheduler = AsyncIOScheduler() # default MemoryJobStore -scheduler.add_job(..., replace_existing=True) -``` - -**Startup misfire recovery:** On `start()`, the scheduler checks `system_settings.last_scan_timestamp` in `aniworld.db`. If the last scan is overdue (>23h but <25h ago), an immediate rescan is triggered. This replaces APScheduler's built-in misfire handling which required a separate SQLite database. - -**Grace period:** If the server was down for more than 25 hours, no automatic recovery occurs to avoid surprise rescans after long downtime. - -**Health endpoint:** `GET /health` returns `scheduler_next_run` and `scheduler_last_run` for external monitors (Uptime Kuma, Prometheus, etc.). - -**If server is down too long:** Manual trigger via `POST /api/scheduler/trigger-rescan` or wait for next scheduled run. - -### Database Session Management - -`get_async_session_factory()` returns a **new AsyncSession instance** directly (not a factory). The function name is historical — callers receive the session immediately: - -```python -# Correct usage: -db = get_async_session_factory() # db IS the session -await db.execute(...) -await db.commit() -await db.close() -``` - -Do NOT call the result again with `()` — that tries to call an `AsyncSession` object, causing `'AsyncSession' object is not callable`. - -For context manager usage, prefer `get_db_session()` (auto-commits) or `get_transactional_session()` (manual commit). - -### Health Check Endpoints - -The application provides health check endpoints for monitoring and container orchestration: - -#### `GET /health` -Basic health check returning service status and startup health check results. - -**Response fields:** -- `status`: "healthy", "degraded", or "unhealthy" based on startup checks -- `timestamp`: ISO timestamp of the check -- `series_app_initialized`: Whether the series app is loaded -- `anime_directory_configured`: Whether anime_directory is set -- `scheduler_next_run` / `scheduler_last_run`: Scheduler times -- `checks`: Detailed startup check results (ffmpeg, DNS, anime_directory) - -#### `GET /health/ready` -Readiness check for container orchestrators (Kubernetes, Docker Swarm). - -**Response when ready:** -```json -{ - "status": "ready", - "ready": true, - "timestamp": "2024-01-01T00:00:00", - "checks": {...} -} -``` - -**Response when not ready (503):** -```json -{ - "status": "not_ready", - "ready": false, - "timestamp": "2024-01-01T00:00:00", - "critical_failures": ["anime_directory: not configured"], - "checks": {...} -} -``` - -#### `GET /health/detailed` -Comprehensive health check including database, filesystem, and system metrics. - -#### Startup Health Checks - -On application startup, the following checks are performed: - -| Check | Failure Status | Impact | -|-------|---------------|--------| -| `ffmpeg` | warning | HLS downloads may fail | -| `dns_aniworld` | warning | Provider requests may fail | -| `dns_tmdb` | warning | TMDB API calls may fail | -| `anime_directory` | error | Download service disabled | - -DNS checks are warnings because failures can be transient. anime_directory errors disable the download service to prevent failures. - -### Troubleshooting Development Issues - -#### Scheduler missed a run - -1. Server was down at scheduled time (03:00 UTC by default). -2. On restart, the scheduler checks `last_scan_timestamp` — if overdue by 23-25h, it triggers immediately. -3. If server was down >25 hours, missed job is skipped to avoid surprise rescans. -4. Trigger manually: `POST /api/scheduler/trigger-rescan` -5. Monitor next run: `GET /health` → `scheduler_next_run` - -#### Scheduler not firing (no events at scheduled time) - -If the scheduler appears configured but never triggers: - -1. **Check application logs for scheduler startup:** - ``` - grep "Scheduler service started" fastapi_app.log - ``` - - If missing, the scheduler failed to start — check for errors above this line - - If present, scheduler started successfully - -2. **Verify the job is registered:** - ``` - grep "Scheduler started with cron trigger" fastapi_app.log - ``` - -3. **Verify APScheduler events in logs:** - ``` - grep "apscheduler.executors.default" fastapi_app.log - ``` - - `Running job` = job triggered - - `executed successfully` = job completed - - No output = job never fired - -4. **Test manual trigger:** - ```bash - curl -X POST http://localhost:8000/api/scheduler/trigger-rescan -H "Authorization: Bearer <token>" - ``` - - If manual trigger works but cron doesn't, the issue is APScheduler configuration - -5. **Check next_run_time via health endpoint:** - ```bash - curl http://localhost:8000/health | jq .scheduler_next_run - ``` - - If `null`, the job is not scheduled - - If set, the scheduler knows when to run next - -6. **Check timezone handling:** - - APScheduler uses UTC internally - - The schedule_time config (e.g., "03:00") is interpreted as UTC - - If you expect local time, adjust the schedule_time accordingly - -#### Startup health check failures - -If `/health` returns `unhealthy` status: - -1. **anime_directory error**: Directory not configured or not writable - - Check `ANIME_DIRECTORY` environment variable - - Verify directory exists and permissions allow write access - - Download service will not initialize until resolved - -2. **ffmpeg warning**: ffmpeg not found in PATH - - HLS stream downloads will fail - - Install ffmpeg: `apt install ffmpeg` or `brew install ffmpeg` - -3. **DNS warnings**: Domain resolution failed - - Check network connectivity - - DNS failures are transient — warnings don't block startup - - Retry later to verify: `GET /health` - -### Provider Failure Handling - -Download providers (VOE, Doodstream, Vidmoly, Vidoza, SpeedFiles, Streamtape, -Luluvdo) regularly break: URLs expire, sites change their player markup, geo -blocks appear, and `yt-dlp` extractors lag behind upstream changes. The -`AniworldLoader.download()` flow is designed to fail fast and rotate. - -**Rotation order** - -1. The episode page is scraped for the providers AniWorld actually advertises. -2. Results are ordered by the preference in `DEFAULT_PROVIDERS` - (`provider_config.py`); providers not listed run last. -3. For each candidate the loader: - 1. Calls `_check_url_alive()` — HEAD probe with GET fallback. Any 4xx - response or connection error skips the provider immediately. - 2. Resolves the redirect via `_resolve_direct_link()` to obtain a direct - stream URL plus headers. Provider-specific extractors (e.g. `VOE`) are - preferred; unknown providers fall back to the embed URL so `yt-dlp` can - attempt extraction. - 3. Tries `_try_direct_stream()` — straight `requests.get(stream=True)` when - `Content-Type` is `video/*` or `application/octet-stream`. This avoids - `yt-dlp` entirely for direct MP4 links. - 4. Falls back to `yt-dlp` with the ffmpeg downloader for HLS streams. -4. On any failure, temp files are cleaned and the loop moves to the next - provider. When the chain is exhausted, the loader logs - `All download providers failed for S{season}E{episode} ...; tried=[...]` - to both the application log and `logs/download_errors.log`. - -**Do not hardcode provider URLs.** Provider domains shift constantly (e.g. -Doodstream alternates between `dood.li`, `dood.so`, `dood.la`). Only the -referer hints in `PROVIDER_HEADERS` are persisted — discovery still happens -at runtime through AniWorld's redirect endpoint. - -### HLS Stream Handling - -HLS (HTTP Live Streaming) manifests (`.m3u8`) require yt-dlp to use the -`ffmpeg` downloader with `--hls-use-mpegts`. Both providers configure this -automatically: - -```python -ydl_opts = { - "downloader": "ffmpeg", # Use ffmpeg instead of native - "hls_use_mpegts": True, # Write transport stream (.ts) segments -} -``` - -**Why this matters**: Without ffmpeg, yt-dlp logs: -`"Live HLS streams are not supported by the native downloader"` - -**Requirements**: -- ffmpeg must be installed and in PATH (`which ffmpeg`) -- Install: `apt install ffmpeg` (Debian/Ubuntu) or `brew install ffmpeg` (macOS) -- Startup health check (see Health Check Endpoints) verifies ffmpeg presence - -**Trade-offs**: -- HLS downloads are slower than direct MP4 (reassembly of .ts segments) -- Requires more disk space during download -- May need post-processing if .ts format is not desired - -**Detection**: VOE provider extracts HLS URLs via `HLS_PATTERN` regex. Other -providers let yt-dlp auto-detect from URL/content-type. - -### Updating yt-dlp - -When extractors break (typical symptoms: every provider HEAD probe succeeds -but `yt-dlp` raises `Unable to extract` or `HTTP Error 404`): - -1. Check the upstream tracker first: https://github.com/yt-dlp/yt-dlp/issues -2. Upgrade in the conda environment: - ```bash - conda run -n AniWorld pip install --upgrade yt-dlp - ``` -3. Smoke-test against a known-good episode before pinning a new floor in - `requirements.txt` (`yt-dlp>=YYYY.MM.DD`). -4. Re-run the provider test suite: - ```bash - conda run -n AniWorld python -m pytest tests/unit/test_aniworld_provider.py -v - ``` -5. If a specific extractor is removed upstream, drop the provider from - `DEFAULT_PROVIDERS` rather than patching `yt-dlp` in tree. - -### User Notification on Total Failure - -`SeriesApp.download_episode()` already emits a `download_status="failed"` -WebSocket event when `loader.download()` returns `False`. Operators should -forward this to `notification_service.notify_download_failed()` so users see -a HIGH-priority alert. The loader keeps the failure detail in -`logs/download_errors.log` for post-mortem. - -## Series Storage - -### Overview - -Series metadata now stored in the database (SQLAlchemy ORM). -Legacy files (`key` and `data` per folder) are deprecated but preserved -for backward compatibility. - -### Architecture - -- **Database**: Single source of truth for all series metadata -- **In-Memory Cache**: SeriesApp maintains a cache for performance -- **Filesystem**: Only used for episode files themselves, not metadata - -### Migration - -First startup after upgrade automatically imports any legacy -series files into the database. - -### Legacy Files - -- `key` file: Contains series provider key (deprecated) -- `data` file: Contains Serie JSON object (deprecated) - -Both are safe to delete after migration; not needed for normal operation. - diff --git a/docs/InstructionsLogging.md b/docs/InstructionsLogging.md deleted file mode 100644 index 70b2e0e..0000000 --- a/docs/InstructionsLogging.md +++ /dev/null @@ -1,94 +0,0 @@ -# Logging Instructions - -This document describes how to write and refactor logging across the AniWorld codebase to make logs **human-readable**, **debug-friendly**, and **noise-free**. - -> ✅ Goal: Logs should help a developer understand what happened, why it happened, and what to inspect next — without overwhelming them with duplicates or irrelevant details. - ---- - -## 1. Principles for Great Logs - -### 1.1 Use the Right Log Level - -- `DEBUG`: Detailed internal state useful when debugging a specific issue (e.g., decision points, returned values, request/response payloads). Not for normal operation. -- `INFO`: High-level events that represent what the system is doing (e.g., "Import started", "New series added", "Config reloaded"). Use sparingly. -- `WARNING`: Something unexpected happened, but the system can continue (e.g., missing optional file, fallback behavior). -- `ERROR`: An operation failed and needs attention (e.g., exception caught, failed database write). -- `CRITICAL`: The system is in an unusable state (e.g., config corruption, failed startup). - -### 1.2 Keep Logs Human-Readable - -- Write messages in a clear, descriptive sentence-style format. -- Avoid cryptic codes or single-word log messages. -- Prefer `logger.debug("... %s", value)`-style formatting over f-strings to avoid unnecessary work when the log level is disabled. - -### 1.3 Avoid Log Spam - -- Don’t log inside hot loops unless you explicitly aggregate and log a summary (e.g., "Processed 124 files, 3 failures"). -- Avoid repeated/logging the same event at the same level (e.g., do not log "Retrying" 10 times at INFO; log once at INFO and then use DEBUG for each retry). -- Use rate limiting or debounce patterns for logs that can fire rapidly (e.g., external service health checks). -- Prefer a single higher-level log with context rather than many low-level logs that clutter output. - -### 1.4 Log Objects Usefully - -- When logging objects, log the minimal useful representation (e.g., ID, name, status) rather than the full object or its memory address. -- If an object has a `.dict()`, `.to_dict()`, or `.as_dict()` helper (common in Pydantic models), log that rather than relying on `repr()`. -- Add a `__repr__` or `__str__` implementation to domain models that returns a helpful, concise string with key identifiers. -- Use structured logging (e.g., `logger.info("Series added", extra={"series_id": series.id, "title": series.title})`) where supported. -- For exceptions, prefer `logger.exception("Failed to ...")` to capture stack traces. - ---- - -## 2. Refactoring Existing Logs - -When improving or refactoring existing log statements, aim to make them: - -- **Actionable**: A developer reading the log should know what happened and what to check next. -- **Non-redundant**: Remove duplicates and ensure only one log records the same high-level event at a given level. -- **Context-rich**: Include identifiers (e.g., `series_id`, `file_path`, `user_id`) and key state that explains why a decision was made. -- **Level-appropriate**: Downgrade noisy INFO logs to DEBUG, and elevate critical failures to ERROR/CRITICAL. - -### 2.1 Refactor Checklist - -1. **Locate noisy logs**: Search for repeated messages (e.g., "Start", "Done") and determine whether they should be DEBUG or removed. -2. **Replace ad-hoc prints**: Remove `print()` statements or `print(obj)` and replace with `logger.*` calls. -3. **Use structured context**: If a function logs multiple related messages, include the same context in each (e.g., `extra={"series_id": series.id}`) or use a context manager that attaches it. -4. **Validate object output**: Ensure any logged object produces a useful representation (add methods or translate to dict). If not, log the key fields explicitly. -5. **Batch repetitive events**: If a loop logs per item, consider collecting stats and logging a summary at the end. - -## 3. Adding New Logs - -When adding logs to new code paths: - -- Log **important state transitions** (e.g., "Queue started", "Download completed", "Config reloaded"). -- For error paths, include what failed and why (e.g., "Could not load config from X: {exc}"). -- Prefer logging at the boundaries of operations, not deep inside utility functions unless it aids debugging. -- Write logs in full sentences, with a clear subject, verb, and object. - ---- - -## 4. Example Patterns - -```python -logger.info("Import completed", extra={"series_id": series.id, "count": len(imported)}) - -logger.debug( - "Fetched feed items", - extra={"feed_url": feed.url, "item_count": len(items)}, -) - -try: - result = download_episode(episode) -except Exception: - logger.exception("Failed to download episode %s", episode.id) -``` - -> 💡 When in doubt, favor **fewer, richer logs** over many noisy logs. - ---- - -## 5. Logging Audit Task List - -For a guided checklist of files and logging improvements, see **`docs/tasks.md`**. This is where we track which files have been reviewed and which logging items still need attention. - -> ✅ After applying the guidelines above, update `docs/tasks.md` to indicate which tasks are complete. diff --git a/docs/MIGRATION_GUIDE.md b/docs/MIGRATION_GUIDE.md deleted file mode 100644 index 5a5e91b..0000000 --- a/docs/MIGRATION_GUIDE.md +++ /dev/null @@ -1,111 +0,0 @@ -# Migration Guide: File-Based to Database Storage - -## Overview - -This guide covers the transition from file-based series metadata storage to the new database-backed system introduced in v2.0. - -## What Changed - -**Before v2.0**: Series metadata stored in `key` and `data` files alongside anime folders. - -**After v2.0**: All metadata stored in SQLite database (`aniworld.db`). Files are deprecated but still supported for backward compatibility during migration. - -## Automated Migration - -The application automatically migrates on first startup: - -1. Scans anime directory for `key` and `data` files -2. Parses legacy files into `AnimeSeries` and `Episode` records -3. Loads series into in-memory cache -4. Logs migration results - -**No manual action required.** - -## Manual Verification - -After first startup with the new version: - -1. **Check logs** for: `"Migrated X series from files to DB"` -2. **Verify series count**: UI shows same number of series as before -3. **Confirm episodes**: Episode counts match expected totals - -```bash -# Check migration log -grep "Migrated" logs/app.log - -# Verify series via API -curl http://localhost:8000/api/anime | jq '.total' -``` - -## After Migration - -### Safe to Delete - -Once verified, these files can be removed: - -``` -<anime_folder>/ -├── Attack on Titan (2013)/ -│ ├── key # ❌ Can delete -│ ├── data # ❌ Can delete -│ └── Season 1/ -│ └── ... -``` - -**Deleting these files does not affect the database.** The metadata now lives in `aniworld.db`. - -### Backup (Recommended) - -Before deleting, backup the files: - -```bash -# Create backup directory -mkdir -p backup/legacy_series_files - -# Copy all key and data files -find /path/to/anime -name "key" -o -name "data" | while read f; do - cp "$f" "backup/legacy_series_files/" -done -``` - -## Reverting (Not Recommended) - -If you must revert to file-based storage: - -1. **Restore from database backup** (if available) -2. **Export manually** (no export script exists) - -**Warning**: File-based storage is deprecated and will be removed in v3.0.0. - -## Troubleshooting - -### Series Not Appearing After Migration - -1. Check logs for migration errors: `grep -i error logs/app.log` -2. Verify `key` and `data` files exist and are readable -3. Manually trigger rescan: `POST /api/scheduler/trigger-rescan` - -### Duplicate Series - -1. Check for duplicate `key` files (same series in multiple folders) -2. Verify series key uniqueness in database: - -```bash -sqlite3 aniworld.db "SELECT key, COUNT(*) FROM anime_series GROUP BY key HAVING COUNT(*) > 1;" -``` - -### Missing Episodes - -1. Trigger targeted scan for affected series -2. Check episode sync logs -3. Verify file permissions on anime directory - -## Deprecation Timeline - -| Version | Status | -|---------|--------| -| v2.0.x | Legacy files supported, migration automated | -| v2.1.x | Legacy files still supported, warnings in logs | -| v3.0.0 | **Legacy files removed** - database only | - -Upgrade to v3.0.0 before legacy file support ends. \ No newline at end of file diff --git a/docs/NAVIGATION.md b/docs/NAVIGATION.md deleted file mode 100644 index cf49aee..0000000 --- a/docs/NAVIGATION.md +++ /dev/null @@ -1,174 +0,0 @@ -# Navigation & Redirect Logic - -This document describes the setup flow navigation, covering how users progress from initial setup through to the main application. - -## Overview - -The application uses a middleware-based redirect system to ensure users complete setup before accessing the main app. The flow involves multiple pages handling setup completion, unresolved folder detection, and initialization. - -## Setup Flow - -``` -┌─────────────────────────────────────────────────────────────────────┐ -│ SETUP FLOW │ -├─────────────────────────────────────────────────────────────────────┤ -│ │ -│ /setup ──► /loading ──┬──► /setup/unresolved ──► /loading │ -│ │ │ │ │ │ -│ │ │ │ │ │ -│ ▼ ▼ ▼ ▼ │ -│ (first time) (WebSocket) (has folders) (all resolved) │ -│ │ │ │ -│ ▼ │ │ -│ /login ◄───────────────────┴──────────────────────┤ -│ │ -└─────────────────────────────────────────────────────────────────────┘ -``` - -## Middleware: SetupRedirectMiddleware - -**File:** `src/server/middleware/setup_redirect.py` - -The middleware intercepts all requests and redirects to `/setup` if: -- No master password is configured -- Configuration file is missing or invalid - -### Exempt Paths (always accessible) - -| Path | Purpose | -|------|---------| -| `/setup` | Initial setup page | -| `/setup/unresolved` | Unresolved folder resolution | -| `/loading` | Initialization progress page | -| `/login` | Authentication | -| `/api/auth/*` | Auth endpoints | -| `/api/config/*` | Config API | -| `/api/health` | Health check | -| `/static/*` | Static assets | - -### Middleware Logic - -1. **Setup incomplete** → Redirect to `/setup` -2. **Setup complete, accessing `/setup`** → Redirect to `/loading` -3. **Setup complete, accessing `/loading`** → Allow access (page handles its own redirect) -4. **API requests during setup** → Return 503 with `setup_url` - -## Pages - -### 1. Setup Page (`/setup`) - -**File:** `src/server/web/templates/setup.html` - -Handles initial configuration: -- Master password creation -- Anime directory selection -- Database initialization - -**Post-completion flow:** -- Redirects to `/loading` to begin initialization - -### 2. Loading Page (`/loading`) - -**File:** `src/server/web/templates/loading.html` - -Shows initialization progress via WebSocket: -- Series scanning -- Database population -- Logo/image loading - -**Post-initialization flow:** -```javascript -async function checkUnresolvedAndProceed() { - // Fetch unresolved folders via API - const res = await fetch('/api/setup/unresolved', { - headers: { 'Authorization': `Bearer ${token}` } - }); - const folders = await res.json(); - - if (folders.length > 0) { - // Has unresolved folders → go to resolution page - window.location.href = '/setup/unresolved'; - } else { - // No unresolved folders → go to login - window.location.href = '/login'; - } -} -``` - -### 3. Unresolved Folders Page (`/setup/unresolved`) - -**File:** `src/server/web/templates/unresolved.html` - -Allows manual resolution of folders that couldn't be auto-matched: -- Shows list of unresolved folders -- Provides search suggestions -- Input field for entering provider key -- Resolve/delete actions - -**Post-resolution flow:** -```javascript -function checkEmptyList() { - if (listEl.children.length === 0) { - // All folders resolved → return to loading - setTimeout(() => { window.location.href = '/loading'; }, 2000); - } -} -``` - -### 4. Login Page (`/login`) - -**File:** `src/server/web/templates/login.html` - -Authentication page. After successful login → redirect to `/` (main app). - -## API Endpoints - -### Unresolved Folders API - -| Method | Endpoint | Description | -|--------|----------|-------------| -| `GET` | `/api/setup/unresolved` | List all unresolved folders | -| `GET` | `/api/setup/unresolved/{folder_name}` | Get specific folder details | -| `POST` | `/api/setup/unresolved/{folder_name}/resolve` | Resolve with provider key | -| `POST` | `/api/setup/unresolved/{folder_name}/search` | Re-search for matches | -| `DELETE` | `/api/setup/unresolved/{folder_name}` | Remove folder from tracking | - -### Auth API - -| Method | Endpoint | Description | -|--------|----------|-------------| -| `POST` | `/api/auth/setup` | Create master password | -| `POST` | `/api/auth/login` | Authenticate | -| `POST` | `/api/auth/logout` | End session | - -## Key Files - -| File | Purpose | -|------|---------| -| `src/server/middleware/setup_redirect.py` | Redirect middleware | -| `src/server/controllers/page_controller.py` | Page route handlers | -| `src/server/web/templates/setup.html` | Setup template | -| `src/server/web/templates/loading.html` | Loading template | -| `src/server/web/templates/unresolved.html` | Unresolved folders template | -| `src/server/api/setup_endpoints.py` | Unresolved folders API | -| `src/server/database/service.py` | UnresolvedFolderService | - -## Common Issues - -### Redirect Loop - -**Symptom:** Browser keeps redirecting between pages. - -**Causes:** -1. `loading.html` always redirected to `/setup/unresolved` without checking if any exist -2. `unresolved.html` redirected to `/` which middleware redirected back to `/login` - -**Fix:** See the navigation logic updates in loading.html and unresolved.html. - -### Can't Access Unresolved Page After Setup - -**Symptom:** Middleware redirects to `/login` instead of allowing access to `/setup/unresolved`. - -**Cause:** `/setup/unresolved` is in the exempt paths but the request may not be reaching it due to completion check timing. - -**Fix:** The middleware allows access to `/loading` which handles the redirect to `/setup/unresolved` after initialization. \ No newline at end of file diff --git a/docs/NFO_GUIDE.md b/docs/NFO_GUIDE.md deleted file mode 100644 index 7da522e..0000000 --- a/docs/NFO_GUIDE.md +++ /dev/null @@ -1,905 +0,0 @@ -# NFO Metadata Guide - -## Document Purpose - -This guide explains how to use the NFO metadata feature to enrich your anime library with TMDB metadata and artwork for Plex, Jellyfin, Emby, and Kodi. - ---- - -## 1. Overview - -### What are NFO Files? - -NFO files are XML documents that contain metadata about TV shows and episodes. Media servers like Plex, Jellyfin, Emby, and Kodi use these files to display information about your library without needing to scrape external sources. - -### Features - -- **Automatic NFO Creation**: Generate NFO files during downloads -- **TMDB Integration**: Fetch metadata from The Movie Database -- **Image Downloads**: Poster, fanart, and logo images -- **Batch Operations**: Create/update NFO files for multiple anime -- **Web UI**: Manage NFO settings and operations -- **API Access**: Programmatic NFO management - ---- - -## 2. Getting Started - -### 2.1 Obtain TMDB API Key - -1. Create a free account at https://www.themoviedb.org -2. Navigate to https://www.themoviedb.org/settings/api -3. Request an API key (select "Developer" option) -4. Copy your API key (v3 auth) - -### 2.2 Configure NFO Settings - -#### Via Web Interface - -1. Open http://127.0.0.1:8000 -2. Click **Configuration** button -3. Scroll to **NFO Settings** section -4. Enter your TMDB API key -5. Click **Test Connection** to verify -6. Configure options: - - **Auto-create during downloads**: Enable to create NFO files automatically - - **Update on library scan**: Enable to refresh existing NFO files - - **Download poster**: Episode and show poster images (poster.jpg) - - **Download logo**: Show logo images (logo.png) - - **Download fanart**: Background artwork (fanart.jpg) - - **Image size**: Select w500 (recommended), w780, or original -7. Click **Save** - -#### Via Environment Variables - -Add to your `.env` file: - -```bash -TMDB_API_KEY=your_api_key_here -NFO_AUTO_CREATE=true -NFO_UPDATE_ON_SCAN=false -NFO_DOWNLOAD_POSTER=true -NFO_DOWNLOAD_LOGO=false -NFO_DOWNLOAD_FANART=false -NFO_IMAGE_SIZE=w500 -``` - -#### Via config.json - -Edit `data/config.json`: - -```json -{ - "nfo": { - "tmdb_api_key": "your_api_key_here", - "auto_create": true, - "update_on_scan": false, - "download_poster": true, - "download_logo": false, - "download_fanart": false, - "image_size": "w500" - } -} -``` - ---- - -## 3. Using NFO Features - -### 3.1 Automatic NFO Creation - -With `auto_create` enabled, NFO files are created automatically when downloading episodes: - -1. Add episodes to download queue -2. Start queue processing -3. NFO files are created after successful downloads -4. Images are downloaded based on configuration - -### 3.2 Manual NFO Creation - -#### Via Web Interface - -1. Navigate to the main page -2. Click **Create NFO** button next to an anime -3. Wait for completion notification - -#### Via API - -```bash -curl -X POST "http://127.0.0.1:8000/api/nfo/create" \ - -H "Authorization: Bearer YOUR_JWT_TOKEN" \ - -H "Content-Type: application/json" \ - -d '{ - "anime_id": 123, - "folder_path": "/path/to/anime/Attack on Titan" - }' -``` - -### 3.3 Batch NFO Creation - -Create NFO files for multiple anime at once: - -```bash -curl -X POST "http://127.0.0.1:8000/api/nfo/batch/create" \ - -H "Authorization: Bearer YOUR_JWT_TOKEN" \ - -H "Content-Type: application/json" \ - -d '{ - "anime_ids": [123, 456, 789] - }' -``` - -### 3.4 Update Existing NFO Files - -Update NFO files with latest TMDB metadata: - -```bash -curl -X POST "http://127.0.0.1:8000/api/nfo/update" \ - -H "Authorization: Bearer YOUR_JWT_TOKEN" \ - -H "Content-Type: application/json" \ - -d '{ - "anime_id": 123, - "folder_path": "/path/to/anime/Attack on Titan", - "force": true - }' -``` - -### 3.5 Check NFO Status - -Check which anime have NFO files: - -```bash -curl -X GET "http://127.0.0.1:8000/api/nfo/check?folder_path=/path/to/anime" \ - -H "Authorization: Bearer YOUR_JWT_TOKEN" -``` - -Response: - -```json -{ - "has_tvshow_nfo": true, - "episode_nfos": [ - { - "season": 1, - "episode": 1, - "has_nfo": true, - "file_path": "/path/to/anime/Season 1/S01E01.nfo" - } - ], - "missing_episodes": [], - "total_episodes": 25, - "nfo_count": 25 -} -``` - -### 3.6 Fallback Behavior When TMDB is Unavailable - -When TMDB lookup fails (network issues, API errors, or no match found), the system creates a **minimal NFO** to ensure the series is still tracked. This behavior applies to: - -- Manual NFO creation via API -- Batch NFO creation operations -- Automatic NFO creation during downloads - -**What a minimal NFO contains:** - -```xml -<?xml version="1.0" encoding="UTF-8" standalone="yes"?> -<tvshow> - <title>Series Name - 2024 - No metadata available for Series Name. TMDB lookup failed. - -``` - -**Limitations of minimal NFOs:** -- No poster, logo, or fanart images -- No rating, genre, or studio information -- No TMDB or other provider IDs -- May not display correctly in some media servers - -**To upgrade a minimal NFO:** -1. Use the Update endpoint (`PUT /api/nfo/{serie_id}/update`) when TMDB is available -2. Or delete the NFO and recreate it with full metadata - ---- - -## 4. File Structure - -### 4.1 NFO File Locations - -NFO files are created in the anime directory: - -``` -/path/to/anime/Attack on Titan/ -├── tvshow.nfo # Show metadata -├── poster.jpg # Show poster (optional) -├── logo.png # Show logo (optional) -├── fanart.jpg # Show fanart (optional) -├── Season 1/ -│ ├── S01E01.mkv -│ ├── S01E01.nfo # Episode metadata -│ ├── S01E01-thumb.jpg # Episode thumbnail (optional) -│ ├── S01E02.mkv -│ └── S01E02.nfo -└── Season 2/ - ├── S02E01.mkv - └── S02E01.nfo -``` - -### 4.2 tvshow.nfo Format - -```xml - - - Attack on Titan - 進撃の巨人 - Attack on Titan - Attack on Titan - 8.5 - 2013 - Humans are nearly exterminated by giant creatures... - 24 - TV-MA - 2013-04-07 - Ended - Wit Studio - Animation - Action - Sci-Fi & Fantasy - 1429 - 1429 - https://image.tmdb.org/t/p/w500/... - - https://image.tmdb.org/t/p/original/... - - -``` - -**Manual TMDB ID Override**: To skip TMDB search and use a specific ID directly, include `YOUR_ID` in the NFO. This is useful when: -- TMDB search fails for your series (e.g., new or obscure anime) -- You already know the correct TMDB ID -- You want to avoid rate limiting from repeated searches - -Aniworld reads `` element and `` first. If found, it uses the ID directly instead of searching. - -### 4.3 Episode NFO Format - -```xml - - - To You, in 2000 Years: The Fall of Shiganshina, Part 1 - Attack on Titan - 1 - 1 - 1 - 1 - After a hundred years of peace... - 24 - 2013-04-07 - 8.2 - 63056 - https://image.tmdb.org/t/p/w500/... - -``` - ---- - -## 5. Folder Naming Convention - -### 5.1 Expected Format - -After the daily folder scan (when **Update on library scan** is enabled), Aniworld validates every series folder against its `tvshow.nfo` metadata. If the folder name does not match the expected convention, it is automatically renamed. - -**Format:** - -``` -{title} ({year}) -``` - -**Examples:** - -| NFO `` | NFO `<year>` | Expected Folder Name | -|---------------|--------------|----------------------| -| `Attack on Titan` | `2013` | `Attack on Titan (2013)` | -| `One Piece` | `1999` | `One Piece (1999)` | -| `Demon Slayer: Kimetsu no Yaiba` | `2019` | `Demon Slayer Kimetsu no Yaiba (2019)` | - -### 5.2 Sanitization Rules - -Illegal filesystem characters are removed or replaced to ensure cross-platform compatibility: - -- Removed: `< > : " / \ | ? *` and null bytes -- Control characters stripped -- Multiple spaces collapsed to one -- Leading/trailing dots and whitespace trimmed -- Maximum length: 200 characters (truncated at word boundary if possible) - -### 5.3 Skip Conditions - -A folder is **not** renamed when any of the following apply: - -- `tvshow.nfo` is missing `<title>` or `<year>` (or they are empty) -- The series has an **active or pending download** -- The target folder name already exists (duplicate) -- The resulting path would exceed the OS path-length limit -- The app lacks write permission to the anime directory - -All skipped and renamed actions are logged. - ---- - -## 6. Poster Check - -### 6.1 Overview - -During the daily folder scan, Aniworld checks every series folder for a valid `poster.jpg`. If the file is missing or smaller than 1 KB, the application attempts to re-download it from the URL stored in the series' `tvshow.nfo` file. - -### 6.2 How It Works - -1. **Scan** — After folder renaming, the scan iterates over all series folders that contain a `tvshow.nfo`. -2. **Validate** — For each folder, it checks whether `poster.jpg` exists and is at least 1 KB. -3. **Parse NFO** — If the poster is missing or too small, the scan reads `tvshow.nfo` and looks for a `<thumb aspect="poster">` (or any `<thumb>`) URL. -4. **Download** — If a URL is found, the poster is downloaded using `ImageDownloader` with a concurrency limit of 3 simultaneous downloads. -5. **Validate Download** — The downloaded image is validated with PIL to ensure it is not corrupted. - -### 6.3 Skip Conditions - -A folder is **not** processed for poster download when any of the following apply: - -- `tvshow.nfo` does not exist in the folder. -- `poster.jpg` already exists and is ≥ 1 KB. -- No `<thumb>` URL is found in the NFO (the NFO may have been created before thumb tags were added). -- The `nfo.download_poster` setting is `false` (poster checks are still performed, but downloads are skipped if the setting is disabled; see [CONFIGURATION.md](CONFIGURATION.md)). - -### 6.4 Logging - -Every poster check action is logged: - -- **INFO** — When a poster is successfully downloaded. -- **WARNING** — When a download fails or no URL is found. -- **ERROR** — When an unexpected exception occurs during download. - ---- - -## 7. API Reference - -### 5.1 Check NFO Status - -**Endpoint**: `GET /api/nfo/check` - -**Query Parameters**: - -- `folder_path` (required): Absolute path to anime directory - -**Response**: - -```json -{ - "has_tvshow_nfo": true, - "episode_nfos": [ - { - "season": 1, - "episode": 1, - "has_nfo": true, - "file_path": "/path/to/S01E01.nfo" - } - ], - "missing_episodes": [], - "total_episodes": 25, - "nfo_count": 25 -} -``` - -### 5.2 Create NFO Files - -**Endpoint**: `POST /api/nfo/create` - -**Request Body**: - -```json -{ - "anime_id": 123, - "folder_path": "/path/to/anime/Attack on Titan" -} -``` - -**Response**: - -```json -{ - "success": true, - "message": "NFO files created successfully", - "files_created": ["tvshow.nfo", "S01E01.nfo", "S01E02.nfo"], - "images_downloaded": ["poster.jpg", "S01E01-thumb.jpg"] -} -``` - -### 5.3 Update NFO Files - -**Endpoint**: `POST /api/nfo/update` - -**Request Body**: - -```json -{ - "anime_id": 123, - "folder_path": "/path/to/anime", - "force": false -} -``` - -**Response**: - -```json -{ - "success": true, - "message": "NFO files updated successfully", - "files_updated": ["tvshow.nfo", "S01E01.nfo"] -} -``` - -### 5.4 View NFO Content - -**Endpoint**: `GET /api/nfo/view` - -**Query Parameters**: - -- `file_path` (required): Absolute path to NFO file - -**Response**: - -```json -{ - "content": "<?xml version=\"1.0\"...?>", - "file_path": "/path/to/tvshow.nfo", - "exists": true -} -``` - -### 5.5 Get Media Status - -**Endpoint**: `GET /api/nfo/media/status` - -**Query Parameters**: - -- `folder_path` (required): Absolute path to anime directory - -**Response**: - -```json -{ - "poster_exists": true, - "poster_path": "/path/to/poster.jpg", - "logo_exists": false, - "logo_path": null, - "fanart_exists": true, - "fanart_path": "/path/to/fanart.jpg", - "episode_thumbs": [ - { - "season": 1, - "episode": 1, - "exists": true, - "path": "/path/to/S01E01-thumb.jpg" - } - ] -} -``` - -### 5.6 Download Media - -**Endpoint**: `POST /api/nfo/media/download` - -**Request Body**: - -```json -{ - "folder_path": "/path/to/anime", - "anime_id": 123, - "download_poster": true, - "download_logo": false, - "download_fanart": false, - "image_size": "w500" -} -``` - -**Response**: - -```json -{ - "success": true, - "message": "Media downloaded successfully", - "downloaded": ["poster.jpg", "S01E01-thumb.jpg"] -} -``` - -### 5.7 Batch Create NFO - -**Endpoint**: `POST /api/nfo/batch/create` - -**Request Body**: - -```json -{ - "anime_ids": [123, 456, 789] -} -``` - -**Response**: - -```json -{ - "success": true, - "results": [ - { - "anime_id": 123, - "success": true, - "message": "Created successfully" - }, - { - "anime_id": 456, - "success": false, - "error": "Folder not found" - } - ] -} -``` - -### 5.8 Find Missing NFOs - -**Endpoint**: `GET /api/nfo/missing` - -**Response**: - -```json -{ - "anime_list": [ - { - "anime_id": 123, - "title": "Attack on Titan", - "folder_path": "/path/to/anime/Attack on Titan", - "missing_tvshow_nfo": false, - "missing_episode_count": 3, - "total_episodes": 25 - } - ] -} -``` - ---- - -## 6. Troubleshooting - -### 6.1 NFO Files Not Created - -**Problem**: NFO files are not being created during downloads. - -**Solutions**: - -1. Verify TMDB API key is configured correctly -2. Check `auto_create` is enabled in settings -3. Ensure anime directory has write permissions -4. Check logs for error messages -5. Test TMDB connection using "Test Connection" button - -### 6.2 Invalid TMDB API Key - -**Problem**: TMDB validation fails with "Invalid API key". - -**Solutions**: - -1. Verify API key is copied correctly (no extra spaces) -2. Ensure you're using the v3 API key (not v4) -3. Check API key is active on TMDB website -4. Try regenerating API key on TMDB - -### 6.3 Images Not Downloading - -**Problem**: NFO files are created but images are missing. - -**Solutions**: - -1. Enable image downloads in settings (poster/logo/fanart) -2. Verify TMDB API key is valid -3. Check network connectivity to TMDB servers -4. Ensure sufficient disk space -5. Check file permissions in anime directory - -### 6.4 Incorrect Metadata - -**Problem**: NFO contains wrong show information. - -**Solutions**: - -1. Verify anime title matches TMDB exactly -2. Use TMDB ID if available for accurate matching -3. Update NFO files with `force=true` to refresh metadata -4. Check TMDB website for correct show information - -### 6.5 Permission Errors - -**Problem**: "Permission denied" when creating NFO files. - -**Solutions**: - -1. Check anime directory permissions: `chmod 755 /path/to/anime` -2. Ensure application user has write access -3. Verify directory ownership: `chown -R user:group /path/to/anime` -4. Check parent directories are accessible - -### 6.6 Slow NFO Creation - -**Problem**: NFO creation takes a long time. - -**Solutions**: - -1. Reduce image size (use w500 instead of original) -2. Disable unnecessary images (logo, fanart) -3. Create NFOs in batches during off-peak hours -4. Check network speed to TMDB servers -5. Verify disk I/O performance - -### 6.7 TMDB Lookup Fails for My Series - -**Problem**: TMDB search fails with "No results found" for a valid series. - -**Solutions**: - -1. **Check if series exists on TMDB**: Visit https://www.themoviedb.org and search for your series -2. **Use manual ID override**: Add TMDB ID directly to `tvshow.nfo`: - ```xml - <?xml version="1.0" encoding="UTF-8" standalone="yes"?> - <tvshow> - <title>Your Series Name - 12345 - 12345 - - ``` - Aniworld will use this ID directly instead of searching. - -3. **Try alternative titles**: Some anime have different titles (Japanese, romaji, English). If you have access to the folder, rename it to match the TMDB title. - -4. **Add to existing NFO**: If `tvshow.nfo` exists but has no TMDB ID, edit it to add: - ```xml - YOUR_TMDB_ID - ``` - Then use the Update endpoint to refresh metadata. - -5. **Check for rate limiting**: If many lookups fail at once, you may be hitting TMDB rate limits. Wait and retry later. - -6. **Verify API key**: Ensure your TMDB API key is valid and has not exceeded usage limits. - ---- - -## 7. Best Practices - -### 7.1 Configuration Recommendations - -- **Image Size**: Use `w500` for optimal balance of quality and storage -- **Auto-create**: Enable for new downloads -- **Update on scan**: Disable to avoid unnecessary TMDB API calls -- **Poster**: Always enable for show and episode thumbnails -- **Logo/Fanart**: Enable only if your media server supports them - -### 7.2 Maintenance - -- **Regular Updates**: Update NFO files quarterly to get latest metadata -- **Backup**: Include NFO files in your backup strategy -- **Validation**: Periodically check missing NFOs using `/api/nfo/missing` -- **API Rate Limits**: Be mindful of TMDB API rate limits when batch processing - -### 7.3 Performance - -- **Batch Operations**: Use batch endpoints for multiple anime -- **Off-Peak Processing**: Create NFOs during low-activity periods -- **Image Optimization**: Use smaller image sizes for large libraries -- **Selective Updates**: Only update NFOs when metadata changes - -### 7.4 Media Server Integration - -#### Plex - -- Use "Personal Media Shows" agent -- Enable "Local Media Assets" scanner -- Place NFO files in anime directories -- Refresh metadata after creating NFOs - -#### Jellyfin - -- Use "NFO" metadata provider -- Enable in Library settings -- Order providers: NFO first, then online sources -- Scan library after NFO creation - -#### Emby - -- Enable "NFO" metadata reader -- Configure in Library advanced settings -- Use "Prefer embedded metadata" option -- Refresh metadata after updates - -#### Kodi - -- NFO files are automatically detected -- No additional configuration needed -- Update library to see changes - ---- - -## 8. Advanced Usage - -### 8.1 Custom NFO Templates - -You can customize NFO generation by modifying the NFO service: - -```python -# src/core/services/nfo_creator.py -def generate_tvshow_nfo(self, metadata: dict) -> str: - # Add custom fields or modify structure - pass -``` - -### 8.2 Bulk Operations - -Create NFOs for entire library: - -```bash -# Get all anime without NFOs -curl -X GET "http://127.0.0.1:8000/api/nfo/missing" \ - -H "Authorization: Bearer $TOKEN" \ - | jq -r '.anime_list[].anime_id' \ - | xargs -I{} curl -X POST "http://127.0.0.1:8000/api/nfo/batch/create" \ - -H "Authorization: Bearer $TOKEN" \ - -H "Content-Type: application/json" \ - -d '{"anime_ids": [{}]}' -``` - -### 8.3 Scheduled Updates - -Use the scheduler API to refresh NFOs automatically: - -```bash -# Schedule weekly NFO updates (rescan runs Sunday at 03:00) -curl -X POST "http://127.0.0.1:8000/api/scheduler/config" \ - -H "Authorization: Bearer $TOKEN" \ - -H "Content-Type: application/json" \ - -d '{ - "enabled": true, - "schedule_time": "03:00", - "schedule_days": ["sun"], - "auto_download_after_rescan": false - }' -``` - ---- - -## 9. Related Documentation - -- [API.md](API.md) - Complete API reference -- [CONFIGURATION.md](CONFIGURATION.md) - All configuration options -- [ARCHITECTURE.md](ARCHITECTURE.md) - System architecture -- [DEVELOPMENT.md](DEVELOPMENT.md) - Development guide - ---- - -## 10. Tag Reference - -The table below lists every XML tag written to `tvshow.nfo` and its source in -the TMDB API response. All tags are written whenever the NFO is created or -updated via `create_tvshow_nfo()` / `update_tvshow_nfo()`. - -| NFO tag | TMDB source field | Required | -| --------------- | ----------------------------------------------------- | -------- | -| `title` | `name` | ✅ | -| `originaltitle` | `original_name` | ✅ | -| `showtitle` | `name` (same as `title`) | ✅ | -| `sorttitle` | `name` (same as `title`) | ✅ | -| `year` | First 4 chars of `first_air_date` | ✅ | -| `plot` | `overview` | ✅ | -| `outline` | `overview` (same as `plot`) | ✅ | -| `tagline` | `tagline` | optional | -| `runtime` | `episode_run_time[0]` | ✅ | -| `premiered` | `first_air_date` | ✅ | -| `status` | `status` | ✅ | -| `mpaa` | US content rating from `content_ratings.results` | optional | -| `fsk` | DE content rating (written as `mpaa` when preferred) | optional | -| `imdbid` | `external_ids.imdb_id` | ✅ | -| `tmdbid` | `id` | ✅ | -| `tvdbid` | `external_ids.tvdb_id` | optional | -| `genre` | `genres[].name` (one element per genre) | ✅ | -| `studio` | `networks[].name` (one element per network) | ✅ | -| `country` | `origin_country[]` or `production_countries[].name` | ✅ | -| `actor` | `credits.cast[]` (top 10, with name/role/thumb) | ✅ | -| `watched` | Always `false` on creation | ✅ | -| `dateadded` | System clock at creation time (`YYYY-MM-DD HH:MM:SS`) | ✅ | - -The mapping logic lives in `src/core/utils/nfo_mapper.py` (`tmdb_to_nfo_model`). -The XML serialisation lives in `src/core/utils/nfo_generator.py` -(`generate_tvshow_nfo`). - ---- - -## 11. Automatic NFO Repair - -NFO repair now runs as part of the scheduled daily folder scan rather than on every -startup. When the scheduler triggers `FolderScanService.run_folder_scan()`, the first -step is `perform_nfo_repair_scan(background_loader=None)`. Each incomplete NFO is -queued as a background `asyncio` task, so the scan returns quickly while repairs -continue asynchronously. - -### How It Works - -1. **Scan** — `perform_nfo_repair_scan()` in - `src/server/services/initialization_service.py` is called from - `FolderScanService.run_folder_scan()` (`src/server/services/folder_scan_service.py`). -2. **Detect** — `nfo_needs_repair(nfo_path)` from - `src/core/services/nfo_repair_service.py` parses each `tvshow.nfo` with - `lxml` and checks for the 13 required tags listed below. -3. **Repair** — Series whose NFO is incomplete are queued for background reload - via `asyncio.create_task`. Each task creates its own isolated - :class:`NFOService` / :class:`TMDBClient` so concurrent tasks never share an - ``aiohttp`` session — this prevents "Connector is closed" errors when many repairs - run in parallel. A semaphore caps TMDB concurrency at 3 to stay within rate limits. - -### Tags Checked (13 required) - -| XPath | Tag name | -| ----------------- | --------------- | -| `./title` | `title` | -| `./originaltitle` | `originaltitle` | -| `./year` | `year` | -| `./plot` | `plot` | -| `./runtime` | `runtime` | -| `./premiered` | `premiered` | -| `./status` | `status` | -| `./imdbid` | `imdbid` | -| `./genre` | `genre` | -| `./studio` | `studio` | -| `./country` | `country` | -| `./actor/name` | `actor/name` | -| `./watched` | `watched` | - -### Log Messages - -| Message | Meaning | -| ----------------------------------------------------------- | ------------------------------------------------- | -| `NFO repair scan complete: 0 of N series queued for repair` | All NFOs are complete — no action needed | -| `NFO repair scan complete: X of N series queued for repair` | X series had incomplete NFOs and have been queued | -| `NFO repair scan skipped: TMDB API key not configured` | Set `tmdb_api_key` in `data/config.json` | -| `NFO repair scan skipped: anime directory not configured` | Set `anime_directory` in `data/config.json` | - -### Triggering a Manual Repair - -You can also repair a single series on demand via the API: - -```http -POST /api/nfo/update/{series_key} -``` - -This calls `NFOService.update_tvshow_nfo()` directly and overwrites the existing -`tvshow.nfo` with fresh data from TMDB. - -### Source Files - -| File | Purpose | -| ----------------------------------------------- | ---------------------------------------------------------------------------------------------- | -| `src/core/services/nfo_repair_service.py` | `REQUIRED_TAGS`, `parse_nfo_tags`, `find_missing_tags`, `nfo_needs_repair`, `NfoRepairService` | -| `src/server/services/folder_scan_service.py` | `perform_nfo_repair_scan` — invoked during the scheduled daily folder scan | - ---- - -## 12. Support - -### Getting Help - -- Check logs in `logs/` directory for error details -- Review [TESTING.md](TESTING.md) for test coverage -- Consult [DATABASE.md](DATABASE.md) for NFO status schema - -### Common Issues - -See section 6 (Troubleshooting) for solutions to common problems. - -### TMDB Resources - -- TMDB API Documentation: https://developers.themoviedb.org/3 -- TMDB Support: https://www.themoviedb.org/talk -- TMDB API Status: https://status.themoviedb.org/ diff --git a/docs/README.md b/docs/README.md deleted file mode 100644 index 5c57b87..0000000 --- a/docs/README.md +++ /dev/null @@ -1,39 +0,0 @@ -# Aniworld Documentation - -## Overview - -This directory contains all documentation for the Aniworld anime download manager project. - -## Documentation Structure - -| Document | Purpose | Target Audience | -| ---------------------------------------- | ---------------------------------------------- | ---------------------------------- | -| [ARCHITECTURE.md](ARCHITECTURE.md) | System architecture and design decisions | Architects, Senior Developers | -| [API.md](API.md) | REST API reference and WebSocket documentation | Frontend Developers, API Consumers | -| [DEVELOPMENT.md](DEVELOPMENT.md) | Developer setup and contribution guide | All Developers | -| [DEPLOYMENT.md](DEPLOYMENT.md) | Deployment and operations guide | DevOps, System Administrators | -| [DATABASE.md](DATABASE.md) | Database schema and data models | Backend Developers | -| [TESTING.md](TESTING.md) | Testing strategy and guidelines | QA Engineers, Developers | -| [SECURITY.md](SECURITY.md) | Security considerations and guidelines | Security Engineers, All Developers | -| [CONFIGURATION.md](CONFIGURATION.md) | Configuration options reference | Operators, Developers | -| [CHANGELOG.md](CHANGELOG.md) | Version history and changes | All Stakeholders | -| [TROUBLESHOOTING.md](TROUBLESHOOTING.md) | Common issues and solutions | Support, Operators | -| [features.md](features.md) | Feature list and capabilities | Product Owners, Users | -| [instructions.md](instructions.md) | AI agent development instructions | AI Agents, Developers | - -## Documentation Standards - -- All documentation uses Markdown format -- Keep documentation up-to-date with code changes -- Include code examples where applicable -- Use clear, concise language -- Include diagrams for complex concepts (use Mermaid syntax) - -## Contributing to Documentation - -When adding or updating documentation: - -1. Follow the established format in each document -2. Update the README.md if adding new documents -3. Ensure cross-references are valid -4. Review for spelling and grammar diff --git a/docs/TESTING.md b/docs/TESTING.md deleted file mode 100644 index d923887..0000000 --- a/docs/TESTING.md +++ /dev/null @@ -1,146 +0,0 @@ -# Testing Documentation - -## Document Purpose - -This document describes the testing strategy, guidelines, and practices for the Aniworld project. - -### What This Document Contains - -- **Testing Strategy**: Overall approach to quality assurance -- **Test Categories**: Unit, integration, API, performance, security tests -- **Test Structure**: Organization of test files and directories -- **Writing Tests**: Guidelines for writing effective tests -- **Fixtures and Mocking**: Shared test utilities and mock patterns -- **Running Tests**: Commands and configurations -- **Coverage Requirements**: Minimum coverage thresholds -- **CI/CD Integration**: How tests run in automation -- **Test Data Management**: Managing test fixtures and data -- **Best Practices**: Do's and don'ts for testing - -### What This Document Does NOT Contain - -- Production deployment (see [DEPLOYMENT.md](DEPLOYMENT.md)) -- Security audit procedures (see [SECURITY.md](SECURITY.md)) -- Bug tracking and issue management -- Performance benchmarking results - -### Target Audience - -- Developers writing tests -- QA Engineers -- CI/CD Engineers -- Code reviewers - ---- - -## Sections to Document - -1. Testing Philosophy - - Test pyramid approach - - Quality gates -2. Test Categories - - Unit Tests (`tests/unit/`) - - Integration Tests (`tests/integration/`) - - API Tests (`tests/api/`) - - Frontend Tests (`tests/frontend/`) - - Performance Tests (`tests/performance/`) - - Security Tests (`tests/security/`) -3. Test Structure and Naming - - File naming conventions - - Test function naming - - Test class organization -4. Running Tests - - pytest commands - - Running specific tests - - Verbose output - - Coverage reports -5. Fixtures and Conftest - - Shared fixtures - - Database fixtures - - Mock services -6. Mocking Guidelines - - What to mock - - Mock patterns - - External service mocks - -### Mocking the Download Queue - -Use `MockQueueRepository` for testing download queue functionality: - -```python -from src.server.models.download import DownloadItem, EpisodeIdentifier - -class MockQueueRepository: - def __init__(self): - self._items: Dict[str, DownloadItem] = {} -``` - -### Testing SetupService - -SetupService handles series key resolution from folder names during library setup. Test file: `tests/unit/test_setup_service.py`. - -Key methods tested: -- `_extract_year_from_folder_name()` — parses `(YYYY)` suffix -- `_extract_title_from_folder_name()` — strips year suffix -- `_resolve_key_via_search()` — resolves provider key via fuzzy title matching - -```python -@pytest.mark.asyncio -async def test_returns_key_when_single_exact_match(self): - """Search returns 1 result with same name → returns key.""" - mock_series_app = AsyncMock() - mock_series_app.search.return_value = [ - {'title': 'Attack on Titan', 'link': '/anime/stream/attack-on-titan'} - ] - - with patch('src.server.services.setup_service.get_series_app', return_value=mock_series_app): - result = await SetupService._resolve_key_via_search("Attack on Titan") - - assert result == 'attack-on-titan' -``` - -### Mocking aiohttp Sessions - -When testing code that uses `aiohttp.ClientSession`: - -```python -from unittest.mock import AsyncMock, MagicMock, patch -from aiohttp import ClientSession - -# Mock aiohttp session for testing -class MockAiohttpSession: - def __init__(self): - self.closed = False - - async def close(self): - self.closed = True - - def get(self, url, **kwargs): - mock_response = AsyncMock() - mock_response.status = 200 - mock_response.json = AsyncMock(return_value={"data": "test"}) - mock_response.__aenter__ = AsyncMock(return_value=mock_response) - mock_response.__aexit__ = AsyncMock(return_value=None) - return mock_response - -# Use in fixture -@pytest.fixture -async def mock_tmdb_session(): - session = MockAiohttpSession() - yield session - # Cleanup verification - assert session.closed, "Session was not closed" -``` - -**Key points:** -- Always verify `session.closed` is `True` after context manager exits -- Mock `__aenter__` and `__aexit__` for response context managers -- Set `closed = False` on mock session for unclosed warning tests - -7. Coverage Requirements -8. CI/CD Integration -9. Writing Good Tests - - Arrange-Act-Assert pattern - - Test isolation - - Edge cases -10. Common Pitfalls to Avoid diff --git a/docs/diagrams/README.md b/docs/diagrams/README.md deleted file mode 100644 index 1e6144d..0000000 --- a/docs/diagrams/README.md +++ /dev/null @@ -1,23 +0,0 @@ -# Architecture Diagrams - -This directory contains architecture diagram source files for the Aniworld documentation. - -## Diagrams - -### System Architecture (Mermaid) - -See [system-architecture.mmd](system-architecture.mmd) for the system overview diagram. - -### Rendering - -Diagrams can be rendered using: - -- Mermaid Live Editor: https://mermaid.live/ -- VS Code Mermaid extension -- GitHub/GitLab native Mermaid support - -## Formats - -- `.mmd` - Mermaid diagram source files -- `.svg` - Exported vector graphics (add when needed) -- `.png` - Exported raster graphics (add when needed) diff --git a/docs/diagrams/download-flow.mmd b/docs/diagrams/download-flow.mmd deleted file mode 100644 index 66800ea..0000000 --- a/docs/diagrams/download-flow.mmd +++ /dev/null @@ -1,44 +0,0 @@ -%%{init: {'theme': 'base'}}%% -sequenceDiagram - participant Client - participant FastAPI - participant AuthMiddleware - participant DownloadService - participant ProgressService - participant WebSocketService - participant SeriesApp - participant Database - - Note over Client,Database: Download Flow - - %% Add to queue - Client->>FastAPI: POST /api/queue/add - FastAPI->>AuthMiddleware: Validate JWT - AuthMiddleware-->>FastAPI: OK - FastAPI->>DownloadService: add_to_queue() - DownloadService->>Database: save_item() - Database-->>DownloadService: item_id - DownloadService-->>FastAPI: [item_ids] - FastAPI-->>Client: 201 Created - - %% Start queue - Client->>FastAPI: POST /api/queue/start - FastAPI->>AuthMiddleware: Validate JWT - AuthMiddleware-->>FastAPI: OK - FastAPI->>DownloadService: start_queue_processing() - - loop For each pending item - DownloadService->>SeriesApp: download_episode() - - loop Progress updates - SeriesApp->>ProgressService: emit("progress_updated") - ProgressService->>WebSocketService: broadcast_to_room() - WebSocketService-->>Client: WebSocket message - end - - SeriesApp-->>DownloadService: completed - DownloadService->>Database: update_status() - end - - DownloadService-->>FastAPI: OK - FastAPI-->>Client: 200 OK diff --git a/docs/diagrams/system-architecture.mmd b/docs/diagrams/system-architecture.mmd deleted file mode 100644 index 83ca2ce..0000000 --- a/docs/diagrams/system-architecture.mmd +++ /dev/null @@ -1,88 +0,0 @@ -%%{init: {'theme': 'base', 'themeVariables': { 'primaryColor': '#4a90d9'}}}%% -flowchart TB - subgraph Clients["Client Layer"] - Browser["Web Browser
(HTML/CSS/JS)"] - CLI["CLI Client
(Main.py)"] - end - - subgraph Server["Server Layer (FastAPI)"] - direction TB - Middleware["Middleware
Auth, Rate Limit, Error Handler"] - - subgraph API["API Routers"] - AuthAPI["/api/auth"] - AnimeAPI["/api/anime"] - QueueAPI["/api/queue"] - ConfigAPI["/api/config"] - SchedulerAPI["/api/scheduler"] - HealthAPI["/health"] - WebSocketAPI["/ws"] - end - - subgraph Services["Services"] - AuthService["AuthService"] - AnimeService["AnimeService"] - DownloadService["DownloadService"] - ConfigService["ConfigService"] - ProgressService["ProgressService"] - WebSocketService["WebSocketService"] - end - end - - subgraph Core["Core Layer"] - SeriesApp["SeriesApp"] - SeriesCache["SeriesCache
(In-Memory)"] - SerieScanner["SerieScanner"] - SerieList["SerieList"] - end - - subgraph Data["Data Layer"] - SQLite[("SQLite
aniworld.db")] - ConfigJSON[(config.json)] - FileSystem[(File System
Anime Episodes)] - LegacyFiles[("Legacy Files
key/data
(Deprecated)")] - end - - subgraph External["External"] - Provider["Anime Provider
(aniworld.to)"] - end - - %% Client connections - Browser -->|HTTP/WebSocket| Middleware - CLI -->|Direct| SeriesApp - - %% Middleware to API - Middleware --> API - - %% API to Services - AuthAPI --> AuthService - AnimeAPI --> AnimeService - QueueAPI --> DownloadService - ConfigAPI --> ConfigService - SchedulerAPI --> AnimeService - WebSocketAPI --> WebSocketService - - %% Services to Core - AnimeService --> SeriesApp - DownloadService --> SeriesApp - - %% Services to Data - AuthService --> ConfigJSON - ConfigService --> ConfigJSON - DownloadService --> SQLite - AnimeService --> SQLite - - %% Core to Data - SeriesApp --> SeriesCache - SeriesCache -.->|Cached Series| SQLite - SeriesApp --> SerieScanner - SeriesApp --> SerieList - SerieScanner -->|Scan Episodes| FileSystem - SerieScanner -->|Detect Series| SQLite - SerieScanner -->|Migrate Legacy| LegacyFiles - SerieScanner --> Provider - - %% Event flow - ProgressService -.->|Events| WebSocketService - DownloadService -.->|Progress| ProgressService - WebSocketService -.->|Broadcast| Browser diff --git a/docs/features.md b/docs/features.md deleted file mode 100644 index 90f28f9..0000000 --- a/docs/features.md +++ /dev/null @@ -1,118 +0,0 @@ -# Aniworld Web Application Features - -## Recent Updates - -### Enhanced Setup and Settings Pages (Latest) - -The application now features a comprehensive configuration system that allows users to configure all settings during initial setup or modify them later through the settings modal: - -**Setup Page Enhancements:** - -- Single-page setup with all configuration options organized into clear sections -- Real-time password strength indicator for security -- Form validation with helpful error messages -- Comprehensive settings including: general, security, scheduler, logging, backup, and NFO metadata - -**Settings Modal Enhancements:** - -- All configuration fields are now editable through the main application's config modal -- Organized into logical sections with clear labels and help text -- Real-time saving with immediate feedback -- Configuration validation to prevent invalid settings -- Full control over cron-based scheduler (time, days of week, auto-download), logging options, and backup settings - ---- - -## Authentication & Security - -- **Master Password Login**: Secure access to the application with a master password system -- **JWT Token Sessions**: Stateless authentication with JSON Web Tokens -- **Rate Limiting**: Built-in protection against brute force attacks - -## Configuration Management - -- **Enhanced Setup Page**: Comprehensive initial configuration interface with all settings in one place: - - General Settings: Application name and data directory configuration - - Security Settings: Master password setup with strength indicator - - Anime Directory: Primary directory path for anime storage - - Scheduler Settings: Enable/disable scheduler, configure daily run time, select days of week, and optionally auto-download missing episodes after rescan - - Logging Settings: Configure log level, file path, file size limits, and backup count - - Backup Settings: Enable automatic backups with configurable path and retention period - - NFO Settings: TMDB API key, auto-creation options, and media file download preferences -- **Enhanced Settings/Config Modal**: Comprehensive configuration interface accessible from main page: - - General Settings: Edit application name and data directory - - Anime Directory: Modify anime storage location with browse functionality - - Scheduler Configuration: Enable/disable, set cron run time (`HH:MM`), select active days of the week, and toggle auto-download after rescan - - Logging Configuration: Full control over logging level, file rotation, and backup count - - Backup Configuration: Configure automatic backup settings including path and retention - - NFO Settings: Complete control over TMDB integration and media file downloads - - Configuration Validation: Validate configuration for errors before saving - - Backup Management: Create, restore, and manage configuration backups - - Export/Import: Export configuration for backup or transfer to another instance - -## User Interface - -- **Dark Mode**: Toggle between light and dark themes for better user experience -- **Responsive Design**: Mobile-friendly interface with touch support -- **Real-time Updates**: WebSocket-based live notifications and progress tracking - -## Anime Management - -- **Anime Library Page**: Display list of anime series with missing episodes -- **Library Filters**: - - "Missing Episodes Only" (shows only series with missing episodes, including series that currently have no downloaded episodes) - - "No Episodes" (shows series that are present in the library but have zero downloaded episodes) - - "Show All Series" (overrides other filters to show every series) -- **Database-Backed Series Storage**: All series metadata and missing episodes stored in SQLite database -- **Automatic Database Synchronization**: Series loaded from database on startup, stays in sync with filesystem -- **Series Selection**: Select individual anime series and add episodes to download queue -- **Anime Search**: Search for anime series using integrated providers -- **Library Scanning**: Automated scanning for missing episodes with database persistence -- **Episode Tracking**: Missing episodes tracked in database, automatically updated during scans -- **NFO Status Indicators**: Visual badges showing NFO and media file status for each series - -## NFO Metadata Management - -- **TMDB Integration**: Automatic metadata fetching from The Movie Database (TMDB) -- **Auto-Create NFO Files**: Automatically generate tvshow.nfo files during downloads -- **Media File Downloads**: Automatic download of poster.jpg, logo.png, and fanart.jpg -- **NFO Status Tracking**: Database tracking of NFO creation and update timestamps -- **Manual NFO Creation**: Create NFO files and download media for existing anime -- **NFO Updates**: Update existing NFO files with latest TMDB metadata -- **Batch Operations**: Create NFO files for multiple anime at once -- **NFO Content Viewing**: View generated NFO file content in the UI -- **Media Server Compatibility**: Kodi, Plex, Jellyfin, and Emby compatible format -- **Configuration Options**: Customize which media files to download and image quality - -## Download Management - -- **Download Queue Page**: View and manage the current download queue with organized sections -- **Queue Organization**: Displays downloads organized by status (pending, active, completed, failed) -- **NFO Integration**: Automatic NFO and media file creation before episode downloads -- **Manual Start/Stop Control**: User manually starts downloads one at a time with Start/Stop buttons -- **FIFO Queue Processing**: First-in, first-out queue order (no priority or reordering) -- **Single Download Mode**: Only one download active at a time, new downloads must be manually started -- **Download Status Display**: Real-time status updates and progress of current download -- **Queue Operations**: Add and remove items from the pending queue -- **Completed Downloads List**: Separate section for completed downloads with clear button -- **Failed Downloads List**: Separate section for failed downloads with retry and clear options -- **Retry Failed Downloads**: Automatically retry failed downloads with configurable limits -- **Clear Completed**: Remove completed downloads from the queue -- **Clear Failed**: Remove failed downloads from the queue -- **Queue Statistics**: Real-time counters for pending, active, completed, and failed items - -## Real-time Communication - -- **WebSocket Support**: Real-time notifications for download progress and queue updates -- **Progress Tracking**: Live progress updates for downloads and scans -- **System Notifications**: Real-time system messages and alerts - -## Folder Management - -- **Fuzzy Series Key Resolution**: Automatic series key resolution from folder names using fuzzy title matching — tolerates title variations like `(TV)`, `(OVA)`, `(Movie)` suffixes and uses similarity matching to resolve provider keys during library setup - -## Core Functionality Overview - -The web application provides a complete interface for managing anime downloads with user-friendly pages for configuration, library management, search capabilities, and download monitoring. All operations are tracked in real-time with comprehensive progress reporting and error handling. - -**NFO Metadata Features**: The application now includes full support for generating Kodi/Plex/Jellyfin/Emby compatible metadata files (tvshow.nfo) with automatic TMDB integration. NFO files are created automatically during downloads or can be managed manually through the UI. The system tracks NFO status in the database and provides comprehensive API endpoints for programmatic access. Media files (poster, logo, fanart) are automatically downloaded based on configuration settings. diff --git a/docs/helper b/docs/helper deleted file mode 100644 index e69de29..0000000 diff --git a/docs/instructions.md b/docs/instructions.md deleted file mode 100644 index ef23c73..0000000 --- a/docs/instructions.md +++ /dev/null @@ -1,119 +0,0 @@ -# Aniworld Web Application Development Instructions - -This document provides detailed tasks for AI agents to implement a modern web application for the Aniworld anime download manager. All tasks should follow the coding guidelines specified in the project's copilot instructions. - -## Project Overview - -The goal is to create a FastAPI-based web application that provides a modern interface for the existing Aniworld anime download functionality. The core anime logic should remain in `SeriesApp.py` while the web layer provides REST API endpoints and a responsive UI. - -## Architecture Principles - -- **Single Responsibility**: Each file/class has one clear purpose -- **Dependency Injection**: Use FastAPI's dependency system -- **Clean Separation**: Web layer calls core logic, never the reverse -- **File Size Limit**: Maximum 500 lines per file -- **Type Hints**: Use comprehensive type annotations -- **Error Handling**: Proper exception handling and logging - -## Additional Implementation Guidelines - -### Code Style and Standards - -- **Type Hints**: Use comprehensive type annotations throughout all modules -- **Docstrings**: Follow PEP 257 for function and class documentation -- **Error Handling**: Implement custom exception classes with meaningful messages -- **Logging**: Use structured logging with appropriate log levels -- **Security**: Validate all inputs and sanitize outputs -- **Performance**: Use async/await patterns for I/O operations - -## 📞 Escalation - -If you encounter: - -- Architecture issues requiring design decisions -- Tests that conflict with documented requirements -- Breaking changes needed -- Unclear requirements or expectations - -**Document the issue and escalate rather than guessing.** - ---- - -## � Credentials - -**Admin Login:** - -- Username: `admin` -- Password: `Hallo123!` - ---- - -## �📚 Helpful Commands - -```bash -# Run all tests -conda run -n AniWorld python -m pytest tests/ -v --tb=short - -# Run specific test file -conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py -v - -# Run specific test class -conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py::TestWebSocketService -v - -# Run specific test -conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py::TestWebSocketService::test_broadcast_download_progress -v - -# Run with extra verbosity -conda run -n AniWorld python -m pytest tests/ -vv - -# Run with full traceback -conda run -n AniWorld python -m pytest tests/ -v --tb=long - -# Run and stop at first failure -conda run -n AniWorld python -m pytest tests/ -v -x - -# Run tests matching pattern -conda run -n AniWorld python -m pytest tests/ -v -k "auth" - -# Show all print statements -conda run -n AniWorld python -m pytest tests/ -v -s - -#Run app -conda run -n AniWorld python -m uvicorn src.server.fastapi_app:app --host 127.0.0.1 --port 8000 --reload -``` - ---- - -## Implementation Notes - -1. **Incremental Development**: Implement features incrementally, testing each component thoroughly before moving to the next -2. **Code Review**: Review all generated code for adherence to project standards -3. **Documentation**: Document all public APIs and complex logic -4. **Testing**: Maintain test coverage above 80% for all new code -5. **Performance**: Profile and optimize critical paths, especially download and streaming operations -6. **Security**: Regular security audits and dependency updates -7. **Monitoring**: Implement comprehensive monitoring and alerting -8. **Maintenance**: Plan for regular maintenance and updates - ---- - -## Task Completion Checklist - -For each task completed: - -- [ ] Implementation follows coding standards -- [ ] Unit tests written and passing -- [ ] Integration tests passing -- [ ] Documentation updated -- [ ] Error handling implemented -- [ ] Logging added -- [ ] Security considerations addressed -- [ ] Performance validated -- [ ] Code reviewed -- [ ] Task marked as complete in instructions.md -- [ ] Infrastructure.md updated and other docs -- [ ] Changes committed to git; keep your messages in git short and clear -- [ ] Take the next task - ---- - diff --git a/docs/key b/docs/key deleted file mode 100644 index 5582160..0000000 --- a/docs/key +++ /dev/null @@ -1,51 +0,0 @@ -API key : 299ae8f630a31bda814263c551361448 - -/mnt/server/serien/Serien/ - -{ - "name": "Aniworld", - "data_dir": "data", - "scheduler": { - "enabled": true, - "interval_minutes": 60, - "schedule_time": "03:00", - "schedule_days": [ - "mon", - "tue", - "wed", - "thu", - "fri", - "sat", - "sun" - ], - "auto_download_after_rescan": true, - "folder_scan_enabled": true - }, - "logging": { - "level": "INFO", - "file": null, - "max_bytes": null, - "backup_count": 3 - }, - "backup": { - "enabled": false, - "path": "data/backups", - "keep_days": 30 - }, - "nfo": { - "tmdb_api_key": "9bc3e547caff878615cbdba2cc421d37", - "auto_create": true, - "update_on_scan": true, - "download_poster": true, - "download_logo": true, - "download_fanart": true, - "image_size": "original" - }, - "other": { - "master_password_hash": "$pbkdf2-sha256$29000$HQNASKk1xpgTAgAgJGRMaQ$73TOCCM0UEZONyNXQEPa3SmIoXeG6C1l5mMFDNgYfMQ", - "anime_directory": "/data" - }, - "version": "1.0.0" -} - - diff --git a/docs/runner.csx b/docs/runner.csx deleted file mode 100644 index 45ae2a1..0000000 --- a/docs/runner.csx +++ /dev/null @@ -1,154 +0,0 @@ -#!/usr/bin/env dotnet-script -#nullable enable - -using System; -using System.IO; -using System.Diagnostics; -using System.Threading; -using System.Text; -using System.Text.RegularExpressions; -using System.Linq; -using System.Collections.Generic; - -// ── Ctrl+C: kill active process and exit cleanly ────────────────────────────── -var cts = new CancellationTokenSource(); -Process? activeProcess = null; - -Console.CancelKeyPress += (_, e) => -{ - e.Cancel = true; - Console.WriteLine("\n[runner] Interrupted — shutting down..."); - cts.Cancel(); - try { activeProcess?.Kill(entireProcessTree: true); } catch { } -}; - -// ── Paths ───────────────────────────────────────────────────────────────────── -var repoRoot = Directory.GetCurrentDirectory(); -var tasksFile = Path.Combine(repoRoot, "Docs", "Tasks.md"); - -if (!File.Exists(tasksFile)) -{ - Console.Error.WriteLine($"[runner] ERROR: Tasks.md not found at {tasksFile}"); - Console.Error.WriteLine("[runner] Run this script from the repository root."); - Environment.Exit(1); -} - -// ── Read & split by "---" separator lines ──────────────────────────────────── -var content = File.ReadAllText(tasksFile); -var items = Regex - .Split(content, @"\r?\n---\r?\n") - .Select(s => s.Trim()) - .Where(s => s.Length > 0) - .ToList(); - -Console.WriteLine($"[runner] Found {items.Count} section(s) in Tasks.md"); - -// ── Helper: run copilot and stream output, return full output ───────────────── -async Task RunCopilot(IEnumerable extraArgs, string prompt) -{ - var output = new StringBuilder(); - - var argList = new List { "launch", "copilot", "--model", "minimax-m2.7:cloud", "--yes", "--", "--allow-all-tools" }; - argList.AddRange(extraArgs); - argList.Add("-p"); - argList.Add(prompt); - - var psi = new ProcessStartInfo("ollama") - { - WorkingDirectory = repoRoot, - RedirectStandardOutput = true, - RedirectStandardError = true, - UseShellExecute = false, - }; - foreach (var a in argList) - psi.ArgumentList.Add(a); - - activeProcess = new Process { StartInfo = psi }; - - activeProcess.OutputDataReceived += (_, e) => - { - if (e.Data is null) return; - Console.WriteLine(e.Data); - output.AppendLine(e.Data); - }; - activeProcess.ErrorDataReceived += (_, e) => - { - if (e.Data is null) return; - Console.Error.WriteLine(e.Data); - output.AppendLine(e.Data); - }; - - activeProcess.Start(); - activeProcess.BeginOutputReadLine(); - activeProcess.BeginErrorReadLine(); - - await activeProcess.WaitForExitAsync(cts.Token); - activeProcess = null; - - return output.ToString(); -} - -// ── Main loop ───────────────────────────────────────────────────────────────── -for (int i = 0; i < items.Count; i++) -{ - var item = items[i]; - if (cts.IsCancellationRequested) break; - - Console.WriteLine(); - Console.WriteLine("[runner] ══════════════════════════════════════════════"); - Console.WriteLine($"[runner] Task:\n{item}"); - Console.WriteLine("[runner] ══════════════════════════════════════════════"); - Console.WriteLine(); - - // Step 1 — run the task prompt - await RunCopilot(Enumerable.Empty(), $"/caveman full"); - await RunCopilot(new[] { "--continue" }, $"read ./Docs/instructions.md. {item}"); - if (cts.IsCancellationRequested) break; - - // Step 2 — confirm completion in the same chat session - Console.WriteLine("\n[runner] Asking for task confirmation...\n"); - var confirmation = await RunCopilot( - new[] { "--continue" }, - "are you sure tasks is done. reply with yes" - ); - if (cts.IsCancellationRequested) break; - - // Step 3 — check for "yes" in the reply, with retry logic for issue resolution - int maxRetries = 3; - int retryCount = 0; - bool taskConfirmed = confirmation.Contains("yes", StringComparison.OrdinalIgnoreCase); - - while (!taskConfirmed && retryCount < maxRetries) - { - retryCount++; - Console.WriteLine($"\n[runner] Attempt {retryCount}/{maxRetries}: Resolving remaining issues and running tests...\n"); - - confirmation = await RunCopilot( - new[] { "--continue" }, - "resolve any remaining issues, make sure all tests are running and pass. then confirm with yes if done" - ); - if (cts.IsCancellationRequested) break; - - taskConfirmed = confirmation.Contains("yes", StringComparison.OrdinalIgnoreCase); - } - - if (!taskConfirmed) - { - Console.WriteLine($"\n[runner] Task not confirmed as done after {maxRetries} attempts. Stopping."); - break; - } - - // Step 4 — commit the work - Console.WriteLine("\n[runner] Task confirmed. Making git commit...\n"); - - await RunCopilot(Enumerable.Empty(), $"/caveman full"); - await RunCopilot(new[] { "--continue" }, "make git commit"); - if (cts.IsCancellationRequested) break; - - // Step 5 — remove completed task from Tasks.md - var remaining = items.Skip(i + 1).ToList(); - File.WriteAllText(tasksFile, string.Join("\n\n---\n\n", remaining)); - Console.WriteLine("[runner] Removed completed task from Tasks.md"); -} - -Console.WriteLine("\n[runner] Finished."); diff --git a/docs/tasks.md b/docs/tasks.md deleted file mode 100644 index 0dafa03..0000000 --- a/docs/tasks.md +++ /dev/null @@ -1,178 +0,0 @@ -# Tasks - -## 1. Scheduled Folder Scan - -### Task 1.1: Add folder scan scheduler configuration - -**Where is that found** -- `src/server/models/config.py` (`SchedulerConfig`) -- `data/config.json` (example/default config) -- `src/server/web/templates/setup.html` (setup UI) -- `src/server/api/auth.py` (config save endpoint, if it validates scheduler fields) - -**Goal. How it should be** -Add a new boolean field `folder_scan_enabled` (default `false`) to `SchedulerConfig`. When `true`, the scheduler will execute the folder maintenance routine during its scheduled run. Add the field to the setup page as a checkbox. Ensure existing configs without this field load successfully (Pydantic default handles this). - -**Possible traps and issues** -- Backward compatibility: old `data/config.json` files must load without errors. Pydantic defaults solve this, but verify by loading an old config. -- The setup page JavaScript must include the new field in the payload sent to `/api/config`. -- Do not confuse this with `auto_download_after_rescan` — this is a separate toggle. - -**Docs changes needed** -- `docs/CONFIGURATION.md`: Document the new `scheduler.folder_scan_enabled` option. -- `docs/ARCHITECTURE.md`: Mention folder scan in the scheduler section. - -**Why this is needed** -Users need an opt-in toggle to enable automatic daily folder maintenance (NFO repair, folder renaming, poster checks) without forcing it on everyone. - ---- - -### Task 1.2: Create FolderScanService skeleton - -**Where is that found** -- New file: `src/server/services/folder_scan_service.py` -- `src/server/services/scheduler_service.py` (to call it) - -**Goal. How it should be** -Create a new `FolderScanService` class with a single async entry point `async def run_folder_scan(self) -> None`. The method should: -1. Log start/completion with structlog. -2. Check prerequisites (`settings.anime_directory` exists, `settings.tmdb_api_key` is set). -3. Skip gracefully with a warning log if prerequisites are missing. -4. Use a module-level semaphore (similar to `_NFO_REPAIR_SEMAPHORE`) to limit concurrent TMDB operations to 3. - -Keep the implementation empty for the sub-tasks (1.3–1.5) to fill in. Just add the skeleton and the semaphore. - -**Possible traps and issues** -- Circular imports: `folder_scan_service.py` will import from `initialization_service`, `config.settings`, etc. Keep imports inside methods or at the bottom if circular issues arise. -- The service should follow the singleton pattern like `SchedulerService` and `DownloadService` if it holds state, or be stateless. For simplicity, make it a plain class instantiated per call or a module-level function set. -- Exception handling: any unhandled exception in the scheduled task should be caught and logged so it doesn't crash the scheduler. - -**Docs changes needed** -- `docs/ARCHITECTURE.md`: Add `folder_scan_service.py` to the services list. - -**Why this is needed** -Encapsulates the new daily maintenance logic in its own module, keeping `scheduler_service.py` clean and allowing the folder scan to be tested independently. - ---- - -### Task 1.3: Integrate NFO repair into folder scan - -**Where is that found** -- `src/server/services/folder_scan_service.py` -- `src/server/services/initialization_service.py` (`perform_nfo_repair_scan`) - -**Goal. How it should be** -Inside `FolderScanService.run_folder_scan()`, call `perform_nfo_repair_scan(background_loader=None)` as the first step. Reuse the existing function exactly — do not copy its logic. Log a message before and after the call. - -**Possible traps and issues** -- `perform_nfo_repair_scan` spawns `asyncio.create_task` for each repair. When called from the scheduler, these background tasks will still run after `run_folder_scan` returns. This is fine, but log that repairs are queued. -- The function already handles missing `tmdb_api_key` and `anime_directory`, so the caller doesn't need to double-check, but the skeleton from Task 1.2 already checks prerequisites. -- `perform_nfo_repair_scan` imports `nfo_needs_repair` and `NfoRepairService` inside the function, so no heavy import-time dependencies. - -**Docs changes needed** -- `docs/NFO_GUIDE.md`: Update the "Automatic NFO Repair" section to state that repair now runs as part of the scheduled folder scan instead of every startup. - -**Why this is needed** -Reuses the existing, tested NFO repair logic. Moves NFO repair from startup blocking to scheduled background maintenance. - ---- - -### Task 1.4: Validate and rename series folders - -**Where is that found** -- `src/server/services/folder_scan_service.py` -- `src/core/services/nfo_repair_service.py` (for `parse_nfo_tags` or similar NFO parsing) -- `src/server/database/models.py` / `src/server/database/system_settings_service.py` (if folder paths are stored in DB) - -**Goal. How it should be** -After NFO repair, iterate over every subfolder in `settings.anime_directory` that contains a `tvshow.nfo`. For each folder: -1. Parse the NFO to extract `` and `<year>` text values. -2. Compute the expected folder name: `f"{title} ({year})"`. -3. Sanitize the expected name for filesystem safety (remove/replace illegal characters like `/`, `\`, `:`, etc.). -4. Compare with the current folder name (`series_dir.name`). -5. If different, rename the folder using `series_dir.rename(expected_path)`. -6. If the series path is stored in the database (check `anime_service` or DB models), update the database record to point to the new path. - -Skip folders where title or year is missing/empty. Log every rename action. - -**Possible traps and issues** -- **Database path consistency**: If `Series` or `Episode` models store absolute or relative paths, renaming the folder on disk without updating the DB will break downloads, NFO updates, and the web UI. Must verify whether paths are stored in the DB and update them. -- **Active downloads**: A series currently being downloaded should not be renamed. Check the download queue or lock status before renaming. If no lock mechanism exists, this is a major trap — document it. -- **Filesystem permissions**: The app may not have write permission to the anime directory. Catch `PermissionError` and `OSError` and log gracefully. -- **Special characters**: Titles like `"A / B"` or `"Show: Subtitle"` contain characters illegal in folder names. Define a sanitization function (e.g., replace `/` with `-`, remove trailing dots on Windows, etc.). -- **Duplicate names**: Two different series could sanitize to the same name. Check if target path already exists before renaming. -- **Path length limits**: Very long titles might exceed OS path limits. - -**Docs changes needed** -- `docs/NFO_GUIDE.md`: Add a section "Folder Naming Convention" explaining the `<title> (<year>)` format. -- `docs/CONFIGURATION.md`: Mention that enabling folder scan will rename folders. - -**Why this is needed** -Enforces a consistent, predictable folder naming scheme across the library, making it easier for media center apps (Kodi, Jellyfin, Plex) to match metadata. - ---- - -### Task 1.5: Check and download missing poster.jpg - -**Where is that found** -- `src/server/services/folder_scan_service.py` -- `src/core/utils/image_downloader.py` (`ImageDownloader`) -- `src/core/services/nfo_service.py` or `src/core/services/nfo_repair_service.py` (to get poster URL from NFO or TMDB) - -**Goal. How it should be** -After folder renaming, iterate over series folders again (or combine with Task 1.4 loop). For each folder: -1. Check if `poster.jpg` exists and has a size ≥ `ImageDownloader.min_file_size` (1 KB by default). -2. If missing or too small: - a. Parse `tvshow.nfo` for `<thumb aspect="poster">` or `<thumb>` URL. - b. If no URL in NFO, skip (do not query TMDB again to keep tasks small; the NFO should already have it after repair). - c. Use `ImageDownloader` (with context manager) to download the image to `series_dir / "poster.jpg"`. - d. Validate the downloaded image with `ImageDownloader._validate_image` (or similar existing validation). -3. Use the existing `_NFO_REPAIR_SEMAPHORE` or a new `POSTER_DOWNLOAD_SEMAPHORE` to limit concurrent downloads to 3. - -**Possible traps and issues** -- **TMDB rate limiting**: Even downloading images hits TMDB CDN. The semaphore limits concurrency. -- **Invalid images**: A download might produce a 0-byte or corrupted file. `ImageDownloader` already validates with PIL; reuse that. -- **NFO without thumb URL**: If the NFO was created before thumb tags were added, there may be no URL. In that case, skip and log. A future task could query TMDB directly. -- **Write permissions**: Same as Task 1.4. -- **Async session sharing**: `ImageDownloader` manages its own `aiohttp` session. Use `async with ImageDownloader() as downloader:` to ensure cleanup. - -**Docs changes needed** -- `docs/NFO_GUIDE.md`: Add "Poster Check" subsection under folder scan. -- `docs/CONFIGURATION.md`: Mention that `nfo.download_poster` setting also affects scheduled poster checks. - -**Why this is needed** -Ensures every series has artwork, which is required by most media center front-ends for a polished library view. - ---- - -## 2. Remove startup NFO repair - -### Task 2.1: Remove perform_nfo_repair_scan from startup lifespan - -**Where is that found** -- `src/server/fastapi_app.py` (lifespan startup block, lines ~245 and ~319) -- `src/server/services/initialization_service.py` (keep the function, just remove the call site) -- `tests/integration/test_nfo_repair_startup.py` -- `tests/unit/test_initialization_service.py` (tests that call `perform_nfo_repair_scan` directly can stay, but integration tests verifying startup wiring must change) - -**Goal. How it should be** -1. In `src/server/fastapi_app.py`, remove the import of `perform_nfo_repair_scan` from the `initialization_service` import block. -2. Remove the line `await perform_nfo_repair_scan(background_loader)` from the lifespan startup sequence. -3. Update `tests/integration/test_nfo_repair_startup.py`: - - Remove or modify `test_perform_nfo_repair_scan_imported_in_lifespan` and `test_perform_nfo_repair_scan_called_after_media_scan` since the startup wiring is gone. - - Replace with a test that verifies `perform_nfo_repair_scan` is NOT called during startup (or simply delete the file if it has no other purpose). -4. `tests/unit/test_initialization_service.py` tests for `perform_nfo_repair_scan` can remain because they test the function itself, not the startup wiring. - -**Possible traps and issues** -- **Test failures**: `test_nfo_repair_startup.py` will fail immediately after the code change. It must be updated in the same PR. -- **Documentation drift**: `docs/NFO_GUIDE.md`, `docs/CHANGELOG.md`, and `docs/ARCHITECTURE.md` all describe the startup NFO repair behavior. If docs are not updated, users will expect repair on every start. -- **Background loader parameter**: The `background_loader` variable was created partly for `perform_nfo_repair_scan`. After removal, check if `background_loader` is still needed for other startup steps (yes — `perform_media_scan_if_needed` uses it). Do not remove `background_loader` entirely. -- **Import cleanup**: Ensure no unused imports remain in `fastapi_app.py` after removal. - -**Docs changes needed** -- `docs/NFO_GUIDE.md`: Update section 11 "Automatic NFO Repair" to remove startup references and state it runs via scheduler. -- `docs/CHANGELOG.md`: Add an entry under "Changed" or "Removed" noting that startup NFO repair is replaced by scheduled folder scan. -- `docs/ARCHITECTURE.md`: Update the startup sequence description. - -**Why this is needed** -Running `perform_nfo_repair_scan` on every startup slows down server restarts, especially for large libraries. Moving it to a scheduled task keeps startup fast while still ensuring regular maintenance.