Lukas
486c5440f2
Added documentation for API, architecture, configuration, database, development guide, testing, and navigation. Includes helper scripts, diagrams, and guides for NFO files and migration.
179 lines
11 KiB
Markdown
179 lines
11 KiB
Markdown
# Tasks
|
||
|
||
## 1. Scheduled Folder Scan
|
||
|
||
### Task 1.1: Add folder scan scheduler configuration
|
||
|
||
**Where is that found**
|
||
- `src/server/models/config.py` (`SchedulerConfig`)
|
||
- `data/config.json` (example/default config)
|
||
- `src/server/web/templates/setup.html` (setup UI)
|
||
- `src/server/api/auth.py` (config save endpoint, if it validates scheduler fields)
|
||
|
||
**Goal. How it should be**
|
||
Add a new boolean field `folder_scan_enabled` (default `false`) to `SchedulerConfig`. When `true`, the scheduler will execute the folder maintenance routine during its scheduled run. Add the field to the setup page as a checkbox. Ensure existing configs without this field load successfully (Pydantic default handles this).
|
||
|
||
**Possible traps and issues**
|
||
- Backward compatibility: old `data/config.json` files must load without errors. Pydantic defaults solve this, but verify by loading an old config.
|
||
- The setup page JavaScript must include the new field in the payload sent to `/api/config`.
|
||
- Do not confuse this with `auto_download_after_rescan` — this is a separate toggle.
|
||
|
||
**Docs changes needed**
|
||
- `docs/CONFIGURATION.md`: Document the new `scheduler.folder_scan_enabled` option.
|
||
- `docs/ARCHITECTURE.md`: Mention folder scan in the scheduler section.
|
||
|
||
**Why this is needed**
|
||
Users need an opt-in toggle to enable automatic daily folder maintenance (NFO repair, folder renaming, poster checks) without forcing it on everyone.
|
||
|
||
---
|
||
|
||
### Task 1.2: Create FolderScanService skeleton
|
||
|
||
**Where is that found**
|
||
- New file: `src/server/services/folder_scan_service.py`
|
||
- `src/server/services/scheduler_service.py` (to call it)
|
||
|
||
**Goal. How it should be**
|
||
Create a new `FolderScanService` class with a single async entry point `async def run_folder_scan(self) -> None`. The method should:
|
||
1. Log start/completion with structlog.
|
||
2. Check prerequisites (`settings.anime_directory` exists, `settings.tmdb_api_key` is set).
|
||
3. Skip gracefully with a warning log if prerequisites are missing.
|
||
4. Use a module-level semaphore (similar to `_NFO_REPAIR_SEMAPHORE`) to limit concurrent TMDB operations to 3.
|
||
|
||
Keep the implementation empty for the sub-tasks (1.3–1.5) to fill in. Just add the skeleton and the semaphore.
|
||
|
||
**Possible traps and issues**
|
||
- Circular imports: `folder_scan_service.py` will import from `initialization_service`, `config.settings`, etc. Keep imports inside methods or at the bottom if circular issues arise.
|
||
- The service should follow the singleton pattern like `SchedulerService` and `DownloadService` if it holds state, or be stateless. For simplicity, make it a plain class instantiated per call or a module-level function set.
|
||
- Exception handling: any unhandled exception in the scheduled task should be caught and logged so it doesn't crash the scheduler.
|
||
|
||
**Docs changes needed**
|
||
- `docs/ARCHITECTURE.md`: Add `folder_scan_service.py` to the services list.
|
||
|
||
**Why this is needed**
|
||
Encapsulates the new daily maintenance logic in its own module, keeping `scheduler_service.py` clean and allowing the folder scan to be tested independently.
|
||
|
||
---
|
||
|
||
### Task 1.3: Integrate NFO repair into folder scan
|
||
|
||
**Where is that found**
|
||
- `src/server/services/folder_scan_service.py`
|
||
- `src/server/services/initialization_service.py` (`perform_nfo_repair_scan`)
|
||
|
||
**Goal. How it should be**
|
||
Inside `FolderScanService.run_folder_scan()`, call `perform_nfo_repair_scan(background_loader=None)` as the first step. Reuse the existing function exactly — do not copy its logic. Log a message before and after the call.
|
||
|
||
**Possible traps and issues**
|
||
- `perform_nfo_repair_scan` spawns `asyncio.create_task` for each repair. When called from the scheduler, these background tasks will still run after `run_folder_scan` returns. This is fine, but log that repairs are queued.
|
||
- The function already handles missing `tmdb_api_key` and `anime_directory`, so the caller doesn't need to double-check, but the skeleton from Task 1.2 already checks prerequisites.
|
||
- `perform_nfo_repair_scan` imports `nfo_needs_repair` and `NfoRepairService` inside the function, so no heavy import-time dependencies.
|
||
|
||
**Docs changes needed**
|
||
- `docs/NFO_GUIDE.md`: Update the "Automatic NFO Repair" section to state that repair now runs as part of the scheduled folder scan instead of every startup.
|
||
|
||
**Why this is needed**
|
||
Reuses the existing, tested NFO repair logic. Moves NFO repair from startup blocking to scheduled background maintenance.
|
||
|
||
---
|
||
|
||
### Task 1.4: Validate and rename series folders
|
||
|
||
**Where is that found**
|
||
- `src/server/services/folder_scan_service.py`
|
||
- `src/core/services/nfo_repair_service.py` (for `parse_nfo_tags` or similar NFO parsing)
|
||
- `src/server/database/models.py` / `src/server/database/system_settings_service.py` (if folder paths are stored in DB)
|
||
|
||
**Goal. How it should be**
|
||
After NFO repair, iterate over every subfolder in `settings.anime_directory` that contains a `tvshow.nfo`. For each folder:
|
||
1. Parse the NFO to extract `<title>` and `<year>` text values.
|
||
2. Compute the expected folder name: `f"{title} ({year})"`.
|
||
3. Sanitize the expected name for filesystem safety (remove/replace illegal characters like `/`, `\`, `:`, etc.).
|
||
4. Compare with the current folder name (`series_dir.name`).
|
||
5. If different, rename the folder using `series_dir.rename(expected_path)`.
|
||
6. If the series path is stored in the database (check `anime_service` or DB models), update the database record to point to the new path.
|
||
|
||
Skip folders where title or year is missing/empty. Log every rename action.
|
||
|
||
**Possible traps and issues**
|
||
- **Database path consistency**: If `Series` or `Episode` models store absolute or relative paths, renaming the folder on disk without updating the DB will break downloads, NFO updates, and the web UI. Must verify whether paths are stored in the DB and update them.
|
||
- **Active downloads**: A series currently being downloaded should not be renamed. Check the download queue or lock status before renaming. If no lock mechanism exists, this is a major trap — document it.
|
||
- **Filesystem permissions**: The app may not have write permission to the anime directory. Catch `PermissionError` and `OSError` and log gracefully.
|
||
- **Special characters**: Titles like `"A / B"` or `"Show: Subtitle"` contain characters illegal in folder names. Define a sanitization function (e.g., replace `/` with `-`, remove trailing dots on Windows, etc.).
|
||
- **Duplicate names**: Two different series could sanitize to the same name. Check if target path already exists before renaming.
|
||
- **Path length limits**: Very long titles might exceed OS path limits.
|
||
|
||
**Docs changes needed**
|
||
- `docs/NFO_GUIDE.md`: Add a section "Folder Naming Convention" explaining the `<title> (<year>)` format.
|
||
- `docs/CONFIGURATION.md`: Mention that enabling folder scan will rename folders.
|
||
|
||
**Why this is needed**
|
||
Enforces a consistent, predictable folder naming scheme across the library, making it easier for media center apps (Kodi, Jellyfin, Plex) to match metadata.
|
||
|
||
---
|
||
|
||
### Task 1.5: Check and download missing poster.jpg
|
||
|
||
**Where is that found**
|
||
- `src/server/services/folder_scan_service.py`
|
||
- `src/core/utils/image_downloader.py` (`ImageDownloader`)
|
||
- `src/core/services/nfo_service.py` or `src/core/services/nfo_repair_service.py` (to get poster URL from NFO or TMDB)
|
||
|
||
**Goal. How it should be**
|
||
After folder renaming, iterate over series folders again (or combine with Task 1.4 loop). For each folder:
|
||
1. Check if `poster.jpg` exists and has a size ≥ `ImageDownloader.min_file_size` (1 KB by default).
|
||
2. If missing or too small:
|
||
a. Parse `tvshow.nfo` for `<thumb aspect="poster">` or `<thumb>` URL.
|
||
b. If no URL in NFO, skip (do not query TMDB again to keep tasks small; the NFO should already have it after repair).
|
||
c. Use `ImageDownloader` (with context manager) to download the image to `series_dir / "poster.jpg"`.
|
||
d. Validate the downloaded image with `ImageDownloader._validate_image` (or similar existing validation).
|
||
3. Use the existing `_NFO_REPAIR_SEMAPHORE` or a new `POSTER_DOWNLOAD_SEMAPHORE` to limit concurrent downloads to 3.
|
||
|
||
**Possible traps and issues**
|
||
- **TMDB rate limiting**: Even downloading images hits TMDB CDN. The semaphore limits concurrency.
|
||
- **Invalid images**: A download might produce a 0-byte or corrupted file. `ImageDownloader` already validates with PIL; reuse that.
|
||
- **NFO without thumb URL**: If the NFO was created before thumb tags were added, there may be no URL. In that case, skip and log. A future task could query TMDB directly.
|
||
- **Write permissions**: Same as Task 1.4.
|
||
- **Async session sharing**: `ImageDownloader` manages its own `aiohttp` session. Use `async with ImageDownloader() as downloader:` to ensure cleanup.
|
||
|
||
**Docs changes needed**
|
||
- `docs/NFO_GUIDE.md`: Add "Poster Check" subsection under folder scan.
|
||
- `docs/CONFIGURATION.md`: Mention that `nfo.download_poster` setting also affects scheduled poster checks.
|
||
|
||
**Why this is needed**
|
||
Ensures every series has artwork, which is required by most media center front-ends for a polished library view.
|
||
|
||
---
|
||
|
||
## 2. Remove startup NFO repair
|
||
|
||
### Task 2.1: Remove perform_nfo_repair_scan from startup lifespan
|
||
|
||
**Where is that found**
|
||
- `src/server/fastapi_app.py` (lifespan startup block, lines ~245 and ~319)
|
||
- `src/server/services/initialization_service.py` (keep the function, just remove the call site)
|
||
- `tests/integration/test_nfo_repair_startup.py`
|
||
- `tests/unit/test_initialization_service.py` (tests that call `perform_nfo_repair_scan` directly can stay, but integration tests verifying startup wiring must change)
|
||
|
||
**Goal. How it should be**
|
||
1. In `src/server/fastapi_app.py`, remove the import of `perform_nfo_repair_scan` from the `initialization_service` import block.
|
||
2. Remove the line `await perform_nfo_repair_scan(background_loader)` from the lifespan startup sequence.
|
||
3. Update `tests/integration/test_nfo_repair_startup.py`:
|
||
- Remove or modify `test_perform_nfo_repair_scan_imported_in_lifespan` and `test_perform_nfo_repair_scan_called_after_media_scan` since the startup wiring is gone.
|
||
- Replace with a test that verifies `perform_nfo_repair_scan` is NOT called during startup (or simply delete the file if it has no other purpose).
|
||
4. `tests/unit/test_initialization_service.py` tests for `perform_nfo_repair_scan` can remain because they test the function itself, not the startup wiring.
|
||
|
||
**Possible traps and issues**
|
||
- **Test failures**: `test_nfo_repair_startup.py` will fail immediately after the code change. It must be updated in the same PR.
|
||
- **Documentation drift**: `docs/NFO_GUIDE.md`, `docs/CHANGELOG.md`, and `docs/ARCHITECTURE.md` all describe the startup NFO repair behavior. If docs are not updated, users will expect repair on every start.
|
||
- **Background loader parameter**: The `background_loader` variable was created partly for `perform_nfo_repair_scan`. After removal, check if `background_loader` is still needed for other startup steps (yes — `perform_media_scan_if_needed` uses it). Do not remove `background_loader` entirely.
|
||
- **Import cleanup**: Ensure no unused imports remain in `fastapi_app.py` after removal.
|
||
|
||
**Docs changes needed**
|
||
- `docs/NFO_GUIDE.md`: Update section 11 "Automatic NFO Repair" to remove startup references and state it runs via scheduler.
|
||
- `docs/CHANGELOG.md`: Add an entry under "Changed" or "Removed" noting that startup NFO repair is replaced by scheduled folder scan.
|
||
- `docs/ARCHITECTURE.md`: Update the startup sequence description.
|
||
|
||
**Why this is needed**
|
||
Running `perform_nfo_repair_scan` on every startup slows down server restarts, especially for large libraries. Moving it to a scheduled task keeps startup fast while still ensuring regular maintenance.
|