Compare commits

..

107 Commits

Author SHA1 Message Date
dbaf80e941 chore: bump version 2026-06-03 21:49:02 +02:00
4fc597c5de fix: disable startup media scan
Skip incomplete series check on server startup to reduce
startup time. Can be re-enabled by restoring _check_media_scan_status().
2026-06-03 21:47:55 +02:00
a77bb371df chore: bump version 2026-06-03 21:41:30 +02:00
420d10bb34 Fix async/await bug in folder_rename_service
- _update_series_folder: make async, await AnimeSeriesService.update()
- test: fix mock method name get_by_key -> get_by_folder

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-03 21:19:03 +02:00
e29918488c fix: correct key_resolution_service import path in scheduler
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-03 21:14:12 +02:00
9c3f03d610 refactor(scheduler): separate scheduler logic from scan/rescan logic
- Extract rescan logic into new RescanService (src/server/services/rescan_service.py)
- SchedulerService now only handles APScheduler cron scheduling
- Move scheduler sub-services (folder_rename, folder_scan, key_resolution) to scheduler/ folder
- Keep RescanOrchestrator as backward-compatible alias
- Update all imports across api/, server/, and test files
2026-06-03 20:58:30 +02:00
9d64241230 fix(db): add missing legacy_key_cleanup_completed column migration
Add migrate_schema_if_needed() to handle adding missing columns to existing
tables for backward compatibility. Automatically adds legacy_key_cleanup_completed
BOOLEAN column to system_settings table if missing, preventing 'no such column'
errors on startup for existing databases.
2026-06-03 20:22:59 +02:00
49cd84f3e5 chore: bump version 2026-06-02 20:59:42 +02:00
e46759347e backup 2026-06-02 20:59:13 +02:00
75f743e6cc fix: fetch series name from provider when scanning
Avoid 'Series name cannot be empty' error when a Serie is loaded
from a serie_file with an empty name by fetching the title from the
provider after year-fetching in the scan() method.

fix #?
2026-06-02 20:57:27 +02:00
4dc5ffa19e copy version to docker file 2026-06-02 20:53:11 +02:00
1649a22418 chore: bump version 2026-06-02 20:39:18 +02:00
246752e2fc Add dynamic version from Docker/VERSION file
- Create version.py utility to read version from Docker/VERSION
- Replace hardcoded version '1.0.1' with APP_VERSION from version.py
- Add version logging on FastAPI startup
- Use APP_VERSION in health endpoints and template context
2026-06-02 20:38:42 +02:00
84b24ed79e chore: bump version 2026-06-02 20:33:01 +02:00
bf3954587a fix(folder_rename_service): use get_by_folder instead of get_by_key when looking up by folder name
Update_database_paths and duplicate folder cleanup were using get_by_key()
(provider key lookup) instead of get_by_folder() when operating on folder names.
This caused orphaned DB records when removing duplicate folders like 'Hells Paradise'
that mapped to an already-existing 'Hell\'s Paradise (2023)'.
2026-06-02 20:09:47 +02:00
ed8f5cae10 chore: bump version 2026-06-01 21:38:37 +02:00
a54c285994 fix(folder_scan): await NFO repair before folder rename
folder_rename_service depends on clean NFO files but repair tasks
were fire-and-forget. Now collect all repair tasks and await them
with asyncio.gather before validate_and_rename_series_folders runs.

Also update tests that mock asyncio.create_task to also mock
asyncio.gather since perform_nfo_repair_scan now awaits tasks.
2026-06-01 21:37:28 +02:00
c58b42dfa5 feat(services): add key resolution for orphaned anime folders
- Add key_resolution_service.py to resolve provider keys for folders without key/data files
- Search anime provider and match folder names (case-insensitive, exact match required)
- Only save to DB if exactly one match found; otherwise skip
- Add comprehensive unit tests (28 tests)
- Integrate into scheduler_service after nfo_repair scan
- Update ARCHITECTURE.md documentation
2026-06-01 20:43:13 +02:00
6dfb24de7e backup 2026-06-01 20:07:58 +02:00
6021cdef28 feat: add anime metadata editing and NFO diagnostics
- Add PUT /anime/{key} endpoint for updating anime key, tmdb_id, tvdb_id
- Add NFO diagnostics and repair endpoints (GET/POST /nfo/diagnostics)
- Add edit modal UI with context menu integration
- Add frontend JS modules for context-menu and edit-modal
- Add comprehensive tests for edit, rename, and NFO repair flows
2026-05-31 18:31:56 +02:00
5517ccbab0 style: reformat folder_rename_service import 2026-05-30 12:20:40 +02:00
94ed013172 Revert "feat: add manual TMDB/TVDB ID entry for failed lookups"
This reverts commit 30858f441c.
2026-05-30 12:17:48 +02:00
76b849fc91 chore: bump version 2026-05-30 12:02:48 +02:00
00b26c8cbc fix: validate generated keys before creating Serie objects
- Add is_valid_key check in SerieScanner._read_data_from_file() to prevent
  passing invalid keys to Serie constructor (caused ValueError)
- Improve error message for key generation failures
- Add warning log before removing duplicate source folders in rename service
2026-05-30 11:42:19 +02:00
a6f2399aca chore: bump version 2026-05-29 19:25:30 +02:00
cf001563b3 refactor: add folder rename configuration and service
Add configurable folder rename patterns via settings with anime_folder_rename_regex and custom_pattern options. Integrate into SerieScanner and SeriesApp for consistent episode organization.
2026-05-29 19:24:09 +02:00
38c12638a4 fix HLS stream warning by disabling native downloader and retrying with ffmpeg
- Set hls_prefer_native: False to skip yt-dlp's native HLS downloader which emits
  'Live HLS streams are not supported' warning
- Add retry logic that catches HLS-related exceptions and retries with
  downloader=ffmpeg and hls_use_mpegts=True
2026-05-29 18:53:47 +02:00
765e43c684 fix(key_utils): drop apostrophes in generate_key_from_folder 2026-05-29 18:20:20 +02:00
5190d32665 chore: bump version 2026-05-28 22:03:52 +02:00
4e6afa31b5 Remove legacy key file support after DB migration
- SerieScanner: Remove key file fallback, keep data file fallback
- SystemSettings: Add legacy_key_cleanup_completed flag
- initialization_service: Add cleanup task to remove key files from folders with DB entries
- Tests updated to reflect key file removal from legacy path

Key files caused duplicate key errors on folder rename. DB is now sole source of truth.
2026-05-28 22:01:37 +02:00
1ef59c5283 feat: add duplicate folder detection and /duplicate-folders API endpoint
- Add DuplicateFolderGroup and DuplicateFoldersResponse Pydantic models
- Add /duplicate-folders GET endpoint for listing pre-existing duplicates
- Add _scan_for_pre_existing_duplicates() function for NFO-based detection
- Add _try_merge_duplicate_group() for auto-merging empty/symlink-only duplicates
- Integrate duplicate detection into validate_and_rename_series_folders workflow
- Skip rename for flagged duplicates to prevent data loss during merge
2026-05-28 21:46:08 +02:00
239341629c Add orphaned folder cleanup after rename
- Add _cleanup_orphaned_folder() to delete/move old folder contents after rename
- Empty folders: delete directly via rmdir()
- Non-empty folders: move contents to new path, then delete old folder
- Handle PermissionError and OSError gracefully with logging
- Add dry_run parameter to preview changes without applying them
- Add --dry-run support to validate_and_rename_series_folders()
- Add unit tests for _cleanup_orphaned_folder and dry-run mode
- All 66 related tests pass

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-28 21:24:13 +02:00
51b7f349f8 fix(scheduler): strip null legacy alias fields from config.json on save
SchedulerConfig.__init__ maps legacy auto_download/folder_scan keys to the
primary auto_download_after_rescan/folder_scan_enabled fields. However,
model_dump() was including auto_download=null and folder_scan=null in
serialised output. When this was written to config.json and reloaded,
those keys were present (albeit null), so the alias mapping was skipped
and the primary fields retained default False values instead of the
configured True values.

Fix:
- Override SchedulerConfig.model_dump() to drop None-valued alias fields
  before returning the serialised dict.
- ConfigService.save_config() re-serialises the scheduler field through
  its overridden model_dump() so the fix applies when writing to disk.

Tests added:
- test_roundtrip_excludes_none_alias_fields: verifies model_dump omits
  null auto_download/folder_scan keys.
- test_save_and_load_scheduler_flags_roundtrip: end-to-end roundtrip
  through ConfigService confirms raw JSON and loaded values match.

Pre-existing failure in test_core_error_handler.py is unrelated.
2026-05-28 21:18:16 +02:00
14b8ef7f06 Add Step 4 fallback: generate key from folder name
- SerieScanner: generate key from folder when no key/data files exist
- Handle edge cases: non-Latin characters, special symbols in folder names
- anime_service: expose loading_status and loading_error fields
- Update tests to match new fallback behavior
2026-05-28 18:48:43 +02:00
7abba0dae2 Fix download provider errors with exponential backoff and playmogo support
- Add exponential backoff retry logic to RecoveryStrategies (1s, 2s, 4s...)
- Add TimeoutError to network failure handling for HTTPS timeouts
- Add playmogo.com referer header for Doodstream provider
- Add Optional import to error_handler.py
- Add sanitize_url_for_logging utility function

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-28 18:47:05 +02:00
30858f441c feat: add manual TMDB/TVDB ID entry for failed lookups
- Add PATCH /api/anime/{key}/metadata-ids endpoint to update IDs
- Add POST /api/anime/{key}/refresh-nfo endpoint to force NFO regeneration
- Add Edit Metadata IDs modal in frontend
- Add showEditMetadataModal, saveMetadataIds, refreshSeriesNfo JS functions
- Add edit-metadata-btn to series cards with database icon
- IDs validated as positive integers or null

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-28 18:38:34 +02:00
33f63ca304 feat(SerieScanner): add folder ignore patterns for non-anime content
- Add NFO_FOLDER_IGNORE_PATTERNS setting to skip TV shows like
  The Last of Us, Loki, Chernobyl, Star Trek Discovery
- Update SerieScanner.__find_mp4_files() to skip ignored folders
- Update SerieList.load_series() to skip ignored folders
- Add should_ignore_folder() method for pattern matching
- Add folder_ignore_patterns property for pattern parsing
- Add comprehensive tests for ignore pattern functionality
- Update NFO_GUIDE.md with ignore patterns documentation
- Update CONFIGURATION.md with new setting

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-28 18:11:45 +02:00
fe9284b80e feat(SerieScanner): add warning event for duplicate series keys
- Add on_warning event system with subscribe/unsubscribe methods
- Change duplicate key handling from error to warning
- Fire on_warning event when duplicate series detected
- Include metadata: key, duplicate_folder, existing_folder
2026-05-28 18:05:07 +02:00
12e5526991 chore: remove obsolete migration script
Migration logic moved to serie_scanner. No longer needed.
2026-05-28 17:55:00 +02:00
bc87bee416 refactor(scheduler): drop separate scheduler.db in favour of MemoryJobStore
Scheduler used a separate SQLite file (scheduler.db) only to persist one
cron job. This was originally required because APScheduler's
SQLAlchemyJobStore is sync-only, creating an async/sync driver conflict
when accessing the same file.

The job is rebuilt from config.json on every startup regardless
(replace_existing=True), so the persisted state only served misfire
detection. Moved misfire detection into the app layer by querying
system_settings.last_scan_timestamp on startup: if the last scan is
>23h but <25h ago, an immediate rescan is triggered.

Change summary:
- Remove SQLAlchemyJobStore; use default MemoryJobStore instead
- Add _check_missed_run() that reads last_scan_timestamp from aniworld.db
- Update docs/DEVELOPMENT.md scheduler troubleshooting section
- Update the scheduler unit test that verified SQLAlchemyJobStore
2026-05-27 22:09:18 +02:00
7ded5a6e4d chore: bump version 2026-05-27 21:41:45 +02:00
d596902ca3 Parse existing NFO for TMDB ID to skip redundant search
Check existing tvshow.nfo for TMDB ID before querying TMDB API.
If found, fetch details directly using cached ID instead of searching.
Reduces API calls and improves performance for already-indexed series.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-27 21:22:24 +02:00
d358a07290 fix async handling in SerieScanner and add image_downloader cleanup
- SerieScanner.scan() now detects running event loop and uses create_task()
  when already in async context, avoids RuntimeError
- NFOService.close() now also closes image_downloader to prevent resource leaks
- Add integration tests for TMDBClient lifecycle management

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-27 20:47:29 +02:00
b9c55f9e7a fix: remove double-call on AsyncSession in SerieScanner
get_async_session_factory() returns session directly, not factory.
Calling result again with () caused 'AsyncSession' object is not callable.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-27 20:19:34 +02:00
fc4e52f1a2 chore: bump version 2026-05-26 20:23:20 +02:00
6d30747f25 Fix stale data file updates on download completion
When episodes are downloaded successfully, the in-memory Serie.episodeDict
is updated, but the deprecated data file was not being synced. This caused
UI to show episodes as missing when already downloaded.

Changes:
- Update data file in _remove_episode_from_memory when download completes
- DB is authoritative; data file is optional backup (deprecated)
- Gracefully skip update if data file doesn't exist

New integration tests for episode download sync:
- Verify episode removed from missing list after download
- Verify in-memory cache updated after download
- Verify data file updated after download (when it exists)
- Verify downloads work without data file
2026-05-26 18:57:04 +02:00
ceb6a2aeb4 Rename sync_series_from_data_files to sync_legacy_series_to_db
- Rename function to reflect its legacy status
- Add deprecation warning log on execution
- Update all callers (initialization_service, api/config, fastapi_app)
- Update tests to use new name
- Add deprecation notice to DEVELOPMENT.md

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-26 18:45:22 +02:00
53d6da5dac Add database loading methods to SerieList
- Add load_all_from_db() for bulk loading series from DB
- Add _load_single_series_from_db() for loading single series by folder
- Add invalidate_cache() to clear in-memory cache
- Add tests for all new methods

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-26 18:26:25 +02:00
102d83e947 feat(scanner): replace file writes with DB persistence for series
- SerieScanner.scan() now calls _persist_serie_to_db() instead of serie.save_to_file()
- Added _sync_episodes_to_db() helper to handle episode CRUD during sync
- EpisodeService gains delete_by_series() for targeted episode deletion
- SerieList gains add_to_db() async method for DB-based series addition
- test_serie_scanner_db_writes.py covers create/update/preserve/sync scenarios
- DATABASE.md updated with Series Persistence Flow section

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-26 18:12:01 +02:00
841368bf85 feat(SerieScanner): DB lookup primary, deprecate key file fallback
DB now source of truth for folder -> Serie resolution.

Changes:
- AnimeSeriesService.get_by_folder(): new async lookup by folder name
- SerieScanner.__read_data_from_file(): query DB first, then provider callback, then legacy key file (temporary, removed v3.0.0)
- Serie: reconstruct from DB record with episode dict
- Key file: warn on use, scheduled removal v3.0.0

Add unit tests for DB hit/miss/callback/fallback edge cases

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-26 17:56:37 +02:00
cbd53ef2a0 feat: add legacy key/data file migration to database
- Add migration_legacy_files_completed flag to SystemSettings model
- Create legacy_file_migration service to migrate series from key/data files
- Integrate legacy migration into initialization_service startup flow
- Add integration tests for legacy file migration
- Update DATABASE.md documentation with migration details
- Fix various test and service issues (nfo_repair, tmdb_client, download_service)
- Add test_database_schema unit tests
2026-05-26 17:44:42 +02:00
50a77976d5 chore: bump version 2026-05-26 13:28:12 +02:00
dfc28b8e66 fix(scheduler): ensure scheduler starts after setup/config update
Add ensure_started() to SchedulerService as idempotent entry point.
Start scheduler in auth setup run_initialization() after NFO scan.
Sync anime_directory and start scheduler in config update endpoint.
Add unit and endpoint tests for ensure_started() behavior.
2026-05-26 13:23:48 +02:00
6c9605e896 chore: bump version 2026-05-26 08:58:37 +02:00
3947f6d266 refactor(scheduler): replace structlog with std logging and add extensive diagnostics
- Switch scheduler_service from structlog to standard logging for consistency
- Add detailed lifecycle logging in SchedulerService (start, stop, rescan)
- Add debug logging in fastapi_app scheduler initialization
- Fix test_add_series_episodes to mock EpisodeService.get_by_series
2026-05-26 07:51:22 +02:00
a3176f5ac1 chore: bump version 2026-05-25 21:31:46 +02:00
9a81b04b65 fix: downloaded episodes no longer appear as missing
Use the database as the authoritative source for missing-episode lists so
that episodes marked is_downloaded=True are never shown as missing, even
when the in-memory state is stale.

Key changes:
- EpisodeService.get_by_series() gains only_missing flag
- AnimeService uses DB-backed episodeDict and preserves downloaded episodes
  during sync, skipping them when adding/removing missing episodes
- DownloadService broadcasts series_updated after marking an episode downloaded
  so the frontend reflects the change immediately
- Frontend filters out series with zero missing episodes client-side and
  fixes renderSeries to respect the active filter
- Unit tests updated to assert the broadcast is sent
2026-05-25 21:30:31 +02:00
a336733ea9 fix(nfo): add year field to series and create missing NFO files
- Add missing year field when building series list in anime_service
- Add _create_missing_nfo to generate minimal NFO for series without one
- Update perform_nfo_repair_scan to detect and create missing NFOs
- Add semaphore-protected async creation with TMDB rate limiting

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-25 16:32:54 +02:00
ca93bb740a feat(providers): detect HTML encoding before parsing
Add chardet-based _decode_html_content() to aniworld_provider. Apply
to all BeautifulSoup parsing calls to prevent decoding warnings on
pages with mismatched encoding declarations. Falls back to utf-8
with errors='replace' when confidence < 0.7.

Also fix test_enhanced_provider HLS test signature and add HLS
pattern unit tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-25 16:30:36 +02:00
d5e955a731 Add year extraction from folder names for existing series
- New migration script: populate year from folder (YYYY) pattern
- SerieScanner: refactor year extraction logic
- anime_service: pass year when syncing from data files
2026-05-25 15:30:28 +02:00
e2a373816a feat(nfo): add minimal NFO fallback when TMDB fails
- Add create_minimal_nfo() method to NFOService for fallback when TMDB lookup fails
- Update API endpoints (single and batch) to use minimal NFO fallback on TMDBAPIError
- Document fallback behavior in NFO_GUIDE.md section 3.6
- Add unit tests for minimal NFO creation (11 tests passing)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-25 15:19:50 +02:00
a115215416 fix(providers): rotate, probe and fall back on 404
Iterate providers actually advertised on the episode page (ordered by
SUPPORTED_PROVIDERS preference) instead of always re-resolving VOE.
Each candidate is HEAD-probed before yt-dlp runs, so dead links are
skipped immediately; direct video URLs use a streaming fast path that
bypasses yt-dlp; total failure now logs the exhausted provider list.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-25 14:32:10 +02:00
c579235af0 feat(download): persist retry state and dead-letter
Retry count and queue status were in-memory only and lost on
restart, so failed downloads could not be safely resumed and
permanently-failed episodes silently blocked re-queueing via the
episode-id unique index.

- Add status + retry_count columns to DownloadQueueItem
- Replace unique(episode_id) with unique(episode_id, status) so
  permanently_failed rows do not block new pending entries
- Add PERMANENTLY_FAILED to DownloadStatus enum
- Persist retry_count on each failure; mark permanently_failed once
  max_retries reached
- QueueRepository reads status/retry_count from DB instead of
  defaulting to PENDING/0
- Stop double-incrementing retry_count in retry_failed_items;
  increment only happens in _process_download on failure

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-25 14:24:31 +02:00
0ba2587bc8 refactor(download): mark episode downloaded instead of deleting
Change _remove_episode_from_missing_list to set is_downloaded=True
and populate file_path via EpisodeService.mark_downloaded, instead of
deleting the Episode row. Preserves download history so queries can
distinguish series with downloaded episodes from completely unwatched
series.

- Pass serie_folder to construct file_path
- Look up series_id via AnimeSeriesService.get_by_key
- Update tests to mock mark_downloaded path
- Document episode lifecycle in docs/DEVELOPMENT.md

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-25 14:14:33 +02:00
bda1fe4445 Fix scheduler next_run_time None check; add debug logging
- Fix race condition: next_run_time only available after scheduler.start()
- Handle None gracefully in logging
- Add debug logging to _perform_rescan and _run_rescan_job
- Document scheduler troubleshooting in DEVELOPMENT.md
2026-05-25 13:48:34 +02:00
810346bc8b chore: bump version 2026-05-24 21:17:39 +02:00
daa937bcb7 Fix test isolation: clear logging handlers and reset propagate flags
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-23 22:44:40 +02:00
1c505bd722 Use ffmpeg for HLS streams in aniworld provider
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-23 22:26:48 +02:00
3551838887 Add startup health checks and /health/ready endpoint
- Add _run_startup_health_checks() function in fastapi_app.py
  - Check ffmpeg availability (warning)
  - Check DNS resolution for aniworld.to and api.themoviedb.org (warning)
  - Check anime_directory configuration and writability (error)
- Store startup checks in app.state for health endpoint access
- Add /health/ready endpoint for container orchestrators
  - Returns not_ready with 503 when critical failures present
  - Includes critical_failures list for debugging
- Update /health endpoint to include startup check results
  - Status reflects worst check (error > warning > ok)
- Document health check endpoints in DEVELOPMENT.md
- Add unit tests for startup health checks
- Add unit tests for /health/ready endpoint

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-23 22:12:03 +02:00
9a20541598 feat(NFO): add TMDB search fallback with alt_titles support
- New _search_with_fallback() method tries multiple strategies:
  1. Primary query with year filter (de-DE locale)
  2. Alternative titles with ja-JP / en-US locales
  3. English search (en-US)
  4. Search without year constraint
  5. Punctuation-normalized query
- create_nfo() accepts new alt_titles param for Japanese/title fallback
- Better match rate for anime with non-English titles

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-23 21:57:00 +02:00
3f7651404d fix(tmdb): harden aiohttp session lifecycle
- Add async context manager to NFOService wrapping TMDBClient + ImageDownloader
- Add TMDBClient.__del__ warning when session leaks
- Log exc_info on session recreation for traceback visibility
- Document async-with usage in docs/DEVELOPMENT.md and docs/TESTING.md
- Add unit tests covering leak detection, context-manager cleanup, and connector-closed warning

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-23 21:34:26 +02:00
bee24406e6 Add runner.csx script 2026-05-23 21:28:54 +02:00
31eb0026cf Add queue deduplication to prevent duplicate entries
- In-memory dedup in add_to_queue() using _pending_by_episode dict
- Batch-local dedup via seen_in_batch set (handles duplicates within single call)
- Database unique index on episode_id via __table_args__
- 5-minute cooldown in _auto_download_missing() to prevent rapid re-triggers
- Updated _add_to_pending_queue() and _remove_from_pending_queue() to track episode keys
- Added TestQueueDeduplication with 4 test cases
- Updated DEVELOPMENT.md and TESTING.md with queue dedup docs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-23 21:27:41 +02:00
24ea12bbaf Add Docs/runner.csx 2026-05-23 21:19:15 +02:00
e74b602f60 Add ffmpeg for HLS stream download support
- Add ffmpeg to Dockerfile.app for container HLS support
- Configure yt-dlp with --downloader ffmpeg --hls-use-mpegts
- Add startup health check warns if ffmpeg missing
- Update DEVELOPMENT.md with ffmpeg prerequisites and troubleshooting
- Add tests for ffmpeg HLS options and health check
2026-05-23 21:18:39 +02:00
db65e28854 backup 2026-05-21 22:10:07 +02:00
11e231a4ab chore: bump version 2026-05-21 21:42:13 +02:00
a11f8c4fa0 fix(vpn): add explicit host route for health-check target
Without a /32 route in the main table, CHECK_HOST (1.1.1.1) fell through
to the VPN default route where source-address selection was defeated by
the priority-100 'from ETH0_IP' policy rule, causing pings to bypass
wg0 and be dropped by the kill switch.

Also add secondary google.com ping to distinguish IP vs DNS failures.
2026-05-21 21:41:51 +02:00
cf5a06af11 chore: bump version 2026-05-21 21:22:08 +02:00
e07f75432e backup 2026-05-21 21:21:13 +02:00
1696d5c65b chore: bump version 2026-05-21 21:04:51 +02:00
c8b386f47a chore: bump version 2026-05-20 20:00:45 +02:00
3888da352a feat(tmdb): improve rate limiting and retry resilience
- Increase max_retries from 3 to 5 with exponential backoff capped at 30s
- Add per-second rate limiter (~35 req/s) to stay under TMDB's ~40/s limit
- Replace small semaphore (4) with larger one (30) + token-bucket throttle
- Abort retries immediately on DNS/name-resolution failures
- Increase rate-limit fallback wait from default to max(delay*2, 10)s
2026-05-20 20:00:11 +02:00
06e104db42 chore: bump version 2026-05-20 19:41:58 +02:00
d4594bd1d9 chore: bump version 2026-05-20 19:40:17 +02:00
d866e836f6 backup 2026-05-20 19:39:08 +02:00
195dae13cb test: add integration tests for NFO content and repair
- test_add_anime_nfo_content.py: verify required NFO tags after anime add
- test_sacrificial_princess_nfo.py: test full NFO generation and repair path
2026-05-20 19:38:43 +02:00
51be777e7d fix: strip all trailing year suffixes to prevent duplication
- series.py: use regex to remove all trailing (YYYY) before appending year
- nfo_service.py: _extract_year_from_name strips all trailing year suffixes
- nfo_repair_service.py: add _read_tmdb_id() helper to extract TMDB ID from NFO
2026-05-20 19:38:37 +02:00
7930e49701 fix: prevent duplicate year suffixes in series name and folder creation
Apply the same duplicate-year prevention logic to additional code paths:

- Serie.name_with_year property: skip adding year suffix if name already ends with it
- add_series API endpoint: avoid duplicating year in folder_name_with_year
- Add integration test for Serie.name_with_year idempotency
- Add API test for add_series endpoint year deduplication

Complements the folder_rename_service fix for comprehensive coverage.
2026-05-19 21:25:21 +02:00
75c22fe296 fix(folder-rename): prevent duplicate year suffixes in series folder names
Use regex to strip all trailing year suffixes before adding the canonical
one, preventing duplication like 'Show (2021) (2021) (2021)'.

- Add regex pattern (\s*\(\d{4}\))+\s*$ to remove all existing year suffixes
- Ensure idempotent behavior across multiple folder rename runs
- Add 7 unit tests covering the bug cases and edge scenarios

Fixes: 86 Eighty Six (2021) (2021)..., Alma-chan (2025) (2025)...
2026-05-19 21:24:07 +02:00
7bcd0600d5 chore: bump version 2026-05-18 09:57:51 +02:00
a333329ae2 backup 2026-05-18 09:56:59 +02:00
363f7899f8 refactor(logging): reduce download log spam and set INFO level
- Pass app logger to yt-dlp so internal [download] progress lines
  are routed through the INFO-level logger instead of stdout.
- Throttle download_progress_handler debug logging to avoid
  flooding logs on every fragment tick.
- Switch key provider lifecycle messages to INFO (start/complete)
  while keeping verbose details at DEBUG.
- Set debug_enabled=False in development config so dev mode
  does not emit extra debug noise.
- Update config docstring example from DEBUG to INFO.
2026-05-18 09:56:19 +02:00
a08a8f7408 backup 2026-05-17 18:57:12 +02:00
54ac5e9ab7 chore: release v1.1.4 2026-05-17 18:51:09 +02:00
c93ac3e7b8 chore: release v1.1.3 2026-05-17 18:40:37 +02:00
68c4335348 feat(vpn): add startup connectivity checks and PersistentKeepalive
Add check_vpn_connectivity() that runs once after wg0 comes up:
- Waits for handshake (up to 15s) and prints public key if missing
- Measures RX bytes before/after curl to detect server-side routing issues
- Tests DNS resolution and dumps resolv.conf on failure
- On failure prints exact server-side commands to fix (sysctl, iptables, wg)

Add PersistentKeepalive=25 to wg0.conf to keep NAT mappings alive.
2026-05-17 18:40:24 +02:00
be87f2e230 chore: release v1.1.2 2026-05-17 18:31:59 +02:00
c56e0f507d fix(vpn): fix DNS iptables rules and add NET_RAW cap
DNS OUTPUT was restricted to -o wg0, but routing decision happens
after iptables OUTPUT — so DNS to VPN-internal addresses (198.18.0.x)
was blocked before the kernel selected the outgoing interface.
Allow DNS unconditionally; routing still sends it through wg0.

Add NET_RAW capability so ping works inside the container.
2026-05-17 18:31:38 +02:00
cb0a36ccc2 chore: release v1.1.1 2026-05-16 21:47:05 +02:00
3644b16447 feat(vpn): add version logging from VERSION file
- Read version from /etc/wireguard/VERSION instead of hardcoding
- Copy VERSION file into container image during build
- Update VERSION to v1.1.0
2026-05-16 21:46:19 +02:00
d5116e378e chore: release v0.1.0 2026-05-16 21:41:40 +02:00
50a7083ce5 fix(vpn): support AllowedIPs=0.0.0.0/0 and multi-DNS configs
- Parse AllowedIPs dynamically from WireGuard config instead of hardcoding routes
- Remove auto-created default route by wg setconf to prevent breaking endpoint connection
- Fix DNS parsing: write comma-separated DNS servers as separate nameserver lines
- Add test for AllowedIPs route verification and DNS configuration
- Update test to skip container runtime tests when not running as root
2026-05-16 21:41:27 +02:00
52c0ff2337 chore(docs): remove temporary planning file docs/bla 2026-05-16 21:22:44 +02:00
a5fd88e224 chore(vpn): update WireGuard endpoint and credentials
- Rotate to new VPN endpoint (91.148.236.64)
- Update private/public keys and client address
- Switch DNS to 198.18.0.1/0.2
- Add local network route preservation via PostUp/PostDown
- Align nl.conf and wg0.conf configurations
2026-05-16 21:22:04 +02:00
98d4edad14 feat(vpn): dynamic AllowedIPs routing and improved test coverage
- Parse AllowedIPs from WireGuard config in entrypoint.sh
- Add/remove routes dynamically instead of hardcoded 0.0.0.0/1 split
- Handle both 0.0.0.0/0 and custom AllowedIPs
- Add route cleanup on VPN stop (endpoint + AllowedIPs)
- Update test_vpn.py with AllowedIPs route verification
- Allow non-root build-only tests with automatic runtime skip
2026-05-16 21:21:56 +02:00
bc8059b453 feat(docker): add release script and enhance push script
- Add release.sh for automated versioning and image pushing
- Enhance push.sh with target selection (app/vpn/all)
- Add docker/podman engine auto-detection
- Improve usage docs and error handling
2026-05-16 21:21:45 +02:00
128 changed files with 15825 additions and 2279 deletions

View File

@@ -17,6 +17,9 @@ __pycache__/
# Docker files (not needed inside the image)
Docker/
# Exception: VERSION is needed by Dockerfile.app
!Docker/VERSION
# Test and dev files
tests/
Temp/

View File

@@ -13,7 +13,8 @@ RUN apk add --no-cache \
# Create wireguard config directory (config is mounted at runtime)
RUN mkdir -p /etc/wireguard
# Copy entrypoint
# Copy version file and entrypoint
COPY VERSION /etc/wireguard/VERSION
COPY entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh

View File

@@ -2,12 +2,13 @@ FROM python:3.12-slim
WORKDIR /app
# Install system dependencies for compiled Python packages
# Install system dependencies for compiled Python packages and ffmpeg for HLS support
RUN apt-get update && \
apt-get install -y --no-install-recommends \
gcc \
g++ \
libffi-dev \
ffmpeg \
&& rm -rf /var/lib/apt/lists/*
# Install Python dependencies (cached layer)
@@ -19,6 +20,7 @@ COPY src/ ./src/
COPY run_server.py .
COPY pyproject.toml .
COPY data/config.json ./data/config.json
COPY Docker/VERSION ./Docker/VERSION
# Create runtime directories
RUN mkdir -p /app/data/config_backups /app/logs

View File

@@ -1 +1 @@
v0.0.1
v1.3.6

View File

@@ -1,6 +1,14 @@
#!/bin/bash
set -e
VERSION_FILE="/etc/wireguard/VERSION"
if [ -f "$VERSION_FILE" ]; then
VERSION=$(cat "$VERSION_FILE")
else
VERSION="unknown"
fi
echo "[init] VPN Container Entrypoint ${VERSION}"
INTERFACE="wg0"
MOUNT_CONFIG="/etc/wireguard/${INTERFACE}.conf"
CONFIG_DIR="/run/wireguard"
@@ -64,9 +72,11 @@ setup_killswitch() {
iptables -A INPUT -i "$INTERFACE" -j ACCEPT
iptables -A OUTPUT -o "$INTERFACE" -j ACCEPT
# Allow DNS to the VPN DNS server (through wg0)
iptables -A OUTPUT -o "$INTERFACE" -p udp --dport 53 -j ACCEPT
iptables -A OUTPUT -o "$INTERFACE" -p tcp --dport 53 -j ACCEPT
# Allow DNS (VPN DNS servers are routed through wg0; allow before routing decision)
iptables -A OUTPUT -p udp --dport 53 -j ACCEPT
iptables -A OUTPUT -p tcp --dport 53 -j ACCEPT
iptables -A INPUT -p udp --sport 53 -j ACCEPT
iptables -A INPUT -p tcp --sport 53 -j ACCEPT
# Allow DHCP (for container networking)
iptables -A OUTPUT -p udp --dport 67:68 -j ACCEPT
@@ -120,27 +130,46 @@ start_vpn() {
ip link add "$INTERFACE" type wireguard
# Apply the WireGuard config (keys, peer, endpoint)
# Filter out wg-quick directives that wg setconf doesn't understand
wg setconf "$INTERFACE" <(grep -v -i '^\(Address\|DNS\|MTU\|Table\|PreUp\|PostUp\|PreDown\|PostDown\|SaveConfig\)' "$CONFIG_FILE")
# Log public key so it can be verified against the server's peer list
local PUBKEY
PUBKEY=$(wg show "$INTERFACE" public-key 2>/dev/null || echo "unknown")
echo "[vpn] Public key: ${PUBKEY}"
# Assign the address
ip -4 address add "$VPN_ADDRESS" dev "$INTERFACE"
# Set MTU
# Set MTU and bring up
ip link set mtu 1420 up dev "$INTERFACE"
# Find default gateway/interface for the endpoint route
# ── fwmark-based routing (mirrors wg-quick behavior) ──
# WireGuard marks its own encapsulated UDP packets with this fwmark.
# Policy rules then ensure:
# - Normal packets (no mark) → VPN routing table → wg0
# - WireGuard-encapsulated packets (marked) → main table → eth0
local FW_MARK=51820
local FW_TABLE=51820
wg set "$INTERFACE" fwmark "$FW_MARK"
# Remove any auto-created default route on wg0
ip route del default dev "$INTERFACE" 2>/dev/null || true
# VPN routing table: send everything through the tunnel
ip -4 route add default dev "$INTERFACE" table "$FW_TABLE"
# Policy rules:
# 1. Packets NOT marked by WireGuard use the VPN table (→ wg0)
# 2. suppress_prefixlength 0: ignore bare default routes in main table,
# but keep more-specific routes (e.g. LAN, endpoint) working
ip -4 rule add not fwmark "$FW_MARK" table "$FW_TABLE"
ip -4 rule add table main suppress_prefixlength 0
# Find default gateway/interface
DEFAULT_GW=$(ip route | grep '^default' | head -1 | awk '{print $3}')
DEFAULT_IF=$(ip route | grep '^default' | head -1 | awk '{print $5}')
# Route VPN endpoint through the container's default gateway
if [ -n "$DEFAULT_GW" ] && [ -n "$DEFAULT_IF" ]; then
ip route add "$VPN_ENDPOINT/32" via "$DEFAULT_GW" dev "$DEFAULT_IF" 2>/dev/null || true
fi
# Route all traffic through the WireGuard tunnel
ip route add 0.0.0.0/1 dev "$INTERFACE"
ip route add 128.0.0.0/1 dev "$INTERFACE"
# ── Policy routing: ensure responses to incoming LAN traffic go back via eth0 ──
if [ -n "$DEFAULT_GW" ] && [ -n "$DEFAULT_IF" ]; then
# Get the container's eth0 IP address (BusyBox-compatible, no grep -P)
@@ -155,14 +184,37 @@ start_vpn() {
fi
fi
# Set up DNS
# Set up DNS (handle comma-separated DNS servers)
VPN_DNS=$(grep -i '^DNS' "$CONFIG_FILE" | head -1 | sed 's/.*= *//;s/ //g')
if [ -n "$VPN_DNS" ]; then
echo "nameserver $VPN_DNS" > /etc/resolv.conf
echo "[vpn] DNS set to ${VPN_DNS}"
# Clear resolv.conf and add each DNS server on its own line
> /etc/resolv.conf
for dns in $(echo "$VPN_DNS" | tr ',' ' '); do
echo "nameserver $dns" >> /etc/resolv.conf
# Add explicit route to DNS server through wg0 so it's found in main table
# (suppress_prefixlength 0 ignores default routes but allows host routes)
ip -4 route add "$dns" dev "$INTERFACE" 2>/dev/null || true
done
echo "[vpn] DNS set to: ${VPN_DNS}"
fi
# Add explicit host route for the health-check target so it is picked up by
# the 'lookup main suppress_prefixlength 0' rule (same as DNS servers above).
# Without this, CHECK_HOST falls through to the VPN table default route whose
# source-address selection can be defeated by the priority-100 'from ETH0_IP'
# policy rule, causing pings to bypass wg0 and be dropped by the kill switch.
ip -4 route add "${CHECK_HOST}" dev "$INTERFACE" 2>/dev/null || true
echo "[vpn] Health-check route: ${CHECK_HOST}${INTERFACE}"
echo "[vpn] WireGuard interface ${INTERFACE} is up."
echo "[vpn] Main routes:"
ip route show | sed 's/^/[vpn] /'
echo "[vpn] VPN table ($FW_TABLE):"
ip route show table "$FW_TABLE" 2>/dev/null | sed 's/^/[vpn] /'
echo "[vpn] Policy rules:"
ip rule show | sed 's/^/[vpn] /'
echo "[vpn] WireGuard status:"
wg show "$INTERFACE" 2>/dev/null | sed 's/^/[vpn] /'
}
# ──────────────────────────────────────────────
@@ -170,6 +222,21 @@ start_vpn() {
# ──────────────────────────────────────────────
stop_vpn() {
echo "[vpn] Stopping WireGuard interface ${INTERFACE}..."
local FW_MARK=51820
local FW_TABLE=51820
# Remove fwmark-based policy rules
ip -4 rule del not fwmark "$FW_MARK" table "$FW_TABLE" 2>/dev/null || true
ip -4 rule del table main suppress_prefixlength 0 2>/dev/null || true
# Flush VPN routing table
ip -4 route flush table "$FW_TABLE" 2>/dev/null || true
# Remove LAN policy routing
ip -4 rule del table 100 2>/dev/null || true
ip -4 route flush table 100 2>/dev/null || true
ip link del "$INTERFACE" 2>/dev/null || true
}
@@ -185,14 +252,31 @@ health_loop() {
while true; do
sleep "$CHECK_INTERVAL"
if curl -sf --max-time 5 "http://$CHECK_HOST" > /dev/null 2>&1; then
if ping -c 1 -W 5 "$CHECK_HOST" > /dev/null 2>&1; then
if [ "$failures" -gt 0 ]; then
echo "[health] VPN recovered."
failures=0
fi
# Secondary DNS check
if ping -c 1 -W 5 "google.com" > /dev/null 2>&1; then
: # DNS OK — silent
else
echo "[health] WARN google.com unreachable — possible DNS issue"
fi
else
failures=$((failures + 1))
echo "[health] Ping failed ($failures/$max_failures)"
echo "[health] Check failed ($failures/$max_failures) — ping ${CHECK_HOST} failed"
# Secondary check: distinguish IP failure from DNS failure
if ping -c 1 -W 5 "google.com" > /dev/null 2>&1; then
echo "[health] INFO google.com reachable — DNS works, ${CHECK_HOST} may be filtered"
else
echo "[health] INFO google.com also unreachable — DNS or general routing failure"
fi
# Dump WireGuard stats to show if handshake is stale and how much data flows
echo "[health] wg stats:"
wg show "$INTERFACE" 2>/dev/null | grep -E 'latest handshake|transfer|endpoint' | sed 's/^/[health] /' || echo "[health] wg0 not found"
echo "[health] routes:"
ip route show | grep -E 'wg0|default' | sed 's/^/[health] /'
if [ "$failures" -ge "$max_failures" ]; then
echo "[health] VPN appears down. Restarting WireGuard..."
@@ -221,8 +305,83 @@ cleanup() {
trap cleanup SIGTERM SIGINT
# ──────────────────────────────────────────────
# Startup connectivity checks — diagnose issues early
# ──────────────────────────────────────────────
check_vpn_connectivity() {
echo "[check] ── Startup connectivity checks ──"
# 1. Wait for WireGuard handshake (up to 15s)
local elapsed=0
local handshake_ts=0
echo "[check] Waiting for WireGuard handshake (up to 15s)..."
while [ "$elapsed" -lt 15 ]; do
handshake_ts=$(wg show "$INTERFACE" latest-handshakes 2>/dev/null | awk '{print $2}' | head -1)
if [ -n "$handshake_ts" ] && [ "$handshake_ts" != "0" ]; then
local age=$(( $(date +%s) - handshake_ts ))
echo "[check] OK Handshake established ${age}s ago"
break
fi
sleep 1
elapsed=$((elapsed + 1))
done
if [ "$elapsed" -ge 15 ]; then
echo "[check] FAIL No WireGuard handshake after 15s — tunnel is not up"
echo "[check] This container's public key (must be on the server):"
echo "[check] PublicKey = $(wg show "$INTERFACE" public-key 2>/dev/null || echo 'unknown')"
echo "[check] AllowedIPs = ${VPN_ADDRESS}"
echo "[check] Verify on server: wg show"
fi
# 2. Check whether traffic actually flows through the tunnel
echo "[check] Testing traffic through tunnel (ping ${CHECK_HOST})..."
local rx_before
rx_before=$(wg show "$INTERFACE" transfer 2>/dev/null | awk '{print $2}' | head -1)
if ping -c 1 -W 8 "${CHECK_HOST}" > /dev/null 2>&1; then
echo "[check] OK Traffic flows — tunnel is fully working"
else
local rx_after
rx_after=$(wg show "$INTERFACE" transfer 2>/dev/null | awk '{print $2}' | head -1)
echo "[check] FAIL ping ${CHECK_HOST} unreachable through tunnel"
if [ -n "$rx_before" ] && [ -n "$rx_after" ]; then
if [ "$rx_after" -le "$rx_before" ]; then
echo "[check] RX bytes unchanged (${rx_before}${rx_after})"
echo "[check] Server receives packets but does NOT route them back"
echo "[check] Fix on VPN server (${VPN_ENDPOINT}):"
echo "[check] sysctl net.ipv4.ip_forward # must output 1"
echo "[check] iptables -t nat -L POSTROUTING -v -n # must have MASQUERADE"
echo "[check] wg show # check peer + AllowedIPs"
else
echo "[check] RX increased (${rx_before}${rx_after}) — tunnel passes data"
echo "[check] Issue may be specific to ${CHECK_HOST} or DNS"
fi
fi
local transfer
transfer=$(wg show "$INTERFACE" transfer 2>/dev/null | awk '{printf "rx=%s tx=%s", $2, $3}')
echo "[check] wg transfer: ${transfer}"
fi
# 3. DNS check
echo "[check] Testing DNS resolution..."
if nslookup 1.1.1.1 > /dev/null 2>&1 || nslookup google.com > /dev/null 2>&1; then
echo "[check] OK DNS resolves"
else
echo "[check] FAIL DNS resolution failed"
echo "[check] resolv.conf: $(tr '\n' ' ' < /etc/resolv.conf)"
echo "[check] Check that DNS servers are reachable through wg0"
echo "[check] ── End of checks ──"
exit 1
fi
echo "[check] ── End of checks ──"
}
# ── Main ──
enable_forwarding
setup_killswitch
start_vpn
check_vpn_connectivity
health_loop

View File

@@ -1,17 +1,16 @@
[Interface]
PrivateKey = iO5spIue/6ciwUoR95hYtuxdtQxV/Q9EOoQ/jHe18kM=
Address = 10.2.0.2/32
DNS = 10.2.0.1
PrivateKey = EPRa2f/v72LvIXAY4yqIRJifsSb+nCcYHqC2rwA94UI=
Address = 100.64.244.78/32
DNS = 198.18.0.1,198.18.0.2
# Route zum VPN-Server direkt über dein lokales Netz
PostUp = ip route add 185.183.34.149 via 192.168.178.1 dev wlp4s0f0
PostUp = ip route add 91.148.236.64 via 192.168.178.1 dev wlp4s0f0
PostUp = ip route add 192.168.178.0/24 via 192.168.178.1 dev wlp4s0f0
PostDown = ip route del 185.183.34.149 via 192.168.178.1 dev wlp4s0f0
PostDown = ip route del 91.148.236.64 via 192.168.178.1 dev wlp4s0f0
PostDown = ip route del 192.168.178.0/24 via 192.168.178.1 dev wlp4s0f0
[Peer]
PublicKey = J4XVdtoBVc/EoI2Yk673Oes97WMnQSH5KfamZNjtM2s=
AllowedIPs = 0.0.0.0/1, 128.0.0.0/1
Endpoint = 185.183.34.149:51820
PublicKey = KgTUh3KLijVluDvNpzDCJJfrJ7EyLzYLmdHCksG4sRg=
AllowedIPs = 0.0.0.0/0
Endpoint = 91.148.236.64:51820

View File

@@ -15,6 +15,7 @@ services:
cap_add:
- NET_ADMIN
- SYS_MODULE
- NET_RAW
sysctls:
- net.ipv4.ip_forward=1
- net.ipv4.conf.all.src_valid_mark=1
@@ -22,7 +23,7 @@ services:
- /server/server_aniworld/wg0.conf:/etc/wireguard/wg0.conf:ro
- /lib/modules:/lib/modules:ro
ports:
- "2000:8000"
- "8000:8000"
environment:
- HEALTH_CHECK_INTERVAL=10
- HEALTH_CHECK_HOST=1.1.1.1
@@ -51,4 +52,5 @@ services:
volumes:
- /server/server_aniworld/data:/app/data
- /server/server_aniworld/logs:/app/logs
- /media/serien/Serien:/data
restart: unless-stopped

View File

@@ -7,6 +7,7 @@ services:
cap_add:
- NET_ADMIN
- SYS_MODULE
- NET_RAW
sysctls:
- net.ipv4.ip_forward=1
- net.ipv4.conf.all.src_valid_mark=1

View File

@@ -1,15 +1,19 @@
#!/usr/bin/env bash
# filepath: /home/lukas/Volume/repo/Aniworld/Docker/push.sh
#
# Build and push Aniworld container images to the Gitea registry.
# Build and push AniWorld container images to the Gitea registry.
#
# Usage:
# ./push.sh # builds & pushes with tag "latest"
# ./push.sh v1.2.3 # builds & pushes with tag "v1.2.3"
# ./push.sh v1.2.3 --no-build # pushes existing images only
# ./push.sh # builds & pushes app with tag "latest"
# ./push.sh app # builds & pushes app image
# ./push.sh vpn # builds & pushes vpn image
# ./push.sh all # builds & pushes both images
# ./push.sh app v1.2.3 # builds & pushes app with tag "v1.2.3"
# ./push.sh vpn v1.2.3 # builds & pushes vpn with tag "v1.2.3"
# ./push.sh all v1.2.3 # builds & pushes both images
# ./push.sh app v1.2.3 --no-build # pushes existing image only
#
# Prerequisites:
# podman login git.lpl-mind.de
# podman login git.lpl-mind.de (or: docker login git.lpl-mind.de)
set -euo pipefail
@@ -23,12 +27,20 @@ PROJECT="aniworld"
APP_IMAGE="${REGISTRY}/${NAMESPACE}/${PROJECT}/app"
VPN_IMAGE="${REGISTRY}/${NAMESPACE}/${PROJECT}/vpn"
TAG="${1:-latest}"
# Parse arguments
TARGET="${1:-app}"
TAG="${2:-latest}"
SKIP_BUILD=false
if [[ "${2:-}" == "--no-build" ]]; then
if [[ "${3:-}" == "--no-build" ]]; then
SKIP_BUILD=true
fi
# Validate target
if [[ "${TARGET}" != "app" && "${TARGET}" != "vpn" && "${TARGET}" != "all" ]]; then
echo "ERROR: Invalid target '${TARGET}'. Must be one of: app, vpn, all" >&2
exit 1
fi
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)"
@@ -36,62 +48,93 @@ PROJECT_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)"
# Helpers
# ---------------------------------------------------------------------------
log() { echo -e "\n>>> $*"; }
err() { echo -e "\nERROR: $*" >&2; exit 1; }
err() { echo -e "\nERROR: $*" >&2; exit 1; }
# Detect container engine (podman preferred, docker fallback)
if command -v podman &>/dev/null; then
ENGINE="podman"
elif command -v docker &>/dev/null; then
ENGINE="docker"
else
err "Neither podman nor docker is installed."
fi
# ---------------------------------------------------------------------------
# Pre-flight checks
# ---------------------------------------------------------------------------
echo "============================================"
echo " Aniworld — Build & Push"
echo " AniWorld — Build & Push"
echo " Engine : ${ENGINE}"
echo " Registry : ${REGISTRY}"
echo " Target : ${TARGET}"
echo " Tag : ${TAG}"
echo "============================================"
command -v podman &>/dev/null || err "podman is not installed."
if ! podman login --get-login "${REGISTRY}" &>/dev/null; then
err "Not logged in. Run:\n podman login ${REGISTRY}"
fi
log "Logging in to ${REGISTRY}"
"${ENGINE}" login "${REGISTRY}"
# ---------------------------------------------------------------------------
# Build
# ---------------------------------------------------------------------------
if [[ "${SKIP_BUILD}" == false ]]; then
build_app() {
log "Building app image → ${APP_IMAGE}:${TAG}"
podman build \
"${ENGINE}" build \
-t "${APP_IMAGE}:${TAG}" \
-f "${SCRIPT_DIR}/Dockerfile.app" \
"${PROJECT_ROOT}"
}
log "Building VPN image → ${VPN_IMAGE}:${TAG}"
podman build \
build_vpn() {
log "Building vpn image → ${VPN_IMAGE}:${TAG}"
"${ENGINE}" build \
-t "${VPN_IMAGE}:${TAG}" \
-f "${SCRIPT_DIR}/Containerfile" \
"${SCRIPT_DIR}"
}
if [[ "${SKIP_BUILD}" == false ]]; then
case "${TARGET}" in
app) build_app ;;
vpn) build_vpn ;;
all) build_app; build_vpn ;;
esac
fi
# ---------------------------------------------------------------------------
# Push
# ---------------------------------------------------------------------------
log "Pushing ${APP_IMAGE}:${TAG}"
podman push "${APP_IMAGE}:${TAG}"
push_app() {
log "Pushing ${APP_IMAGE}:${TAG}"
"${ENGINE}" push "${APP_IMAGE}:${TAG}"
}
log "Pushing ${VPN_IMAGE}:${TAG}"
podman push "${VPN_IMAGE}:${TAG}"
push_vpn() {
log "Pushing ${VPN_IMAGE}:${TAG}"
"${ENGINE}" push "${VPN_IMAGE}:${TAG}"
}
case "${TARGET}" in
app) push_app ;;
vpn) push_vpn ;;
all) push_app; push_vpn ;;
esac
# ---------------------------------------------------------------------------
# Summary
# ---------------------------------------------------------------------------
echo ""
echo "============================================"
echo " Push complete!"
echo " Push complete!"
echo ""
echo " Images:"
echo " ${APP_IMAGE}:${TAG}"
echo " ${VPN_IMAGE}:${TAG}"
case "${TARGET}" in
app) echo " ${APP_IMAGE}:${TAG}" ;;
vpn) echo " ${VPN_IMAGE}:${TAG}" ;;
all) echo " ${APP_IMAGE}:${TAG}"; echo " ${VPN_IMAGE}:${TAG}" ;;
esac
echo ""
echo " Deploy on server:"
echo " podman login ${REGISTRY}"
echo " podman-compose -f podman-compose.prod.yml pull"
echo " podman-compose -f podman-compose.prod.yml up -d"
echo " ${ENGINE} login ${REGISTRY}"
echo " ${ENGINE} compose -f Docker/podman-compose.prod.yml pull"
echo " ${ENGINE} compose -f Docker/podman-compose.prod.yml up -d"
echo "============================================"

129
Docker/release.sh Normal file
View File

@@ -0,0 +1,129 @@
#!/usr/bin/env bash
#
# Bump the project version and push images to the registry.
#
# Usage:
# ./release.sh
#
# The current version is stored in VERSION (next to this script).
# You will be asked whether to bump major, minor, or patch.
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
VERSION_FILE="${SCRIPT_DIR}/VERSION"
# ---------------------------------------------------------------------------
# Read current version
# ---------------------------------------------------------------------------
if [[ ! -f "${VERSION_FILE}" ]]; then
echo "0.0.0" > "${VERSION_FILE}"
fi
CURRENT="$(cat "${VERSION_FILE}")"
# Strip leading 'v' for arithmetic
VERSION="${CURRENT#v}"
IFS='.' read -r MAJOR MINOR PATCH <<< "${VERSION}"
echo "============================================"
echo " AniWorld — Release"
echo " Current version: v${MAJOR}.${MINOR}.${PATCH}"
echo "============================================"
echo ""
echo "Which image(s) would you like to release?"
echo " 1) app (Dockerfile.app)"
echo " 2) vpn (Containerfile)"
echo " 3) all (both images)"
echo ""
read -rp "Enter choice [1/2/3]: " TARGET_CHOICE
case "${TARGET_CHOICE}" in
1) TARGET="app" ;;
2) TARGET="vpn" ;;
3) TARGET="all" ;;
*)
echo "Invalid choice. Aborting." >&2
exit 1
;;
esac
echo ""
echo "How would you like to bump the version?"
echo " 1) patch (v${MAJOR}.${MINOR}.${PATCH} → v${MAJOR}.${MINOR}.$((PATCH + 1)))"
echo " 2) minor (v${MAJOR}.${MINOR}.${PATCH} → v${MAJOR}.$((MINOR + 1)).0)"
echo " 3) major (v${MAJOR}.${MINOR}.${PATCH} → v$((MAJOR + 1)).0.0)"
echo ""
read -rp "Enter choice [1/2/3]: " CHOICE
case "${CHOICE}" in
1) NEW_TAG="v${MAJOR}.${MINOR}.$((PATCH + 1))" ;;
2) NEW_TAG="v${MAJOR}.$((MINOR + 1)).0" ;;
3) NEW_TAG="v$((MAJOR + 1)).0.0" ;;
*)
echo "Invalid choice. Aborting." >&2
exit 1
;;
esac
echo ""
echo "New version: ${NEW_TAG}"
echo "Target: ${TARGET}"
read -rp "Confirm? [y/N]: " CONFIRM
if [[ ! "${CONFIRM}" =~ ^[yY]$ ]]; then
echo "Aborted."
exit 0
fi
# ---------------------------------------------------------------------------
# Write new version
# ---------------------------------------------------------------------------
echo "${NEW_TAG}" > "${VERSION_FILE}"
echo "Version file updated → ${VERSION_FILE}"
# Keep root package.json in sync.
FRONT_VERSION="${NEW_TAG#v}"
FRONT_PKG="${SCRIPT_DIR}/../package.json"
if [[ -f "${FRONT_PKG}" ]]; then
sed -i "s/\"version\": \"[^\"]*\"/\"version\": \"${FRONT_VERSION}\"/" "${FRONT_PKG}"
echo "package.json version updated → ${FRONT_VERSION}"
else
echo "Warning: package.json not found, skipping package.json version sync" >&2
fi
# Keep root pyproject.toml in sync.
BACKEND_PYPROJECT="${SCRIPT_DIR}/../pyproject.toml"
if [[ -f "${BACKEND_PYPROJECT}" ]]; then
# Update version under [project] section if present
if grep -q '^\[project\]' "${BACKEND_PYPROJECT}"; then
sed -i "/^\[project\]/,/^\[/ s/^version = \".*\"/version = \"${FRONT_VERSION}\"/" "${BACKEND_PYPROJECT}"
else
sed -i "s/^version = \".*\"/version = \"${FRONT_VERSION}\"/" "${BACKEND_PYPROJECT}"
fi
echo "pyproject.toml version updated → ${FRONT_VERSION}"
else
echo "Warning: pyproject.toml not found, skipping pyproject.toml version sync" >&2
fi
# ---------------------------------------------------------------------------
# Push containers
# ---------------------------------------------------------------------------
bash "${SCRIPT_DIR}/push.sh" "${TARGET}" "${NEW_TAG}"
bash "${SCRIPT_DIR}/push.sh" "${TARGET}"
# ---------------------------------------------------------------------------
# Git tag (local only; push after container build)
# ---------------------------------------------------------------------------
cd "${SCRIPT_DIR}/.."
git add Docker/VERSION package.json pyproject.toml
git commit -m "chore: bump version"
git tag -a "${NEW_TAG}" -m "Release ${NEW_TAG}"
echo "Local git commit + tag ${NEW_TAG} created."
# ---------------------------------------------------------------------------
# Push git commits & tag
# ---------------------------------------------------------------------------
git push origin HEAD
git push origin "${NEW_TAG}"
echo "Git commit and tag ${NEW_TAG} pushed."

View File

@@ -6,23 +6,29 @@ Verifies:
2. The container starts and becomes healthy.
3. The public IP inside the VPN differs from the host IP.
4. Kill switch blocks traffic when WireGuard is down.
5. AllowedIPs routes are set dynamically from the config.
Requirements:
- podman installed
- Root/sudo (NET_ADMIN capability)
- Root/sudo (NET_ADMIN capability) for container runtime tests
- A valid WireGuard config at ./wg0.conf (or ./nl.conf)
Usage:
# Build-only test (no sudo needed):
python3 -m pytest test_vpn.py::TestVPNImage::test_00_build_image -v
# Full integration test (requires sudo):
sudo python3 -m pytest test_vpn.py -v
# or
sudo python3 test_vpn.py
"""
import logging
import os
import subprocess
import sys
import time
import unittest
import os
logger = logging.getLogger(__name__)
@@ -35,6 +41,11 @@ STARTUP_TIMEOUT = 30 # seconds to wait for VPN to come up
HEALTH_POLL_INTERVAL = 2 # seconds between health checks
def is_root() -> bool:
"""Check if running as root."""
return os.geteuid() == 0
def run(cmd: list[str], timeout: int = 30, check: bool = True) -> subprocess.CompletedProcess:
"""Run a command and return the result."""
return subprocess.run(cmd, capture_output=True, text=True, timeout=timeout, check=check)
@@ -55,6 +66,7 @@ class TestVPNImage(unittest.TestCase):
"""Test suite for the WireGuard VPN container."""
host_ip: str = ""
container_id: str = ""
@classmethod
def setUpClass(cls):
@@ -84,6 +96,12 @@ class TestVPNImage(unittest.TestCase):
assert result.returncode == 0, f"Build failed:\n{result.stderr}"
logger.info("Image built successfully.")
# Skip container runtime tests if not root
if not is_root():
logger.warning("Not running as root — skipping container runtime tests.")
cls.container_id = ""
return
# ── 3. Start the container ──
logger.info("Starting container '%s'...", CONTAINER_NAME)
result = run(
@@ -120,6 +138,8 @@ class TestVPNImage(unittest.TestCase):
@classmethod
def tearDownClass(cls):
"""Stop and remove the container."""
if not is_root():
return
logger.info("Cleaning up test container...")
subprocess.run(["podman", "rm", "-f", CONTAINER_NAME], capture_output=True, check=False)
logger.info("Cleanup complete.")
@@ -144,10 +164,22 @@ class TestVPNImage(unittest.TestCase):
)
return result.stdout.strip()
def _skip_if_not_root(self):
"""Skip test if not running as root."""
if not is_root():
self.skipTest("This test requires root/sudo privileges")
# ── Tests ────────────────────────────────────────────────
def test_00_build_image(self):
"""The image builds successfully."""
# This is already verified in setUpClass, just confirm here
result = run(["podman", "images", "--format", "{{.Repository}}:{{.Tag}}"])
self.assertIn(IMAGE_NAME, result.stdout, "Image was not built")
def test_01_ip_differs_from_host(self):
"""Public IP inside VPN is different from host IP."""
self._skip_if_not_root()
vpn_ip = self._get_vpn_ip()
logger.info("VPN public IP: %s", vpn_ip)
logger.info("Host public IP: %s", self.host_ip)
@@ -161,12 +193,42 @@ class TestVPNImage(unittest.TestCase):
def test_02_wireguard_interface_exists(self):
"""The wg0 interface is present in the container."""
self._skip_if_not_root()
result = podman_exec(CONTAINER_NAME, ["wg", "show", "wg0"])
self.assertEqual(result.returncode, 0, f"wg show failed:\n{result.stderr}")
self.assertIn("peer", result.stdout.lower(), "No peer information in wg show output")
# AllowedIPs should be present in wg show output
self.assertIn("allowed ips", result.stdout.lower(), "AllowedIPs not found in wg show output")
def test_03_kill_switch_blocks_traffic(self):
def test_03_allowedips_routes_set(self):
"""Routes are set dynamically based on AllowedIPs from config."""
self._skip_if_not_root()
# Check that routes exist for the AllowedIPs
result = podman_exec(CONTAINER_NAME, ["ip", "route", "show", "dev", "wg0"])
self.assertEqual(result.returncode, 0, f"ip route show failed:\n{result.stderr}")
# The config has AllowedIPs = 0.0.0.0/0, which should result in:
# 0.0.0.0/1 dev wg0 and 128.0.0.0/1 dev wg0
self.assertIn("0.0.0.0/1", result.stdout, "Route 0.0.0.0/1 not found")
self.assertIn("128.0.0.0/1", result.stdout, "Route 128.0.0.0/1 not found")
# Make sure there is NO default route through wg0 (Table = off should prevent this)
self.assertNotIn("default dev wg0", result.stdout, "Default route through wg0 found — Table = off not working!")
logger.info("AllowedIPs routes verified: %s", result.stdout.strip())
def test_03b_dns_configured(self):
"""DNS is configured correctly with multiple nameserver lines."""
self._skip_if_not_root()
result = podman_exec(CONTAINER_NAME, ["cat", "/etc/resolv.conf"])
self.assertEqual(result.returncode, 0, f"cat /etc/resolv.conf failed:\n{result.stderr}")
# Should have two separate nameserver lines, not one with commas
self.assertIn("nameserver 198.18.0.1", result.stdout, "DNS 198.18.0.1 not found")
self.assertIn("nameserver 198.18.0.2", result.stdout, "DNS 198.18.0.2 not found")
# Make sure there are no commas in nameserver lines
self.assertNotIn("nameserver 198.18.0.1,198.18.0.2", result.stdout, "DNS servers written on one line with comma!")
logger.info("DNS config verified: %s", result.stdout.strip())
def test_04_kill_switch_blocks_traffic(self):
"""When WireGuard is down, traffic is blocked (kill switch)."""
self._skip_if_not_root()
# Bring down the WireGuard interface by deleting it
down_result = podman_exec(CONTAINER_NAME, ["ip", "link", "del", "wg0"], timeout=10)
self.assertEqual(down_result.returncode, 0, f"ip link del wg0 failed:\n{down_result.stderr}")

View File

@@ -1,10 +1,18 @@
[Interface]
PrivateKey = iO5spIue/6ciwUoR95hYtuxdtQxV/Q9EOoQ/jHe18kM=
Address = 10.2.0.2/32
DNS = 10.2.0.1
PrivateKey = EPRa2f/v72LvIXAY4yqIRJifsSb+nCcYHqC2rwA94UI=
Address = 100.64.244.78/32
#DNS = 198.18.0.1,198.18.0.2
DNS = 8.8.8.8
# Route zum VPN-Server direkt über dein lokales Netz
PostUp = ip route add 91.148.236.64 via 192.168.178.1 dev wlp4s0f0
PostUp = ip route add 192.168.178.0/24 via 192.168.178.1 dev wlp4s0f0
PostDown = ip route del 91.148.236.64 via 192.168.178.1 dev wlp4s0f0
PostDown = ip route del 192.168.178.0/24 via 192.168.178.1 dev wlp4s0f0
[Peer]
PublicKey = J4XVdtoBVc/EoI2Yk673Oes97WMnQSH5KfamZNjtM2s=
PublicKey = KgTUh3KLijVluDvNpzDCJJfrJ7EyLzYLmdHCksG4sRg=
AllowedIPs = 0.0.0.0/0
Endpoint = 185.183.34.149:51820
Endpoint = 91.148.236.64:51820
PersistentKeepalive = 25

View File

@@ -293,7 +293,7 @@ The FastAPI lifespan function (`src/server/fastapi_app.py`) runs the following s
9. Scheduler service started
+-- Cron-based library rescans configured
+-- Optional: auto-download missing episodes after rescan
+-- Optional: folder maintenance (NFO repair, renaming, poster checks) during scheduled runs
+-- Optional: folder maintenance (NFO repair, key resolution, renaming, poster checks) during scheduled runs
```
### 12.2 Temp Folder Guarantee

View File

@@ -41,6 +41,15 @@ This changelog follows [Keep a Changelog](https://keepachangelog.com/) principle
### Added
- **Encoding detection for HTML parsing** (`src/core/providers/aniworld_provider.py`):
Added `_decode_html_content()` function that uses `chardet` to detect the actual
encoding of HTML content before parsing. Falls back to UTF-8 with `errors='replace'`
to handle pages with mismatched encoding declarations. Applied to all BeautifulSoup
parsing calls to prevent "Some characters could not be decoded" warnings.
- **chardet dependency**: Added `chardet>=5.2.0` to `requirements.txt` for encoding detection.
### Added
- **Temp file cleanup after every download** (`src/core/providers/aniworld_provider.py`,
`src/core/providers/enhanced_provider.py`): Module-level helper
`_cleanup_temp_file()` removes the working temp file and any yt-dlp `.part`
@@ -130,6 +139,10 @@ This changelog follows [Keep a Changelog](https://keepachangelog.com/) principle
- Modified `src/server/api/anime.py` to save scanned episodes to database
- Episodes table properly tracks missing episodes with automatic cleanup
### Deprecated
- **Legacy Series Files (key/data)**: File-based series storage is deprecated. `key` and `data` files in anime folders will be removed in v3.0.0. Database storage is now the primary method. See [docs/MIGRATION_GUIDE.md](docs/MIGRATION_GUIDE.md) for details.
---
## Sections for Each Release

View File

@@ -83,17 +83,23 @@ Source: [src/server/database/models.py](../src/server/database/models.py), [src/
### 3.2 anime_series
Stores anime series metadata.
Stores anime series metadata. Corresponds to the core `Serie` class.
| Column | Type | Constraints | Description |
| ------------ | ------------- | -------------------------- | ------------------------------------------------------- |
| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Internal database ID |
| `key` | VARCHAR(255) | UNIQUE, NOT NULL, INDEX | **Primary identifier** - provider-assigned URL-safe key |
| `name` | VARCHAR(500) | NOT NULL, INDEX | Display name of the series |
| `site` | VARCHAR(500) | NOT NULL | Provider site URL |
| `folder` | VARCHAR(1000) | NOT NULL | Filesystem folder name (metadata only) |
| `created_at` | DATETIME | NOT NULL, DEFAULT NOW | Record creation timestamp |
| `updated_at` | DATETIME | NOT NULL, ON UPDATE NOW | Last update timestamp |
| Column | Type | Constraints | Description |
| ---------------- | ------------- | -------------------------- | ------------------------------------------------------- |
| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Internal database ID |
| `key` | VARCHAR(255) | UNIQUE, NOT NULL, INDEX | **Primary identifier** - provider-assigned URL-safe key |
| `name` | VARCHAR(500) | NOT NULL, INDEX | Display name of the series |
| `site` | VARCHAR(500) | NOT NULL | Provider site URL |
| `folder` | VARCHAR(1000) | NOT NULL | Filesystem folder name (metadata only) |
| `year` | INTEGER | NULLABLE | Release year of the series |
| `nfo_path` | VARCHAR(1000) | NULLABLE | Path to tvshow.nfo metadata file |
| `tmdb_id` | INTEGER | NULLABLE, INDEX | TMDB (The Movie Database) ID for metadata |
| `tvdb_id` | INTEGER | NULLABLE, INDEX | TVDB (TheTVDB) ID for metadata |
| `has_nfo` | BOOLEAN | NOT NULL, DEFAULT FALSE | Whether tvshow.nfo exists |
| `loading_status` | VARCHAR(50) | NOT NULL, DEFAULT 'completed' | Status: pending, loading_episodes, loading_nfo, completed, failed |
| `created_at` | DATETIME | NOT NULL, DEFAULT NOW | Record creation timestamp |
| `updated_at` | DATETIME | NOT NULL, ON UPDATE NOW | Last update timestamp |
**Identifier Convention:**
@@ -101,7 +107,13 @@ Stores anime series metadata.
- `folder` is **metadata only** for filesystem operations (e.g., `"Attack on Titan (2013)"`)
- `id` is used only for database relationships
Source: [src/server/database/models.py](../src/server/database/models.py#L23-L87)
**EpisodeDict Mapping:**
The `episodeDict` (season → episode numbers mapping) is stored as individual `Episode` records:
- Each `Episode` has `season` and `episode_number` columns
- Relationship: `AnimeSeries.episodes` returns all Episode records for that series
Source: [src/server/database/models.py](../src/server/database/models.py#L23-L150)
### 3.3 episodes
@@ -441,7 +453,187 @@ items = await db.execute(
---
## 12. Database Location
## 12. Series Storage: Database vs Files (Deprecated)
### File-Based Storage (Removed in v2.0)
Prior to v2.0, series metadata was stored in two files per anime folder:
| File | Contents |
| -------- | ------------------------------------------------------- |
| `key` | Series provider key (e.g., `"attack-on-titan"`) |
| `data` | JSON serialization of `Serie` object |
File structure example:
```
/anime/Attack on Titan (2013)/
├── key # Contains: attack-on-titan
├── data # Contains: {"key": "...", "name": "...", "episodeDict": {...}}
├── Season 1/
│ └── ...
```
### Database Storage (Current)
Since v2.0, all series metadata is stored in the `anime_series` table with `Episode` records for episode tracking. This provides:
- **ACID transactions** for data consistency
- **Foreign key constraints** (cascade delete)
- **Indexed queries** for fast lookups
- **No filesystem dependency** for metadata
### Migration from Files to Database
The `Serie.save_to_file()` and `Serie.load_from_file()` methods are deprecated but still functional for backward compatibility during migration:
```python
from src.core.entities.series import Serie
# Old file-based loading (deprecated)
serie = Serie.load_from_file("/anime/Attack on Titan (2013)/data")
# New database-based loading
from src.server.database.service import AnimeSeriesService
serie = await AnimeSeriesService.get_by_key(db, "attack-on-titan")
```
### Removing File Dependencies
After verifying database schema supports all fields, file-based storage can be removed:
1. ✅ Schema verified: All `Serie` fields have corresponding DB columns
2. ✅ Migration complete: All existing series migrated to database
3. ❌ File cleanup: Remove `key` and `data` files (pending)
**Note:** The `save_to_file()` and `load_from_file()` methods will be removed in v3.0.0.
---
## 12. Series Persistence Flow
When a directory scan discovers or updates series, the scanner persists data to the database instead of writing to disk files.
### Scan Flow
```
Scan Directory
Find MP4 Files → Extract Serie Key
Check DB for Existing Series (by key)
├─── EXISTS ──────────────────────► Update Series Metadata
│ │
│ ▼
│ Sync Episodes to DB
│ │
│◄──────────────────────────────────────┘
└─── NEW ───────────────────────────► Create New Series Record
Create Episode Records
Return to Scan Loop
```
### Key Methods
**SerieScanner._persist_serie_to_db()**
- Called after `get_missing_episodes_and_season()` computes episodeDict
- Uses `AnimeSeriesService.get_by_key()` to check if series exists
- If exists: calls `AnimeSeriesService.update()` + `_sync_episodes_to_db()`
- If new: calls `AnimeSeriesService.create()` + creates episodes
**SerieScanner._sync_episodes_to_db()**
- Gets existing episodes from DB via `EpisodeService.get_by_series()`
- Compares with new episodeDict
- Removes episodes no longer missing (unless `is_downloaded=True`)
- Adds new missing episodes
- Preserves `is_downloaded=True` episodes when removing missing ones
**SerieList.add_to_db()**
- Used when adding a new discovered series via API
- Creates filesystem folder + database record + episode records
### Episode Sync Logic
```python
# For each episode in DB but not in new episodeDict:
if episode.is_downloaded:
# Keep - file exists, don't remove
pass
else:
# Remove - no longer missing
EpisodeService.delete()
# For each episode in new episodeDict but not in DB:
# Add as new missing episode
EpisodeService.create(is_downloaded=False)
```
### Transaction Handling
- DB operations use their own session with commit/rollback
- If DB write fails, error is logged and scan continues
- File-based `save_to_file()` no longer called during scan
### Migration Path
1. v2.x: Scanner writes to both DB (primary) and files (fallback)
2. v3.0: Scanner writes only to DB, file methods removed
---
## 13. Series Persistence
### Schema
**AnimeSeries Table**: Stores series metadata (key, name, site, folder, year)
| Column | Type | Constraints | Description |
|-----------|--------------|---------------------------|----------------------|
| `id` | INTEGER | PRIMARY KEY | Auto-increment |
| `key` | VARCHAR(255) | UNIQUE, NOT NULL | Series provider key |
| `name` | VARCHAR(500) | NOT NULL | Display name |
| `site` | VARCHAR(500) | | Provider site URL |
| `folder` | VARCHAR(1000)| | Filesystem folder |
**Episode Table**: Stores per-episode metadata (season, episode_number, is_downloaded)
| Column | Type | Constraints | Description |
|-----------------|--------------|---------------------------|----------------------|
| `id` | INTEGER | PRIMARY KEY | Auto-increment |
| `series_id` | INTEGER | FOREIGN KEY → anime_series| Parent series |
| `season` | INTEGER | NOT NULL | Season number |
| `episode_number`| INTEGER | NOT NULL | Episode number |
| `is_downloaded` | BOOLEAN | DEFAULT FALSE | Download status |
### Relationships
- `AnimeSeries.episodes` → List of Episode objects (one-to-many)
- `Episode.series` → Parent AnimeSeries (many-to-one)
- Cascade delete: Deleting a series removes all its episodes
### Queries
```python
# Get all series with episodes
AnimeSeriesService.get_all(db, with_episodes=True)
# Get by provider key
AnimeSeriesService.get_by_key(db, key)
# Get by folder path
AnimeSeriesService.get_by_folder(db, folder)
```
---
## 14. Database Location
| Environment | Default Location |
| ----------- | ------------------------------------------------- |

View File

@@ -61,4 +61,376 @@ This document provides guidance for developers working on the Aniworld project.
- Commit message format
- Pull request process
8. Common Development Tasks
### Adding Queue Deduplication
The download queue prevents duplicate entries at two levels:
**In-Memory Deduplication** (`src/server/services/download_service.py`):
- `_pending_by_episode` dict tracks pending episodes: key = `(serie_id, season, episode)`
- `_add_to_pending_queue()` updates the dict when adding items
- `add_to_queue()` checks this dict before adding episodes (includes batch-local dedup)
- `_remove_from_pending_queue()` cleans up the dict when items are removed
**Database Constraint** (`src/server/models.py`):
- `DownloadQueueItem` has a unique index on `episode_id` via `__table_args__`
- Prevents duplicate queue entries at the database level
- Unique constraint: `Index("ix_download_queue_episode_pending", "episode_id", unique=True)`
**Scheduler Cooldown** (`src/server/services/scheduler_service.py`):
- `_last_auto_download_time` tracks when auto-download last ran
- 5-minute cooldown prevents rapid re-triggers
- Checked at start of `_auto_download_missing()`
### Episode Lifecycle
Episodes transition through states stored in the `episodes` table:
| State | `is_downloaded` | `file_path` | Description |
|-------|----------------|-------------|-------------|
| Missing | `False` | `NULL` | Episode not yet downloaded |
| Downloaded | `True` | Set | Episode exists on disk |
**State Transitions:**
1. **Missing → Downloaded**: When download completes, `_remove_episode_from_missing_list()` calls `EpisodeService.mark_downloaded()` to set `is_downloaded=True` and populate `file_path`. The episode record is NOT deleted.
**Query Implications:**
- `get_series_with_missing_episodes()`: Filters for `is_downloaded=False` to find series with undownloaded episodes
- `get_series_with_no_episodes()`: Finds series with `is_downloaded=False` episodes but NO `is_downloaded=True` episodes (completely unwatched series)
### Mocking the Download Queue
When testing components that use the download queue:
```python
# Mock repository for unit tests
class MockQueueRepository:
def __init__(self):
self._items: Dict[str, DownloadItem] = {}
async def save_item(self, item: DownloadItem) -> DownloadItem:
self._items[item.id] = item
return item
async def get_all_items(self) -> List[DownloadItem]:
return list(self._items.values())
# Use in fixture
@pytest.fixture
def mock_queue_repository():
return MockQueueRepository()
@pytest.fixture
def download_service(mock_anime_service, mock_queue_repository):
return DownloadService(
anime_service=mock_anime_service,
queue_repository=mock_queue_repository,
max_retries=3,
)
```
9. Troubleshooting Development Issues
### Async Context Managers for aiohttp
All `aiohttp.ClientSession` usages must be wrapped in `async with`:
```python
# Correct — session properly closed on exit
async with TMDBClient(api_key="key") as client:
result = await client.search_tv_show("Show")
# Wrong — session may leak if exception occurs
client = TMDBClient(api_key="key")
result = await client.search_tv_show("Show")
await client.close() # May not be called if exception raised earlier
```
**Why:**
- `aiohttp.ClientSession` holds TCP connections that must be explicitly closed
- If exception occurs before `close()`, session leaks
- Context manager guarantees `__aexit__` runs even on exceptions
**Services that use aiohttp:**
- `TMDBClient` — has `__aenter__`/`__aexit__`, use `async with`
- `ImageDownloader` — has `__aenter__`/`__aexit__`, use `async with`
- `NFOService` — wraps both above, use `async with`
**Verification:**
- Missing context manager usage triggers `__del__` warning on garbage collection
- Integration tests verify no "Unclosed client session" errors in logs
### Scheduler Persistence and Recovery
The scheduler uses APScheduler's in-memory job store. Jobs are reconstructed from `config.json` on every startup — no separate database is needed.
```python
# Jobs are built from config on startup — no persistence DB required
scheduler = AsyncIOScheduler() # default MemoryJobStore
scheduler.add_job(..., replace_existing=True)
```
**Startup misfire recovery:** On `start()`, the scheduler checks `system_settings.last_scan_timestamp` in `aniworld.db`. If the last scan is overdue (>23h but <25h ago), an immediate rescan is triggered. This replaces APScheduler's built-in misfire handling which required a separate SQLite database.
**Grace period:** If the server was down for more than 25 hours, no automatic recovery occurs to avoid surprise rescans after long downtime.
**Health endpoint:** `GET /health` returns `scheduler_next_run` and `scheduler_last_run` for external monitors (Uptime Kuma, Prometheus, etc.).
**If server is down too long:** Manual trigger via `POST /api/scheduler/trigger-rescan` or wait for next scheduled run.
### Database Session Management
`get_async_session_factory()` returns a **new AsyncSession instance** directly (not a factory). The function name is historical — callers receive the session immediately:
```python
# Correct usage:
db = get_async_session_factory() # db IS the session
await db.execute(...)
await db.commit()
await db.close()
```
Do NOT call the result again with `()` — that tries to call an `AsyncSession` object, causing `'AsyncSession' object is not callable`.
For context manager usage, prefer `get_db_session()` (auto-commits) or `get_transactional_session()` (manual commit).
### Health Check Endpoints
The application provides health check endpoints for monitoring and container orchestration:
#### `GET /health`
Basic health check returning service status and startup health check results.
**Response fields:**
- `status`: "healthy", "degraded", or "unhealthy" based on startup checks
- `timestamp`: ISO timestamp of the check
- `series_app_initialized`: Whether the series app is loaded
- `anime_directory_configured`: Whether anime_directory is set
- `scheduler_next_run` / `scheduler_last_run`: Scheduler times
- `checks`: Detailed startup check results (ffmpeg, DNS, anime_directory)
#### `GET /health/ready`
Readiness check for container orchestrators (Kubernetes, Docker Swarm).
**Response when ready:**
```json
{
"status": "ready",
"ready": true,
"timestamp": "2024-01-01T00:00:00",
"checks": {...}
}
```
**Response when not ready (503):**
```json
{
"status": "not_ready",
"ready": false,
"timestamp": "2024-01-01T00:00:00",
"critical_failures": ["anime_directory: not configured"],
"checks": {...}
}
```
#### `GET /health/detailed`
Comprehensive health check including database, filesystem, and system metrics.
#### Startup Health Checks
On application startup, the following checks are performed:
| Check | Failure Status | Impact |
|-------|---------------|--------|
| `ffmpeg` | warning | HLS downloads may fail |
| `dns_aniworld` | warning | Provider requests may fail |
| `dns_tmdb` | warning | TMDB API calls may fail |
| `anime_directory` | error | Download service disabled |
DNS checks are warnings because failures can be transient. anime_directory errors disable the download service to prevent failures.
### Troubleshooting Development Issues
#### Scheduler missed a run
1. Server was down at scheduled time (03:00 UTC by default).
2. On restart, the scheduler checks `last_scan_timestamp` — if overdue by 23-25h, it triggers immediately.
3. If server was down >25 hours, missed job is skipped to avoid surprise rescans.
4. Trigger manually: `POST /api/scheduler/trigger-rescan`
5. Monitor next run: `GET /health``scheduler_next_run`
#### Scheduler not firing (no events at scheduled time)
If the scheduler appears configured but never triggers:
1. **Check application logs for scheduler startup:**
```
grep "Scheduler service started" fastapi_app.log
```
- If missing, the scheduler failed to start — check for errors above this line
- If present, scheduler started successfully
2. **Verify the job is registered:**
```
grep "Scheduler started with cron trigger" fastapi_app.log
```
3. **Verify APScheduler events in logs:**
```
grep "apscheduler.executors.default" fastapi_app.log
```
- `Running job` = job triggered
- `executed successfully` = job completed
- No output = job never fired
4. **Test manual trigger:**
```bash
curl -X POST http://localhost:8000/api/scheduler/trigger-rescan -H "Authorization: Bearer <token>"
```
- If manual trigger works but cron doesn't, the issue is APScheduler configuration
5. **Check next_run_time via health endpoint:**
```bash
curl http://localhost:8000/health | jq .scheduler_next_run
```
- If `null`, the job is not scheduled
- If set, the scheduler knows when to run next
6. **Check timezone handling:**
- APScheduler uses UTC internally
- The schedule_time config (e.g., "03:00") is interpreted as UTC
- If you expect local time, adjust the schedule_time accordingly
#### Startup health check failures
If `/health` returns `unhealthy` status:
1. **anime_directory error**: Directory not configured or not writable
- Check `ANIME_DIRECTORY` environment variable
- Verify directory exists and permissions allow write access
- Download service will not initialize until resolved
2. **ffmpeg warning**: ffmpeg not found in PATH
- HLS stream downloads will fail
- Install ffmpeg: `apt install ffmpeg` or `brew install ffmpeg`
3. **DNS warnings**: Domain resolution failed
- Check network connectivity
- DNS failures are transient — warnings don't block startup
- Retry later to verify: `GET /health`
### Provider Failure Handling
Download providers (VOE, Doodstream, Vidmoly, Vidoza, SpeedFiles, Streamtape,
Luluvdo) regularly break: URLs expire, sites change their player markup, geo
blocks appear, and `yt-dlp` extractors lag behind upstream changes. The
`AniworldLoader.download()` flow is designed to fail fast and rotate.
**Rotation order**
1. The episode page is scraped for the providers AniWorld actually advertises.
2. Results are ordered by the preference in `DEFAULT_PROVIDERS`
(`provider_config.py`); providers not listed run last.
3. For each candidate the loader:
1. Calls `_check_url_alive()` — HEAD probe with GET fallback. Any 4xx
response or connection error skips the provider immediately.
2. Resolves the redirect via `_resolve_direct_link()` to obtain a direct
stream URL plus headers. Provider-specific extractors (e.g. `VOE`) are
preferred; unknown providers fall back to the embed URL so `yt-dlp` can
attempt extraction.
3. Tries `_try_direct_stream()` — straight `requests.get(stream=True)` when
`Content-Type` is `video/*` or `application/octet-stream`. This avoids
`yt-dlp` entirely for direct MP4 links.
4. Falls back to `yt-dlp` with the ffmpeg downloader for HLS streams.
4. On any failure, temp files are cleaned and the loop moves to the next
provider. When the chain is exhausted, the loader logs
`All download providers failed for S{season}E{episode} ...; tried=[...]`
to both the application log and `logs/download_errors.log`.
**Do not hardcode provider URLs.** Provider domains shift constantly (e.g.
Doodstream alternates between `dood.li`, `dood.so`, `dood.la`). Only the
referer hints in `PROVIDER_HEADERS` are persisted — discovery still happens
at runtime through AniWorld's redirect endpoint.
### HLS Stream Handling
HLS (HTTP Live Streaming) manifests (`.m3u8`) require yt-dlp to use the
`ffmpeg` downloader with `--hls-use-mpegts`. Both providers configure this
automatically:
```python
ydl_opts = {
"downloader": "ffmpeg", # Use ffmpeg instead of native
"hls_use_mpegts": True, # Write transport stream (.ts) segments
}
```
**Why this matters**: Without ffmpeg, yt-dlp logs:
`"Live HLS streams are not supported by the native downloader"`
**Requirements**:
- ffmpeg must be installed and in PATH (`which ffmpeg`)
- Install: `apt install ffmpeg` (Debian/Ubuntu) or `brew install ffmpeg` (macOS)
- Startup health check (see Health Check Endpoints) verifies ffmpeg presence
**Trade-offs**:
- HLS downloads are slower than direct MP4 (reassembly of .ts segments)
- Requires more disk space during download
- May need post-processing if .ts format is not desired
**Detection**: VOE provider extracts HLS URLs via `HLS_PATTERN` regex. Other
providers let yt-dlp auto-detect from URL/content-type.
### Updating yt-dlp
When extractors break (typical symptoms: every provider HEAD probe succeeds
but `yt-dlp` raises `Unable to extract` or `HTTP Error 404`):
1. Check the upstream tracker first: https://github.com/yt-dlp/yt-dlp/issues
2. Upgrade in the conda environment:
```bash
conda run -n AniWorld pip install --upgrade yt-dlp
```
3. Smoke-test against a known-good episode before pinning a new floor in
`requirements.txt` (`yt-dlp>=YYYY.MM.DD`).
4. Re-run the provider test suite:
```bash
conda run -n AniWorld python -m pytest tests/unit/test_aniworld_provider.py -v
```
5. If a specific extractor is removed upstream, drop the provider from
`DEFAULT_PROVIDERS` rather than patching `yt-dlp` in tree.
### User Notification on Total Failure
`SeriesApp.download_episode()` already emits a `download_status="failed"`
WebSocket event when `loader.download()` returns `False`. Operators should
forward this to `notification_service.notify_download_failed()` so users see
a HIGH-priority alert. The loader keeps the failure detail in
`logs/download_errors.log` for post-mortem.
## Series Storage
### Overview
Series metadata now stored in the database (SQLAlchemy ORM).
Legacy files (`key` and `data` per folder) are deprecated but preserved
for backward compatibility.
### Architecture
- **Database**: Single source of truth for all series metadata
- **In-Memory Cache**: SeriesApp maintains a cache for performance
- **Filesystem**: Only used for episode files themselves, not metadata
### Migration
First startup after upgrade automatically imports any legacy
series files into the database.
### Legacy Files
- `key` file: Contains series provider key (deprecated)
- `data` file: Contains Serie JSON object (deprecated)
Both are safe to delete after migration; not needed for normal operation.

111
docs/MIGRATION_GUIDE.md Normal file
View File

@@ -0,0 +1,111 @@
# Migration Guide: File-Based to Database Storage
## Overview
This guide covers the transition from file-based series metadata storage to the new database-backed system introduced in v2.0.
## What Changed
**Before v2.0**: Series metadata stored in `key` and `data` files alongside anime folders.
**After v2.0**: All metadata stored in SQLite database (`aniworld.db`). Files are deprecated but still supported for backward compatibility during migration.
## Automated Migration
The application automatically migrates on first startup:
1. Scans anime directory for `key` and `data` files
2. Parses legacy files into `AnimeSeries` and `Episode` records
3. Loads series into in-memory cache
4. Logs migration results
**No manual action required.**
## Manual Verification
After first startup with the new version:
1. **Check logs** for: `"Migrated X series from files to DB"`
2. **Verify series count**: UI shows same number of series as before
3. **Confirm episodes**: Episode counts match expected totals
```bash
# Check migration log
grep "Migrated" logs/app.log
# Verify series via API
curl http://localhost:8000/api/anime | jq '.total'
```
## After Migration
### Safe to Delete
Once verified, these files can be removed:
```
<anime_folder>/
├── Attack on Titan (2013)/
│ ├── key # ❌ Can delete
│ ├── data # ❌ Can delete
│ └── Season 1/
│ └── ...
```
**Deleting these files does not affect the database.** The metadata now lives in `aniworld.db`.
### Backup (Recommended)
Before deleting, backup the files:
```bash
# Create backup directory
mkdir -p backup/legacy_series_files
# Copy all key and data files
find /path/to/anime -name "key" -o -name "data" | while read f; do
cp "$f" "backup/legacy_series_files/"
done
```
## Reverting (Not Recommended)
If you must revert to file-based storage:
1. **Restore from database backup** (if available)
2. **Export manually** (no export script exists)
**Warning**: File-based storage is deprecated and will be removed in v3.0.0.
## Troubleshooting
### Series Not Appearing After Migration
1. Check logs for migration errors: `grep -i error logs/app.log`
2. Verify `key` and `data` files exist and are readable
3. Manually trigger rescan: `POST /api/scheduler/trigger-rescan`
### Duplicate Series
1. Check for duplicate `key` files (same series in multiple folders)
2. Verify series key uniqueness in database:
```bash
sqlite3 aniworld.db "SELECT key, COUNT(*) FROM anime_series GROUP BY key HAVING COUNT(*) > 1;"
```
### Missing Episodes
1. Trigger targeted scan for affected series
2. Check episode sync logs
3. Verify file permissions on anime directory
## Deprecation Timeline
| Version | Status |
|---------|--------|
| v2.0.x | Legacy files supported, migration automated |
| v2.1.x | Legacy files still supported, warnings in logs |
| v3.0.0 | **Legacy files removed** - database only |
Upgrade to v3.0.0 before legacy file support ends.

View File

@@ -171,6 +171,35 @@ Response:
}
```
### 3.6 Fallback Behavior When TMDB is Unavailable
When TMDB lookup fails (network issues, API errors, or no match found), the system creates a **minimal NFO** to ensure the series is still tracked. This behavior applies to:
- Manual NFO creation via API
- Batch NFO creation operations
- Automatic NFO creation during downloads
**What a minimal NFO contains:**
```xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<tvshow>
<title>Series Name</title>
<year>2024</year>
<plot>No metadata available for Series Name. TMDB lookup failed.</plot>
</tvshow>
```
**Limitations of minimal NFOs:**
- No poster, logo, or fanart images
- No rating, genre, or studio information
- No TMDB or other provider IDs
- May not display correctly in some media servers
**To upgrade a minimal NFO:**
1. Use the Update endpoint (`PUT /api/nfo/{serie_id}/update`) when TMDB is available
2. Or delete the NFO and recreate it with full metadata
---
## 4. File Structure
@@ -217,6 +246,7 @@ NFO files are created in the anime directory:
<genre>Action</genre>
<genre>Sci-Fi & Fantasy</genre>
<uniqueid type="tmdb">1429</uniqueid>
<tmdbid>1429</tmdbid>
<thumb aspect="poster">https://image.tmdb.org/t/p/w500/...</thumb>
<fanart>
<thumb>https://image.tmdb.org/t/p/original/...</thumb>
@@ -224,6 +254,13 @@ NFO files are created in the anime directory:
</tvshow>
```
**Manual TMDB ID Override**: To skip TMDB search and use a specific ID directly, include `<tmdbid>YOUR_ID</tmdbid>` in the NFO. This is useful when:
- TMDB search fails for your series (e.g., new or obscure anime)
- You already know the correct TMDB ID
- You want to avoid rate limiting from repeated searches
Aniworld reads `<tmdbid>` element and `<uniqueid type="tmdb">` first. If found, it uses the ID directly instead of searching.
### 4.3 Episode NFO Format
```xml
@@ -600,6 +637,36 @@ Every poster check action is logged:
4. Check network speed to TMDB servers
5. Verify disk I/O performance
### 6.7 TMDB Lookup Fails for My Series
**Problem**: TMDB search fails with "No results found" for a valid series.
**Solutions**:
1. **Check if series exists on TMDB**: Visit https://www.themoviedb.org and search for your series
2. **Use manual ID override**: Add TMDB ID directly to `tvshow.nfo`:
```xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<tvshow>
<title>Your Series Name</title>
<tmdbid>12345</tmdbid>
<uniqueid type="tmdb">12345</uniqueid>
</tvshow>
```
Aniworld will use this ID directly instead of searching.
3. **Try alternative titles**: Some anime have different titles (Japanese, romaji, English). If you have access to the folder, rename it to match the TMDB title.
4. **Add to existing NFO**: If `tvshow.nfo` exists but has no TMDB ID, edit it to add:
```xml
<tmdbid>YOUR_TMDB_ID</tmdbid>
```
Then use the Update endpoint to refresh metadata.
5. **Check for rate limiting**: If many lookups fail at once, you may be hitting TMDB rate limits. Wait and retry later.
6. **Verify API key**: Ensure your TMDB API key is valid and has not exceeded usage limits.
---
## 7. Best Practices

View File

@@ -62,6 +62,90 @@ This document describes the testing strategy, guidelines, and practices for the
- What to mock
- Mock patterns
- External service mocks
### Mocking the Download Queue
Use `MockQueueRepository` for testing download queue functionality:
```python
from src.server.models.download import DownloadItem, EpisodeIdentifier
class MockQueueRepository:
def __init__(self):
self._items: Dict[str, DownloadItem] = {}
async def save_item(self, item: DownloadItem) -> DownloadItem:
self._items[item.id] = item
return item
async def get_item(self, item_id: str) -> Optional[DownloadItem]:
return self._items.get(item_id)
async def get_all_items(self) -> List[DownloadItem]:
return list(self._items.values())
async def set_error(self, item_id: str, error: str) -> bool:
if item_id in self._items:
self._items[item_id].error = error
return True
return False
async def delete_item(self, item_id: str) -> bool:
if item_id in self._items:
del self._items[item_id]
return True
return False
async def clear_all(self) -> int:
count = len(self._items)
self._items.clear()
return count
```
**Key points:**
- The mock uses in-memory storage, no database required
- All async methods are implemented (even if just pass-through)
- `save_item` uses `item.id` as key (must be set before calling)
- Suitable for unit tests only (no persistence)
### Mocking aiohttp Sessions
When testing code that uses `aiohttp.ClientSession`:
```python
from unittest.mock import AsyncMock, MagicMock, patch
from aiohttp import ClientSession
# Mock aiohttp session for testing
class MockAiohttpSession:
def __init__(self):
self.closed = False
async def close(self):
self.closed = True
def get(self, url, **kwargs):
mock_response = AsyncMock()
mock_response.status = 200
mock_response.json = AsyncMock(return_value={"data": "test"})
mock_response.__aenter__ = AsyncMock(return_value=mock_response)
mock_response.__aexit__ = AsyncMock(return_value=None)
return mock_response
# Use in fixture
@pytest.fixture
async def mock_tmdb_session():
session = MockAiohttpSession()
yield session
# Cleanup verification
assert session.closed, "Session was not closed"
```
**Key points:**
- Always verify `session.closed` is `True` after context manager exits
- Mock `__aenter__` and `__aexit__` for response context managers
- Set `closed = False` on mock session for unclosed warning tests
7. Coverage Requirements
8. CI/CD Integration
9. Writing Good Tests

View File

@@ -1,10 +0,0 @@
review frontend code and check for architektre issues
write the tasks in Task.md
for each task add the following informations
where is that found
goal. how it should be
possibale traps and issues
docs changes needed
why this is needed

View File

@@ -31,14 +31,16 @@ flowchart TB
subgraph Core["Core Layer"]
SeriesApp["SeriesApp"]
SeriesCache["SeriesCache<br/>(In-Memory)"]
SerieScanner["SerieScanner"]
SerieList["SerieList"]
end
subgraph Data["Data Layer"]
SQLite[(SQLite<br/>aniworld.db)]
SQLite[("SQLite<br/>aniworld.db")]
ConfigJSON[(config.json)]
FileSystem[(File System<br/>Anime Directory)]
FileSystem[(File System<br/>Anime Episodes)]
LegacyFiles[("Legacy Files<br/>key/data<br/>(Deprecated)")]
end
subgraph External["External"]
@@ -71,9 +73,13 @@ flowchart TB
AnimeService --> SQLite
%% Core to Data
SeriesApp --> SeriesCache
SeriesCache -.->|Cached Series| SQLite
SeriesApp --> SerieScanner
SeriesApp --> SerieList
SerieScanner --> FileSystem
SerieScanner -->|Scan Episodes| FileSystem
SerieScanner -->|Detect Series| SQLite
SerieScanner -->|Migrate Legacy| LegacyFiles
SerieScanner --> Provider
%% Event flow

0
docs/helper Normal file
View File

View File

@@ -2,3 +2,50 @@ API key : 299ae8f630a31bda814263c551361448
/mnt/server/serien/Serien/
{
"name": "Aniworld",
"data_dir": "data",
"scheduler": {
"enabled": true,
"interval_minutes": 60,
"schedule_time": "03:00",
"schedule_days": [
"mon",
"tue",
"wed",
"thu",
"fri",
"sat",
"sun"
],
"auto_download_after_rescan": true,
"folder_scan_enabled": true
},
"logging": {
"level": "INFO",
"file": null,
"max_bytes": null,
"backup_count": 3
},
"backup": {
"enabled": false,
"path": "data/backups",
"keep_days": 30
},
"nfo": {
"tmdb_api_key": "9bc3e547caff878615cbdba2cc421d37",
"auto_create": true,
"update_on_scan": true,
"download_poster": true,
"download_logo": true,
"download_fanart": true,
"image_size": "original"
},
"other": {
"master_password_hash": "$pbkdf2-sha256$29000$HQNASKk1xpgTAgAgJGRMaQ$73TOCCM0UEZONyNXQEPa3SmIoXeG6C1l5mMFDNgYfMQ",
"anime_directory": "/data"
},
"version": "1.0.0"
}

154
docs/runner.csx Normal file
View File

@@ -0,0 +1,154 @@
#!/usr/bin/env dotnet-script
#nullable enable
using System;
using System.IO;
using System.Diagnostics;
using System.Threading;
using System.Text;
using System.Text.RegularExpressions;
using System.Linq;
using System.Collections.Generic;
// ── Ctrl+C: kill active process and exit cleanly ──────────────────────────────
var cts = new CancellationTokenSource();
Process? activeProcess = null;
Console.CancelKeyPress += (_, e) =>
{
e.Cancel = true;
Console.WriteLine("\n[runner] Interrupted — shutting down...");
cts.Cancel();
try { activeProcess?.Kill(entireProcessTree: true); } catch { }
};
// ── Paths ─────────────────────────────────────────────────────────────────────
var repoRoot = Directory.GetCurrentDirectory();
var tasksFile = Path.Combine(repoRoot, "Docs", "Tasks.md");
if (!File.Exists(tasksFile))
{
Console.Error.WriteLine($"[runner] ERROR: Tasks.md not found at {tasksFile}");
Console.Error.WriteLine("[runner] Run this script from the repository root.");
Environment.Exit(1);
}
// ── Read & split by "---" separator lines ────────────────────────────────────
var content = File.ReadAllText(tasksFile);
var items = Regex
.Split(content, @"\r?\n---\r?\n")
.Select(s => s.Trim())
.Where(s => s.Length > 0)
.ToList();
Console.WriteLine($"[runner] Found {items.Count} section(s) in Tasks.md");
// ── Helper: run copilot and stream output, return full output ─────────────────
async Task<string> RunCopilot(IEnumerable<string> extraArgs, string prompt)
{
var output = new StringBuilder();
var argList = new List<string> { "launch", "copilot", "--model", "minimax-m2.7:cloud", "--yes", "--", "--allow-all-tools" };
argList.AddRange(extraArgs);
argList.Add("-p");
argList.Add(prompt);
var psi = new ProcessStartInfo("ollama")
{
WorkingDirectory = repoRoot,
RedirectStandardOutput = true,
RedirectStandardError = true,
UseShellExecute = false,
};
foreach (var a in argList)
psi.ArgumentList.Add(a);
activeProcess = new Process { StartInfo = psi };
activeProcess.OutputDataReceived += (_, e) =>
{
if (e.Data is null) return;
Console.WriteLine(e.Data);
output.AppendLine(e.Data);
};
activeProcess.ErrorDataReceived += (_, e) =>
{
if (e.Data is null) return;
Console.Error.WriteLine(e.Data);
output.AppendLine(e.Data);
};
activeProcess.Start();
activeProcess.BeginOutputReadLine();
activeProcess.BeginErrorReadLine();
await activeProcess.WaitForExitAsync(cts.Token);
activeProcess = null;
return output.ToString();
}
// ── Main loop ─────────────────────────────────────────────────────────────────
for (int i = 0; i < items.Count; i++)
{
var item = items[i];
if (cts.IsCancellationRequested) break;
Console.WriteLine();
Console.WriteLine("[runner] ══════════════════════════════════════════════");
Console.WriteLine($"[runner] Task:\n{item}");
Console.WriteLine("[runner] ══════════════════════════════════════════════");
Console.WriteLine();
// Step 1 — run the task prompt
await RunCopilot(Enumerable.Empty<string>(), $"/caveman full");
await RunCopilot(new[] { "--continue" }, $"read ./Docs/instructions.md. {item}");
if (cts.IsCancellationRequested) break;
// Step 2 — confirm completion in the same chat session
Console.WriteLine("\n[runner] Asking for task confirmation...\n");
var confirmation = await RunCopilot(
new[] { "--continue" },
"are you sure tasks is done. reply with yes"
);
if (cts.IsCancellationRequested) break;
// Step 3 — check for "yes" in the reply, with retry logic for issue resolution
int maxRetries = 3;
int retryCount = 0;
bool taskConfirmed = confirmation.Contains("yes", StringComparison.OrdinalIgnoreCase);
while (!taskConfirmed && retryCount < maxRetries)
{
retryCount++;
Console.WriteLine($"\n[runner] Attempt {retryCount}/{maxRetries}: Resolving remaining issues and running tests...\n");
confirmation = await RunCopilot(
new[] { "--continue" },
"resolve any remaining issues, make sure all tests are running and pass. then confirm with yes if done"
);
if (cts.IsCancellationRequested) break;
taskConfirmed = confirmation.Contains("yes", StringComparison.OrdinalIgnoreCase);
}
if (!taskConfirmed)
{
Console.WriteLine($"\n[runner] Task not confirmed as done after {maxRetries} attempts. Stopping.");
break;
}
// Step 4 — commit the work
Console.WriteLine("\n[runner] Task confirmed. Making git commit...\n");
await RunCopilot(Enumerable.Empty<string>(), $"/caveman full");
await RunCopilot(new[] { "--continue" }, "make git commit");
if (cts.IsCancellationRequested) break;
// Step 5 — remove completed task from Tasks.md
var remaining = items.Skip(i + 1).ToList();
File.WriteAllText(tasksFile, string.Join("\n\n---\n\n", remaining));
Console.WriteLine("[runner] Removed completed task from Tasks.md");
}
Console.WriteLine("\n[runner] Finished.");

View File

@@ -1,6 +1,6 @@
{
"name": "aniworld-web",
"version": "0.0.1",
"version": "1.3.6",
"description": "Aniworld Anime Download Manager - Web Frontend",
"type": "module",
"scripts": {

View File

@@ -22,6 +22,7 @@ APScheduler>=3.10.4
Events>=0.5
requests>=2.31.0
beautifulsoup4>=4.12.0
chardet>=5.2.0
fake-useragent>=1.4.0
yt-dlp>=2024.1.0
urllib3>=2.0.0

View File

@@ -1,3 +1,4 @@
import re
import secrets
from typing import Optional
@@ -114,6 +115,40 @@ class Settings(BaseSettings):
validation_alias="NFO_PREFER_FSK_RATING",
description="Prefer German FSK rating over MPAA rating in NFO files"
)
nfo_folder_ignore_patterns: str = Field(
default="The Last of Us|Loki|Chernobyl|Star Trek Discovery|Marvel|Matrix|Fast & Furious|Jurassic|James Bond|Mission: Impossible|Bourne|Hunger Games|Die Hard|John Wick|Pacific Rim|Guardians of the Galaxy|Avengers|Batman|Superman|Wonder Woman|Spider-Man|X-Men|Fantastic Four|Terminator|Predator|Rambo|Rocky|Expendables|Tomb Raider|Jumanji|Jurassic Park|Pirates of the Caribbean|Harry Potter|Lord of the Rings|Hobbit|Game of Thrones|Westworld|Stranger Things|Breaking Bad|Better Call Saul|Sherlock|Downton Abbey|The Crown|Bridgerton|Sex Education|Normal People|Emily in Paris|The Witcher|Servant|Lucifer|Dark|Shadow and Bone|Grimm|Fairytale",
validation_alias="NFO_FOLDER_IGNORE_PATTERNS",
description="Regex patterns for folder names to skip during scan (pipe-separated)"
)
@property
def folder_ignore_patterns(self) -> list[str]:
"""Parse ignore patterns from comma-separated string into list.
Returns:
List of regex patterns to skip during folder scanning.
"""
if not self.nfo_folder_ignore_patterns:
return []
return [
pattern.strip()
for pattern in self.nfo_folder_ignore_patterns.split("|")
if pattern.strip()
]
def should_ignore_folder(self, folder_name: str) -> bool:
"""Check if folder should be ignored based on ignore patterns.
Args:
folder_name: Name of folder to check.
Returns:
True if folder matches any ignore pattern, False otherwise.
"""
for pattern in self.folder_ignore_patterns:
if re.search(pattern, folder_name, re.IGNORECASE):
return True
return False
@property
def allowed_origins(self) -> list[str]:
@@ -134,5 +169,23 @@ class Settings(BaseSettings):
]
return [origin.strip() for origin in raw.split(",") if origin.strip()]
@property
def scan_key_overrides(self) -> dict[str, str]:
"""Return scan key overrides from config.json.
Maps folder names to provider keys for cases where auto-generated
keys from folder names are incorrect.
Returns:
Dict mapping folder names to provider keys.
"""
from src.server.services.config_service import ConfigService
try:
config_service = ConfigService()
config = config_service.load_config()
return config.scan_key_overrides or {}
except Exception:
return {}
settings = Settings()

File diff suppressed because it is too large Load Diff

View File

@@ -166,7 +166,10 @@ class SeriesApp:
self.loaders = Loaders()
self.loader = self.loaders.GetLoader(key="aniworld.to")
self.serie_scanner = SerieScanner(
directory_to_search, self.loader, db_lookup=db_lookup
directory_to_search,
self.loader,
db_lookup=db_lookup,
scan_key_overrides=settings.scan_key_overrides,
)
# Skip automatic loading from data files - series will be loaded
# from database by the service layer during application setup
@@ -445,9 +448,12 @@ class SeriesApp:
try:
def download_progress_handler(progress_info):
"""Handle download progress events from loader."""
logger.debug(
"download_progress_handler called with: %s", progress_info
)
# Throttle progress logging to avoid spam
status = progress_info.get("status", "")
if status in ("downloading", "finished"):
logger.debug(
"download_progress_handler called with: %s", progress_info
)
downloaded = progress_info.get('downloaded_bytes', 0)
total_bytes = (

View File

@@ -1,320 +1,531 @@
"""Utilities for loading and managing stored anime series metadata.
This module provides the SerieList class for managing collections of anime
series metadata. It uses file-based storage only.
Note:
This module is part of the core domain layer and has no database
dependencies. All database operations are handled by the service layer.
"""
from __future__ import annotations
import logging
import os
import warnings
from json import JSONDecodeError
from typing import Dict, Iterable, List, Optional
from src.core.entities.series import Serie
logger = logging.getLogger(__name__)
class SerieList:
"""
Represents the collection of cached series stored on disk.
Series are identified by their unique 'key' (provider identifier).
The 'folder' is metadata only and not used for lookups.
This class manages in-memory series data loaded from filesystem.
It has no database dependencies - all persistence is handled by
the service layer.
Example:
# File-based mode
serie_list = SerieList("/path/to/anime")
series = serie_list.get_all()
Attributes:
directory: Path to the anime directory
keyDict: Internal dictionary mapping serie.key to Serie objects
"""
def __init__(
self,
base_path: str,
skip_load: bool = False
) -> None:
"""Initialize the SerieList.
Args:
base_path: Path to the anime directory
skip_load: If True, skip automatic loading of series from files.
Useful when planning to load from database instead.
"""
self.directory: str = base_path
# Internal storage using serie.key as the dictionary key
self.keyDict: Dict[str, Serie] = {}
# Only auto-load from files if not skipping
if not skip_load:
self.load_series()
def add(self, serie: Serie, use_sanitized_folder: bool = True) -> str:
"""
Persist a new series if it is not already present (file-based mode).
Uses serie.key for identification. Creates the filesystem folder
using either the sanitized display name (default) or the existing
folder property.
Args:
serie: The Serie instance to add
use_sanitized_folder: If True (default), use serie.sanitized_folder
for the filesystem folder name based on display name.
If False, use serie.folder as-is for backward compatibility.
Returns:
str: The folder path that was created/used
Note:
This method creates data files on disk. For database storage,
use add_to_db() instead.
"""
if self.contains(serie.key):
# Return existing folder path
existing = self.keyDict[serie.key]
return os.path.join(self.directory, existing.folder)
# Determine folder name to use
if use_sanitized_folder:
folder_name = serie.sanitized_folder
# Update the serie's folder property to match what we create
serie.folder = folder_name
else:
folder_name = serie.folder
data_path = os.path.join(self.directory, folder_name, "data")
anime_path = os.path.join(self.directory, folder_name)
os.makedirs(anime_path, exist_ok=True)
if not os.path.isfile(data_path):
serie.save_to_file(data_path)
# Store by key, not folder
self.keyDict[serie.key] = serie
return anime_path
def contains(self, key: str) -> bool:
"""
Return True when a series identified by ``key`` already exists.
Args:
key: The unique provider identifier for the series
Returns:
True if the series exists in the collection
"""
return key in self.keyDict
def load_series(self) -> None:
"""Populate the in-memory map with metadata discovered on disk."""
logger.info("Scanning anime folders in %s", self.directory)
try:
entries: Iterable[str] = os.listdir(self.directory)
except OSError as error:
logger.error(
"Unable to scan directory %s: %s",
self.directory,
error,
)
return
nfo_stats = {"total": 0, "with_nfo": 0, "without_nfo": 0}
media_stats = {
"with_poster": 0,
"without_poster": 0,
"with_logo": 0,
"without_logo": 0,
"with_fanart": 0,
"without_fanart": 0
}
for anime_folder in entries:
anime_path = os.path.join(self.directory, anime_folder, "data")
if os.path.isfile(anime_path):
logger.debug("Found data file for folder %s", anime_folder)
serie = self._load_data(anime_folder, anime_path)
if serie:
nfo_stats["total"] += 1
# Check for NFO file
nfo_file_path = os.path.join(
self.directory, anime_folder, "tvshow.nfo"
)
if os.path.isfile(nfo_file_path):
serie.nfo_path = nfo_file_path
nfo_stats["with_nfo"] += 1
else:
nfo_stats["without_nfo"] += 1
logger.debug(
"Series '%s' (key: %s) is missing tvshow.nfo",
serie.name,
serie.key
)
# Check for media files
folder_path = os.path.join(self.directory, anime_folder)
poster_path = os.path.join(folder_path, "poster.jpg")
if os.path.isfile(poster_path):
media_stats["with_poster"] += 1
else:
media_stats["without_poster"] += 1
logger.debug(
"Series '%s' (key: %s) is missing poster.jpg",
serie.name,
serie.key
)
logo_path = os.path.join(folder_path, "logo.png")
if os.path.isfile(logo_path):
media_stats["with_logo"] += 1
else:
media_stats["without_logo"] += 1
logger.debug(
"Series '%s' (key: %s) is missing logo.png",
serie.name,
serie.key
)
fanart_path = os.path.join(folder_path, "fanart.jpg")
if os.path.isfile(fanart_path):
media_stats["with_fanart"] += 1
else:
media_stats["without_fanart"] += 1
logger.debug(
"Series '%s' (key: %s) is missing fanart.jpg",
serie.name,
serie.key
)
continue
logger.warning(
"Skipping folder %s because no metadata file was found",
anime_folder,
)
# Log summary statistics
if nfo_stats["total"] > 0:
logger.info(
"NFO scan complete: %d series total, %d with NFO, %d without NFO",
nfo_stats["total"],
nfo_stats["with_nfo"],
nfo_stats["without_nfo"]
)
logger.info(
"Media scan complete: Poster (%d/%d), Logo (%d/%d), Fanart (%d/%d)",
media_stats["with_poster"],
nfo_stats["total"],
media_stats["with_logo"],
nfo_stats["total"],
media_stats["with_fanart"],
nfo_stats["total"]
)
def _load_data(self, anime_folder: str, data_path: str) -> Optional[Serie]:
"""
Load a single series metadata file into the in-memory collection.
Args:
anime_folder: The folder name (for logging only)
data_path: Path to the metadata file
Returns:
Serie: The loaded Serie object, or None if loading failed
"""
try:
serie = Serie.load_from_file(data_path)
# Store by key, not folder
self.keyDict[serie.key] = serie
logger.debug(
"Successfully loaded metadata for %s (key: %s)",
anime_folder,
serie.key
)
return serie
except (OSError, JSONDecodeError, KeyError, ValueError) as error:
logger.error(
"Failed to load metadata for folder %s from %s: %s",
anime_folder,
data_path,
error,
)
return None
def GetMissingEpisode(self) -> List[Serie]:
"""Return all series that still contain missing episodes."""
return [
serie
for serie in self.keyDict.values()
if serie.episodeDict
]
def get_missing_episodes(self) -> List[Serie]:
"""PEP8-friendly alias for :meth:`GetMissingEpisode`."""
return self.GetMissingEpisode()
def GetList(self) -> List[Serie]:
"""Return all series instances stored in the list."""
return list(self.keyDict.values())
def get_all(self) -> List[Serie]:
"""PEP8-friendly alias for :meth:`GetList`."""
return self.GetList()
def get_by_key(self, key: str) -> Optional[Serie]:
"""
Get a series by its unique provider key.
This is the primary method for series lookup.
Args:
key: The unique provider identifier (e.g., "attack-on-titan")
Returns:
The Serie instance if found, None otherwise
"""
return self.keyDict.get(key)
def get_by_folder(self, folder: str) -> Optional[Serie]:
"""
Get a series by its folder name.
.. deprecated:: 2.0.0
Use :meth:`get_by_key` instead. Folder-based lookups will be
removed in version 3.0.0. The `folder` field is metadata only
and should not be used for identification.
This method is provided for backward compatibility only.
Prefer using get_by_key() for new code.
Args:
folder: The filesystem folder name (e.g., "Attack on Titan (2013)")
Returns:
The Serie instance if found, None otherwise
"""
warnings.warn(
"get_by_folder() is deprecated and will be removed in v3.0.0. "
"Use get_by_key() instead. The 'folder' field is metadata only.",
DeprecationWarning,
stacklevel=2
)
for serie in self.keyDict.values():
if serie.folder == folder:
return serie
return None
"""Utilities for loading and managing stored anime series metadata.
This module provides the SerieList class for managing collections of anime
series metadata. It supports loading from both filesystem (legacy) and
database (primary).
Note:
This module is part of the core domain layer. Database operations
are handled by the service layer via add_to_db().
"""
from __future__ import annotations
import logging
import os
import warnings
from json import JSONDecodeError
from typing import Dict, Iterable, List, Optional
from src.config.settings import settings
from src.core.entities.series import Serie
logger = logging.getLogger(__name__)
class SerieList:
"""
Represents the collection of cached series stored on disk.
Series are identified by their unique 'key' (provider identifier).
The 'folder' is metadata only and not used for lookups.
This class manages in-memory series data loaded from filesystem.
It has no database dependencies - all persistence is handled by
the service layer.
Example:
# File-based mode
serie_list = SerieList("/path/to/anime")
series = serie_list.get_all()
Attributes:
directory: Path to the anime directory
keyDict: Internal dictionary mapping serie.key to Serie objects
"""
def __init__(
self,
base_path: str,
skip_load: bool = False
) -> None:
"""Initialize the SerieList.
Args:
base_path: Path to the anime directory
skip_load: If True, skip automatic loading of series from files.
Useful when planning to load from database instead.
"""
self.directory: str = base_path
# Internal storage using serie.key as the dictionary key
self.keyDict: Dict[str, Serie] = {}
# Only auto-load from files if not skipping
if not skip_load:
self.load_series()
def add(self, serie: Serie, use_sanitized_folder: bool = True) -> str:
"""
Persist a new series if it is not already present (file-based mode).
Uses serie.key for identification. Creates the filesystem folder
using either the sanitized display name (default) or the existing
folder property.
Args:
serie: The Serie instance to add
use_sanitized_folder: If True (default), use serie.sanitized_folder
for the filesystem folder name based on display name.
If False, use serie.folder as-is for backward compatibility.
Returns:
str: The folder path that was created/used
Note:
This method creates data files on disk. For database storage,
use add_to_db() instead.
"""
if self.contains(serie.key):
# Return existing folder path
existing = self.keyDict[serie.key]
return os.path.join(self.directory, existing.folder)
# Determine folder name to use
if use_sanitized_folder:
folder_name = serie.sanitized_folder
# Update the serie's folder property to match what we create
serie.folder = folder_name
else:
folder_name = serie.folder
data_path = os.path.join(self.directory, folder_name, "data")
anime_path = os.path.join(self.directory, folder_name)
os.makedirs(anime_path, exist_ok=True)
if not os.path.isfile(data_path):
serie.save_to_file(data_path)
# Store by key, not folder
self.keyDict[serie.key] = serie
return anime_path
async def add_to_db(self, serie: Serie) -> bool:
"""Persist a new series to the database.
Creates the filesystem folder using serie.folder, then persists
the series metadata to the database.
Args:
serie: The Serie instance to add
Returns:
True if successful, False otherwise
"""
try:
from src.server.database.connection import get_async_session_factory
from src.server.database.service import AnimeSeriesService, EpisodeService
folder_name = serie.folder
anime_path = os.path.join(self.directory, folder_name)
os.makedirs(anime_path, exist_ok=True)
session_factory = get_async_session_factory()
db = session_factory()
try:
existing = await AnimeSeriesService.get_by_key(db, serie.key)
if existing:
logger.debug(
"Series '%s' (key=%s) already exists in DB, skipping",
serie.name, serie.key
)
return True
anime_series = await AnimeSeriesService.create(
db=db,
key=serie.key,
name=serie.name,
site=serie.site,
folder=folder_name,
year=serie.year
)
for season, eps in serie.episodeDict.items():
for ep in eps:
await EpisodeService.create(
db=db,
series_id=anime_series.id,
season=season,
episode_number=ep
)
await db.commit()
self.keyDict[serie.key] = serie
logger.info(
"Persisted series '%s' to database",
serie.name
)
return True
except Exception as e:
await db.rollback()
logger.error(
"Failed to persist series '%s' to DB: %s",
serie.key, e, exc_info=True
)
return False
finally:
await db.close()
except Exception as e:
logger.error(
"Could not add series '%s' to DB (DB unavailable?): %s",
serie.key, e
)
return False
def contains(self, key: str) -> bool:
"""
Return True when a series identified by ``key`` already exists.
Args:
key: The unique provider identifier for the series
Returns:
True if the series exists in the collection
"""
return key in self.keyDict
def load_series(self) -> None:
"""Populate the in-memory map with metadata discovered on disk."""
logger.info("Scanning anime folders in %s", self.directory)
try:
entries: Iterable[str] = os.listdir(self.directory)
except OSError as error:
logger.error(
"Unable to scan directory %s: %s",
self.directory,
error,
)
return
nfo_stats = {"total": 0, "with_nfo": 0, "without_nfo": 0}
media_stats = {
"with_poster": 0,
"without_poster": 0,
"with_logo": 0,
"without_logo": 0,
"with_fanart": 0,
"without_fanart": 0
}
for anime_folder in entries:
if settings.should_ignore_folder(anime_folder):
logger.debug("Skipping ignored folder: %s", anime_folder)
continue
anime_path = os.path.join(self.directory, anime_folder, "data")
if os.path.isfile(anime_path):
logger.debug("Found data file for folder %s", anime_folder)
serie = self._load_data(anime_folder, anime_path)
if serie:
nfo_stats["total"] += 1
# Check for NFO file
nfo_file_path = os.path.join(
self.directory, anime_folder, "tvshow.nfo"
)
if os.path.isfile(nfo_file_path):
serie.nfo_path = nfo_file_path
nfo_stats["with_nfo"] += 1
else:
nfo_stats["without_nfo"] += 1
logger.debug(
"Series '%s' (key: %s) is missing tvshow.nfo",
serie.name,
serie.key
)
# Check for media files
folder_path = os.path.join(self.directory, anime_folder)
poster_path = os.path.join(folder_path, "poster.jpg")
if os.path.isfile(poster_path):
media_stats["with_poster"] += 1
else:
media_stats["without_poster"] += 1
logger.debug(
"Series '%s' (key: %s) is missing poster.jpg",
serie.name,
serie.key
)
logo_path = os.path.join(folder_path, "logo.png")
if os.path.isfile(logo_path):
media_stats["with_logo"] += 1
else:
media_stats["without_logo"] += 1
logger.debug(
"Series '%s' (key: %s) is missing logo.png",
serie.name,
serie.key
)
fanart_path = os.path.join(folder_path, "fanart.jpg")
if os.path.isfile(fanart_path):
media_stats["with_fanart"] += 1
else:
media_stats["without_fanart"] += 1
logger.debug(
"Series '%s' (key: %s) is missing fanart.jpg",
serie.name,
serie.key
)
continue
logger.warning(
"Skipping folder %s because no metadata file was found",
anime_folder,
)
# Log summary statistics
if nfo_stats["total"] > 0:
logger.info(
"NFO scan complete: %d series total, %d with NFO, %d without NFO",
nfo_stats["total"],
nfo_stats["with_nfo"],
nfo_stats["without_nfo"]
)
logger.info(
"Media scan complete: Poster (%d/%d), Logo (%d/%d), Fanart (%d/%d)",
media_stats["with_poster"],
nfo_stats["total"],
media_stats["with_logo"],
nfo_stats["total"],
media_stats["with_fanart"],
nfo_stats["total"]
)
def _load_data(self, anime_folder: str, data_path: str) -> Optional[Serie]:
"""
Load a single series metadata file into the in-memory collection.
Args:
anime_folder: The folder name (for logging only)
data_path: Path to the metadata file
Returns:
Serie: The loaded Serie object, or None if loading failed
"""
try:
serie = Serie.load_from_file(data_path)
# Store by key, not folder
self.keyDict[serie.key] = serie
logger.debug(
"Successfully loaded metadata for %s (key: %s)",
anime_folder,
serie.key
)
return serie
except (OSError, JSONDecodeError, KeyError, ValueError) as error:
logger.error(
"Failed to load metadata for folder %s from %s: %s",
anime_folder,
data_path,
error,
)
return None
def GetMissingEpisode(self) -> List[Serie]:
"""Return all series that still contain missing episodes."""
return [
serie
for serie in self.keyDict.values()
if serie.episodeDict
]
def get_missing_episodes(self) -> List[Serie]:
"""PEP8-friendly alias for :meth:`GetMissingEpisode`."""
return self.GetMissingEpisode()
def GetList(self) -> List[Serie]:
"""Return all series instances stored in the list."""
return list(self.keyDict.values())
def get_all(self) -> List[Serie]:
"""PEP8-friendly alias for :meth:`GetList`."""
return self.GetList()
def get_by_key(self, key: str) -> Optional[Serie]:
"""
Get a series by its unique provider key.
This is the primary method for series lookup.
Args:
key: The unique provider identifier (e.g., "attack-on-titan")
Returns:
The Serie instance if found, None otherwise
"""
return self.keyDict.get(key)
def get_by_folder(self, folder: str) -> Optional[Serie]:
"""
Get a series by its folder name.
.. deprecated:: 2.0.0
Use :meth:`get_by_key` instead. Folder-based lookups will be
removed in version 3.0.0. The `folder` field is metadata only
and should not be used for identification.
This method is provided for backward compatibility only.
Prefer using get_by_key() for new code.
Args:
folder: The filesystem folder name (e.g., "Attack on Titan (2013)")
Returns:
The Serie instance if found, None otherwise
"""
warnings.warn(
"get_by_folder() is deprecated and will be removed in v3.0.0. "
"Use get_by_key() instead. The 'folder' field is metadata only.",
DeprecationWarning,
stacklevel=2
)
for serie in self.keyDict.values():
if serie.folder == folder:
return serie
return None
async def load_all_from_db(self) -> int:
"""Load all series from database into in-memory cache.
Retrieves all anime series from the database with their episodes
and populates the in-memory keyDict for fast access.
This method replaces file-based loading. Use after initialization
when database is ready.
Returns:
int: Number of series loaded into cache
"""
from src.server.database.connection import get_async_session_factory
from src.server.database.service import AnimeSeriesService
try:
session_factory = get_async_session_factory()
db = session_factory()
try:
anime_series_list = await AnimeSeriesService.get_all(
db, with_episodes=True
)
count = 0
for anime_series in anime_series_list:
episode_dict: Dict[int, List[int]] = {}
if anime_series.episodes:
for ep in anime_series.episodes:
if ep.season not in episode_dict:
episode_dict[ep.season] = []
episode_dict[ep.season].append(ep.episode_number)
serie = Serie(
key=anime_series.key,
name=anime_series.name,
site=anime_series.site,
folder=anime_series.folder,
episodeDict=episode_dict,
year=anime_series.year
)
self.keyDict[serie.key] = serie
count += 1
logger.info(
"Loaded %d series from database into in-memory cache",
count
)
return count
finally:
await db.close()
except RuntimeError:
logger.warning(
"Database not available, skipping DB load"
)
return 0
async def _load_single_series_from_db(
self,
anime_folder: str
) -> Optional[Serie]:
"""Load a single series from database by folder name.
Looks up a series in the database by its folder name and adds
it to the in-memory cache.
Args:
anime_folder: The filesystem folder name to look up
Returns:
Serie if found and loaded, None otherwise
"""
from src.server.database.connection import get_async_session_factory
from src.server.database.service import AnimeSeriesService
try:
session_factory = get_async_session_factory()
db = session_factory()
try:
anime_series = await AnimeSeriesService.get_by_folder(
db, anime_folder
)
if not anime_series:
logger.debug(
"Series with folder '%s' not found in DB",
anime_folder
)
return None
episode_dict: Dict[int, List[int]] = {}
if anime_series.episodes:
for ep in anime_series.episodes:
if ep.season not in episode_dict:
episode_dict[ep.season] = []
episode_dict[ep.season].append(ep.episode_number)
serie = Serie(
key=anime_series.key,
name=anime_series.name,
site=anime_series.site,
folder=anime_series.folder,
episodeDict=episode_dict,
year=anime_series.year
)
self.keyDict[serie.key] = serie
logger.debug(
"Loaded series '%s' (key=%s) from DB",
serie.name, serie.key
)
return serie
finally:
await db.close()
except RuntimeError:
logger.warning(
"Database not available, cannot load series '%s'",
anime_folder
)
return None
def invalidate_cache(self) -> None:
"""Clear the in-memory cache.
Use after database modifications to force reload from DB
on next access.
"""
self.keyDict.clear()
logger.debug("SerieList in-memory cache invalidated")
def reload(self) -> None:
"""Reload series from filesystem (legacy mode).
Warning:
This method uses file-based loading and should only be
used as fallback when database is not available.
"""
self.load_series()

View File

@@ -271,7 +271,11 @@ class Serie:
'Dororo (2025)'
"""
if self._year:
return f"{self._name} ({self._year})"
import re
year_suffix = f" ({self._year})"
# Strip ALL trailing year suffixes before appending to prevent duplication
clean_name = re.sub(r'(\s*\(\d{4}\))+\s*$', '', self._name).strip()
return f"{clean_name}{year_suffix}"
return self._name
@property

View File

@@ -7,7 +7,7 @@ errors in provider operations with automatic retry mechanisms.
import functools
import logging
from typing import Any, Callable, TypeVar
from typing import Any, Callable, Optional, TypeVar
logger = logging.getLogger(__name__)
@@ -42,41 +42,85 @@ class DownloadError(Exception):
class RecoveryStrategies:
"""Strategies for handling errors and recovering from failures."""
@staticmethod
def handle_network_failure(
func: Callable, *args: Any, **kwargs: Any
) -> Any:
"""Handle network failures with basic retry logic."""
max_retries = 3
for attempt in range(max_retries):
try:
return func(*args, **kwargs)
except (NetworkError, ConnectionError):
if attempt == max_retries - 1:
raise
logger.warning(
"Network error on attempt %d, retrying...",
attempt + 1,
)
continue
def __init__(
self,
max_retries: int = 3,
base_delay: float = 1.0,
max_delay: float = 60.0,
exponential_base: float = 2.0,
) -> None:
"""Initialize recovery strategies.
@staticmethod
def handle_download_failure(
Args:
max_retries: Maximum number of retry attempts.
base_delay: Initial delay between retries in seconds.
max_delay: Maximum delay between retries in seconds.
exponential_base: Base for exponential backoff multiplier.
"""
self.max_retries = max_retries
self.base_delay = base_delay
self.max_delay = max_delay
self.exponential_base = exponential_base
def _calculate_delay(self, attempt: int) -> float:
"""Calculate delay for given retry attempt using exponential backoff.
Args:
attempt: Zero-based retry attempt number.
Returns:
Delay in seconds before next retry.
"""
delay = self.base_delay * (self.exponential_base ** attempt)
return min(delay, self.max_delay)
def handle_network_failure(
self,
func: Callable, *args: Any, **kwargs: Any
) -> Any:
"""Handle download failures with retry logic."""
max_retries = 2
for attempt in range(max_retries):
"""Handle network failures with exponential backoff retry logic."""
last_error: Optional[Exception] = None
for attempt in range(self.max_retries):
try:
return func(*args, **kwargs)
except DownloadError:
if attempt == max_retries - 1:
raise
logger.warning(
"Download error on attempt %d, retrying...",
attempt + 1,
)
continue
except (NetworkError, ConnectionError, TimeoutError) as exc:
last_error = exc
if attempt < self.max_retries - 1:
delay = self._calculate_delay(attempt)
logger.warning(
"Network error on attempt %d/%d, retrying in %.1fs: %s",
attempt + 1, self.max_retries, delay, exc
)
import time
time.sleep(delay)
continue
if last_error:
raise last_error
raise NetworkError("Network failure after retries")
def handle_download_failure(
self,
func: Callable, *args: Any, **kwargs: Any
) -> Any:
"""Handle download failures with exponential backoff retry logic."""
last_error: Optional[Exception] = None
for attempt in range(self.max_retries):
try:
return func(*args, **kwargs)
except DownloadError as exc:
last_error = exc
if attempt < self.max_retries - 1:
delay = self._calculate_delay(attempt)
logger.warning(
"Download error on attempt %d/%d, retrying in %.1fs: %s",
attempt + 1, self.max_retries, delay, exc
)
import time
time.sleep(delay)
continue
if last_error:
raise last_error
raise DownloadError("Download failed after retries")
class FileCorruptionDetector:

View File

@@ -9,6 +9,7 @@ import threading
from pathlib import Path
from urllib.parse import quote
import chardet
import requests
from bs4 import BeautifulSoup
from events import Events
@@ -80,6 +81,37 @@ if not download_error_logger.handlers:
noKeyFound_logger = logging.getLogger()
def _decode_html_content(content: bytes) -> str:
"""Decode HTML content with encoding detection.
Uses chardet to detect the actual encoding of the content,
falling back to utf-8 with replacement error handling.
Args:
content: Raw HTML bytes from the response
Returns:
Decoded string content
"""
detected = chardet.detect(content)
encoding = detected.get('encoding', 'utf-8')
confidence = detected.get('confidence', 0)
if confidence < 0.7:
logger.debug(
"Low encoding confidence (%.2f) for detected encoding '%s', using utf-8",
confidence,
encoding
)
encoding = 'utf-8'
try:
return content.decode(encoding, errors='replace')
except Exception as exc:
logger.warning("Failed to decode content with %s: %s, using utf-8 replace", encoding, exc)
return content.decode('utf-8', errors='replace')
class AniworldLoader(Loader):
def __init__(self) -> None:
self.SUPPORTED_PROVIDERS = DEFAULT_PROVIDERS
@@ -90,7 +122,10 @@ class AniworldLoader(Loader):
self.LULUVDO_USER_AGENT = LULUVDO_USER_AGENT
self.PROVIDER_HEADERS = {
ProviderType.VIDMOLY.value: ['Referer: "https://vidmoly.to"'],
ProviderType.DOODSTREAM.value: ['Referer: "https://dood.li/"'],
ProviderType.DOODSTREAM.value: [
'Referer: "https://dood.li/"',
'Referer: "https://playmogo.com/"',
],
ProviderType.VOE.value: [f"User-Agent: {self.RANDOM_USER_AGENT}"],
ProviderType.LULUVDO.value: [
f"User-Agent: {self.LULUVDO_USER_AGENT}",
@@ -231,7 +266,7 @@ class AniworldLoader(Loader):
language_code = self._get_language_key(language)
episode_soup = BeautifulSoup(
self._get_episode_html(season, episode, key).content,
_decode_html_content(self._get_episode_html(season, episode, key).content),
'html.parser'
)
change_language_box_div = episode_soup.find(
@@ -249,6 +284,118 @@ class AniworldLoader(Loader):
logger.debug("Available languages for S%02dE%03d: %s, requested: %s, available: %s", season, episode, languages, language_code, is_available)
return is_available
def _check_url_alive(
self,
url: str,
headers: dict | None = None,
timeout: int = 10,
) -> bool:
"""Probe a provider URL with HEAD before committing to yt-dlp.
Skips dead providers quickly so the failover loop never blocks
waiting for yt-dlp to fail on a 404. Falls back to a streaming
GET when HEAD is not allowed by the upstream server.
Args:
url: URL to probe.
headers: Optional headers to forward with the probe.
timeout: Per-request timeout (seconds).
Returns:
True when the URL responds with a non-4xx status, else False.
"""
try:
response = self.session.head(
url,
headers=headers,
timeout=timeout,
allow_redirects=True,
)
if response.status_code == 405:
response = self.session.get(
url,
headers=headers,
timeout=timeout,
stream=True,
allow_redirects=True,
)
response.close()
if 400 <= response.status_code < 500:
logger.warning(
"Provider URL returned HTTP %s: %s",
response.status_code, url
)
return False
return True
except requests.RequestException as exc:
logger.warning("Provider URL unreachable %s: %s", url, exc)
return False
def _try_direct_stream(
self,
link: str,
output_path: str,
headers: dict | None,
timeout: int,
) -> bool:
"""Stream a direct video URL to disk without yt-dlp.
Used as a fast-path when the resolved provider link already points
at a downloadable video file (``Content-Type: video/*`` or
``application/octet-stream``). HLS and other non-video payloads
are rejected so the caller can fall back to yt-dlp.
Args:
link: Direct download URL.
output_path: Destination file path.
headers: Optional HTTP headers.
timeout: Per-request timeout (seconds).
Returns:
True on a successful save, False when the link is not a
direct video or the download fails.
"""
try:
with self.session.get(
link,
headers=headers,
timeout=timeout,
stream=True,
) as response:
if not response.ok:
logger.debug(
"Direct stream HEAD returned %s for %s",
response.status_code, link[:80]
)
return False
content_type = response.headers.get("Content-Type", "")
if not (
content_type.startswith("video/")
or content_type == "application/octet-stream"
):
logger.debug(
"Direct stream skipped, Content-Type=%s",
content_type
)
return False
logger.info(
"Direct stream download starting (type=%s)",
content_type
)
with open(output_path, "wb") as fh:
for chunk in response.iter_content(chunk_size=1024 * 1024):
if self._cancel_flag.is_set():
logger.info(
"Cancellation detected during direct stream"
)
return False
if chunk:
fh.write(chunk)
return True
except requests.RequestException as exc:
logger.warning("Direct stream download failed: %s", exc)
return False
def download(
self,
base_directory: str,
@@ -259,7 +406,12 @@ class AniworldLoader(Loader):
language: str = "German Dub"
) -> bool:
"""Download episode to specified directory.
Iterates the providers actually advertised on the episode page
(ordered by SUPPORTED_PROVIDERS preference), probing each URL
before attempting an extraction so dead providers are skipped
immediately instead of stalling yt-dlp on a 404.
Args:
base_directory: Base download directory path
serie_folder: Filesystem folder name (metadata only, used for
@@ -308,12 +460,78 @@ class AniworldLoader(Loader):
temp_path = os.path.join(temp_dir, output_file)
logger.debug("Temporary path: %s", temp_path)
for provider in self.SUPPORTED_PROVIDERS:
logger.debug("Attempting download with provider: %s", provider)
link, header = self._get_direct_link_from_provider(
candidate_providers = self._select_providers_for_episode(
season, episode, key, language
)
if not candidate_providers:
logger.error(
"No providers advertised for S%02dE%03d (%s) in %s",
season, episode, key, language
)
logger.debug("Direct link obtained from provider")
self.clear_cache()
return False
tried: list[str] = []
for provider_name, redirect_url in candidate_providers:
tried.append(provider_name)
logger.debug("Attempting download with provider: %s", provider_name)
probe_headers = {"User-Agent": self.RANDOM_USER_AGENT}
if not self._check_url_alive(
redirect_url,
headers=probe_headers,
timeout=self.DEFAULT_REQUEST_TIMEOUT,
):
logger.info(
"Skipping provider %s, redirect URL not reachable",
provider_name
)
continue
try:
resolved = self._resolve_direct_link(
redirect_url, provider_name
)
except Exception as exc:
logger.warning(
"Provider %s link resolution failed: %s: %s",
provider_name, type(exc).__name__, exc
)
continue
if resolved is None:
logger.info(
"Provider %s returned no direct link", provider_name
)
continue
link, header = resolved
if self._cancel_flag.is_set():
logger.info("Cancellation requested before download start")
_cleanup_temp_file(temp_path)
self.clear_cache()
return False
if self._try_direct_stream(
link,
temp_path,
header,
self.DEFAULT_REQUEST_TIMEOUT,
) and os.path.exists(temp_path):
logger.debug(
"Direct stream succeeded with provider %s", provider_name
)
shutil.copyfile(temp_path, output_path)
os.remove(temp_path)
logger.info(
"Download completed successfully (direct): %s",
output_file
)
self.clear_cache()
return True
_cleanup_temp_file(temp_path)
cancel_flag = self._cancel_flag
@@ -321,7 +539,6 @@ class AniworldLoader(Loader):
if cancel_flag.is_set():
logger.info("Cancellation detected in progress hook")
raise DownloadCancelled("Download cancelled by user")
# Fire the event for progress
self.events.download_progress(d)
ydl_opts = {
@@ -331,7 +548,12 @@ class AniworldLoader(Loader):
'no_warnings': True,
'progress_with_newline': False,
'nocheckcertificate': True,
'logger': logger,
'progress_hooks': [events_progress_hook],
# yt-dlp defaults to native HLS downloader which warns about
# "Live HLS streams are not supported" - disable to go
# straight to ffmpeg, avoiding the warning
'hls_prefer_native': False,
}
if header:
@@ -339,9 +561,11 @@ class AniworldLoader(Loader):
logger.debug("Using custom headers for download")
try:
logger.debug("Starting YoutubeDL download")
logger.info(
"Starting yt-dlp download with %s: %s",
provider_name, output_file
)
logger.debug("Download link: %s...", link[:100])
logger.debug("YDL options: %s", ydl_opts)
with YoutubeDL(ydl_opts) as ydl:
info = ydl.extract_info(link, download=True)
@@ -352,39 +576,185 @@ class AniworldLoader(Loader):
if os.path.exists(temp_path):
logger.debug("Moving file from temp to final destination")
# Use copyfile instead of copy to avoid metadata permission issues
shutil.copyfile(temp_path, output_path)
os.remove(temp_path)
logger.info("Download completed successfully: %s", output_file)
logger.info(
"Download completed successfully: %s", output_file
)
self.clear_cache()
return True
else:
logger.error("Download failed: temp file not found at %s", temp_path)
self.clear_cache()
return False
except BrokenPipeError as e:
logger.error(
"Broken pipe error with provider %s: %s. "
"This usually means the stream connection was closed.",
provider, e
"Download failed: temp file not found at %s", temp_path
)
except DownloadCancelled:
logger.info("Download cancelled by user")
_cleanup_temp_file(temp_path)
self.clear_cache()
return False
except BrokenPipeError as exc:
logger.error(
"Broken pipe error with provider %s: %s",
provider_name, exc
)
_cleanup_temp_file(temp_path)
continue
except Exception as e:
except Exception as exc:
# Check if this is an HLS-related failure that might succeed
# with additional ffmpeg options
exc_str = str(exc).lower()
is_hls_related = (
'hls' in exc_str or
'live' in exc_str or
'native downloader' in exc_str
)
if is_hls_related and 'ffmpeg' not in str(ydl_opts.get('downloader', '')):
logger.info(
"HLS stream detected, retrying with ffmpeg options: %s",
output_file
)
# Retry with ffmpeg explicitly set
retry_opts = ydl_opts.copy()
retry_opts['downloader'] = 'ffmpeg'
retry_opts['hls_use_mpegts'] = True
try:
with YoutubeDL(retry_opts) as ydl:
info = ydl.extract_info(link, download=True)
if os.path.exists(temp_path):
shutil.copyfile(temp_path, output_path)
os.remove(temp_path)
logger.info(
"Download completed successfully (retry): %s",
output_file
)
self.clear_cache()
return True
except Exception:
_cleanup_temp_file(temp_path)
# Continue to next provider if retry also fails
continue
logger.error(
"YoutubeDL download failed with provider %s: %s: %s",
provider, type(e).__name__, e
provider_name, type(exc).__name__, exc
)
_cleanup_temp_file(temp_path)
continue
break
# If we get here, all providers failed
logger.error("All download providers failed")
logger.error(
"All download providers failed for S%02dE%03d (%s) in %s. "
"Tried: %s. Episode may be unavailable on the source site.",
season, episode, key, language, ", ".join(tried) or "none"
)
download_error_logger.error(
"All providers failed for %s S%02dE%03d (%s); tried=%s",
key, season, episode, language, tried
)
_cleanup_temp_file(temp_path)
self.clear_cache()
return False
def _select_providers_for_episode(
self,
season: int,
episode: int,
key: str,
language: str,
) -> list[tuple[str, str]]:
"""Return ``[(provider_name, redirect_url), ...]`` for an episode.
Filters by requested language and orders results by
``SUPPORTED_PROVIDERS`` preference so the failover chain matches
operator expectations. Returns an empty list when nothing is
advertised on the page.
"""
if not self.is_language(season, episode, key, language):
logger.warning(
"Language %s not advertised for S%02dE%03d (%s)",
language, season, episode, key
)
return []
language_code = self._get_language_key(language)
providers = self._get_provider_from_html(season, episode, key)
ordered: list[tuple[str, str]] = []
preferred = list(self.SUPPORTED_PROVIDERS)
for name in preferred:
lang_map = providers.get(name)
if lang_map and language_code in lang_map:
ordered.append((name, lang_map[language_code]))
for name, lang_map in providers.items():
if name in preferred:
continue
if language_code in lang_map:
ordered.append((name, lang_map[language_code]))
return ordered
def _resolve_direct_link(
self,
redirect_url: str,
provider_name: str,
) -> tuple[str, dict] | None:
"""Resolve a provider redirect URL into a direct stream link.
Follows the redirect to the embedded player, then delegates to a
provider-specific extractor (when registered) or returns the
embed URL itself so yt-dlp can attempt extraction.
Args:
redirect_url: AniWorld redirect URL.
provider_name: Provider key (e.g. ``"VOE"``).
Returns:
``(direct_link, headers)`` tuple or None when extraction fails.
"""
try:
embedded = self.session.get(
redirect_url,
timeout=self.DEFAULT_REQUEST_TIMEOUT,
headers={"User-Agent": self.RANDOM_USER_AGENT},
allow_redirects=True,
).url
except requests.RequestException as exc:
logger.warning(
"Failed resolving redirect for %s: %s", provider_name, exc
)
return None
try:
extractor = self.Providers.GetProvider(provider_name)
except (KeyError, AttributeError):
extractor = None
if extractor is not None:
try:
return extractor.get_link(
embedded, self.DEFAULT_REQUEST_TIMEOUT
)
except Exception as exc:
logger.warning(
"Custom extractor %s failed: %s",
provider_name, exc
)
return None
header_list = self.PROVIDER_HEADERS.get(provider_name)
header_dict = self._parse_provider_headers(header_list)
return embedded, header_dict
@staticmethod
def _parse_provider_headers(
header_list: list | None,
) -> dict[str, str]:
"""Convert legacy ``"Name: value"`` header strings to a dict."""
if not header_list:
return {}
parsed: dict[str, str] = {}
for entry in header_list:
if not isinstance(entry, str) or ":" not in entry:
continue
name, _, value = entry.partition(":")
parsed[name.strip()] = value.strip().strip('"')
return parsed
def get_site_key(self) -> str:
"""Get the site key for this provider."""
return "aniworld.to"
@@ -393,7 +763,7 @@ class AniworldLoader(Loader):
"""Get anime title from series key."""
logger.debug("Getting title for key: %s", key)
soup = BeautifulSoup(
self._get_key_html(key).content,
_decode_html_content(self._get_key_html(key).content),
'html.parser'
)
title_div = soup.find('div', class_='series-title')
@@ -424,7 +794,7 @@ class AniworldLoader(Loader):
logger.debug("Getting year for key: %s", key)
try:
soup = BeautifulSoup(
self._get_key_html(key).content,
_decode_html_content(self._get_key_html(key).content),
'html.parser'
)
@@ -538,7 +908,7 @@ class AniworldLoader(Loader):
"""
logger.debug("Extracting providers from HTML for S%02dE%03d (%s)", season, episode, key)
soup = BeautifulSoup(
self._get_episode_html(season, episode, key).content,
_decode_html_content(self._get_episode_html(season, episode, key).content),
'html.parser'
)
providers: dict[str, dict[int, str]] = {}
@@ -661,7 +1031,7 @@ class AniworldLoader(Loader):
base_url = f"{self.ANIWORLD_TO}/anime/stream/{safe_slug}/"
logger.debug("Base URL: %s", base_url)
response = requests.get(base_url, timeout=self.DEFAULT_REQUEST_TIMEOUT)
soup = BeautifulSoup(response.content, 'html.parser')
soup = BeautifulSoup(_decode_html_content(response.content), 'html.parser')
season_meta = soup.find('meta', itemprop='numberOfSeasons')
number_of_seasons = int(season_meta['content']) if season_meta else 0
@@ -676,7 +1046,7 @@ class AniworldLoader(Loader):
season_url,
timeout=self.DEFAULT_REQUEST_TIMEOUT,
)
soup = BeautifulSoup(response.content, 'html.parser')
soup = BeautifulSoup(_decode_html_content(response.content), 'html.parser')
episode_links = soup.find_all('a', href=True)
unique_links = set(

View File

@@ -88,7 +88,10 @@ class EnhancedAniWorldLoader(Loader):
self.PROVIDER_HEADERS = {
ProviderType.VIDMOLY.value: ['Referer: "https://vidmoly.to"'],
ProviderType.DOODSTREAM.value: ['Referer: "https://dood.li/"'],
ProviderType.DOODSTREAM.value: [
'Referer: "https://dood.li/"',
'Referer: "https://playmogo.com/"',
],
ProviderType.VOE.value: [f'User-Agent: {self.RANDOM_USER_AGENT}'],
ProviderType.LULUVDO.value: [
f'User-Agent: {self.LULUVDO_USER_AGENT}',
@@ -566,6 +569,10 @@ class EnhancedAniWorldLoader(Loader):
"nocheckcertificate": True,
"socket_timeout": self.download_timeout,
"http_chunk_size": 1024 * 1024, # 1MB chunks
"logger": self.logger,
# Use ffmpeg for HLS streams and transport stream format
"downloader": "ffmpeg",
"hls_use_mpegts": True,
}
if headers:
ydl_opts['http_headers'] = headers

View File

@@ -16,6 +16,7 @@ from typing import Dict, List
from lxml import etree
from src.core.services.nfo_service import NFOService
from src.core.services.tmdb_client import TMDBAPIError
logger = logging.getLogger(__name__)
@@ -120,6 +121,37 @@ def nfo_needs_repair(nfo_path: Path) -> bool:
return bool(find_missing_tags(nfo_path))
def _read_tmdb_id(nfo_path: Path) -> int | None:
"""Return the TMDB ID stored in an existing NFO, or ``None``.
Checks both ``<tmdbid>`` and ``<uniqueid type="tmdb">`` elements.
Args:
nfo_path: Absolute path to the ``tvshow.nfo`` file.
Returns:
Integer TMDB ID, or ``None`` if not found or not parseable.
"""
if not nfo_path.exists():
return None
try:
root = etree.parse(str(nfo_path)).getroot()
for uniqueid in root.findall(".//uniqueid"):
if uniqueid.get("type") == "tmdb" and uniqueid.text:
return int(uniqueid.text)
tmdbid_elem = root.find(".//tmdbid")
if tmdbid_elem is not None and tmdbid_elem.text:
return int(tmdbid_elem.text)
except (etree.XMLSyntaxError, ValueError):
pass
except Exception: # pylint: disable=broad-except
pass
return None
class NfoRepairService:
"""Service that detects and repairs incomplete tvshow.nfo files.
@@ -171,10 +203,26 @@ class NfoRepairService:
", ".join(missing),
)
await self._nfo_service.update_tvshow_nfo(
series_name,
download_media=False,
)
try:
await self._nfo_service.update_tvshow_nfo(
series_name,
download_media=False,
)
except TMDBAPIError as e:
if "No TMDB ID found" in str(e):
# No TMDB ID in existing NFO — create new one via search
logger.info(
"NFO has no TMDB ID, creating new NFO via TMDB search"
)
await self._nfo_service.create_tvshow_nfo(
serie_name=series_name,
serie_folder=series_name,
download_poster=False,
download_logo=False,
download_fanart=False,
)
else:
raise
logger.info("NFO repair completed: %s", series_name)
return True

View File

@@ -10,6 +10,7 @@ Example:
import logging
import re
import unicodedata
from pathlib import Path
from typing import Any, Dict, List, Optional, Tuple
@@ -19,6 +20,7 @@ from src.core.services.tmdb_client import TMDBAPIError, TMDBClient
from src.core.utils.image_downloader import ImageDownloader
from src.core.utils.nfo_generator import generate_tvshow_nfo
from src.core.utils.nfo_mapper import tmdb_to_nfo_model
from src.core.entities.nfo_models import TVShowNFO
logger = logging.getLogger(__name__)
@@ -53,6 +55,18 @@ class NFOService:
self.image_size = image_size
self.auto_create = auto_create
async def __aenter__(self) -> "NFOService":
"""Enter async context manager."""
await self.tmdb_client.__aenter__()
await self.image_downloader.__aenter__()
return self
async def __aexit__(self, exc_type, exc_val, exc_tb):
"""Exit async context manager and cleanup resources."""
await self.tmdb_client.close()
await self.image_downloader.close()
return False
def has_nfo(self, serie_folder: str) -> bool:
"""Check if tvshow.nfo exists for a series.
@@ -83,11 +97,12 @@ class NFOService:
>>> _extract_year_from_name("Attack on Titan")
("Attack on Titan", None)
"""
# Match year in parentheses at the end: (YYYY)
# Match the last year in parentheses at the end: (YYYY)
match = re.search(r'\((\d{4})\)\s*$', serie_name)
if match:
year = int(match.group(1))
clean_name = serie_name[:match.start()].strip()
# Strip ALL trailing year suffixes to get a fully clean name
clean_name = re.sub(r'(\s*\(\d{4}\))+\s*$', '', serie_name).strip()
return clean_name, year
return serie_name, None
@@ -110,7 +125,8 @@ class NFOService:
year: Optional[int] = None,
download_poster: bool = True,
download_logo: bool = True,
download_fanart: bool = True
download_fanart: bool = True,
alt_titles: Optional[List[str]] = None
) -> Path:
"""Create tvshow.nfo by scraping TMDB.
@@ -122,6 +138,7 @@ class NFOService:
download_poster: Whether to download poster.jpg
download_logo: Whether to download logo.png
download_fanart: Whether to download fanart.jpg
alt_titles: Alternative titles (e.g., Japanese title) for fallback search
Returns:
Path to created NFO file
@@ -146,63 +163,89 @@ class NFOService:
logger.info("Creating series folder: %s", folder_path)
folder_path.mkdir(parents=True, exist_ok=True)
# Check for existing NFO with TMDB ID to skip search
nfo_path = folder_path / "tvshow.nfo"
existing_ids = None
if nfo_path.exists():
try:
existing_ids = self.parse_nfo_ids(nfo_path)
if existing_ids.get("tmdb_id"):
logger.info(
"Found existing TMDB ID %s in NFO, using directly",
existing_ids["tmdb_id"]
)
except Exception as e:
logger.debug("Could not parse existing NFO IDs: %s", e)
try:
await self.tmdb_client._ensure_session()
# Search for TV show with clean name (without year)
logger.debug("Searching TMDB for: %s", search_name)
search_results = await self.tmdb_client.search_tv_show(search_name)
if not search_results.get("results"):
raise TMDBAPIError(f"No results found for: {search_name}")
# Find best match (consider year if provided)
tv_show = self._find_best_match(search_results["results"], search_name, year)
tv_id = tv_show["id"]
# Use existing TMDB ID if found, otherwise search
if existing_ids and existing_ids.get("tmdb_id"):
tv_id = existing_ids["tmdb_id"]
logger.info("Fetching details directly for TMDB ID: %s", tv_id)
details = await self.tmdb_client.get_tv_show_details(
tv_id,
append_to_response="credits,external_ids,images"
)
content_ratings = await self.tmdb_client.get_tv_show_content_ratings(tv_id)
tv_show = {"id": tv_id, "name": details.get("name", serie_name)}
search_source = "nfo_override"
else:
# Search for TV show - try multiple strategies
tv_show, search_source = await self._search_with_fallback(
search_name, year, alt_titles
)
tv_id = tv_show["id"]
logger.info("Found match: %s (ID: %s)", tv_show['name'], tv_id)
# Get detailed information with multi-language image support
details = await self.tmdb_client.get_tv_show_details(
tv_id,
append_to_response="credits,external_ids,images"
)
# Skip if we already fetched details via nfo_override
if search_source != "nfo_override":
details = await self.tmdb_client.get_tv_show_details(
tv_id,
append_to_response="credits,external_ids,images"
)
# Get content ratings for FSK
content_ratings = await self.tmdb_client.get_tv_show_content_ratings(tv_id)
# Get content ratings for FSK
content_ratings = await self.tmdb_client.get_tv_show_content_ratings(tv_id)
# Enrich with fallback languages for empty overview/tagline
# Pass search result overview as last resort fallback
search_overview = tv_show.get("overview") or None
if not search_overview:
try:
logger.debug(
"No overview in German search result, trying en-US search fallback for: %s",
search_name,
)
en_search_results = await self.tmdb_client.search_tv_show(
search_name,
language="en-US",
)
if en_search_results.get("results"):
en_match = self._find_best_match(
en_search_results["results"], search_name, year
# Enrich with fallback languages for empty overview/tagline
# Pass search result overview as last resort fallback
search_overview = tv_show.get("overview") or None
if not search_overview:
try:
logger.debug(
"No overview in German search result, trying en-US search fallback for: %s",
search_name,
)
search_overview = en_match.get("overview") or None
if search_overview:
logger.info(
"Using en-US search overview fallback for %s",
search_name,
en_search_results = await self.tmdb_client.search_tv_show(
search_name,
language="en-US",
)
if en_search_results.get("results"):
en_match = self._find_best_match(
en_search_results["results"], search_name, year
)
except (TMDBAPIError, Exception) as exc:
logger.warning(
"Failed en-US search fallback for overview: %s",
exc,
)
search_overview = en_match.get("overview") or None
if search_overview:
logger.info(
"Using en-US search overview fallback for %s",
search_name,
)
except (TMDBAPIError, Exception) as exc:
logger.warning(
"Failed en-US search fallback for overview: %s",
exc,
)
details = await self._enrich_details_with_fallback(
details, search_overview=search_overview
)
details = await self._enrich_details_with_fallback(
details, search_overview=search_overview
)
else:
# When using nfo_override, content_ratings already fetched
pass
# Convert TMDB data to TVShowNFO model
nfo_model = tmdb_to_nfo_model(
@@ -412,6 +455,62 @@ class NFOService:
logger.error("Error parsing NFO file %s: %s", nfo_path, e)
return result
def parse_nfo_year(self, nfo_path: Path) -> Optional[int]:
"""Parse year from an existing NFO file.
Extracts year from <year> or <premiered> elements.
Args:
nfo_path: Path to tvshow.nfo file
Returns:
Year as integer if found, None otherwise.
Example:
>>> year = nfo_service.parse_nfo_year(Path("/anime/series/tvshow.nfo"))
>>> print(year)
2013
"""
if not nfo_path.exists():
logger.debug("NFO file not found: %s", nfo_path)
return None
try:
tree = etree.parse(str(nfo_path))
root = tree.getroot()
# Try <year> element first
year_elem = root.find(".//year")
if year_elem is not None and year_elem.text:
try:
year = int(year_elem.text)
if 1900 <= year <= 2100:
logger.debug("Found year in NFO: %d", year)
return year
except ValueError:
pass
# Fallback: try <premiered> element (format: YYYY-MM-DD)
premiered_elem = root.find(".//premiered")
if premiered_elem is not None and premiered_elem.text:
if premiered_elem.text and len(premiered_elem.text) >= 4:
try:
year = int(premiered_elem.text[:4])
if 1900 <= year <= 2100:
logger.debug("Found year from premiered in NFO: %d", year)
return year
except ValueError:
pass
logger.debug("No year found in NFO: %s", nfo_path)
except etree.XMLSyntaxError as e:
logger.error("Invalid XML in NFO file %s: %s", nfo_path, e)
except Exception as e: # pylint: disable=broad-except
logger.error("Error parsing year from NFO file %s: %s", nfo_path, e)
return None
async def _enrich_details_with_fallback(
self,
@@ -518,6 +617,161 @@ class NFOService:
# Return first result (usually best match)
return results[0]
async def _search_with_fallback(
self,
primary_query: str,
year: Optional[int],
alt_titles: Optional[List[str]] = None
) -> Tuple[Dict[str, Any], str]:
"""Search TMDB with fallback strategies.
Tries multiple search strategies in order:
1. Primary query with year filter
2. Alternative titles (e.g., Japanese name)
3. Multi-language search (en-US)
4. Search without year constraint
5. Punctuation-normalized search
Args:
primary_query: Primary search term
year: Release year for filtering
alt_titles: Alternative titles to try if primary fails
Returns:
Tuple of (matched TV show dict, source description string)
Raises:
TMDBAPIError: If all search strategies fail
"""
search_strategies = [
# Strategy 1: Primary query as-is
{"query": primary_query, "year": year, "lang": "de-DE", "desc": "primary"},
]
# Strategy 2: Try alt titles (typically Japanese)
if alt_titles:
for alt in alt_titles:
if alt != primary_query:
search_strategies.append(
{"query": alt, "year": year, "lang": "ja-JP", "desc": f"alt_title:{alt}"}
)
search_strategies.append(
{"query": alt, "year": year, "lang": "en-US", "desc": f"alt_title:{alt}"}
)
# Strategy 3: Try English search
search_strategies.append(
{"query": primary_query, "year": year, "lang": "en-US", "desc": "english"}
)
# Strategy 4: Try without year constraint
if year:
search_strategies.append(
{"query": primary_query, "year": None, "lang": "de-DE", "desc": "no_year"}
)
# Strategy 5: Normalize punctuation
normalized = self._normalize_query_for_search(primary_query)
if normalized != primary_query:
search_strategies.append(
{"query": normalized, "year": year, "lang": "de-DE", "desc": f"normalized:{normalized}"}
)
# Strategy 6: Try search/multi for series indexed as movies
search_strategies.append(
{"query": primary_query, "year": year, "lang": "en-US", "desc": "multi_search", "use_multi": True}
)
last_error = None
for strategy in search_strategies:
query = strategy["query"]
lang = strategy["lang"]
desc = strategy["desc"]
use_multi = strategy.get("use_multi", False)
try:
logger.debug(
"TMDB search attempt: query='%s', lang=%s, year=%s, strategy=%s",
query, lang, strategy["year"], desc
)
# Use search/multi for multi_search strategy
if use_multi:
search_results = await self.tmdb_client.search_multi(
query,
language=lang
)
# Filter for TV shows only
if search_results.get("results"):
tv_results = [
r for r in search_results["results"]
if r.get("media_type") == "tv"
]
if tv_results:
search_results["results"] = tv_results
else:
search_results["results"] = []
else:
search_results = await self.tmdb_client.search_tv_show(
query,
language=lang
)
if search_results.get("results"):
# Apply year filter if we have one
results = search_results["results"]
if strategy["year"]:
year_filtered = [
r for r in results
if r.get("first_air_date", "").startswith(str(strategy["year"]))
]
if year_filtered:
match = year_filtered[0]
else:
# Year didn't match, still use first result but log it
match = results[0]
logger.debug(
"Year %s not found in results for '%s', using: %s",
strategy["year"], query, match["name"]
)
else:
match = results[0]
logger.info(
"TMDB search succeeded: '%s' found via strategy '%s' (ID: %s)",
match["name"], desc, match["id"]
)
return match, desc
else:
logger.debug("No results for '%s' via %s", query, desc)
except TMDBAPIError as e:
last_error = e
logger.debug("Search strategy '%s' failed: %s", desc, e)
continue
# All strategies exhausted
raise TMDBAPIError(
f"No results found for: {primary_query} (tried {len(search_strategies)} strategies)"
)
def _normalize_query_for_search(self, query: str) -> str:
"""Normalize query by removing punctuation and special chars.
Args:
query: Original search query
Returns:
Query with punctuation removed
"""
# Remove common punctuation but keep CJK characters
normalized = unicodedata.normalize('NFKC', query)
# Remove punctuation but not CJK
normalized = re.sub(r'[^\w\s\u3000-\u9fff\u4e00-\u9faf]', '', normalized)
# Collapse multiple spaces
normalized = re.sub(r'\s+', ' ', normalized).strip()
return normalized
async def _download_media_files(
@@ -585,3 +839,53 @@ class NFOService:
async def close(self):
"""Clean up resources."""
await self.tmdb_client.close()
await self.image_downloader.close()
async def create_minimal_nfo(
self,
serie_name: str,
serie_folder: str,
year: Optional[int] = None
) -> Path:
"""Create minimal tvshow.nfo when TMDB lookup fails.
Creates a basic NFO with just the title (and year if available)
so the series is tracked even without TMDB metadata.
Args:
serie_name: Name of the series (may include year in parentheses)
serie_folder: Series folder name
year: Optional release year
Returns:
Path to created NFO file
Raises:
FileNotFoundError: If series folder doesn't exist
"""
# Extract year from name if not provided
clean_name, extracted_year = self._extract_year_from_name(serie_name)
if year is None and extracted_year is not None:
year = extracted_year
folder_path = self.anime_directory / serie_folder
if not folder_path.exists():
logger.info("Creating series folder: %s", folder_path)
folder_path.mkdir(parents=True, exist_ok=True)
# Create minimal NFO model with just title and year
nfo_model = TVShowNFO(
title=clean_name,
year=year,
plot=f"No metadata available for {clean_name}. TMDB lookup failed."
)
# Generate XML
nfo_xml = generate_tvshow_nfo(nfo_model)
# Save NFO file
nfo_path = folder_path / "tvshow.nfo"
nfo_path.write_text(nfo_xml, encoding="utf-8")
logger.info("Created minimal NFO (no TMDB): %s", nfo_path)
return nfo_path

View File

@@ -129,6 +129,9 @@ class SeriesManagerService:
if not self.nfo_service:
return
nfo_exists = False
ids = {}
try:
folder_path = Path(self.anime_directory) / serie_folder
nfo_path = folder_path / "tvshow.nfo"
@@ -195,22 +198,49 @@ class SeriesManagerService:
logger.info(
f"Creating NFO for '{serie_name}' ({serie_folder})"
)
await self.nfo_service.create_tvshow_nfo(
serie_name=serie_name,
serie_folder=serie_folder,
year=year,
download_poster=self.download_poster,
download_logo=self.download_logo,
download_fanart=self.download_fanart
)
logger.info("Successfully created NFO for '%s'", serie_name)
try:
await self.nfo_service.create_tvshow_nfo(
serie_name=serie_name,
serie_folder=serie_folder,
year=year,
download_poster=self.download_poster,
download_logo=self.download_logo,
download_fanart=self.download_fanart
)
logger.info("Successfully created NFO for '%s'", serie_name)
except TMDBAPIError as create_error:
# TMDB lookup failed, create minimal NFO to track the series
logger.warning(
"TMDB lookup failed for '%s', creating minimal NFO: %s",
serie_name, create_error
)
try:
await self.nfo_service.create_minimal_nfo(
serie_name=serie_name,
serie_folder=serie_folder,
year=year
)
logger.info("Created minimal NFO for '%s'", serie_name)
except Exception as minimal_error:
logger.error(
"Failed to create minimal NFO for '%s': %s",
serie_name, minimal_error
)
elif nfo_exists:
logger.debug(
f"NFO exists for '{serie_name}', skipping download"
)
except TMDBAPIError as e:
logger.error("TMDB API error processing '%s': %s", serie_name, e)
# Only log at ERROR if no NFO exists and we have no IDs
# If NFO exists with IDs, this is just a lookup failure, log at DEBUG
if nfo_exists and (ids.get("tmdb_id") or ids.get("tvdb_id")):
logger.debug(
"TMDB API lookup failed for '%s' (has NFO with IDs): %s",
serie_name, e
)
else:
logger.error("TMDB API error processing '%s': %s", serie_name, e)
except Exception as e:
logger.error(
f"Unexpected error processing NFO for '{serie_name}': {e}",

View File

@@ -12,6 +12,7 @@ Example:
import asyncio
import logging
import time
from pathlib import Path
from typing import Any, Dict, List, Optional
@@ -38,6 +39,7 @@ class TMDBClient:
DEFAULT_BASE_URL = "https://api.themoviedb.org/3"
DEFAULT_IMAGE_BASE_URL = "https://image.tmdb.org/t/p"
NEGATIVE_CACHE_TTL = 86400 # 24 hours
def __init__(
self,
@@ -63,6 +65,12 @@ class TMDBClient:
self.max_connections = max_connections
self.session: Optional[aiohttp.ClientSession] = None
self._cache: Dict[str, Any] = {}
self._negative_cache: Dict[str, float] = {} # query -> timestamp when cached
# TMDB allows ~40 req/s; use 30 concurrent + per-second throttle to stay safe
self._semaphore = asyncio.Semaphore(30)
self._rate_limit_lock = asyncio.Lock()
self._request_timestamps: List[float] = []
self._max_requests_per_second = 35 # Stay under TMDB's ~40/s limit
async def __aenter__(self):
"""Async context manager entry."""
@@ -83,7 +91,7 @@ class TMDBClient:
self,
endpoint: str,
params: Optional[Dict[str, Any]] = None,
max_retries: int = 3
max_retries: int = 5
) -> Dict[str, Any]:
"""Make an async request to TMDB API with retries.
@@ -110,58 +118,100 @@ class TMDBClient:
logger.debug("Cache hit for %s", endpoint)
return self._cache[cache_key]
# Check negative cache (cached empty results)
negative_cache_key = f"{endpoint}:{str(sorted(params.items()))}"
if negative_cache_key in self._negative_cache:
if time.monotonic() - self._negative_cache[negative_cache_key] < self.NEGATIVE_CACHE_TTL:
logger.debug("Negative cache hit for %s (cached empty result)", endpoint)
return {"results": []}
else:
# Expired negative cache entry
del self._negative_cache[negative_cache_key]
delay = 1
last_error = None
for attempt in range(max_retries):
try:
# Re-ensure session before each attempt in case it was closed
await self._ensure_session()
if self.session is None:
raise TMDBAPIError("Session is not available")
logger.debug("TMDB API request: %s (attempt %s)", endpoint, attempt + 1)
async with self.session.get(url, params=params, timeout=aiohttp.ClientTimeout(total=60)) as resp:
if resp.status == 401:
raise TMDBAPIError("Invalid TMDB API key")
elif resp.status == 404:
raise TMDBAPIError(f"Resource not found: {endpoint}")
elif resp.status == 429:
# Rate limit - wait longer
retry_after = int(resp.headers.get('Retry-After', delay * 2))
logger.warning("Rate limited, waiting %ss", retry_after)
await asyncio.sleep(retry_after)
continue
resp.raise_for_status()
data = await resp.json()
self._cache[cache_key] = data
return data
except asyncio.TimeoutError as e:
last_error = e
if attempt < max_retries - 1:
logger.warning("Request timeout (attempt %s), retrying in %ss", attempt + 1, delay)
await asyncio.sleep(delay)
delay *= 2
else:
logger.error("Request timed out after %s attempts", max_retries)
except (aiohttp.ClientError, AttributeError) as e:
last_error = e
# If connector/session was closed, try to recreate it
if "Connector is closed" in str(e) or isinstance(e, AttributeError):
logger.warning("Session issue detected, recreating session: %s", e)
self.session = None
# Rate limiting: ensure we don't exceed ~35 requests/second
async with self._rate_limit_lock:
now = time.monotonic()
# Remove timestamps older than 1 second
self._request_timestamps = [
ts for ts in self._request_timestamps if now - ts < 1.0
]
if len(self._request_timestamps) >= self._max_requests_per_second:
sleep_time = 1.0 - (now - self._request_timestamps[0])
if sleep_time > 0:
logger.debug("Rate throttling: waiting %.2fs", sleep_time)
await asyncio.sleep(sleep_time)
self._request_timestamps.append(time.monotonic())
async with self._semaphore:
for attempt in range(max_retries):
try:
# Re-ensure session before each attempt in case it was closed
await self._ensure_session()
if attempt < max_retries - 1:
logger.warning("Request failed (attempt %s): %s, retrying in %ss", attempt + 1, e, delay)
await asyncio.sleep(delay)
delay *= 2
else:
logger.error("Request failed after %s attempts: %s", max_retries, e)
if self.session is None:
raise TMDBAPIError("Session is not available")
logger.debug("TMDB API request: %s (attempt %s)", endpoint, attempt + 1)
async with self.session.get(url, params=params, timeout=aiohttp.ClientTimeout(total=60)) as resp:
if resp.status == 401:
raise TMDBAPIError("Invalid TMDB API key")
elif resp.status == 404:
raise TMDBAPIError(f"Resource not found: {endpoint}")
elif resp.status == 429:
# Rate limit - wait longer with exponential backoff
retry_after = int(resp.headers.get('Retry-After', max(delay * 2, 2)))
logger.warning("Rate limited, waiting %ss", retry_after)
await asyncio.sleep(retry_after)
continue
resp.raise_for_status()
data = await resp.json()
self._cache[cache_key] = data
# Cache negative result if empty
if endpoint.startswith("search/") and not data.get("results"):
self._negative_cache[negative_cache_key] = time.monotonic()
logger.debug("Cached negative result for %s", endpoint)
return data
except asyncio.TimeoutError as e:
last_error = e
if attempt < max_retries - 1:
logger.warning("Request timeout (attempt %s), retrying in %ss", attempt + 1, delay)
await asyncio.sleep(delay)
delay *= 2
else:
logger.error("Request timed out after %s attempts", max_retries)
except (aiohttp.ClientError, AttributeError) as e:
last_error = e
# If connector/session was closed, try to recreate it
if "Connector is closed" in str(e) or isinstance(e, AttributeError):
logger.warning(
"Session issue detected, recreating session: %s",
e,
exc_info=True,
)
self.session = None
await self._ensure_session()
# DNS / host-unreachable errors are not transient — abort immediately
error_str = str(e)
if "name resolution" in error_str.lower() or (
isinstance(e, aiohttp.ClientConnectorError) and
"Cannot connect to host" in error_str
):
logger.error("Non-transient connection error, aborting retries: %s", e)
raise TMDBAPIError(f"Request failed after {attempt + 1} attempts: {e}") from e
if attempt < max_retries - 1:
logger.warning("Request failed (attempt %s): %s, retrying in %ss", attempt + 1, e, delay)
await asyncio.sleep(delay)
delay *= 2
else:
logger.error("Request failed after %s attempts: %s", max_retries, e)
raise TMDBAPIError(f"Request failed after {max_retries} attempts: {last_error}")
@@ -190,6 +240,34 @@ class TMDBClient:
{"query": query, "language": language, "page": page}
)
async def search_multi(
self,
query: str,
language: str = "en-US",
page: int = 1
) -> Dict[str, Any]:
"""Search for movies and TV shows by name using TMDB multi search.
Multi search returns both movies and TV shows, useful for anime
that might be indexed as movies on TMDB.
Args:
query: Search query (show name)
language: Language for results (default: English)
page: Page number for pagination
Returns:
Search results with list of movies and TV shows
Example:
>>> results = await client.search_multi("Suzume no Tojimari")
>>> shows = [r for r in results["results"] if r["media_type"] == "tv"]
"""
return await self._request(
"search/multi",
{"query": query, "language": language, "page": page}
)
async def get_tv_show_details(
self,
tv_id: int,
@@ -309,8 +387,38 @@ class TMDBClient:
await self.session.close()
self.session = None
logger.debug("TMDB client session closed")
def __del__(self):
"""Warn if session is unclosed during garbage collection."""
if self.session is not None and not self.session.closed:
logger.warning(
"TMDBClient: unclosed session detected. "
"Use 'async with TMDBClient(...)' or call close() explicitly."
)
def clear_cache(self):
"""Clear the request cache."""
self._cache.clear()
logger.debug("TMDB client cache cleared")
def clear_negative_cache(self):
"""Clear the negative result cache."""
self._negative_cache.clear()
logger.debug("TMDB negative cache cleared")
def cleanup_expired_negative_cache(self) -> int:
"""Remove expired entries from negative cache.
Returns:
Number of entries removed
"""
now = time.monotonic()
expired_keys = [
key for key, timestamp in self._negative_cache.items()
if now - timestamp >= self.NEGATIVE_CACHE_TTL
]
for key in expired_keys:
del self._negative_cache[key]
if expired_keys:
logger.debug("Removed %d expired negative cache entries", len(expired_keys))
return len(expired_keys)

248
src/core/utils/key_utils.py Normal file
View File

@@ -0,0 +1,248 @@
"""Utility functions for generating URL-safe keys from folder names.
This module provides key generation and normalization for anime series,
handling edge cases like non-Latin characters and special symbols.
"""
from __future__ import annotations
import re
import unicodedata
import uuid
from typing import Optional
# Valid key pattern: alphanumeric, hyphens, underscores
# Must be at least 1 char, URL-safe
VALID_KEY_PATTERN = re.compile(r'^[a-zA-Z0-9][a-zA-Z0-9_-]*$')
def normalize_key(key: str) -> str:
"""Normalize a key to a URL-safe format.
Args:
key: The key to normalize
Returns:
Normalized lowercase key with spaces replaced by hyphens
"""
if not key:
return ""
# Convert to lowercase
normalized = key.lower()
# Replace spaces and underscores with hyphens
normalized = re.sub(r'[\s_]+', '-', normalized)
# Remove any characters that aren't alphanumeric or hyphens
normalized = re.sub(r'[^a-z0-9-]', '', normalized)
# Collapse multiple consecutive hyphens
normalized = re.sub(r'-+', '-', normalized)
# Remove leading/trailing hyphens
normalized = normalized.strip('-')
return normalized
def is_valid_key(key: str) -> bool:
"""Check if a key is valid for URL-safe use.
Args:
key: The key to validate
Returns:
True if key is valid (non-empty, URL-safe, alphanumeric start/end, min 2 chars)
"""
if not key or not key.strip():
return False
if len(key) < 2:
return False
return bool(VALID_KEY_PATTERN.match(key))
def generate_key_from_folder(folder_name: str) -> str:
"""Generate a URL-safe key from a folder name.
Handles edge cases:
- Non-Latin characters (Japanese, Chinese, etc.)
- Special characters
- All-invalid names that normalize to empty
Args:
folder_name: The folder name to convert to a key
Returns:
A URL-safe key string. Never returns empty string.
Examples:
>>> generate_key_from_folder("Attack on Titan (2013)")
'attack-on-titan-2013'
>>> generate_key_from_folder("A Time Called You (2023)")
'a-time-called-you-2023'
>>> generate_key_from_folder("25-sai no Joshikousei (2018)")
'25-sai-no-joshikousei-2018'
"""
if not folder_name or not folder_name.strip():
raise ValueError("Folder name cannot be empty")
# Step 1: Unicode NFC normalization (preserves international chars)
normalized = unicodedata.normalize('NFC', folder_name.strip())
# Step 2: Extract alphanumeric parts, preserving international chars
# This keeps Japanese/Chinese characters but removes special symbols
parts = []
for char in normalized:
# Keep Unicode alphanumeric characters (letters/numbers from any script)
if char.isalnum():
parts.append(char)
elif char.isspace():
parts.append(' ')
# Handle apostrophes - treat as part of word (remove, don't replace with space)
# This normalizes e.g., "Hell's" -> "Hells"
# Includes: ' (0x27), ' (0x2018), ' (0x2019), ' (0x02BC), ` (0x0060)
elif char in ("'", "'", "'", "'", "`", """, """):
pass # Skip - drop the apostrophe
else:
parts.append(' ')
working = ''.join(parts)
# Step 3: Split into words and normalize each
words = working.split()
# Step 4: Convert to lowercase and create hyphenated key
key = '-'.join(word.lower() for word in words if word)
# Step 5: If we got a valid key, return it
if key and is_valid_key(key):
return key
# Step 6: Try just alphanumeric characters
alphanumeric_only = re.sub(r'[^a-zA-Z0-9\s]', '', working)
words = alphanumeric_only.split()
key = '-'.join(word.lower() for word in words if word)
if key and is_valid_key(key):
return key
# Step 7: Last resort - use folder name directly with transliteration
# Try to convert non-ASCII to ASCII equivalents
try:
# Use NFD normalization and strip combining characters
# This effectively Latinizes some characters
nfd_form = unicodedata.normalize('NFD', folder_name)
latinized = ''.join(
char for char in nfd_form
if unicodedata.category(char) != 'Mn' # Strip combining marks
)
# Remove non-ASCII letters
latinized = re.sub(r'[^a-zA-Z0-9\s]', ' ', latinized)
words = latinized.split()
key = '-'.join(word.lower() for word in words if word)
if key and is_valid_key(key):
return key
except Exception:
pass
# Step 8: Absolute fallback - generate UUID-based key
# Use first 8 chars of UUID for brevity
uuid_key = uuid.uuid4().hex[:8]
# Try to extract any meaningful words from the original name
meaningful_parts = []
for char in folder_name:
if char.isalnum():
meaningful_parts.append(char.lower())
elif len(meaningful_parts) > 0:
meaningful_parts.append('-')
fallback_base = ''.join(meaningful_parts).strip('-')
if fallback_base and len(fallback_base) >= 2:
# Combine meaningful parts with UUID for uniqueness
# Truncate meaningful parts if too long
if len(fallback_base) > 20:
fallback_base = fallback_base[:20]
return f"{fallback_base}-{uuid_key}"
return f"series-{uuid_key}"
def validate_key_uniqueness(
key: str,
existing_keys: set[str],
) -> tuple[bool, str]:
"""Validate that a key is unique among existing keys.
Args:
key: The key to validate
existing_keys: Set of keys that already exist
Returns:
Tuple of (is_valid, error_message)
"""
if not key or not key.strip():
return False, "Key cannot be empty"
stripped = key.strip()
if len(stripped) < 2:
return False, "Key must be at least 2 characters"
if not is_valid_key(stripped):
return False, "Key must be URL-safe (alphanumeric, hyphens, underscores only)"
if stripped in existing_keys:
return False, f"Key '{stripped}' is already in use"
return True, ""
def sanitize_key_for_url(key: str) -> str:
"""Sanitize a key for safe URL usage.
Args:
key: The key to sanitize
Returns:
URL-safe version of the key
"""
if not key:
return ""
# Replace spaces with hyphens first
sanitized = key.replace(' ', '-')
# Remove any characters that could cause URL issues (keep alphanumerics, hyphens, underscores)
sanitized = re.sub(r'[^\w\-]', '', sanitized)
# Collapse multiple hyphens
sanitized = re.sub(r'-+', '-', sanitized)
return sanitized.strip('-')
def sanitize_url_for_logging(url: str, max_length: int = 100) -> str:
"""Sanitize a URL for safe logging by removing sensitive query parameters.
Removes or truncates query parameters that may contain tokens, keys,
or other sensitive data while preserving enough structure for debugging.
Args:
url: The URL to sanitize
max_length: Maximum length of the returned URL string
Returns:
Sanitized URL safe for logging
"""
if not url:
return ""
# Truncate if too long
if len(url) > max_length:
return url[:max_length] + "..."
return url

View File

@@ -1,12 +1,15 @@
import logging
import warnings
from pathlib import Path
from typing import Any, List, Optional
from fastapi import APIRouter, Depends, HTTPException, status
from pydantic import BaseModel, Field
from sqlalchemy.ext.asyncio import AsyncSession
from src.config.settings import settings
from src.core.entities.series import Serie
from src.core.utils.key_utils import generate_key_from_folder, is_valid_key
from src.server.database.service import AnimeSeriesService
from src.server.exceptions import (
BadRequestError,
@@ -14,11 +17,16 @@ from src.server.exceptions import (
ServerError,
ValidationError,
)
from src.server.models.anime import AnimeMetadataUpdate
from src.server.services.anime_service import AnimeService, AnimeServiceError
from src.server.services.background_loader_service import BackgroundLoaderService
from src.server.services.scheduler.folder_rename_service import (
_scan_for_pre_existing_duplicates,
)
from src.server.utils.dependencies import (
get_anime_service,
get_background_loader_service,
get_database_session,
get_optional_database_session,
get_series_app,
require_auth,
@@ -70,6 +78,100 @@ async def get_anime_status(
) from exc
class DuplicateFolderGroup(BaseModel):
"""A group of duplicate folders for the same series.
Attributes:
key: Series key (provider-assigned unique identifier)
folders: List of folder names that are duplicates
folder_count: Number of duplicate folders
"""
key: str = Field(..., description="Series key (unique identifier)")
folders: List[str] = Field(..., description="List of duplicate folder names")
folder_count: int = Field(..., description="Number of duplicate folders")
class DuplicateFoldersResponse(BaseModel):
"""Response model for duplicate folders listing.
Attributes:
total_groups: Total number of duplicate groups found
duplicate_groups: List of duplicate folder groups
message: Human-readable summary
"""
total_groups: int = Field(..., description="Total number of duplicate groups")
duplicate_groups: List[DuplicateFolderGroup] = Field(
..., description="List of duplicate folder groups"
)
message: str = Field(..., description="Human-readable summary")
@router.get("/duplicate-folders", response_model=DuplicateFoldersResponse)
async def get_duplicate_folders(
_auth: dict = Depends(require_auth),
) -> DuplicateFoldersResponse:
"""List all pre-existing duplicate folder groups.
Scans the anime directory for folders with tvshow.nfo files that
map to the same series key. Returns groups of duplicates for
manual review and cleanup.
Returns:
DuplicateFoldersResponse with groups of duplicate folders
Note:
Not all duplicate folders are safe to merge - some may belong
to different releases (e.g., dubbed vs. subbed). Review carefully
before taking action.
"""
try:
if not settings.anime_directory:
return DuplicateFoldersResponse(
total_groups=0,
duplicate_groups=[],
message="Anime directory not configured",
)
anime_dir = Path(settings.anime_directory)
if not anime_dir.is_dir():
return DuplicateFoldersResponse(
total_groups=0,
duplicate_groups=[],
message=f"Anime directory not found: {anime_dir}",
)
duplicates = _scan_for_pre_existing_duplicates(anime_dir)
groups = [
DuplicateFolderGroup(
key=dup.key,
folders=dup.folders,
folder_count=dup.count,
)
for dup in duplicates
]
if groups:
message = (
f"Found {len(groups)} duplicate group(s). "
"Review carefully - some duplicates may be different releases "
"(e.g., dubbed vs. subbed)."
)
else:
message = "No duplicate folders found."
return DuplicateFoldersResponse(
total_groups=len(groups),
duplicate_groups=groups,
message=message,
)
except Exception as exc:
logger.error("Failed to scan for duplicate folders: %s", str(exc))
raise ServerError(
message=f"Failed to scan for duplicates: {str(exc)}"
) from exc
class AnimeSummary(BaseModel):
"""Summary of an anime series with missing episodes.
@@ -730,7 +832,11 @@ async def add_series(
# Create folder name with year if available
if year:
folder_name_with_year = f"{name} ({year})"
year_suffix = f" ({year})"
if name.endswith(year_suffix):
folder_name_with_year = name
else:
folder_name_with_year = f"{name}{year_suffix}"
else:
folder_name_with_year = name
@@ -1084,3 +1190,75 @@ async def get_anime(
# Maximum allowed input size for security
MAX_INPUT_LENGTH = 100000 # 100KB
@router.put("/{anime_key}")
async def update_anime_metadata(
anime_key: str,
body: AnimeMetadataUpdate,
_auth: dict = Depends(require_auth),
db: AsyncSession = Depends(get_database_session),
) -> dict:
"""Update anime metadata (key, tmdb_id, tvdb_id).
Args:
anime_key: Current series key to update
body: Fields to update (all optional)
_auth: Authentication dependency
db: Database session
Returns:
Updated series metadata
Raises:
HTTPException 404: Series not found
HTTPException 409: Key conflict (new key already exists)
HTTPException 422: Validation error
"""
series = await AnimeSeriesService.get_by_key(db, anime_key)
if not series:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Series with key '{anime_key}' not found",
)
updates = {}
if body.key is not None and body.key != anime_key:
existing = await AnimeSeriesService.get_by_key(db, body.key)
if existing:
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail=f"A series with key '{body.key}' already exists",
)
updates["key"] = body.key
if body.tmdb_id is not None:
updates["tmdb_id"] = body.tmdb_id
if body.tvdb_id is not None:
updates["tvdb_id"] = body.tvdb_id
if not updates:
return {
"key": series.key,
"tmdb_id": series.tmdb_id,
"tvdb_id": series.tvdb_id,
"message": "No changes",
}
updated = await AnimeSeriesService.update(db, series.id, **updates)
await db.commit()
logger.info(
"Updated metadata for '%s': %s",
anime_key,
updates,
)
return {
"key": updated.key,
"tmdb_id": updated.tmdb_id,
"tvdb_id": updated.tvdb_id,
"message": "Metadata updated successfully",
}

View File

@@ -163,6 +163,22 @@ async def setup_auth(req: SetupRequest):
# Perform NFO scan if configured
await perform_nfo_scan_if_needed(progress_service)
# Start scheduler if anime_directory is now set
try:
from src.server.services.scheduler.scheduler_service import (
get_scheduler_service,
)
scheduler_svc = get_scheduler_service()
logger.info("Starting scheduler after initialization")
await scheduler_svc.ensure_started()
logger.info("Scheduler started successfully during setup")
except Exception as sched_exc:
logger.warning(
"Failed to start scheduler during setup: %s", sched_exc
)
# Continue — scheduler failure should not break initialization
# Send completion event
from src.server.services.progress_service import ProgressType
await progress_service.start_progress(

View File

@@ -1,7 +1,10 @@
import logging
from typing import Any, Dict, List, Optional
from fastapi import APIRouter, Depends, HTTPException, status
logger = logging.getLogger(__name__)
from src.server.models.config import AppConfig, ConfigUpdate, ValidationResult
from src.server.services.config_service import (
ConfigBackupError,
@@ -28,16 +31,53 @@ def get_config(auth: Optional[dict] = Depends(require_auth)) -> AppConfig:
@router.put("", response_model=AppConfig)
def update_config(
async def update_config(
update: ConfigUpdate, auth: dict = Depends(require_auth)
) -> AppConfig:
"""Apply an update to the configuration and persist it.
Creates automatic backup before applying changes.
Creates automatic backup before applying changes. If anime_directory
is configured, starts the scheduler service.
"""
try:
config_service = get_config_service()
return config_service.update_config(update)
updated_config = config_service.update_config(update)
# Sync anime_directory to settings if it was updated
from src.config.settings import settings as app_settings
anime_dir_changed = False
if update.other and update.other.get("anime_directory"):
anime_dir = update.other.get("anime_directory")
if anime_dir and not app_settings.anime_directory:
app_settings.anime_directory = str(anime_dir)
anime_dir_changed = True
logger.info("Synced anime_directory from config: %s", anime_dir)
# Start scheduler if anime_directory was just configured
if anime_dir_changed:
try:
from src.server.services.scheduler.scheduler_service import (
get_scheduler_service,
)
scheduler_svc = get_scheduler_service()
logger.info(
"Starting scheduler after anime_directory configuration"
)
await scheduler_svc.ensure_started()
logger.info(
"Scheduler started successfully after config update"
)
except Exception as sched_exc:
logger.warning(
"Failed to start scheduler after config update: %s",
sched_exc,
)
# Config was already saved, don't fail the request
return updated_config
except ConfigValidationError as e:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
@@ -244,9 +284,9 @@ async def update_directory(
try:
import structlog
from src.server.services.anime_service import sync_series_from_data_files
from src.server.services.anime_service import sync_legacy_series_to_db
logger = structlog.get_logger(__name__)
sync_count = await sync_series_from_data_files(directory, logger)
sync_count = await sync_legacy_series_to_db(directory, logger)
logger.info(
"Directory updated: synced series from data files",
directory=directory,

View File

@@ -5,7 +5,7 @@ from datetime import datetime
from typing import Any, Dict, Optional
import psutil
from fastapi import APIRouter, Depends, HTTPException
from fastapi import APIRouter, Depends, HTTPException, Request
from pydantic import BaseModel
from sqlalchemy import text
from sqlalchemy.ext.asyncio import AsyncSession
@@ -17,15 +17,21 @@ logger = logging.getLogger(__name__)
router = APIRouter(prefix="/health", tags=["health"])
from src.server.utils.version import APP_VERSION
class HealthStatus(BaseModel):
"""Basic health status response."""
status: str
timestamp: str
version: str = "1.0.1"
version: str = APP_VERSION
service: str = "aniworld-api"
series_app_initialized: bool = False
anime_directory_configured: bool = False
scheduler_next_run: Optional[str] = None
scheduler_last_run: Optional[str] = None
checks: Optional[Dict[str, Any]] = None
class DatabaseHealth(BaseModel):
@@ -60,7 +66,7 @@ class DetailedHealthStatus(BaseModel):
status: str
timestamp: str
version: str = "1.0.1"
version: str = APP_VERSION
dependencies: DependencyHealth
startup_time: datetime
@@ -171,29 +177,92 @@ def get_system_metrics() -> SystemMetrics:
@router.get("", response_model=HealthStatus)
async def basic_health_check() -> HealthStatus:
async def basic_health_check(request: Request) -> HealthStatus:
"""Basic health check endpoint.
This endpoint does not depend on anime_directory configuration
and should always return 200 OK for basic health monitoring.
Includes service information for identification.
Includes scheduler next/last run times for monitoring tools.
Includes startup health check results.
Returns:
HealthStatus: Simple health status with timestamp and service info.
"""
from src.config.settings import settings
from src.server.utils.dependencies import _series_app
# Get scheduler status for health monitoring
scheduler_status: dict = {}
try:
from src.server.services.scheduler.scheduler_service import (
get_scheduler_service,
)
scheduler_status = get_scheduler_service().get_status()
except Exception:
pass
# Get startup checks from app state
checks = getattr(request.app.state, "startup_checks", None)
# Determine overall status based on checks
overall_status = "healthy"
if checks:
for check_name, check_data in checks.items():
if check_data.get("status") == "error":
overall_status = "unhealthy"
break
elif check_data.get("status") == "warning":
overall_status = "degraded"
logger.debug("Basic health check requested")
return HealthStatus(
status="healthy",
status=overall_status,
timestamp=datetime.now().isoformat(),
service="aniworld-api",
series_app_initialized=_series_app is not None,
anime_directory_configured=bool(settings.anime_directory),
scheduler_next_run=scheduler_status.get("next_run"),
scheduler_last_run=scheduler_status.get("last_run"),
checks=checks,
)
@router.get("/ready")
async def ready_check(request: Request) -> Dict[str, Any]:
"""Readiness check endpoint for container orchestrators.
Returns 503 if critical dependencies are not available.
This endpoint is used by Kubernetes, Docker Swarm, etc. to determine
if the container should receive traffic.
Returns:
dict: Readiness status with checks details.
"""
checks = getattr(request.app.state, "startup_checks", {})
critical_failures = []
for check_name, check_data in checks.items():
if check_data.get("status") == "error":
critical_failures.append(f"{check_name}: {check_data.get('message')}")
if critical_failures:
return {
"status": "not_ready",
"ready": False,
"timestamp": datetime.now().isoformat(),
"critical_failures": critical_failures,
"checks": checks,
}
return {
"status": "ready",
"ready": True,
"timestamp": datetime.now().isoformat(),
"checks": checks,
}
@router.get("/detailed", response_model=DetailedHealthStatus)
async def detailed_health_check(
db: AsyncSession = Depends(get_database_session),

View File

@@ -16,6 +16,11 @@ from src.core.entities.series import Serie
from src.core.SeriesApp import SeriesApp
from src.core.services.nfo_factory import get_nfo_factory
from src.core.services.nfo_service import NFOService
from src.core.services.nfo_repair_service import (
REQUIRED_TAGS,
NfoRepairService,
find_missing_tags,
)
from src.core.services.tmdb_client import TMDBAPIError
from src.server.models.nfo import (
MediaDownloadRequest,
@@ -27,8 +32,10 @@ from src.server.models.nfo import (
NFOContentResponse,
NFOCreateRequest,
NFOCreateResponse,
NfoDiagnosticsResponse,
NFOMissingResponse,
NFOMissingSeries,
NfoRepairResponse,
)
from src.server.utils.dependencies import get_series_app, require_auth
from src.server.utils.media import check_media_files, get_media_file_paths
@@ -144,6 +151,27 @@ async def batch_create_nfo(
nfo_path=str(nfo_path)
)
except TMDBAPIError as e:
logger.warning("TMDB API error for %s, creating minimal fallback: %s", serie_id, e)
# TMDB failed, create minimal NFO
try:
serie_folder = serie.ensure_folder_with_year()
except Exception:
serie_folder = serie_folder
serie_name = serie.name or serie_folder
nfo_path = await nfo_service.create_minimal_nfo(
serie_name=serie_name,
serie_folder=serie_folder
)
return NFOBatchResult(
serie_id=serie_id,
serie_folder=serie_folder,
success=True,
message="Created minimal NFO (TMDB lookup failed)",
nfo_path=str(nfo_path)
)
except Exception as e:
logger.error(
f"Error creating NFO for {serie_id}: {e}",
@@ -429,11 +457,42 @@ async def create_nfo(
except HTTPException:
raise
except TMDBAPIError as e:
logger.warning("TMDB API error creating NFO for %s: %s", serie_id, e)
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail=f"TMDB API error: {str(e)}"
) from e
logger.warning("TMDB API error for %s, creating minimal fallback: %s", serie_id, e)
# TMDB failed, create minimal NFO with just folder name
try:
serie_folder = serie.ensure_folder_with_year()
except Exception:
serie_folder = serie_folder
folder_path = Path(settings.anime_directory) / serie_folder
serie_name_fallback = request.serie_name or serie.name or serie_folder
nfo_path = await nfo_service.create_minimal_nfo(
serie_name=serie_name_fallback,
serie_folder=serie_folder,
year=year
)
# Check media files (will likely be empty)
media_status = check_media_files(folder_path)
file_paths = get_media_file_paths(folder_path)
media_files = MediaFilesStatus(
has_poster=media_status.get("poster", False),
has_logo=media_status.get("logo", False),
has_fanart=media_status.get("fanart", False),
poster_path=str(file_paths["poster"]) if file_paths.get("poster") else None,
logo_path=str(file_paths["logo"]) if file_paths.get("logo") else None,
fanart_path=str(file_paths["fanart"]) if file_paths.get("fanart") else None
)
return NFOCreateResponse(
serie_id=serie_id,
serie_folder=serie_folder,
nfo_path=str(nfo_path),
media_files=media_files,
message="Created minimal NFO (TMDB lookup failed)"
)
except Exception as e:
logger.error(
f"Error creating NFO for {serie_id}: {e}",
@@ -756,3 +815,142 @@ async def download_media(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Failed to download media: {str(e)}"
) from e
@router.get("/{serie_key}/diagnostics", response_model=NfoDiagnosticsResponse)
async def get_nfo_diagnostics(
serie_key: str,
_auth: dict = Depends(require_auth),
series_app: SeriesApp = Depends(get_series_app),
) -> NfoDiagnosticsResponse:
"""Get NFO diagnostics showing missing required tags.
Args:
serie_key: Series key identifier
_auth: Authentication dependency
series_app: SeriesApp instance
Returns:
NfoDiagnosticsResponse with has_nfo, missing_tags, required_tags
Raises:
HTTPException 404: Series not found
"""
serie = None
for s in series_app.list.GetList():
if getattr(s, "key", None) == serie_key:
serie = s
break
if not serie:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Series with key '{serie_key}' not found",
)
serie_folder = serie.ensure_folder_with_year()
folder_path = Path(settings.anime_directory) / serie_folder
nfo_path = folder_path / "tvshow.nfo"
required_tag_names = list(REQUIRED_TAGS.values())
if not nfo_path.exists():
return NfoDiagnosticsResponse(
has_nfo=False,
nfo_path=None,
missing_tags=required_tag_names,
required_tags=required_tag_names,
)
missing = find_missing_tags(nfo_path)
return NfoDiagnosticsResponse(
has_nfo=True,
nfo_path=str(nfo_path),
missing_tags=missing,
required_tags=required_tag_names,
)
@router.post("/{serie_key}/repair", response_model=NfoRepairResponse)
async def repair_nfo(
serie_key: str,
_auth: dict = Depends(require_auth),
series_app: SeriesApp = Depends(get_series_app),
nfo_service: NFOService = Depends(get_nfo_service),
) -> NfoRepairResponse:
"""Repair or recreate NFO file for a series.
Detects missing required tags and re-fetches metadata from TMDB.
Args:
serie_key: Series key identifier
_auth: Authentication dependency
series_app: SeriesApp instance
nfo_service: NFO service for TMDB operations
Returns:
NfoRepairResponse with success status and details
Raises:
HTTPException 404: Series not found
HTTPException 400: Cannot repair (e.g., no TMDB data available)
"""
serie = None
for s in series_app.list.GetList():
if getattr(s, "key", None) == serie_key:
serie = s
break
if not serie:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Series with key '{serie_key}' not found",
)
serie_folder = serie.ensure_folder_with_year()
folder_path = Path(settings.anime_directory) / serie_folder
nfo_path = folder_path / "tvshow.nfo"
# Get missing tags before repair for reporting
missing_before = find_missing_tags(nfo_path) if nfo_path.exists() else list(REQUIRED_TAGS.values())
try:
repair_service = NfoRepairService(nfo_service)
if nfo_path.exists():
repaired = await repair_service.repair_series(folder_path, serie_folder)
if not repaired:
return NfoRepairResponse(
success=True,
message="NFO is already complete, no repair needed",
repaired_tags=[],
)
else:
# No NFO exists — create new one
await nfo_service.create_tvshow_nfo(
serie_name=serie.name,
serie_folder=serie_folder,
download_poster=True,
download_logo=True,
download_fanart=True,
)
return NfoRepairResponse(
success=True,
message=f"NFO repaired successfully. Fixed {len(missing_before)} missing tags.",
repaired_tags=missing_before,
)
except TMDBAPIError as e:
logger.warning("NFO repair failed for '%s': %s", serie_key, e)
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Cannot repair NFO: {str(e)}. Ensure TMDB ID is set.",
) from e
except Exception as e:
logger.error("NFO repair error for '%s': %s", serie_key, e, exc_info=True)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Failed to repair NFO: {str(e)}",
) from e

View File

@@ -10,7 +10,7 @@ from fastapi import APIRouter, Depends, HTTPException, status
from src.server.models.config import SchedulerConfig
from src.server.services.config_service import ConfigServiceError, get_config_service
from src.server.services.scheduler_service import get_scheduler_service
from src.server.services.scheduler.scheduler_service import get_scheduler_service
from src.server.utils.dependencies import require_auth
logger = logging.getLogger(__name__)

View File

@@ -39,7 +39,7 @@ def get_settings() -> Union[DevelopmentSettings, ProductionSettings]:
Example:
>>> settings = get_settings()
>>> print(settings.log_level)
DEBUG
INFO
"""
if ENVIRONMENT in {"development", "testing"}:
return get_development_settings()

View File

@@ -215,7 +215,7 @@ class DevelopmentSettings(BaseSettings):
@property
def debug_enabled(self) -> bool:
"""Check if debug mode is enabled."""
return True
return False
@property
def reload_enabled(self) -> bool:

View File

@@ -119,6 +119,11 @@ async def initialize_database(
result["tables_created"] = tables
logger.info("Created %s tables", len(tables))
# Migrate schema if needed (add missing columns to existing tables)
migrations = await migrate_schema_if_needed(engine)
if migrations:
logger.info("Applied %s schema migrations", len(migrations))
# Validate schema if requested
if validate_schema:
validation = await validate_database_schema(engine)
@@ -305,6 +310,66 @@ async def validate_database_schema(
}
# =============================================================================
# Schema Migration
# =============================================================================
async def migrate_schema_if_needed(
engine: Optional[AsyncEngine] = None
) -> List[str]:
"""Migrate database schema to current version if needed.
Handles adding missing columns to existing tables for backward
compatibility with older database schemas.
Args:
engine: Optional database engine (uses default if not provided)
Returns:
List of migration operations performed
"""
if engine is None:
engine = get_engine()
migrations_applied = []
try:
async with engine.connect() as conn:
# Get existing columns in system_settings table
existing_columns = await conn.run_sync(
lambda sync_conn: [
col["name"]
for col in inspect(sync_conn).get_columns("system_settings")
]
)
# Migration: Add legacy_key_cleanup_completed column if missing
if "legacy_key_cleanup_completed" not in existing_columns:
logger.info(
"Migrating system_settings table: "
"adding legacy_key_cleanup_completed column"
)
await conn.execute(
text("""
ALTER TABLE system_settings
ADD COLUMN legacy_key_cleanup_completed BOOLEAN
NOT NULL DEFAULT 0
""")
)
migrations_applied.append("added legacy_key_cleanup_completed")
logger.info(
"Migration complete: added legacy_key_cleanup_completed column"
)
except Exception as e:
logger.error("Schema migration failed: %s", e)
# Don't raise - migration failures shouldn't block startup
# The missing column will be handled gracefully by the application
return migrations_applied
# =============================================================================
# Schema Version Management
# =============================================================================

View File

@@ -15,7 +15,7 @@ from datetime import datetime, timezone
from enum import Enum
from typing import List, Optional
from sqlalchemy import Boolean, DateTime, ForeignKey, Integer, String, Text, func
from sqlalchemy import Boolean, DateTime, ForeignKey, Index, Integer, String, Text, func
from sqlalchemy.orm import Mapped, mapped_column, relationship, validates
from src.server.database.base import Base, TimestampMixin
@@ -83,6 +83,10 @@ class AnimeSeries(Base, TimestampMixin):
Boolean, nullable=False, default=False, server_default="0",
doc="Whether tvshow.nfo file exists for this series"
)
nfo_path: Mapped[Optional[str]] = mapped_column(
String(1000), nullable=True,
doc="Path to the tvshow.nfo metadata file"
)
nfo_created_at: Mapped[Optional[datetime]] = mapped_column(
DateTime(timezone=True), nullable=True,
doc="Timestamp when NFO was first created"
@@ -91,6 +95,7 @@ class AnimeSeries(Base, TimestampMixin):
DateTime(timezone=True), nullable=True,
doc="Timestamp when NFO was last updated"
)
# TMDB (The Movie Database) ID for series metadata
tmdb_id: Mapped[Optional[int]] = mapped_column(
Integer, nullable=True, index=True,
doc="TMDB (The Movie Database) ID for series metadata"
@@ -316,6 +321,7 @@ class DownloadQueueItem(Base, TimestampMixin):
id: Primary key
series_id: Foreign key to AnimeSeries
episode_id: Foreign key to Episode
status: Queue status (pending/downloading/completed/failed/permanently_failed)
error_message: Error description if failed
download_url: Provider download URL
file_destination: Target file path
@@ -347,6 +353,33 @@ class DownloadQueueItem(Base, TimestampMixin):
index=True
)
# Status column to track queue item state
# Allows distinguishing pending items from permanently failed ones
status: Mapped[str] = mapped_column(
String(50), nullable=False, default="pending",
doc="Queue item status: pending, downloading, completed, failed, permanently_failed"
)
# Retry count to track failed download attempts
# Used to determine when to move item to permanently_failed
retry_count: Mapped[int] = mapped_column(
Integer, nullable=False, default=0,
doc="Number of retry attempts for this download"
)
# Unique constraint to prevent duplicate pending queue items per episode
# An episode can only have one PENDING entry at a time
# The status column allows failed items to remain in DB while new
# pending items can be added (application-level dedup still required)
__table_args__ = (
Index(
"ix_download_queue_episode_status",
"episode_id",
"status",
unique=True,
),
)
# Error handling
error_message: Mapped[Optional[str]] = mapped_column(
Text, nullable=True,
@@ -580,6 +613,14 @@ class SystemSettings(Base, TimestampMixin):
Boolean, nullable=False, default=False, server_default="0",
doc="Whether the initial media scan has been completed"
)
migration_legacy_files_completed: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=False, server_default="0",
doc="Whether legacy key/data file migration has been completed"
)
legacy_key_cleanup_completed: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=False, server_default="0",
doc="Whether legacy key file cleanup has been completed"
)
last_scan_timestamp: Mapped[Optional[datetime]] = mapped_column(
DateTime(timezone=True), nullable=True,
doc="Timestamp of the last completed scan"

View File

@@ -169,6 +169,26 @@ class AnimeSeriesService:
)
return result.scalar_one_or_none()
@staticmethod
async def get_by_folder(db: AsyncSession, folder: str) -> Optional[AnimeSeries]:
"""Look up an anime series by its filesystem folder name (async).
Intended as primary lookup for ``SerieScanner`` when scanning
directories, replacing the legacy file-based lookups (key/data files).
Args:
db: Async database session.
folder: Filesystem folder name to match (e.g.
``"Rooster Fighter (2026)"``).
Returns:
``AnimeSeries`` instance or ``None`` if not found.
"""
result = await db.execute(
select(AnimeSeries).where(AnimeSeries.folder == folder)
)
return result.scalar_one_or_none()
@staticmethod
async def get_all(
db: AsyncSession,
@@ -541,6 +561,7 @@ class EpisodeService:
db: AsyncSession,
series_id: int,
season: Optional[int] = None,
only_missing: bool = False,
) -> List[Episode]:
"""Get episodes for a series.
@@ -548,6 +569,9 @@ class EpisodeService:
db: Database session
series_id: Foreign key to AnimeSeries
season: Optional season filter
only_missing: If True, only return episodes where
is_downloaded is False (i.e., missing episodes).
Default False returns all episodes.
Returns:
List of Episode instances
@@ -557,6 +581,9 @@ class EpisodeService:
if season is not None:
query = query.where(Episode.season == season)
if only_missing:
query = query.where(Episode.is_downloaded == False)
query = query.order_by(Episode.season, Episode.episode_number)
result = await db.execute(query)
return list(result.scalars().all())
@@ -622,11 +649,11 @@ class EpisodeService:
@staticmethod
async def delete(db: AsyncSession, episode_id: int) -> bool:
"""Delete episode.
Args:
db: Database session
episode_id: Episode primary key
Returns:
True if deleted, False if not found
"""
@@ -635,6 +662,33 @@ class EpisodeService:
)
return result.rowcount > 0
@staticmethod
async def delete_by_series(
db: AsyncSession,
series_id: int,
season: int,
episode_number: int,
) -> bool:
"""Delete episode by series ID, season, and episode number.
Args:
db: Database session
series_id: Foreign key to AnimeSeries
season: Season number
episode_number: Episode number within season
Returns:
True if deleted, False if not found
"""
result = await db.execute(
delete(Episode).where(
Episode.series_id == series_id,
Episode.season == season,
Episode.episode_number == episode_number,
)
)
return result.rowcount > 0
@staticmethod
async def delete_by_series_and_episode(
db: AsyncSession,
@@ -748,6 +802,8 @@ class DownloadQueueService:
episode_id: int,
download_url: Optional[str] = None,
file_destination: Optional[str] = None,
status: str = "pending",
retry_count: int = 0,
) -> DownloadQueueItem:
"""Add item to download queue.
@@ -757,6 +813,8 @@ class DownloadQueueService:
episode_id: Foreign key to Episode
download_url: Optional provider download URL
file_destination: Optional target file path
status: Queue item status (default: "pending")
retry_count: Number of retry attempts (default: 0)
Returns:
Created DownloadQueueItem instance
@@ -766,13 +824,15 @@ class DownloadQueueService:
episode_id=episode_id,
download_url=download_url,
file_destination=file_destination,
status=status,
retry_count=retry_count,
)
db.add(item)
await db.flush()
await db.refresh(item)
logger.info(
f"Added to download queue: episode_id={episode_id} "
f"for series_id={series_id}"
f"for series_id={series_id}, status={status}"
)
return item
@@ -799,21 +859,24 @@ class DownloadQueueService:
async def get_by_episode(
db: AsyncSession,
episode_id: int,
status_filter: Optional[str] = None,
) -> Optional[DownloadQueueItem]:
"""Get download queue item by episode ID.
Args:
db: Database session
episode_id: Foreign key to Episode
status_filter: Optional status to filter by (e.g., "pending")
Returns:
DownloadQueueItem instance or None if not found
"""
result = await db.execute(
select(DownloadQueueItem).where(
DownloadQueueItem.episode_id == episode_id
)
query = select(DownloadQueueItem).where(
DownloadQueueItem.episode_id == episode_id
)
if status_filter:
query = query.where(DownloadQueueItem.status == status_filter)
result = await db.execute(query)
return result.scalar_one_or_none()
@staticmethod
@@ -873,6 +936,95 @@ class DownloadQueueService:
logger.debug("Set error on download queue item %s", item_id)
return item
@staticmethod
async def set_status(
db: AsyncSession,
item_id: int,
status: str,
) -> Optional[DownloadQueueItem]:
"""Set status on download queue item.
Args:
db: Database session
item_id: Item primary key
status: New status value
Returns:
Updated DownloadQueueItem instance or None if not found
"""
item = await DownloadQueueService.get_by_id(db, item_id)
if not item:
return None
item.status = status
await db.flush()
await db.refresh(item)
logger.debug("Set status on download queue item %s to %s", item_id, status)
return item
@staticmethod
async def increment_retry_count(
db: AsyncSession,
item_id: int,
) -> Optional[DownloadQueueItem]:
"""Increment retry count on download queue item.
Args:
db: Database session
item_id: Item primary key
Returns:
Updated DownloadQueueItem instance or None if not found
"""
item = await DownloadQueueService.get_by_id(db, item_id)
if not item:
return None
item.retry_count += 1
await db.flush()
await db.refresh(item)
logger.debug(
"Incremented retry count on download queue item %s to %s",
item_id, item.retry_count
)
return item
@staticmethod
async def set_status_and_error(
db: AsyncSession,
item_id: int,
status: str,
error_message: Optional[str] = None,
) -> Optional[DownloadQueueItem]:
"""Set status and error message on download queue item atomically.
Args:
db: Database session
item_id: Item primary key
status: New status value
error_message: Optional error description
Returns:
Updated DownloadQueueItem instance or None if not found
"""
item = await DownloadQueueService.get_by_id(db, item_id)
if not item:
return None
item.status = status
if error_message is not None:
item.error_message = error_message
await db.flush()
await db.refresh(item)
logger.debug(
"Set status=%s on download queue item %s, error=%s",
status, item_id, error_message
)
return item
@staticmethod
async def delete(db: AsyncSession, item_id: int) -> bool:
"""Delete download queue item.

View File

@@ -125,6 +125,66 @@ class SystemSettingsService:
settings = await SystemSettingsService.get_or_create(db)
return settings.initial_media_scan_completed
@staticmethod
async def is_migration_legacy_files_completed(db: AsyncSession) -> bool:
"""Check if legacy key/data file migration has been completed.
Args:
db: Database session
Returns:
True if legacy migration is completed, False otherwise
"""
settings = await SystemSettingsService.get_or_create(db)
return settings.migration_legacy_files_completed
@staticmethod
async def mark_migration_legacy_files_completed(
db: AsyncSession,
timestamp: Optional[datetime] = None
) -> None:
"""Mark the legacy key/data file migration as completed.
Args:
db: Database session
timestamp: Optional timestamp to set, defaults to current time
"""
settings = await SystemSettingsService.get_or_create(db)
settings.migration_legacy_files_completed = True
settings.last_scan_timestamp = timestamp or datetime.now(timezone.utc)
await db.commit()
logger.info("Marked legacy files migration as completed")
@staticmethod
async def is_legacy_key_cleanup_completed(db: AsyncSession) -> bool:
"""Check if legacy key file cleanup has been completed.
Args:
db: Database session
Returns:
True if cleanup is completed, False otherwise
"""
settings = await SystemSettingsService.get_or_create(db)
return settings.legacy_key_cleanup_completed
@staticmethod
async def mark_legacy_key_cleanup_completed(
db: AsyncSession,
timestamp: Optional[datetime] = None
) -> None:
"""Mark the legacy key file cleanup as completed.
Args:
db: Database session
timestamp: Optional timestamp to set, defaults to current time
"""
settings = await SystemSettingsService.get_or_create(db)
settings.legacy_key_cleanup_completed = True
settings.last_scan_timestamp = timestamp or datetime.now(timezone.utc)
await db.commit()
logger.info("Marked legacy key file cleanup as completed")
@staticmethod
async def mark_initial_media_scan_completed(
db: AsyncSession,
@@ -154,6 +214,8 @@ class SystemSettingsService:
settings.initial_scan_completed = False
settings.initial_nfo_scan_completed = False
settings.initial_media_scan_completed = False
settings.migration_legacy_files_completed = False
settings.legacy_key_cleanup_completed = False
settings.last_scan_timestamp = None
await db.commit()
logger.info("Reset all scan completion flags")

View File

@@ -38,7 +38,6 @@ from src.server.controllers.page_controller import router as page_router
from src.server.middleware.auth import AuthMiddleware
from src.server.middleware.error_handler import register_exception_handlers
from src.server.middleware.setup_redirect import SetupRedirectMiddleware
from src.server.services.anime_service import sync_series_from_data_files
from src.server.services.progress_service import get_progress_service
from src.server.services.websocket_service import get_websocket_service
@@ -104,6 +103,107 @@ async def _check_incomplete_series_on_startup(background_loader) -> None:
logger.exception("Failed to check incomplete series on startup")
async def _run_startup_health_checks(logger) -> dict:
"""Run startup health checks for critical dependencies.
Checks:
- ffmpeg availability
- DNS resolution for aniworld.to and api.themoviedb.org
- anime_directory configuration and writability
Args:
logger: Logger instance for recording check results.
Returns:
dict: Health check results with status and details for each check.
"""
import asyncio
import shutil
import socket
from typing import Any, Dict
checks: Dict[str, Any] = {
"ffmpeg": {"status": "unknown", "message": None},
"dns_aniworld": {"status": "unknown", "message": None},
"dns_tmdb": {"status": "unknown", "message": None},
"anime_directory": {"status": "unknown", "message": None, "path": None},
}
# Check ffmpeg availability
try:
ffmpeg_path = shutil.which("ffmpeg")
if ffmpeg_path:
checks["ffmpeg"]["status"] = "ok"
checks["ffmpeg"]["message"] = f"Found at {ffmpeg_path}"
logger.debug("ffmpeg health check passed: %s", ffmpeg_path)
else:
checks["ffmpeg"]["status"] = "warning"
checks["ffmpeg"]["message"] = "ffmpeg not found in PATH"
logger.warning("ffmpeg health check failed: not in PATH")
except Exception as e:
checks["ffmpeg"]["status"] = "error"
checks["ffmpeg"]["message"] = str(e)
logger.warning("Could not check ffmpeg: %s", e)
# Check DNS resolution for aniworld.to
try:
socket.gethostbyname("aniworld.to")
checks["dns_aniworld"]["status"] = "ok"
checks["dns_aniworld"]["message"] = "Resolved successfully"
logger.debug("DNS health check passed for aniworld.to")
except socket.gaierror as e:
checks["dns_aniworld"]["status"] = "warning"
checks["dns_aniworld"]["message"] = f"DNS resolution failed: {e}"
logger.warning("DNS health check failed for aniworld.to: %s", e)
except Exception as e:
checks["dns_aniworld"]["status"] = "warning"
checks["dns_aniworld"]["message"] = f"Unexpected error: {e}"
logger.warning("Unexpected DNS error for aniworld.to: %s", e)
# Check DNS resolution for api.themoviedb.org
try:
socket.gethostbyname("api.themoviedb.org")
checks["dns_tmdb"]["status"] = "ok"
checks["dns_tmdb"]["message"] = "Resolved successfully"
logger.debug("DNS health check passed for api.themoviedb.org")
except socket.gaierror as e:
checks["dns_tmdb"]["status"] = "warning"
checks["dns_tmdb"]["message"] = f"DNS resolution failed: {e}"
logger.warning("DNS health check failed for api.themoviedb.org: %s", e)
except Exception as e:
checks["dns_tmdb"]["status"] = "warning"
checks["dns_tmdb"]["message"] = f"Unexpected error: {e}"
logger.warning("Unexpected DNS error for api.themoviedb.org: %s", e)
# Check anime_directory configuration and writability
from src.config.settings import settings
anime_dir = settings.anime_directory
if not anime_dir:
checks["anime_directory"]["status"] = "error"
checks["anime_directory"]["message"] = "anime_directory not configured"
checks["anime_directory"]["path"] = None
logger.error("anime_directory health check failed: not configured")
else:
import os
checks["anime_directory"]["path"] = anime_dir
if not os.path.isdir(anime_dir):
checks["anime_directory"]["status"] = "error"
checks["anime_directory"]["message"] = f"Directory does not exist: {anime_dir}"
logger.error("anime_directory health check failed: %s does not exist", anime_dir)
elif not os.access(anime_dir, os.W_OK):
checks["anime_directory"]["status"] = "error"
checks["anime_directory"]["message"] = f"Directory not writable: {anime_dir}"
logger.error("anime_directory health check failed: %s not writable", anime_dir)
else:
checks["anime_directory"]["status"] = "ok"
checks["anime_directory"]["message"] = f"Directory exists and is writable: {anime_dir}"
logger.debug("anime_directory health check passed: %s", anime_dir)
return checks
@asynccontextmanager
async def lifespan(_application: FastAPI):
"""Manage application lifespan (startup and shutdown).
@@ -114,6 +214,7 @@ async def lifespan(_application: FastAPI):
"""
# Setup logging first with INFO level
logger = setup_logging(log_level="INFO")
logger.info("Starting FastAPI application v%s", APP_VERSION)
# Track successful initialization steps
initialized = {
@@ -297,19 +398,6 @@ async def lifespan(_application: FastAPI):
except Exception as e:
logger.warning("Failed to start background loader service: %s", e)
# Initialize and start scheduler service
try:
from src.server.services.scheduler_service import (
get_scheduler_service,
)
scheduler_service = get_scheduler_service()
await scheduler_service.start()
initialized['scheduler'] = True
logger.info("Scheduler service started")
except Exception as e:
logger.warning("Failed to start scheduler service: %s", e)
# Continue - scheduler is optional
# Run media scan only on first run
await perform_media_scan_if_needed(background_loader)
else:
@@ -317,6 +405,22 @@ async def lifespan(_application: FastAPI):
"Download service initialization skipped - "
"anime directory not configured"
)
# Initialize and start scheduler service (independent of anime_directory)
# The scheduler loads its own config from config.json and the
# anime_directory may be configured there even if the env var is empty.
try:
logger.info("Initializing scheduler service...")
from src.server.services.scheduler.scheduler_service import (
get_scheduler_service,
)
scheduler_service = get_scheduler_service()
logger.info("Scheduler service instance obtained, starting...")
await scheduler_service.start()
initialized['scheduler'] = True
logger.info("Scheduler service started successfully")
except Exception as e:
logger.warning("Failed to start scheduler service: %s", e)
except (OSError, RuntimeError, ValueError) as e:
logger.warning("Failed to initialize services: %s", e)
# Continue startup - services can be initialized later
@@ -329,6 +433,27 @@ async def lifespan(_application: FastAPI):
logger.info(
"API documentation available at http://127.0.0.1:8000/api/docs"
)
# Check for ffmpeg availability and warn if missing
try:
import shutil as _shutil
if _shutil.which("ffmpeg") is None:
logger.warning(
"ffmpeg not found in PATH. HLS streams may fail to download. "
"Install ffmpeg to enable HLS support."
)
else:
logger.debug("ffmpeg found at: %s", _shutil.which("ffmpeg"))
except Exception as _exc:
logger.warning("Could not check for ffmpeg: %s", _exc)
# Run startup health checks and store results for /health endpoint
try:
startup_checks = await _run_startup_health_checks(logger)
app.state.startup_checks = startup_checks
except Exception as _exc:
logger.warning("Could not run startup health checks: %s", _exc)
app.state.startup_checks = {}
except Exception as e:
logger.error("Error during startup: %s", e, exc_info=True)
startup_error = e
@@ -373,7 +498,9 @@ async def lifespan(_application: FastAPI):
# 1. Stop scheduler service (only if initialized)
if initialized['scheduler']:
try:
from src.server.services.scheduler_service import get_scheduler_service
from src.server.services.scheduler.scheduler_service import (
get_scheduler_service,
)
scheduler_service = get_scheduler_service()
logger.info("Stopping scheduler service...")
await asyncio.wait_for(
@@ -418,8 +545,8 @@ async def lifespan(_application: FastAPI):
# 4. Shutdown download service and persist active downloads
try:
from src.server.services.download_service import ( # noqa: E501
_download_service_instance,
from src.server.services.download_service import (
_download_service_instance, # noqa: E501
)
if _download_service_instance is not None:
logger.info("Stopping download service...")
@@ -476,11 +603,13 @@ async def lifespan(_application: FastAPI):
raise startup_error
from src.server.utils.version import APP_VERSION
# Initialize FastAPI app with lifespan
app = FastAPI(
title="Aniworld Download Manager",
description="Modern web interface for Aniworld anime download management",
version="1.0.1",
version=APP_VERSION,
docs_url="/api/docs",
redoc_url="/api/redoc",
lifespan=lifespan

View File

@@ -83,6 +83,30 @@ class AnimeSeriesResponse(BaseModel):
return v
class AnimeMetadataUpdate(BaseModel):
"""Request model for updating anime metadata (key, tmdb_id, tvdb_id)."""
key: Optional[str] = Field(None, description="New series key (URL-safe, lowercase)")
tmdb_id: Optional[int] = Field(None, ge=1, description="TMDB ID (positive integer)")
tvdb_id: Optional[int] = Field(None, ge=1, description="TVDB ID (positive integer)")
@field_validator('key', mode='before')
@classmethod
def validate_key_format(cls, v: Optional[str]) -> Optional[str]:
"""Validate key is URL-safe lowercase with hyphens only."""
if v is None:
return v
v = v.strip().lower()
if not v:
raise ValueError("Key cannot be empty")
if not KEY_PATTERN.match(v):
raise ValueError(
"Key must contain only lowercase letters, numbers, and hyphens. "
"Cannot start or end with a hyphen."
)
return v
class SearchRequest(BaseModel):
"""Request payload for searching series."""

View File

@@ -44,6 +44,18 @@ class SchedulerConfig(BaseModel):
description="Run folder maintenance (NFO repair, folder renaming, "
"poster checks) during the scheduled run.",
)
# Legacy alias fields — read via Pydantic alias
auto_download: Optional[bool] = Field(default=None, alias="auto_download")
folder_scan: Optional[bool] = Field(default=None, alias="folder_scan")
def __init__(self, **data):
super().__init__(**data)
# Map legacy keys to primary fields only when primary key absent from data.
# "key in data" checks for explicit presence (even False/None), not just truthiness.
if self.auto_download is not None and "auto_download_after_rescan" not in data:
object.__setattr__(self, "auto_download_after_rescan", self.auto_download)
if self.folder_scan is not None and "folder_scan_enabled" not in data:
object.__setattr__(self, "folder_scan_enabled", self.folder_scan)
@field_validator("schedule_time")
@classmethod
@@ -69,6 +81,22 @@ class SchedulerConfig(BaseModel):
)
return v
def model_dump(self, **kwargs) -> Dict[str, object]:
"""Serialize, excluding legacy alias fields when they are None.
The alias fields (auto_download, folder_scan) must not be written to
config.json as null entries, otherwise a roundtrip load sees the key
present (哪怕 value is None) and skips the alias-to-primary mapping.
"""
data = super().model_dump(**kwargs)
# Drop None alias fields so they don't pollute config.json.
# They are still settable via the constructor for backward compatibility.
if data.get("auto_download") is None:
data.pop("auto_download", None)
if data.get("folder_scan") is None:
data.pop("folder_scan", None)
return data
class BackupConfig(BaseModel):
"""Configuration for automatic backups of application data."""
@@ -171,6 +199,12 @@ class AppConfig(BaseModel):
logging: LoggingConfig = Field(default_factory=LoggingConfig)
backup: BackupConfig = Field(default_factory=BackupConfig)
nfo: NFOConfig = Field(default_factory=NFOConfig)
scan_key_overrides: Dict[str, str] = Field(
default_factory=dict,
description="Map of folder names to provider keys for scan overrides. "
"Used when auto-generated keys from folder names are incorrect. "
"Format: {\"Folder Name\": \"actual-provider-key\"}"
)
other: Dict[str, object] = Field(
default_factory=dict, description="Arbitrary other settings"
)
@@ -209,6 +243,7 @@ class ConfigUpdate(BaseModel):
logging: Optional[LoggingConfig] = None
backup: Optional[BackupConfig] = None
nfo: Optional[NFOConfig] = None
scan_key_overrides: Optional[Dict[str, str]] = None
other: Optional[Dict[str, object]] = None
def apply_to(self, current: AppConfig) -> AppConfig:
@@ -225,6 +260,8 @@ class ConfigUpdate(BaseModel):
data["backup"] = self.backup.model_dump()
if self.nfo is not None:
data["nfo"] = self.nfo.model_dump()
if self.scan_key_overrides is not None:
data["scan_key_overrides"] = self.scan_key_overrides
if self.other is not None:
merged = dict(current.other or {})
merged.update(self.other)

View File

@@ -22,6 +22,7 @@ class DownloadStatus(str, Enum):
COMPLETED = "completed"
FAILED = "failed"
CANCELLED = "cancelled"
PERMANENTLY_FAILED = "permanently_failed"
class DownloadPriority(str, Enum):

View File

@@ -355,3 +355,29 @@ class NFOMissingResponse(BaseModel):
...,
description="List of series missing NFO"
)
class NfoDiagnosticsResponse(BaseModel):
"""Response for NFO diagnostics showing missing required tags."""
has_nfo: bool = Field(..., description="Whether tvshow.nfo exists")
nfo_path: Optional[str] = Field(None, description="Path to NFO file if exists")
missing_tags: List[str] = Field(
default_factory=list,
description="List of missing required tag names"
)
required_tags: List[str] = Field(
default_factory=list,
description="All required tag names for reference"
)
class NfoRepairResponse(BaseModel):
"""Response after NFO repair attempt."""
success: bool = Field(..., description="Whether repair succeeded")
message: str = Field(..., description="Human-readable result message")
repaired_tags: List[str] = Field(
default_factory=list,
description="Tags that were missing before repair"
)

View File

@@ -498,13 +498,19 @@ class AnimeService:
logger.info("No series found in SeriesApp")
return []
# Build NFO metadata map and filter data from database
nfo_map = {}
series_with_no_episodes = set()
# Build NFO metadata map, episode dict, and filter data from database.
# Using DB as authoritative source for episodeDict ensures that
# episodes marked is_downloaded=True are never shown as missing,
# even if the in-memory state is stale.
nfo_map: dict = {}
db_episode_dict_map: dict[str, dict[int, list[int]]] = {}
series_with_no_episodes: set = set()
async with get_db_session() as db:
# Get all series NFO metadata using service layer
db_series_list = await AnimeSeriesService.get_all(db)
# Single query: load all series with their episodes eagerly
db_series_list = await AnimeSeriesService.get_all(
db, with_episodes=True
)
for db_series in db_series_list:
nfo_created = (
@@ -522,7 +528,23 @@ class AnimeService:
"tmdb_id": db_series.tmdb_id,
"tvdb_id": db_series.tvdb_id,
"series_id": db_series.id,
"loading_status": db_series.loading_status,
"loading_error": db_series.loading_error,
}
# Build episodeDict from DB, skipping is_downloaded=True
# episodes so they are never shown as missing in the UI.
ep_dict: dict[int, list[int]] = {}
if db_series.episodes:
for ep in db_series.episodes:
if ep.is_downloaded:
continue
if ep.season not in ep_dict:
ep_dict[ep.season] = []
ep_dict[ep.season].append(ep.episode_number)
for s in ep_dict:
ep_dict[s].sort()
db_episode_dict_map[db_series.folder] = ep_dict
# If filter is "missing_episodes", get series with any missing episodes
if filter_type == "missing_episodes":
@@ -545,7 +567,12 @@ class AnimeService:
name = getattr(serie, "name", "")
site = getattr(serie, "site", "")
folder = getattr(serie, "folder", "")
episode_dict = getattr(serie, "episodeDict", {}) or {}
# Use DB-backed episodeDict (is_downloaded=True already filtered out)
# with in-memory episodeDict as fallback if the series isn't in DB yet.
episode_dict = db_episode_dict_map.get(
folder,
getattr(serie, "episodeDict", {}) or {}
)
# Apply filter if specified
if filter_type == "missing_episodes":
@@ -571,6 +598,8 @@ class AnimeService:
"tmdb_id": nfo_data.get("tmdb_id"),
"tvdb_id": nfo_data.get("tvdb_id"),
"series_id": nfo_data.get("series_id"),
"loading_status": nfo_data.get("loading_status"),
"loading_error": nfo_data.get("loading_error"),
}
result_list.append(series_dict)
@@ -815,18 +844,24 @@ class AnimeService:
- Adds new missing episodes that are not in the database
- Removes episodes from database that are no longer missing
(i.e., the file has been added to the filesystem)
- Preserves episodes marked as downloaded (is_downloaded=True)
so download history is not lost
"""
from src.server.database.service import AnimeSeriesService, EpisodeService
# Get existing episodes from database
# Get existing episodes from database (all episodes, including downloaded)
existing_episodes = await EpisodeService.get_by_series(db, existing.id)
# Build dict of existing episodes: {season: {ep_num: episode_id}}
# and track which ones are already downloaded
existing_dict: dict[int, dict[int, int]] = {}
downloaded_set: set[tuple[int, int]] = set()
for ep in existing_episodes:
if ep.season not in existing_dict:
existing_dict[ep.season] = {}
existing_dict[ep.season][ep.episode_number] = ep.id
if ep.is_downloaded:
downloaded_set.add((ep.season, ep.episode_number))
# Get new missing episodes from scan
new_dict = serie.episodeDict or {}
@@ -857,9 +892,22 @@ class AnimeService:
# Remove episodes from database that are no longer missing
# (i.e., the episode file now exists on the filesystem)
# BUT: preserve episodes that are already downloaded (is_downloaded=True)
# so we don't lose download history
for season, eps_dict in existing_dict.items():
for ep_num, episode_id in eps_dict.items():
if (season, ep_num) not in new_missing_set:
# Skip already-downloaded episodes — they should stay in DB
# with is_downloaded=True to preserve download history
if (season, ep_num) in downloaded_set:
logger.debug(
"Preserving downloaded episode in database: "
"%s S%02dE%02d",
serie.key,
season,
ep_num
)
continue
await EpisodeService.delete(db, episode_id)
logger.info(
"Removed episode from database (no longer missing): "
@@ -889,6 +937,10 @@ class AnimeService:
This method is called during initialization and after rescans
to ensure the in-memory series list is in sync with the database.
Only episodes where is_downloaded=False are loaded into the
in-memory episodeDict, so downloaded episodes are not shown
as missing.
"""
from src.core.entities.series import Serie
from src.server.database.connection import get_db_session
@@ -903,9 +955,14 @@ class AnimeService:
series_list = []
for anime_series in anime_series_list:
# Build episode_dict from episodes relationship
# Only include episodes that are NOT downloaded (is_downloaded=False)
# so the missing-episode list stays accurate
episode_dict: dict[int, list[int]] = {}
if anime_series.episodes:
for episode in anime_series.episodes:
# Skip downloaded episodes — they are not missing
if episode.is_downloaded:
continue
season = episode.season
if season not in episode_dict:
episode_dict[season] = []
@@ -919,7 +976,8 @@ class AnimeService:
name=anime_series.name,
site=anime_series.site,
folder=anime_series.folder,
episodeDict=episode_dict
episodeDict=episode_dict,
year=anime_series.year
)
series_list.append(serie)
@@ -962,23 +1020,39 @@ class AnimeService:
logger.warning("Series not found in database: %s", series_key)
return 0
# Get existing episodes from database
# Get existing episodes from database (all, including downloaded)
existing_episodes = await EpisodeService.get_by_series(db, series_db.id)
# Build dict of existing episodes: {season: {ep_num: episode_id}}
# and track which ones are already downloaded
existing_dict: dict[int, dict[int, int]] = {}
downloaded_set: set[tuple[int, int]] = set()
for ep in existing_episodes:
if ep.season not in existing_dict:
existing_dict[ep.season] = {}
existing_dict[ep.season][ep.episode_number] = ep.id
if ep.is_downloaded:
downloaded_set.add((ep.season, ep.episode_number))
# Get new missing episodes from in-memory serie
new_dict = serie.episodeDict or {}
# Add new missing episodes that are not in the database
# Skip episodes that are already downloaded (is_downloaded=True)
# so we don't re-add them as missing after they've been downloaded
for season, episode_numbers in new_dict.items():
existing_season_eps = existing_dict.get(season, {})
for ep_num in episode_numbers:
# Skip if already downloaded — don't re-add as missing
if (season, ep_num) in downloaded_set:
logger.debug(
"Skipping already-downloaded episode: "
"%s S%02dE%02d",
series_key,
season,
ep_num,
)
continue
if ep_num not in existing_season_eps:
await EpisodeService.create(
db=db,
@@ -1014,20 +1088,23 @@ class AnimeService:
if hasattr(self._app, 'list') and hasattr(self._app.list, 'keyDict'):
serie = self._app.list.keyDict.get(series_key)
if serie:
# Convert episode dict keys to strings for JSON
missing_episodes = {str(k): v for k, v in (serie.episodeDict or {}).items()}
total_missing = sum(len(eps) for eps in missing_episodes.values())
# Fetch NFO metadata from database
# Fetch NFO metadata and episodes from database.
# Using DB as the authoritative source for missing_episodes
# ensures that episodes marked is_downloaded=True are never
# broadcast as missing, even if in-memory state is stale.
has_nfo = False
nfo_created_at = None
nfo_updated_at = None
tmdb_id = None
tvdb_id = None
missing_episodes: dict[str, list] = {}
try:
from src.server.database.connection import get_db_session
from src.server.database.service import AnimeSeriesService
from src.server.database.service import (
AnimeSeriesService,
EpisodeService,
)
async with get_db_session() as db:
db_series = await AnimeSeriesService.get_by_key(db, series_key)
@@ -1043,12 +1120,31 @@ class AnimeService:
)
tmdb_id = db_series.tmdb_id
tvdb_id = db_series.tvdb_id
# Build missing_episodes from DB, skipping is_downloaded=True
db_episodes = await EpisodeService.get_by_series(
db, db_series.id, only_missing=True
)
for ep in db_episodes:
key_str = str(ep.season)
if key_str not in missing_episodes:
missing_episodes[key_str] = []
missing_episodes[key_str].append(ep.episode_number)
for s in missing_episodes:
missing_episodes[s].sort()
except Exception as e:
logger.warning(
"Could not fetch NFO data for %s: %s",
"Could not fetch series data for %s from DB: %s",
series_key,
str(e)
)
# Fallback to in-memory state
missing_episodes = {
str(k): v
for k, v in (serie.episodeDict or {}).items()
}
total_missing = sum(len(eps) for eps in missing_episodes.values())
series_data = {
"key": serie.key,
@@ -1462,19 +1558,17 @@ def get_anime_service(series_app: SeriesApp) -> AnimeService:
return AnimeService(series_app)
async def sync_series_from_data_files(
async def sync_legacy_series_to_db(
anime_directory: str,
log_instance=None # pylint: disable=unused-argument
) -> int:
"""
Sync series from data files to the database.
One-time legacy sync: import any series from 'data' files
not already in the database.
Scans the anime directory for data files and adds any new series
to the database. Existing series are skipped (no duplicates).
This function is typically called during application startup to ensure
series metadata stored in filesystem data files is available in the
database.
Deprecated: Series are now loaded directly from the database.
This function remains for backwards compatibility with legacy
file-based data during migration.
Args:
anime_directory: Path to the anime directory with data files
@@ -1486,6 +1580,11 @@ async def sync_series_from_data_files(
"""
# Always use structlog for structured logging with keyword arguments
log = structlog.get_logger(__name__)
log.warning(
"sync_legacy_series_to_db is deprecated. "
"Series are now loaded directly from database."
)
try:
from src.server.database.connection import get_db_session
@@ -1550,6 +1649,7 @@ async def sync_series_from_data_files(
name=serie.name,
site=serie.site,
folder=serie.folder,
year=serie.year if hasattr(serie, 'year') else None,
)
# Create Episode records for each episode in episodeDict

View File

@@ -144,7 +144,13 @@ class ConfigService:
# Save configuration with version
data = config.model_dump()
data["version"] = self.CONFIG_VERSION
# Re-serialize SchedulerConfig through its overridden model_dump so
# that None legacy alias fields are stripped before writing to disk.
# Pydantic converts nested models to plain dicts in model_dump() output,
# so we call the override explicitly on the scheduler field.
data["scheduler"] = config.scheduler.model_dump()
# Write to temporary file first for atomic operation
temp_path = self.config_path.with_suffix(".tmp")
try:

View File

@@ -14,6 +14,7 @@ import uuid
from collections import deque
from concurrent.futures import ThreadPoolExecutor
from datetime import datetime, timezone
from pathlib import Path
from typing import TYPE_CHECKING, Dict, List, Optional
import structlog
@@ -68,6 +69,7 @@ class DownloadService:
progress_service: Optional progress service for tracking
"""
self._anime_service = anime_service
self._directory = anime_service._directory
self._max_retries = max_retries
self._progress_service = progress_service or get_progress_service()
@@ -79,6 +81,9 @@ class DownloadService:
self._pending_queue: deque[DownloadItem] = deque()
# Helper dict for O(1) lookup of pending items by ID
self._pending_items_by_id: Dict[str, DownloadItem] = {}
# Helper dict for O(1) lookup of pending items by episode identity
# Key: (serie_id, season, episode), Value: item ID
self._pending_by_episode: Dict[tuple, str] = {}
self._active_download: Optional[DownloadItem] = None
self._completed_items: deque[DownloadItem] = deque(maxlen=100)
self._failed_items: deque[DownloadItem] = deque(maxlen=50)
@@ -165,6 +170,27 @@ class DownloadService:
logger.error("Failed to save item to database: %s", e)
return item
async def _set_status_in_database(
self,
item_id: str,
status: str,
) -> bool:
"""Set status on an item in the database.
Args:
item_id: Download item ID
status: New status value
Returns:
True if update succeeded
"""
try:
repository = self._get_repository()
return await repository.set_status(item_id, status)
except Exception as e:
logger.error("Failed to set status in database: %s", e)
return False
async def _set_error_in_database(
self,
item_id: str,
@@ -186,6 +212,25 @@ class DownloadService:
logger.error("Failed to set error in database: %s", e)
return False
async def _increment_retry_in_database(
self,
item_id: str,
) -> bool:
"""Increment retry count on an item in the database.
Args:
item_id: Download item ID
Returns:
True if update succeeded
"""
try:
repository = self._get_repository()
return await repository.increment_retry(item_id)
except Exception as e:
logger.error("Failed to increment retry in database: %s", e)
return False
async def _delete_from_database(self, item_id: str) -> bool:
"""Delete an item from the database.
@@ -207,30 +252,33 @@ class DownloadService:
series_key: str,
season: int,
episode: int,
serie_folder: Optional[str] = None,
) -> bool:
"""Remove a downloaded episode from the missing episodes list.
"""Mark a downloaded episode as downloaded instead of deleting it.
Called when a download completes successfully to update both:
1. The database (Episode record deleted)
1. The database (Episode record marked is_downloaded=True)
2. The in-memory Serie.episodeDict and series_list cache
This ensures the episode no longer appears as missing in both
the API responses and the UI immediately after download.
the API responses and the UI immediately after download,
while preserving the download history.
Args:
series_key: Unique provider key for the series
season: Season number
episode: Episode number within season
serie_folder: Series folder name (required for file_path)
Returns:
True if episode was removed, False otherwise
True if episode was updated, False otherwise
"""
try:
from src.server.database.connection import get_db_session
from src.server.database.service import EpisodeService
from src.server.database.service import AnimeSeriesService, EpisodeService
logger.info(
"Attempting to remove missing episode from DB: "
"Attempting to mark episode as downloaded in DB: "
"%s S%02dE%02d",
series_key,
season,
@@ -238,28 +286,63 @@ class DownloadService:
)
async with get_db_session() as db:
deleted = await EpisodeService.delete_by_series_and_episode(
# Get series by key to find series_id
series = await AnimeSeriesService.get_by_key(db, series_key)
if not series:
logger.warning(
"Series not found for key: %s", series_key
)
return False
# Get episode by series_id, season, episode_number
ep = await EpisodeService.get_by_episode(
db=db,
series_key=series_key,
series_id=series.id,
season=season,
episode_number=episode,
)
if deleted:
if not ep:
logger.warning(
"Episode not found in DB: %s S%02dE%02d",
series_key,
season,
episode,
)
return False
# Construct file_path if serie_folder provided
file_path = None
if serie_folder:
season_folder = f"Season {season}"
file_path = str(
Path(self._directory) / serie_folder / season_folder
)
# Mark episode as downloaded instead of deleting
updated = await EpisodeService.mark_downloaded(
db=db,
episode_id=ep.id,
file_path=file_path or "",
)
if updated:
logger.info(
"Successfully removed episode from DB missing list: "
"Marked episode as downloaded in DB: "
"%s S%02dE%02d, file_path=%s",
series_key,
season,
episode,
file_path,
)
else:
logger.warning(
"Failed to mark episode as downloaded: "
"%s S%02dE%02d",
series_key,
season,
episode,
)
else:
logger.warning(
"Episode not found in DB missing list "
"(may already be removed): %s S%02dE%02d",
series_key,
season,
episode,
)
return False
# Update in-memory Serie.episodeDict so list_missing is
# immediately consistent without a full DB reload
@@ -270,8 +353,8 @@ class DownloadService:
try:
self._anime_service._cached_list_missing.cache_clear()
logger.debug(
"Cleared list_missing cache after removing "
"%s S%02dE%02d",
"Cleared list_missing cache after marking "
"%s S%02dE%02d as downloaded",
series_key,
season,
episode,
@@ -279,10 +362,35 @@ class DownloadService:
except Exception:
pass
return deleted
# Broadcast real-time update to frontend so the series card
# immediately reflects the new downloaded state (no longer
# shows the episode as missing) without waiting for a full
# reload on DOWNLOAD_COMPLETED.
try:
await self._anime_service._broadcast_series_updated(
series_key
)
logger.debug(
"Broadcast series_updated after marking "
"%s S%02dE%02d as downloaded",
series_key,
season,
episode,
)
except Exception as broadcast_exc:
logger.warning(
"Failed to broadcast series update after marking "
"%s S%02dE%02d as downloaded: %s",
series_key,
season,
episode,
broadcast_exc,
)
return True
except Exception as e:
logger.error(
"Failed to remove episode from missing list: "
"Failed to mark episode as downloaded: "
"%s S%02dE%02d - %s",
series_key,
season,
@@ -358,6 +466,27 @@ class DownloadService:
"missing episodes remaining",
len(app.series_list),
)
# Update deprecated data file if it exists
# DB is authoritative; data file is optional backup
serie_folder = serie.folder
data_path = Path(self._directory) / serie_folder / "data"
if data_path.exists():
try:
import warnings
with warnings.catch_warnings():
warnings.simplefilter("ignore", DeprecationWarning)
serie.save_to_file(str(data_path))
logger.debug(
"Updated data file after download: %s",
data_path,
)
except Exception as e:
logger.warning(
"Failed to update data file %s: %s",
data_path,
e,
)
else:
logger.debug(
"Episode %d not in season %d for %s, "
@@ -409,7 +538,7 @@ class DownloadService:
def _add_to_pending_queue(
self, item: DownloadItem, front: bool = False
) -> None:
"""Add item to pending queue and update helper dict.
"""Add item to pending queue and update helper dicts.
Args:
item: Download item to add
@@ -420,9 +549,12 @@ class DownloadService:
else:
self._pending_queue.append(item)
self._pending_items_by_id[item.id] = item
# Track by episode identity for deduplication
ep_key = (item.serie_id, item.episode.season, item.episode.episode)
self._pending_by_episode[ep_key] = item.id
def _remove_from_pending_queue(self, item_or_id: str) -> Optional[DownloadItem]: # noqa: E501
"""Remove item from pending queue and update helper dict.
"""Remove item from pending queue and update helper dicts.
Args:
item_or_id: Item ID to remove
@@ -442,6 +574,10 @@ class DownloadService:
try:
self._pending_queue.remove(item)
del self._pending_items_by_id[item_id]
# Clean up episode tracking
ep_key = (item.serie_id, item.episode.season, item.episode.episode)
if self._pending_by_episode.get(ep_key) == item_id:
del self._pending_by_episode[ep_key]
return item
except (ValueError, KeyError):
return None
@@ -481,10 +617,35 @@ class DownloadService:
# Initialize queue progress tracking if not already done
await self._init_queue_progress()
# Filter out episodes already in pending queue
episodes_to_add = []
skipped_count = 0
seen_in_batch: set = set() # Track duplicates within this batch
for ep in episodes:
ep_key = (serie_id, ep.season, ep.episode)
if ep_key in self._pending_by_episode or ep_key in seen_in_batch:
logger.debug(
"Skipping duplicate episode in queue",
serie_key=serie_id,
season=ep.season,
episode=ep.episode,
)
skipped_count += 1
continue
seen_in_batch.add(ep_key)
episodes_to_add.append(ep)
if skipped_count > 0:
logger.info(
"Skipped %d duplicate episodes in queue",
skipped_count,
serie_key=serie_id,
)
created_ids = []
try:
for episode in episodes:
for episode in episodes_to_add:
item = DownloadItem(
id=self._generate_item_id(),
serie_id=serie_id,
@@ -976,17 +1137,16 @@ class DownloadService:
if item.retry_count >= self._max_retries:
continue
# Move back to pending
# Move back to pending (retry_count will be incremented
# by _process_download when the item fails again)
self._failed_items.remove(item)
item.status = DownloadStatus.PENDING
item.retry_count += 1
item.error = None
item.progress = None
item.retry_count += 1
self._add_to_pending_queue(item)
retried_ids.append(item.id)
# Status is now managed in-memory only
logger.info(
"Retrying failed item: item_id=%s, retry_count=%d",
item.id,
@@ -994,18 +1154,23 @@ class DownloadService:
)
if retried_ids:
# Notify via progress service
queue_status = await self.get_queue_status()
await self._progress_service.update_progress(
progress_id="download_queue",
message=f"Retried {len(retried_ids)} failed items",
metadata={
"action": "items_retried",
"retried_ids": retried_ids,
"queue_status": queue_status.model_dump(mode="json"),
},
force_broadcast=True,
)
# Notify via progress service if available
try:
queue_status = await self.get_queue_status()
await self._progress_service.update_progress(
progress_id="download_queue",
message=f"Retried {len(retried_ids)} failed items",
metadata={
"action": "items_retried",
"retried_ids": retried_ids,
"queue_status": queue_status.model_dump(mode="json"),
},
force_broadcast=True,
)
except Exception as e:
logger.warning(
"Failed to broadcast retry progress: %s", e
)
return retried_ids
@@ -1084,12 +1249,13 @@ class DownloadService:
# Delete completed item from download queue database
await self._delete_from_database(item.id)
# Remove episode from missing episodes list
# Mark episode as downloaded in missing episodes list
# (both database and in-memory)
removed = await self._remove_episode_from_missing_list(
series_key=item.serie_id,
season=item.episode.season,
episode=item.episode.episode,
serie_folder=item.serie_folder,
)
logger.info(
@@ -1144,17 +1310,35 @@ class DownloadService:
item.status = DownloadStatus.FAILED
item.completed_at = datetime.now(timezone.utc)
item.error = str(e)
# Increment retry count in memory and database
item.retry_count += 1
await self._increment_retry_in_database(item.id)
self._failed_items.append(item)
# Set error in database
await self._set_error_in_database(item.id, str(e))
logger.error(
"Download failed: item_id=%s, error=%s, retry_count=%d",
item.id,
str(e),
item.retry_count,
)
# Check if max retries exceeded - move to dead-letter
if item.retry_count >= self._max_retries:
await self._set_status_in_database(
item.id, DownloadStatus.PERMANENTLY_FAILED.value
)
logger.error(
"Download permanently failed after max retries: "
"item_id=%s, error=%s, retry_count=%d",
item.id,
str(e),
item.retry_count,
)
else:
logger.error(
"Download failed: item_id=%s, error=%s, retry_count=%d",
item.id,
str(e),
item.retry_count,
)
# Note: Failure is already broadcast by AnimeService
# via ProgressService when SeriesApp fires failed event

View File

@@ -1,331 +0,0 @@
"""Folder rename service for validating and renaming series folders.
After NFO repair, this service iterates over every subfolder in
``settings.anime_directory`` that contains a ``tvshow.nfo``. For each
folder it parses the NFO to extract ``<title>`` and ``<year>``, computes
the expected folder name ``f"{title} ({year})"``, sanitises it for
filesystem safety, and renames the folder if the current name differs.
Database records (``AnimeSeries.folder``, ``Episode.file_path``,
``DownloadQueueItem.file_destination``) are updated atomically to
reflect the new paths.
"""
from __future__ import annotations
import logging
from pathlib import Path
from typing import Dict, List, Optional, Tuple
from lxml import etree
from src.config.settings import settings
from src.server.database.connection import get_db_session
from src.server.database.service import (
AnimeSeriesService,
DownloadQueueService,
EpisodeService,
)
from src.server.utils.dependencies import get_download_service
from src.server.utils.filesystem import sanitize_folder_name
logger = logging.getLogger(__name__)
# Characters that are invalid in filesystem paths across platforms
INVALID_PATH_CHARS = '<>:"/\\|?*\x00'
def _parse_nfo_title_and_year(nfo_path: Path) -> Tuple[Optional[str], Optional[str]]:
"""Parse a tvshow.nfo and return (title, year) text values.
Args:
nfo_path: Absolute path to the ``tvshow.nfo`` file.
Returns:
Tuple of (title, year) where either may be ``None`` if missing
or empty.
"""
try:
tree = etree.parse(str(nfo_path))
root = tree.getroot()
title_elem = root.find("./title")
year_elem = root.find("./year")
title = title_elem.text.strip() if title_elem is not None and title_elem.text and title_elem.text.strip() else None
year = year_elem.text.strip() if year_elem is not None and year_elem.text and year_elem.text.strip() else None
return title, year
except etree.XMLSyntaxError as exc:
logger.warning("Malformed XML in %s: %s", nfo_path, exc)
return None, None
except Exception as exc: # pylint: disable=broad-except
logger.warning("Unexpected error parsing %s: %s", nfo_path, exc)
return None, None
def _compute_expected_folder_name(title: str, year: str) -> str:
"""Compute the expected folder name from title and year.
Args:
title: Series title from NFO.
year: Release year from NFO.
Returns:
Sanitised folder name in the format ``"{title} ({year})"``.
"""
raw_name = f"{title} ({year})"
return sanitize_folder_name(raw_name)
def _is_series_being_downloaded(series_folder: str) -> bool:
"""Check whether the given series has an active or pending download.
Args:
series_folder: The series folder name (as stored in the DB).
Returns:
``True`` if the series appears in the active download or the
pending queue.
"""
try:
download_service = get_download_service()
active = download_service._active_download # pylint: disable=protected-access
if active and active.serie_folder == series_folder:
return True
for item in download_service._pending_queue: # pylint: disable=protected-access
if item.serie_folder == series_folder:
return True
return False
except Exception as exc: # pylint: disable=broad-except
logger.warning(
"Could not check download status for %s: %s", series_folder, exc
)
# Safer to skip renaming if we can't verify download status.
return True
async def _update_database_paths(
old_folder: str,
new_folder: str,
anime_dir: Path,
) -> None:
"""Update all database records that reference the old folder path.
Updates:
- ``AnimeSeries.folder`` → ``new_folder``
- ``Episode.file_path`` → adjusted to new folder
- ``DownloadQueueItem.file_destination`` → adjusted to new folder
Args:
old_folder: Previous folder name.
new_folder: New folder name.
anime_dir: Root anime directory path.
"""
old_series_path = anime_dir / old_folder
new_series_path = anime_dir / new_folder
async with get_db_session() as db:
# 1. Update AnimeSeries.folder
series = await AnimeSeriesService.get_by_key(db, old_folder)
if series is None:
# Fallback: try to find by folder name
all_series = await AnimeSeriesService.get_all(db)
for s in all_series:
if s.folder == old_folder:
series = s
break
if series is None:
logger.warning(
"No database record found for folder '%s', skipping DB update",
old_folder,
)
return
await AnimeSeriesService.update(db, series.id, folder=new_folder)
logger.info(
"Updated AnimeSeries.folder: %s%s (id=%s)",
old_folder,
new_folder,
series.id,
)
# 2. Update Episode.file_path for all episodes of this series
episodes = await EpisodeService.get_by_series(db, series.id)
for episode in episodes:
if episode.file_path:
old_file_path = Path(episode.file_path)
# Only update if the path is under the old series folder
try:
old_file_path.relative_to(old_series_path)
new_file_path = new_series_path / old_file_path.relative_to(
old_series_path
)
episode.file_path = str(new_file_path)
logger.debug(
"Updated Episode.file_path: %s%s",
old_file_path,
new_file_path,
)
except ValueError:
# Path is not under old_series_path, skip
pass
await db.flush()
# 3. Update DownloadQueueItem.file_destination for pending items
queue_items = await DownloadQueueService.get_all(db, with_series=True)
for item in queue_items:
if item.series_id == series.id and item.file_destination:
old_dest = Path(item.file_destination)
try:
old_dest.relative_to(old_series_path)
new_dest = new_series_path / old_dest.relative_to(
old_series_path
)
item.file_destination = str(new_dest)
logger.debug(
"Updated DownloadQueueItem.file_destination: %s%s",
old_dest,
new_dest,
)
except ValueError:
pass
await db.flush()
logger.info(
"Database paths updated for series '%s''%s'",
old_folder,
new_folder,
)
async def validate_and_rename_series_folders() -> Dict[str, int]:
"""Validate and rename series folders to match NFO metadata.
Iterates over every subfolder in ``settings.anime_directory`` that
contains a ``tvshow.nfo``. For each folder:
1. Parse the NFO to extract ``<title>`` and ``<year>``.
2. Compute the expected folder name: ``f"{title} ({year})"``.
3. Sanitise the expected name for filesystem safety.
4. Compare with the current folder name.
5. If different, rename the folder and update the database.
Skips folders where title or year is missing/empty. Logs every
rename action.
Returns:
Dictionary with counts:
- ``"scanned"``: total folders scanned
- ``"renamed"``: folders renamed
- ``"skipped"``: folders skipped (missing title/year)
- ``"errors"``: folders that caused an error
"""
if not settings.anime_directory:
logger.warning("Folder rename skipped — anime directory not configured")
return {"scanned": 0, "renamed": 0, "skipped": 0, "errors": 0}
anime_dir = Path(settings.anime_directory)
if not anime_dir.is_dir():
logger.warning(
"Folder rename skipped — anime directory not found: %s", anime_dir
)
return {"scanned": 0, "renamed": 0, "skipped": 0, "errors": 0}
stats = {"scanned": 0, "renamed": 0, "skipped": 0, "errors": 0}
for series_dir in sorted(anime_dir.iterdir()):
if not series_dir.is_dir():
continue
nfo_path = series_dir / "tvshow.nfo"
if not nfo_path.exists():
continue
stats["scanned"] += 1
title, year = _parse_nfo_title_and_year(nfo_path)
if not title or not year:
logger.info(
"Skipping rename for '%s' — missing title or year in NFO",
series_dir.name,
)
stats["skipped"] += 1
continue
expected_name = _compute_expected_folder_name(title, year)
current_name = series_dir.name
if expected_name == current_name:
logger.debug(
"Folder name already correct: '%s'", current_name
)
continue
# Check for active downloads
if _is_series_being_downloaded(current_name):
logger.info(
"Skipping rename for '%s' — series has active or pending downloads",
current_name,
)
stats["skipped"] += 1
continue
expected_path = anime_dir / expected_name
# Check for duplicate target
if expected_path.exists():
logger.warning(
"Cannot rename '%s''%s' — target already exists",
current_name,
expected_name,
)
stats["errors"] += 1
continue
# Check path length limits
if len(str(expected_path)) > 4096:
logger.warning(
"Cannot rename '%s''%s' — path exceeds OS limit",
current_name,
expected_name,
)
stats["errors"] += 1
continue
try:
series_dir.rename(expected_path)
logger.info(
"Renamed folder: '%s''%s'", current_name, expected_name
)
stats["renamed"] += 1
# Update database records
await _update_database_paths(current_name, expected_name, anime_dir)
except PermissionError as exc:
logger.error(
"Permission denied renaming '%s''%s': %s",
current_name,
expected_name,
exc,
)
stats["errors"] += 1
except OSError as exc:
logger.error(
"OS error renaming '%s''%s': %s",
current_name,
expected_name,
exc,
)
stats["errors"] += 1
logger.info(
"Folder rename scan complete: scanned=%d, renamed=%d, skipped=%d, errors=%d",
stats["scanned"],
stats["renamed"],
stats["skipped"],
stats["errors"],
)
return stats

View File

@@ -1,12 +1,14 @@
"""Centralized initialization service for application startup and setup."""
import asyncio
import os
from pathlib import Path
from typing import Callable, Optional
import structlog
from src.config.settings import settings
from src.server.services.anime_service import sync_series_from_data_files
from src.server.services.anime_service import sync_legacy_series_to_db
from src.server.services.legacy_file_migration import migrate_series_from_files_to_db
logger = structlog.get_logger(__name__)
@@ -99,6 +101,151 @@ async def _mark_initial_scan_completed() -> None:
)
async def _check_legacy_migration_status() -> bool:
"""Check if legacy key/data file migration has been completed.
Returns:
bool: True if migration was completed, False otherwise
"""
return await _check_scan_status(
check_method=lambda svc, db: svc.is_migration_legacy_files_completed(db),
scan_type="legacy_migration",
log_completed_msg="Legacy file migration already completed, skipping",
log_not_completed_msg="Legacy file migration not yet run, will check for files"
)
async def _mark_legacy_migration_completed() -> None:
"""Mark the legacy file migration as completed in system settings."""
await _mark_scan_completed(
mark_method=lambda svc, db: svc.mark_migration_legacy_files_completed(db),
scan_type="legacy_migration"
)
async def _check_legacy_key_cleanup_status() -> bool:
"""Check if legacy key file cleanup has been completed.
Returns:
bool: True if cleanup was completed, False otherwise
"""
return await _check_scan_status(
check_method=lambda svc, db: svc.is_legacy_key_cleanup_completed(db),
scan_type="legacy_key_cleanup",
log_completed_msg="Legacy key file cleanup already completed, skipping",
log_not_completed_msg="Legacy key file cleanup not yet run, will clean up key files"
)
async def _mark_legacy_key_cleanup_completed() -> None:
"""Mark the legacy key file cleanup as completed in system settings."""
await _mark_scan_completed(
mark_method=lambda svc, db: svc.mark_legacy_key_cleanup_completed(db),
scan_type="legacy_key_cleanup"
)
async def _migrate_legacy_files() -> int:
"""Migrate series from legacy key/data files to database.
Returns:
int: Number of series migrated
"""
from src.server.database.connection import get_db_session
logger.info("Checking for legacy key/data files to migrate...")
try:
async with get_db_session() as db:
migrated_count = await migrate_series_from_files_to_db(
settings.anime_directory,
db
)
if migrated_count > 0:
logger.info("Migrated %d series from legacy files", migrated_count)
else:
logger.info("No series found in legacy files to migrate")
return migrated_count
except Exception as e:
logger.warning("Failed to migrate legacy files: %s", e)
return 0
async def _cleanup_legacy_key_files() -> int:
"""Remove legacy key files from folders that already have DB entries.
This is a one-time cleanup task that runs at startup after legacy migration.
It removes deprecated 'key' files that cause duplicate key errors when
folders are renamed, since the DB is now the source of truth.
Returns:
int: Number of key files deleted
"""
from src.server.database.connection import get_db_session
from src.server.database.service import AnimeSeriesService
logger.info("Checking for legacy key files to clean up...")
if not settings.anime_directory or not os.path.isdir(settings.anime_directory):
logger.warning(
"Anime directory not configured or does not exist, skipping legacy key cleanup"
)
return 0
deleted_count = 0
scanned_count = 0
try:
async with get_db_session() as db:
# Get all series from DB to know which folders should have key files removed
all_series = await AnimeSeriesService.get_all(db)
# Build a set of known folder names from DB
db_folders: set[str] = {series.folder for series in all_series if series.folder}
for folder_name in db_folders:
folder_path = settings.anime_directory / folder_name
key_file = folder_path / "key"
if not key_file.exists():
continue
scanned_count += 1
try:
key_file.unlink()
deleted_count += 1
logger.info(
"Removed legacy key file",
folder=folder_name,
key_file=str(key_file)
)
except OSError as exc:
logger.warning(
"Could not remove legacy key file",
folder=folder_name,
key_file=str(key_file),
error=str(exc)
)
except Exception as e:
logger.error(
"Legacy key file cleanup failed",
error=str(e),
exc_info=True
)
return deleted_count
logger.info(
"Legacy key file cleanup complete",
scanned=scanned_count,
deleted=deleted_count
)
return deleted_count
async def _sync_anime_folders(progress_service=None) -> int:
"""Scan anime folders and sync series to database.
@@ -118,7 +265,7 @@ async def _sync_anime_folders(progress_service=None) -> int:
metadata={"step_id": "series_sync"}
)
sync_count = await sync_series_from_data_files(settings.anime_directory)
sync_count = await sync_legacy_series_to_db(settings.anime_directory)
logger.info("Data file sync complete. Added %d series.", sync_count)
if progress_service:
@@ -181,18 +328,19 @@ async def _validate_anime_directory(progress_service=None) -> bool:
async def perform_initial_setup(progress_service=None):
"""Perform initial setup including series sync and scan completion marking.
This function is called both during application lifespan startup
and when the setup endpoint is completed. It ensures that:
1. Series are synced from data files to database
2. Initial scan is marked as completed
3. Series are loaded into memory
4. NFO scan is performed if configured
5. Media scan is performed
1. Legacy key/data files are migrated to database (one-time)
2. Series are synced from data files to database
3. Initial scan is marked as completed
4. Series are loaded into memory
5. NFO scan is performed if configured
6. Media scan is performed
Args:
progress_service: Optional ProgressService for emitting updates
Returns:
bool: True if initialization was performed, False if skipped
"""
@@ -225,17 +373,30 @@ async def perform_initial_setup(progress_service=None):
# Perform the actual initialization
try:
# First, run legacy file migration if needed (independent of initial scan)
is_legacy_migration_done = await _check_legacy_migration_status()
if not is_legacy_migration_done:
await _migrate_legacy_files()
await _mark_legacy_migration_completed()
# Sync series from anime folders to database
await _sync_anime_folders(progress_service)
# Clean up legacy key files from folders that now have DB entries
# This runs after migration/sync to ensure DB entries exist before deletion
is_key_cleanup_done = await _check_legacy_key_cleanup_status()
if not is_key_cleanup_done:
await _cleanup_legacy_key_files()
await _mark_legacy_key_cleanup_completed()
# Mark the initial scan as completed
await _mark_initial_scan_completed()
# Load series into memory from database
await _load_series_into_memory(progress_service)
return True
except (OSError, RuntimeError, ValueError) as e:
logger.warning("Failed to perform initial setup: %s", e)
return False
@@ -383,10 +544,9 @@ async def _check_media_scan_status() -> bool:
Returns:
bool: True if media scan was completed, False otherwise
"""
return await _check_scan_status(
check_method=lambda svc, db: svc.is_initial_media_scan_completed(db),
scan_type="media"
)
# DISABLED: Always return True to skip startup scan
# To re-enable, change to: return await _check_scan_status(...)
return True
async def _mark_media_scan_completed() -> None:

View File

@@ -0,0 +1,233 @@
"""One-time migration service for legacy key and data files.
This module provides functionality to migrate series data from legacy
file-based storage (key/data files) to the database. The migration is
designed to be idempotent and run only once per environment.
"""
from __future__ import annotations
import json
import os
from pathlib import Path
from typing import Optional
import structlog
from sqlalchemy.ext.asyncio import AsyncSession
logger = structlog.get_logger(__name__)
async def migrate_series_from_files_to_db(
anime_dir: str,
db: AsyncSession,
) -> int:
"""Migrate series from legacy key/data files to database.
Scans for folders containing legacy 'key' or 'data' files and imports
any series not already in the database. The DB version wins if a series
exists in both places.
Args:
anime_dir: Path to the anime directory
db: Database session
Returns:
Number of series imported
"""
from src.server.database.service import AnimeSeriesService, EpisodeService
if not anime_dir or not os.path.isdir(anime_dir):
logger.warning(
"Anime directory does not exist, skipping legacy migration",
anime_dir=anime_dir
)
return 0
migrated_count = 0
scanned_count = 0
try:
for folder_name in os.listdir(anime_dir):
folder_path = os.path.join(anime_dir, folder_name)
if not os.path.isdir(folder_path):
continue
scanned_count += 1
# Check for 'key' file (single line with series key)
key_file = os.path.join(folder_path, "key")
# Check for 'data' file (JSON with series metadata)
data_file = os.path.join(folder_path, "data")
series_data: Optional[dict] = None
# Try to load from 'data' file first (more complete)
if os.path.isfile(data_file):
series_data = _load_data_file(data_file)
elif os.path.isfile(key_file):
# Fall back to 'key' file - just the key, need to infer other data
series_data = _load_key_file(key_file, folder_name)
if series_data is None:
continue
key = series_data.get("key")
if not key:
logger.warning(
"Skipping folder with no valid key",
folder=folder_name
)
continue
# Check if already in DB
existing = await AnimeSeriesService.get_by_key(db, key)
if existing:
logger.debug(
"Series already in database, skipping",
key=key,
folder=folder_name
)
continue
# Create the series in DB
try:
name = series_data.get("name") or folder_name
site = series_data.get("site", "https://aniworld.to")
folder = series_data.get("folder", folder_name)
year = series_data.get("year")
anime_series = await AnimeSeriesService.create(
db=db,
key=key,
name=name,
site=site,
folder=folder,
year=year,
)
# Create episodes if present
episode_dict = series_data.get("episodeDict", {})
if episode_dict:
for season, episode_numbers in episode_dict.items():
for episode_number in episode_numbers:
await EpisodeService.create(
db=db,
series_id=anime_series.id,
season=season,
episode_number=episode_number,
)
migrated_count += 1
logger.info(
"Migrated series from legacy file",
key=key,
name=name,
folder=folder_name
)
except Exception as e:
logger.warning(
"Failed to migrate series from legacy file",
key=key,
folder=folder_name,
error=str(e)
)
except Exception as e:
logger.error(
"Legacy migration failed",
anime_dir=anime_dir,
error=str(e),
exc_info=True
)
logger.info(
"Legacy file migration complete",
scanned_folders=scanned_count,
migrated=migrated_count
)
return migrated_count
def _load_data_file(data_file_path: str) -> Optional[dict]:
"""Load and parse a legacy 'data' file (JSON).
Args:
data_file_path: Path to the data file
Returns:
Parsed data dict or None if parsing fails
"""
try:
with open(data_file_path, "r", encoding="utf-8") as f:
data = json.load(f)
if not isinstance(data, dict):
logger.warning(
"Data file is not a dictionary",
file=data_file_path
)
return None
# Ensure episodeDict has int keys
if "episodeDict" in data and isinstance(data["episodeDict"], dict):
data["episodeDict"] = {
int(k): v for k, v in data["episodeDict"].items()
}
return data
except json.JSONDecodeError as e:
logger.warning(
"Failed to parse legacy data file (JSON error)",
file=data_file_path,
error=str(e)
)
return None
except Exception as e:
logger.warning(
"Failed to read legacy data file",
file=data_file_path,
error=str(e)
)
return None
def _load_key_file(key_file_path: str, folder_name: str) -> Optional[dict]:
"""Load a legacy 'key' file (single line with series key).
Args:
key_file_path: Path to the key file
folder_name: Folder name to use as fallback name
Returns:
Data dict with key and inferred fields, or None if loading fails
"""
try:
with open(key_file_path, "r", encoding="utf-8") as f:
key = f.read().strip()
if not key:
logger.warning(
"Key file is empty",
file=key_file_path
)
return None
# Infer basic data from key file
return {
"key": key,
"name": folder_name,
"site": "https://aniworld.to",
"folder": folder_name,
"episodeDict": {},
}
except Exception as e:
logger.warning(
"Failed to read legacy key file",
file=key_file_path,
error=str(e)
)
return None

View File

@@ -83,15 +83,12 @@ class QueueRepository:
) -> DownloadItem:
"""Convert database model to DownloadItem.
Note: Since the database model is simplified, status, priority,
progress, and retry_count default to initial values.
Args:
db_item: SQLAlchemy download queue item
item_id: Optional override for item ID
Returns:
Pydantic download item with default status/priority
Pydantic download item with status/retry_count from database
"""
# Get episode info from the related Episode object
episode = db_item.episode
@@ -109,14 +106,14 @@ class QueueRepository:
serie_folder=series.folder if series else "",
serie_name=series.name if series else "",
episode=episode_identifier,
status=DownloadStatus.PENDING, # Default - managed in-memory
priority=DownloadPriority.NORMAL, # Default - managed in-memory
status=DownloadStatus(db_item.status), # From database
priority=DownloadPriority.NORMAL, # Managed in-memory
added_at=db_item.created_at or datetime.now(timezone.utc),
started_at=db_item.started_at,
completed_at=db_item.completed_at,
progress=None, # Managed in-memory
error=db_item.error_message,
retry_count=0, # Managed in-memory
retry_count=db_item.retry_count, # From database
source_url=db_item.download_url,
)
@@ -350,6 +347,110 @@ class QueueRepository:
finally:
if manage_session:
await session.close()
async def set_status(
self,
item_id: str,
status: str,
db: Optional[AsyncSession] = None,
) -> bool:
"""Set status on a download item.
Args:
item_id: Download item ID
status: New status value
db: Optional existing database session
Returns:
True if update succeeded, False if item not found
Raises:
QueueRepositoryError: If update fails
"""
session = db or self._db_session_factory()
manage_session = db is None
try:
result = await DownloadQueueService.set_status(
session,
int(item_id),
status,
)
if manage_session:
await session.commit()
success = result is not None
if success:
logger.debug(
"Set status on queue item: item_id=%s, status=%s",
item_id,
status,
)
return success
except ValueError:
return False
except Exception as e:
if manage_session:
await session.rollback()
logger.error("Failed to set status: %s", e)
raise QueueRepositoryError(f"Failed to set status: {e}") from e
finally:
if manage_session:
await session.close()
async def increment_retry(
self,
item_id: str,
db: Optional[AsyncSession] = None,
) -> bool:
"""Increment retry count on a download item.
Args:
item_id: Download item ID
db: Optional existing database session
Returns:
True if update succeeded, False if item not found
Raises:
QueueRepositoryError: If update fails
"""
session = db or self._db_session_factory()
manage_session = db is None
try:
result = await DownloadQueueService.increment_retry_count(
session,
int(item_id),
)
if manage_session:
await session.commit()
success = result is not None
if success:
logger.debug(
"Incremented retry count on queue item: item_id=%s",
item_id,
)
return success
except ValueError:
return False
except Exception as e:
if manage_session:
await session.rollback()
logger.error("Failed to increment retry: %s", e)
raise QueueRepositoryError(f"Failed to increment retry: {e}") from e
finally:
if manage_session:
await session.close()
async def delete_item(
self,

View File

@@ -0,0 +1,291 @@
"""Rescan service — orchestrates library rescans.
This service handles the actual scan/rescan logic:
- Library rescan via anime_service
- Auto-download of missing episodes (if enabled)
- Folder maintenance scan (if enabled)
- Orphaned folder key resolution
SchedulerService only calls RescanService.execute() — it does not
know about the internal steps.
"""
from __future__ import annotations
import logging
from datetime import datetime, timedelta, timezone
from typing import List, Optional
from src.server.models.config import SchedulerConfig
logger = logging.getLogger(__name__)
_AUTO_DOWNLOAD_COOLDOWN_SECONDS = 300 # 5 minutes
class RescanService:
"""Orchestrates all rescan-related operations.
Encapsulates the full post-rescan workflow so SchedulerService
only needs to call a single execute() method.
"""
def __init__(self, config: Optional[SchedulerConfig] = None) -> None:
"""Initialize the rescan service.
Args:
config: Optional scheduler config. If None, operations that depend
on config flags (auto_download, folder_scan) will be skipped.
"""
self._config = config
self._last_scan_time: Optional[datetime] = None
self._last_auto_download_time: Optional[datetime] = None
@property
def last_scan_time(self) -> Optional[datetime]:
return self._last_scan_time
# ------------------------------------------------------------------
# Public entry point
# ------------------------------------------------------------------
async def execute(self) -> dict:
"""Execute the full rescan workflow.
Runs in order:
1. anime_service.rescan()
2. auto-download (if enabled)
3. folder scan (if enabled)
4. key resolution scan (always, if anime_directory configured)
Returns:
Dict with duration and counts for each step.
"""
from src.server.services.websocket_service import get_websocket_service
scan_start = datetime.now(timezone.utc)
results = {
"started_at": scan_start.isoformat(),
"duration_seconds": 0.0,
"rescan_completed": False,
"auto_download_queued": 0,
"folder_scan_completed": False,
"key_resolution": {"resolved": 0, "skipped": 0, "errors": 0},
}
await self._broadcast("scheduled_rescan_started", {"timestamp": scan_start.isoformat()})
try:
# 1. Main library rescan
await self._run_rescan()
results["rescan_completed"] = True
# 2. Auto-download
if self._config and self._config.auto_download_after_rescan:
try:
queued = await self._run_auto_download()
results["auto_download_queued"] = queued
await self._broadcast("auto_download_started", {"queued_count": queued})
except Exception as exc:
logger.error("Auto-download failed: %s", exc, exc_info=True)
await self._broadcast("auto_download_error", {"error": str(exc)})
# 3. Folder scan
if self._config and self._config.folder_scan_enabled:
try:
await self._run_folder_scan()
results["folder_scan_completed"] = True
except Exception as exc:
logger.error("Folder scan failed: %s", exc, exc_info=True)
await self._broadcast("folder_scan_error", {"error": str(exc)})
# 4. Key resolution scan
try:
key_stats = await self._run_key_resolution()
results["key_resolution"] = key_stats
except Exception as exc:
logger.error("Key resolution scan failed: %s", exc, exc_info=True)
self._last_scan_time = datetime.now(timezone.utc)
results["duration_seconds"] = (self._last_scan_time - scan_start).total_seconds()
await self._broadcast(
"scheduled_rescan_completed",
{
"timestamp": self._last_scan_time.isoformat(),
"duration_seconds": results["duration_seconds"],
},
)
logger.info(
"Scheduled library rescan completed: duration=%.2fs",
results["duration_seconds"],
)
except Exception as exc:
logger.error("Scheduled rescan failed: %s", exc, exc_info=True)
await self._broadcast(
"scheduled_rescan_error",
{"error": str(exc), "timestamp": datetime.now(timezone.utc).isoformat()},
)
raise
return results
# ------------------------------------------------------------------
# Step 1: Library rescan
# ------------------------------------------------------------------
async def _run_rescan(self) -> None:
"""Run the anime service rescan."""
from src.server.utils.dependencies import get_anime_service
anime_service = get_anime_service()
logger.info("Anime service obtained, calling anime_service.rescan()...")
await anime_service.rescan()
logger.info("anime_service.rescan() completed")
# ------------------------------------------------------------------
# Step 2: Auto-download
# ------------------------------------------------------------------
async def _run_auto_download(self) -> int:
"""Queue and start downloads for all series with missing episodes.
Returns:
Number of episodes queued.
"""
from src.server.models.download import EpisodeIdentifier
from src.server.utils.dependencies import (
get_anime_service,
get_download_service,
)
# Cooldown check to prevent rapid re-triggers
now = datetime.now(timezone.utc)
if self._last_auto_download_time is not None:
elapsed = now - self._last_auto_download_time
if elapsed < timedelta(seconds=_AUTO_DOWNLOAD_COOLDOWN_SECONDS):
logger.debug(
"Auto-download skipped: cooldown active (elapsed=%.1fs cooldown=%ds)",
elapsed.total_seconds(),
_AUTO_DOWNLOAD_COOLDOWN_SECONDS,
)
return 0
anime_service = get_anime_service()
download_service = get_download_service()
series_list = anime_service._cached_list_missing()
queued_count = 0
for series in series_list:
episode_dict: dict = series.get("episodeDict") or {}
if not episode_dict:
continue
episodes: List[EpisodeIdentifier] = []
for season_str, ep_numbers in episode_dict.items():
for ep_num in ep_numbers:
episodes.append(
EpisodeIdentifier(season=int(season_str), episode=int(ep_num))
)
if not episodes:
continue
await download_service.add_to_queue(
serie_id=series.get("key", ""),
serie_folder=series.get("folder", series.get("name", "")),
serie_name=series.get("name", ""),
episodes=episodes,
)
queued_count += len(episodes)
logger.info(
"Auto-download queued episodes for series=%s count=%d",
series.get("key"),
len(episodes),
)
if queued_count:
await download_service.start_queue_processing()
logger.info("Auto-download queue processing started: queued=%d", queued_count)
self._last_auto_download_time = datetime.now(timezone.utc)
logger.info("Auto-download completed: queued_count=%d", queued_count)
return queued_count
# ------------------------------------------------------------------
# Step 3: Folder scan
# ------------------------------------------------------------------
async def _run_folder_scan(self) -> None:
"""Run the folder scan maintenance task."""
from src.server.services.scheduler.folder_scan_service import FolderScanService
folder_scan_service = FolderScanService()
await folder_scan_service.run_folder_scan()
logger.info("Folder scan completed successfully")
# ------------------------------------------------------------------
# Step 4: Key resolution
# ------------------------------------------------------------------
async def _run_key_resolution(self) -> dict:
"""Run the orphaned folder key resolution scan.
Returns:
Dict with resolved/skipped/errors counts.
"""
from src.server.services.scheduler.key_resolution_service import (
perform_key_resolution_scan,
)
key_stats = await perform_key_resolution_scan()
logger.info(
"Key resolution scan completed: resolved=%d, skipped=%d, errors=%d",
key_stats["resolved"],
key_stats["skipped"],
key_stats["errors"],
)
return key_stats
# ------------------------------------------------------------------
# Private helpers
# ------------------------------------------------------------------
async def _broadcast(self, event_type: str, data: dict) -> None:
"""Broadcast a WebSocket event to all connected clients."""
try:
from src.server.services.websocket_service import get_websocket_service
ws_service = get_websocket_service()
await ws_service.manager.broadcast({"type": event_type, "data": data})
except Exception as exc:
logger.warning(
"WebSocket broadcast failed: event=%s error=%s", event_type, exc
)
# ---------------------------------------------------------------------------
# Module-level singleton
# ---------------------------------------------------------------------------
_rescan_service: Optional[RescanService] = None
def get_rescan_service(config: Optional[SchedulerConfig] = None) -> RescanService:
"""Return a RescanService singleton (or create with optional config)."""
global _rescan_service
if _rescan_service is None or config is not None:
_rescan_service = RescanService(config=config)
logger.debug("Created new RescanService singleton")
else:
logger.debug("Returning existing RescanService singleton")
return _rescan_service
def reset_rescan_service() -> None:
"""Reset the singleton (used in tests)."""
global _rescan_service
_rescan_service = None

View File

@@ -0,0 +1,45 @@
"""Scheduler services package.
Contains scheduler orchestration and rescan coordination:
- scheduler_service: Cron-based scheduler using APScheduler
- rescan_orchestrator: Legacy alias for RescanService (for backward compatibility)
"""
from __future__ import annotations
from src.server.services.rescan_service import (
RescanService,
get_rescan_service,
reset_rescan_service,
)
# Backward compatibility alias
from src.server.services.scheduler.rescan_orchestrator import (
RescanOrchestrator,
get_rescan_orchestrator,
reset_rescan_orchestrator,
)
from src.server.services.scheduler.scheduler_service import (
SchedulerService,
SchedulerServiceError,
get_scheduler_service,
reset_scheduler_service,
)
__all__ = [
# RescanService (new location)
"RescanService",
"get_rescan_service",
"reset_rescan_service",
# Scheduler
"SchedulerService",
"SchedulerServiceError",
"get_scheduler_service",
"reset_scheduler_service",
# Backward compatibility
"RescanOrchestrator",
"get_rescan_orchestrator",
"reset_rescan_orchestrator",
# Sub-services (still in scheduler folder)
"folder_rename_service",
]

View File

@@ -0,0 +1,739 @@
"""Folder rename service for validating and renaming series folders.
After NFO repair, this service iterates over every subfolder in
``settings.anime_directory`` that contains a ``tvshow.nfo``. For each
folder it parses the NFO to extract ``<title>`` and ``<year>``, computes
the expected folder name ``f"{title} ({year})"``, sanitises it for
filesystem safety, and renames the folder if the current name differs.
Database records (``AnimeSeries.folder``, ``Episode.file_path``,
``DownloadQueueItem.file_destination``) are updated atomically to
reflect the new paths.
"""
from __future__ import annotations
import logging
import re
import shutil
from collections import defaultdict
from dataclasses import dataclass
from pathlib import Path
from typing import Optional
from lxml import etree
from src.config.settings import settings
from src.server.database.connection import get_db_session
from src.server.database.service import (
AnimeSeriesService,
DownloadQueueService,
EpisodeService,
)
from src.server.utils.dependencies import get_download_service
from src.server.utils.filesystem import sanitize_folder_name
logger = logging.getLogger(__name__)
# Pre-compiled pattern for stripping existing year suffixes
_YEAR_SUFFIX_PATTERN = re.compile(r'(\s*\(\d{4}\))+\s*$')
@dataclass
class DuplicateGroup:
"""Represents a group of duplicate folders for the same series.
Attributes:
key: The series key (folder name before rename).
folders: List of folder paths that map to this series.
nfo_paths: List of corresponding NFO file paths.
"""
key: str
folders: list[str]
nfo_paths: list[Path]
@property
def count(self) -> int:
return len(self.folders)
def __repr__(self) -> str:
return f"DuplicateGroup(key={self.key!r}, folders={self.folders})"
@dataclass
class RenameStats:
"""Statistics from a folder rename operation."""
scanned: int = 0
renamed: int = 0
skipped: int = 0
errors: int = 0
def to_dict(self) -> dict[str, int]:
return {"scanned": self.scanned, "renamed": self.renamed, "skipped": self.skipped, "errors": self.errors}
def _scan_for_pre_existing_duplicates(anime_dir: Path) -> list[DuplicateGroup]:
"""Scan anime directory for pre-existing duplicate folders.
Groups folders by the series key extracted from their NFO files.
Folders with the same title+year (same expected name) are flagged as duplicates.
Args:
anime_dir: Path to the anime directory to scan.
Returns:
List of DuplicateGroup objects, one per series with duplicate folders.
"""
groups: dict[str, list[tuple[str, Path]]] = defaultdict(list)
for series_dir in anime_dir.iterdir():
if not series_dir.is_dir():
continue
nfo_path = series_dir / "tvshow.nfo"
if not nfo_path.exists():
continue
title, year = _parse_nfo_title_and_year(nfo_path)
if not title or not year:
continue
expected_name = _compute_expected_folder_name(title, year)
groups[expected_name].append((series_dir.name, nfo_path))
duplicates = []
for key, items in groups.items():
if len(items) > 1:
folders = [item[0] for item in items]
nfo_paths = [item[1] for item in items]
duplicates.append(DuplicateGroup(key=key, folders=folders, nfo_paths=nfo_paths))
return duplicates
def _try_merge_duplicate_group(group: DuplicateGroup, dry_run: bool = False) -> bool:
"""Attempt to merge a duplicate group automatically.
Uses the first folder as the canonical one and removes others if they are
empty or contain only symlinks.
Args:
group: The DuplicateGroup to merge.
dry_run: If True, only log actions without executing them.
Returns:
True if merge was successful, False otherwise.
"""
if len(group.folders) < 2:
return True
canonical = group.folders[0]
to_remove = group.folders[1:]
for folder in to_remove:
folder_path = group.nfo_paths[0].parent.parent / folder
if not folder_path.exists():
continue
try:
contents = list(folder_path.iterdir())
except PermissionError:
logger.warning("Permission denied accessing %s, skip merge", folder_path)
return False
except OSError:
return False
if not contents:
if dry_run:
logger.info("[DRY-RUN] Would delete empty duplicate folder: %s", folder_path)
else:
try:
folder_path.rmdir()
logger.info("Deleted empty duplicate folder: %s", folder_path)
except OSError:
return False
continue
canonical_path = folder_path.parent / canonical
all_symlinks = all(
item.is_symlink() and item.resolve() == canonical_path.resolve()
for item in contents
)
if all_symlinks:
if dry_run:
logger.info("[DRY-RUN] Would remove symlinks in duplicate folder: %s", folder_path)
else:
for item in contents:
item.unlink()
try:
folder_path.rmdir()
logger.info("Removed symlink-only duplicate folder: %s", folder_path)
except OSError:
return False
continue
logger.warning(
"Cannot auto-merge duplicate folders for '%s': %s (manual merge required)",
group.key,
[canonical] + to_remove,
)
return False
return True
def _parse_nfo_title_and_year(nfo_path: Path) -> tuple[Optional[str], Optional[str]]:
"""Parse a tvshow.nfo and return (title, year) text values.
Args:
nfo_path: Absolute path to the ``tvshow.nfo`` file.
Returns:
Tuple of (title, year) where either may be ``None`` if missing
or empty.
"""
try:
tree = etree.parse(str(nfo_path))
root = tree.getroot()
title_elem = root.find("./title")
year_elem = root.find("./year")
title = title_elem.text.strip() if title_elem is not None and title_elem.text and title_elem.text.strip() else None
year = year_elem.text.strip() if year_elem is not None and year_elem.text and year_elem.text.strip() else None
return title, year
except etree.XMLSyntaxError as exc:
logger.warning("Malformed XML in %s: %s", nfo_path, exc)
return None, None
except Exception as exc:
logger.warning("Unexpected error parsing %s: %s", nfo_path, exc)
return None, None
def _compute_expected_folder_name(title: str, year: str) -> str:
"""Compute the expected folder name from title and year.
Removes any existing year suffixes (e.g., "(2021)") before adding the
canonical one to prevent duplication across multiple folder rename runs.
Args:
title: Series title from NFO.
year: Release year from NFO.
Returns:
Sanitised folder name in the format ``"{title} ({year})"``.
"""
clean_title = _YEAR_SUFFIX_PATTERN.sub('', title).strip()
year_suffix = f" ({year})"
raw_name = f"{clean_title}{year_suffix}"
return sanitize_folder_name(raw_name)
def _is_series_being_downloaded(series_folder: str) -> bool:
"""Check whether the given series has an active or pending download.
Args:
series_folder: The series folder name (as stored in the DB).
Returns:
``True`` if the series appears in the active download or the
pending queue.
"""
try:
download_service = get_download_service()
active = download_service._active_download
if active and active.serie_folder == series_folder:
return True
for item in download_service._pending_queue:
if item.serie_folder == series_folder:
return True
return False
except Exception as exc:
logger.warning(
"Could not check download status for %s: %s", series_folder, exc
)
return True
def _remove_key_file(path: Path) -> None:
"""Remove legacy 'key' file from a series folder.
Args:
path: Path to the series folder.
"""
key_file = path / "key"
if key_file.exists():
try:
key_file.unlink()
logger.info("Removed legacy 'key' file after rename: %s", key_file)
except OSError as exc:
logger.warning("Could not remove legacy 'key' file %s: %s", key_file, exc)
def _move_file(item: Path, dest: Path) -> bool:
"""Move a single file or directory to destination.
Args:
item: Source path to move.
dest: Destination path.
Returns:
True if move succeeded, False otherwise.
"""
try:
item.rename(dest)
logger.debug("Moved %s%s", item, dest)
return True
except PermissionError as exc:
logger.warning("Permission denied moving %s: %s", item, exc)
return False
except OSError as exc:
logger.warning("OS error moving %s: %s", item, exc)
return False
def _cleanup_orphaned_folder(old_path: Path, new_path: Path, dry_run: bool = False) -> bool:
"""Clean up orphaned folder after successful rename.
After a folder is successfully renamed to new_path, this function checks
if the old_path still exists (orphaned folder) and removes it. If the
old folder contains files, they are moved to new_path before deletion.
Args:
old_path: The original folder path before rename.
new_path: The new folder path after rename.
dry_run: If True, only log actions without executing them.
Returns:
True if old folder was cleaned up (or would be in dry-run mode),
False if old folder does not exist or cleanup failed.
"""
if not old_path.exists():
logger.debug("Old folder does not exist, no cleanup needed: %s", old_path)
return False
try:
contents = list(old_path.iterdir())
except PermissionError as exc:
logger.warning("Permission denied accessing old folder %s: %s", old_path, exc)
return False
except OSError as exc:
logger.warning("OS error accessing old folder %s: %s", old_path, exc)
return False
if not contents:
if dry_run:
logger.info("[DRY-RUN] Would delete empty orphaned folder: %s", old_path)
return True
try:
old_path.rmdir()
logger.info("Deleted empty orphaned folder: %s", old_path)
return True
except PermissionError as exc:
logger.warning("Permission denied deleting folder %s: %s", old_path, exc)
return False
except OSError as exc:
logger.warning("OS error deleting folder %s: %s", old_path, exc)
return False
if dry_run:
logger.info("[DRY-RUN] Would move %d files from orphaned folder %s to %s",
len(contents), old_path, new_path)
for item in contents:
logger.info("[DRY-RUN] Would move: %s%s", item, new_path / item.name)
logger.info("[DRY-RUN] Would then delete orphaned folder: %s", old_path)
return True
files_moved = 0
errors = 0
for item in contents:
if not _move_file(item, new_path / item.name):
errors += 1
else:
files_moved += 1
if files_moved > 0:
logger.info("Moved %d files from orphaned folder to %s", files_moved, new_path)
try:
old_path.rmdir()
logger.info("Deleted orphaned folder after moving contents: %s", old_path)
return errors == 0
except OSError as exc:
logger.warning("Could not delete orphaned folder %s (may not be empty): %s", old_path, exc)
return False
async def _update_series_folder(db, series, new_folder: str) -> None:
"""Update AnimeSeries.folder in the database.
Args:
db: Database session.
series: The AnimeSeries instance to update.
new_folder: New folder name.
"""
if series is None:
return
await AnimeSeriesService.update(db, series.id, folder=new_folder)
logger.info("Updated AnimeSeries.folder: %s (id=%s)", new_folder, series.id)
def _update_episode_paths(episodes, old_series_path: Path, new_series_path: Path) -> None:
"""Update Episode.file_path for all episodes of a series.
Args:
episodes: List of Episode instances.
old_series_path: Path to the old series folder.
new_series_path: Path to the new series folder.
"""
for episode in episodes:
if not episode.file_path:
continue
old_file_path = Path(episode.file_path)
try:
old_file_path.relative_to(old_series_path)
new_file_path = new_series_path / old_file_path.relative_to(old_series_path)
episode.file_path = str(new_file_path)
logger.debug("Updated Episode.file_path: %s%s", old_file_path, new_file_path)
except ValueError:
pass
def _update_queue_destinations(
queue_items,
series_id,
old_series_path: Path,
new_series_path: Path,
) -> None:
"""Update DownloadQueueItem.file_destination for pending items.
Args:
queue_items: List of DownloadQueueItem instances.
series_id: ID of the series to filter by.
old_series_path: Path to the old series folder.
new_series_path: Path to the new series folder.
"""
for item in queue_items:
if item.series_id != series_id or not item.file_destination:
continue
old_dest = Path(item.file_destination)
try:
old_dest.relative_to(old_series_path)
new_dest = new_series_path / old_dest.relative_to(old_series_path)
item.file_destination = str(new_dest)
logger.debug("Updated DownloadQueueItem.file_destination: %s%s", old_dest, new_dest)
except ValueError:
pass
async def _update_database_paths(
old_folder: str,
new_folder: str,
anime_dir: Path,
) -> None:
"""Update all database records that reference the old folder path.
Updates:
- ``AnimeSeries.folder`` → ``new_folder``
- ``Episode.file_path`` → adjusted to new folder
- ``DownloadQueueItem.file_destination`` → adjusted to new folder
Args:
old_folder: Previous folder name.
new_folder: New folder name.
anime_dir: Root anime directory path.
"""
old_series_path = anime_dir / old_folder
new_series_path = anime_dir / new_folder
async with get_db_session() as db:
series = await AnimeSeriesService.get_by_folder(db, old_folder)
if series is None:
all_series = await AnimeSeriesService.get_all(db)
for s in all_series:
if s.folder == old_folder:
series = s
break
await _update_series_folder(db, series, new_folder)
if series is None:
return
episodes = await EpisodeService.get_by_series(db, series.id)
_update_episode_paths(episodes, old_series_path, new_series_path)
await db.flush()
queue_items = await DownloadQueueService.get_all(db, with_series=True)
_update_queue_destinations(queue_items, series.id, old_series_path, new_series_path)
await db.flush()
logger.info("Database paths updated for series '%s''%s'", old_folder, new_folder)
def _remove_duplicate_target_folder(
series_dir: Path,
current_name: str,
expected_name: str,
expected_path: Path,
) -> bool:
"""Handle the case where the target folder already exists.
Removes the source folder and its DB record to avoid orphaning
episodes/downloads.
Args:
series_dir: Path to the series directory being processed.
current_name: Current folder name.
expected_name: Expected folder name.
expected_path: Path to the expected (target) folder.
Returns:
True if folder was removed successfully, False otherwise.
"""
logger.warning(
"Cannot rename '%s''%s' — target already exists",
current_name,
expected_name,
)
try:
try:
contents = list(series_dir.iterdir())
logger.warning(
"REMOVING folder '%s' with %d items — target '%s' already exists",
current_name,
len(contents),
expected_name,
)
for item in contents:
logger.warning(" Would remove: %s", item)
except OSError as exc:
logger.warning(
"Could not list contents of folder '%s' before removal: %s",
current_name,
exc,
)
shutil.rmtree(series_dir)
logger.info(
"Removed source folder '%s' — series already exists at target",
current_name,
)
# Delete source DB record using synchronous helper
_delete_series_db_record(current_name, expected_name)
return True
except OSError as exc:
logger.error("Failed to remove source folder '%s': %s", current_name, exc)
return False
def _delete_series_db_record(current_name: str, expected_name: str) -> None:
"""Delete the series DB record for a folder that was removed.
Args:
current_name: The folder name to look up in the DB.
expected_name: The target folder name (for logging).
"""
try:
import asyncio
asyncio.run(_delete_series_db_record_async(current_name, expected_name))
except Exception as exc:
logger.warning(
"Could not delete DB record for '%s': %s",
current_name,
exc,
)
async def _delete_series_db_record_async(current_name: str, expected_name: str) -> None:
"""Async helper to delete series DB record.
Args:
current_name: The folder name to look up.
expected_name: The target folder name (for logging).
"""
async with get_db_session() as db:
source_series = await AnimeSeriesService.get_by_folder(db, current_name)
if source_series is None:
all_series = await AnimeSeriesService.get_all(db)
for s in all_series:
if s.folder == current_name:
source_series = s
break
if source_series is not None:
await AnimeSeriesService.delete(db, source_series.id)
logger.info(
"Deleted source DB record for '%s' (id=%s) — target folder '%s' retains DB record",
current_name,
source_series.id,
expected_name,
)
else:
logger.info(
"No DB record found for source folder '%s' — folder removed only",
current_name,
)
async def validate_and_rename_series_folders(dry_run: bool = False) -> dict[str, int]:
"""Validate and rename series folders to match NFO metadata.
Iterates over every subfolder in ``settings.anime_directory`` that
contains a ``tvshow.nfo``. For each folder:
1. Parse the NFO to extract ``<title>`` and ``<year>``.
2. Compute the expected folder name: ``f"{title} ({year})"``.
3. Sanitise the expected name for filesystem safety.
4. Compare with the current folder name.
5. If different, rename the folder and update the database.
Skips folders where title or year is missing/empty. Logs every
rename action.
Args:
dry_run: If True, simulate rename operations without actually
moving folders or updating the database.
Returns:
Dictionary with counts:
- ``"scanned"``: total folders scanned
- ``"renamed"``: folders renamed
- ``"skipped"``: folders skipped (missing title/year)
- ``"errors"``: folders that caused an error
"""
if not settings.anime_directory:
logger.warning("Folder rename skipped — anime directory not configured")
return RenameStats().to_dict()
anime_dir = Path(settings.anime_directory)
if not anime_dir.is_dir():
logger.warning("Folder rename skipped — anime directory not found: %s", anime_dir)
return RenameStats().to_dict()
if dry_run:
logger.info("Running in DRY-RUN mode — no changes will be made")
stats = RenameStats()
pre_existing_duplicates: set[str] = set()
duplicates = _scan_for_pre_existing_duplicates(anime_dir)
for dup_group in duplicates:
if _try_merge_duplicate_group(dup_group, dry_run=dry_run):
logger.info(
"Auto-merged duplicate group for '%s' (%d folders)",
dup_group.key,
dup_group.count,
)
else:
for folder in dup_group.folders:
pre_existing_duplicates.add(folder)
logger.warning(
"Duplicate folders detected for series '%s': %s"
"manual cleanup required (different releases or non-empty duplicates)",
dup_group.key,
dup_group.folders,
)
for series_dir in sorted(anime_dir.iterdir()):
if not series_dir.is_dir():
continue
nfo_path = series_dir / "tvshow.nfo"
if not nfo_path.exists():
continue
stats.scanned += 1
title, year = _parse_nfo_title_and_year(nfo_path)
if not title or not year:
logger.info(
"Skipping rename for '%s' — missing title or year in NFO",
series_dir.name,
)
stats.skipped += 1
continue
expected_name = _compute_expected_folder_name(title, year)
current_name = series_dir.name
if expected_name == current_name:
logger.debug("Folder name already correct: '%s'", current_name)
continue
if _is_series_being_downloaded(current_name):
logger.info(
"Skipping rename for '%s' — series has active or pending downloads",
current_name,
)
stats.skipped += 1
continue
expected_path = anime_dir / expected_name
if current_name in pre_existing_duplicates:
logger.warning(
"Skipping rename for '%s' — pre-existing duplicate folder detected",
current_name,
)
stats.errors += 1
continue
if expected_path.exists():
if _remove_duplicate_target_folder(series_dir, current_name, expected_name, expected_path):
stats.renamed += 1
else:
stats.errors += 1
continue
if len(str(expected_path)) > 4096:
logger.warning(
"Cannot rename '%s''%s' — path exceeds OS limit",
current_name,
expected_name,
)
stats.errors += 1
continue
if dry_run:
logger.info("[DRY-RUN] Would rename folder: '%s''%s'", current_name, expected_name)
stats.renamed += 1
continue
try:
old_path = series_dir
series_dir.rename(expected_path)
logger.info("Renamed folder: '%s''%s'", current_name, expected_name)
stats.renamed += 1
await _update_database_paths(current_name, expected_name, anime_dir)
_remove_key_file(expected_path)
_cleanup_orphaned_folder(old_path, expected_path, dry_run=False)
except PermissionError as exc:
logger.error(
"Permission denied renaming '%s''%s': %s",
current_name,
expected_name,
exc,
)
stats.errors += 1
except OSError as exc:
logger.error(
"OS error renaming '%s''%s': %s",
current_name,
expected_name,
exc,
)
stats.errors += 1
logger.info(
"Folder rename scan complete: scanned=%d, renamed=%d, skipped=%d, errors=%d",
stats.scanned,
stats.renamed,
stats.skipped,
stats.errors,
)
return stats.to_dict()

View File

@@ -28,6 +28,36 @@ _POSTER_DOWNLOAD_SEMAPHORE: asyncio.Semaphore = asyncio.Semaphore(3)
_NFO_REPAIR_SEMAPHORE: asyncio.Semaphore = asyncio.Semaphore(3)
async def _create_missing_nfo(series_dir: Path, series_name: str) -> None:
"""Create minimal NFO for series without one.
Creates a fresh :class:`NFOService` per invocation so concurrent
tasks cannot interfere with each other.
A module-level semaphore limits concurrent TMDB operations to 3.
Args:
series_dir: Absolute path to the series folder.
series_name: Human-readable series name for log messages.
"""
from src.core.services.nfo_factory import NFOServiceFactory
async with _NFO_REPAIR_SEMAPHORE:
try:
factory = NFOServiceFactory()
nfo_service = factory.create()
await nfo_service.create_minimal_nfo(
serie_name=series_name,
serie_folder=series_dir.name,
)
except Exception as exc: # pylint: disable=broad-except
logger.error(
"NFO creation failed for %s: %s",
series_name,
exc,
)
async def _repair_one_series(series_dir: Path, series_name: str) -> None:
"""Repair a single series NFO in isolation.
@@ -63,12 +93,13 @@ async def _repair_one_series(series_dir: Path, series_name: str) -> None:
async def perform_nfo_repair_scan(background_loader=None) -> None:
"""Scan all series folders and repair incomplete tvshow.nfo files.
"""Scan all series folders, repair incomplete and create missing NFO files.
Called from ``FolderScanService.run_folder_scan()`` during the scheduled
daily folder scan (not on every startup). Checks each subfolder of
``settings.anime_directory`` for a ``tvshow.nfo`` and calls
``_repair_one_series`` for every file with absent or empty required tags.
``settings.anime_directory`` for a ``tvshow.nfo``:
- Missing NFOs: creates minimal NFO via ``_create_missing_nfo``
- Incomplete NFOs: repairs via ``_repair_one_series``
Each repair task creates its own isolated :class:`NFOService` /
:class:`TMDBClient` so concurrent tasks never share an ``aiohttp``
@@ -97,26 +128,46 @@ async def perform_nfo_repair_scan(background_loader=None) -> None:
queued = 0
total = 0
missing_nfo_count = 0
repair_tasks: list[asyncio.Task] = []
for series_dir in sorted(anime_dir.iterdir()):
if not series_dir.is_dir():
continue
nfo_path = series_dir / "tvshow.nfo"
series_name = series_dir.name
if not nfo_path.exists():
# Create minimal NFO for series without one
missing_nfo_count += 1
repair_tasks.append(
asyncio.create_task(
_create_missing_nfo(series_dir, series_name),
name=f"nfo_create:{series_name}",
)
)
continue
total += 1
series_name = series_dir.name
if nfo_needs_repair(nfo_path):
queued += 1
# Each task creates its own NFOService so connectors are isolated.
asyncio.create_task(
_repair_one_series(series_dir, series_name),
name=f"nfo_repair:{series_name}",
repair_tasks.append(
asyncio.create_task(
_repair_one_series(series_dir, series_name),
name=f"nfo_repair:{series_name}",
)
)
if repair_tasks:
logger.info(
"NFO repair scan: waiting for %d repair/create tasks to complete",
len(repair_tasks),
)
await asyncio.gather(*repair_tasks, return_exceptions=True)
logger.info("NFO repair scan tasks completed")
logger.info(
"NFO repair scan complete: %d of %d series queued for repair",
"NFO repair scan complete: %d of %d series queued for repair, %d missing NFOs queued for creation",
queued,
total,
missing_nfo_count,
)
@@ -144,14 +195,14 @@ class FolderScanService:
if not self._prerequisites_met():
return
# 1.3 — Repair incomplete NFO files in the background.
# 1.3 — Repair incomplete NFO files (synchronous, waits for completion).
logger.info("Starting NFO repair scan as part of folder scan")
await perform_nfo_repair_scan(background_loader=None)
logger.info("NFO repair scan queued; repairs will continue in background")
logger.info("NFO repair scan complete")
# 1.4 — Validate and rename series folders after NFO repair.
logger.info("Starting folder rename validation")
from src.server.services.folder_rename_service import (
from src.server.services.scheduler.folder_rename_service import (
validate_and_rename_series_folders,
)

View File

@@ -0,0 +1,317 @@
"""Key resolution service for orphaned anime folders.
Attempts to resolve provider keys for anime folders that have no key/data
file and no database entry, by searching the anime provider and matching
folder names to search results.
This service runs after nfo_repair_service during the daily folder scan.
"""
from __future__ import annotations
import asyncio
import re
from pathlib import Path
from typing import Optional
import structlog
from src.config.settings import settings as _settings
logger = structlog.get_logger(__name__)
# Limit concurrent provider searches to avoid rate-limiting.
_SEARCH_SEMAPHORE: asyncio.Semaphore = asyncio.Semaphore(2)
def _strip_year_from_folder(folder_name: str) -> str:
"""Remove trailing year suffix like ' (2020)' from folder name.
Args:
folder_name: Folder name, e.g. 'Rent-A-Girlfriend (2020)'
Returns:
Name without year, e.g. 'Rent-A-Girlfriend'
"""
return re.sub(r"\s*\(\d{4}\)\s*$", "", folder_name).strip()
def _extract_year_from_folder(folder_name: str) -> Optional[int]:
"""Extract year from folder name like 'Anime Name (2020)'.
Returns:
Year as int or None if not present.
"""
match = re.search(r"\((\d{4})\)$", folder_name.strip())
if match:
return int(match.group(1))
return None
def _extract_key_from_link(link: str) -> Optional[str]:
"""Extract provider key from search result link.
Args:
link: Link like '/anime/stream/rent-a-girlfriend' or full URL.
Returns:
Key slug like 'rent-a-girlfriend' or None.
"""
if not link:
return None
if "/anime/stream/" in link:
parts = link.split("/anime/stream/")[-1].split("/")
key = parts[0].strip()
return key if key else None
# If link is just a slug
if "/" not in link and link.strip():
return link.strip()
return None
def _normalize_for_comparison(text: str) -> str:
"""Normalize text for case-insensitive comparison.
Strips whitespace, lowercases, and removes common punctuation
differences that shouldn't affect matching.
Args:
text: Raw text string.
Returns:
Normalized lowercase string.
"""
normalized = text.strip().lower()
# Remove common punctuation that varies between sources
normalized = re.sub(r"[:\-–—]", " ", normalized)
# Collapse multiple spaces
normalized = re.sub(r"\s+", " ", normalized)
return normalized.strip()
async def resolve_key_for_folder(folder_name: str) -> Optional[str]:
"""Attempt to resolve the provider key for a single folder.
Strategy:
1. Strip year suffix from folder name to get search query.
2. Search the anime provider with that query.
3. If exactly ONE result matches the folder name (case-insensitive),
return the key extracted from the result link.
4. If zero or multiple matches, return None (not confident enough).
Args:
folder_name: The anime folder name, e.g. 'Rent-A-Girlfriend (2020)'.
Returns:
The provider key string, or None if resolution is not confident.
"""
search_query = _strip_year_from_folder(folder_name)
if not search_query:
logger.debug("Empty search query after stripping year from '%s'", folder_name)
return None
async with _SEARCH_SEMAPHORE:
try:
loop = asyncio.get_running_loop()
results = await loop.run_in_executor(None, _search_provider, search_query)
except Exception as exc:
logger.warning(
"Provider search failed for '%s': %s", search_query, exc
)
return None
if not results:
logger.debug("No search results for folder '%s'", folder_name)
return None
# Filter results: find exact name matches (case-insensitive)
normalized_query = _normalize_for_comparison(search_query)
exact_matches = []
for result in results:
title = result.get("title") or result.get("name") or ""
normalized_title = _normalize_for_comparison(title)
if normalized_title == normalized_query:
key = _extract_key_from_link(result.get("link", ""))
if key:
exact_matches.append((key, title))
if len(exact_matches) == 1:
resolved_key, matched_title = exact_matches[0]
logger.info(
"Resolved key for folder '%s': key='%s' (matched title: '%s')",
folder_name,
resolved_key,
matched_title,
)
return resolved_key
if len(exact_matches) > 1:
logger.info(
"Multiple exact matches for folder '%s' (%d matches), skipping",
folder_name,
len(exact_matches),
)
else:
logger.debug(
"No exact title match for folder '%s' in %d results",
folder_name,
len(results),
)
return None
def _search_provider(query: str) -> list:
"""Call the anime provider search synchronously.
Args:
query: Search term.
Returns:
List of search result dicts with 'link' and 'title'/'name' fields.
"""
from src.core.providers.provider_factory import Loaders
loader = Loaders().GetLoader("aniworld.to")
return loader.search(query)
async def perform_key_resolution_scan() -> dict[str, int]:
"""Scan all anime folders and resolve missing keys.
Iterates over all subfolders of the anime directory. For each folder
that has no corresponding database entry, attempts to resolve the
provider key via provider search and saves it to the database.
Returns:
Dictionary with counts:
- 'scanned': total folders checked
- 'resolved': keys successfully resolved and saved
- 'skipped': folders already in DB or resolution uncertain
- 'errors': folders that caused errors during resolution
"""
from src.server.database.connection import get_db_session
from src.server.database.service import AnimeSeriesService
stats = {"scanned": 0, "resolved": 0, "skipped": 0, "errors": 0}
if not _settings.anime_directory:
logger.warning("Key resolution scan skipped — anime directory not configured")
return stats
anime_dir = Path(_settings.anime_directory)
if not anime_dir.is_dir():
logger.warning(
"Key resolution scan skipped — anime directory not found: %s",
anime_dir,
)
return stats
# Collect folders that need resolution
folders_to_resolve: list[str] = []
async with get_db_session() as db:
for series_dir in sorted(anime_dir.iterdir()):
if not series_dir.is_dir():
continue
folder_name = series_dir.name
stats["scanned"] += 1
# Check if already in database
existing = await AnimeSeriesService.get_by_folder(db, folder_name)
if existing:
stats["skipped"] += 1
continue
folders_to_resolve.append(folder_name)
if not folders_to_resolve:
logger.info("Key resolution scan: all folders already have DB entries")
return stats
logger.info(
"Key resolution scan: %d folders need resolution", len(folders_to_resolve)
)
# Resolve keys one by one (provider search is rate-limited)
for folder_name in folders_to_resolve:
try:
key = await resolve_key_for_folder(folder_name)
if key:
# Save to database
await _save_resolved_key(folder_name, key)
stats["resolved"] += 1
else:
stats["skipped"] += 1
except Exception as exc:
logger.error(
"Error resolving key for folder '%s': %s",
folder_name,
exc,
)
stats["errors"] += 1
logger.info(
"Key resolution scan complete: scanned=%d, resolved=%d, skipped=%d, errors=%d",
stats["scanned"],
stats["resolved"],
stats["skipped"],
stats["errors"],
)
return stats
async def _save_resolved_key(folder_name: str, key: str) -> None:
"""Save a resolved key to the database.
Creates a new AnimeSeries entry with the resolved key and folder name.
Does NOT write any key/data file to disk.
Args:
folder_name: The anime folder name (e.g. 'Rent-A-Girlfriend (2020)').
key: The resolved provider key (e.g. 'rent-a-girlfriend').
"""
from src.server.database.connection import get_db_session
from src.server.database.service import AnimeSeriesService
name = _strip_year_from_folder(folder_name)
year = _extract_year_from_folder(folder_name)
async with get_db_session() as db:
# Double-check: another task might have resolved it concurrently
existing = await AnimeSeriesService.get_by_folder(db, folder_name)
if existing:
logger.debug(
"Folder '%s' already in DB (resolved concurrently), skipping",
folder_name,
)
return
# Also check if a series with this key already exists
existing_key = await AnimeSeriesService.get_by_key(db, key)
if existing_key:
logger.warning(
"Key '%s' already exists in DB for folder '%s', "
"cannot assign to folder '%s'",
key,
existing_key.folder,
folder_name,
)
return
await AnimeSeriesService.create(
db,
key=key,
name=name,
site="aniworld.to",
folder=folder_name,
year=year,
loading_status="pending",
episodes_loaded=False,
)
logger.info(
"Saved resolved key '%s' for folder '%s' to database",
key,
folder_name,
)

View File

@@ -0,0 +1,293 @@
"""Rescan orchestrator — coordinates all scan/cleanup operations during a rescan.
Extracts the rescan workflow from SchedulerService so scheduling and scan
logic are cleanly separated.
Called by SchedulerService.trigger_rescan() and by _run_rescan_job().
"""
from __future__ import annotations
import logging
from datetime import datetime, timezone
from typing import List, Optional
from src.server.models.config import SchedulerConfig
logger = logging.getLogger(__name__)
class RescanOrchestrator:
"""Coordinates rescan, auto-download, folder scan, and key resolution.
This class encapsulates the entire post-rescan workflow so SchedulerService
only needs to call a single method.
"""
def __init__(self, config: Optional[SchedulerConfig] = None) -> None:
"""Initialize the orchestrator.
Args:
config: Optional scheduler config. If None, operations that depend
on config flags (auto_download, folder_scan) will be skipped.
"""
self._config = config
self._last_scan_time: Optional[datetime] = None
# Cooldown tracking for auto-download to prevent rapid re-triggers
self._last_auto_download_time: Optional[datetime] = None
self._auto_download_cooldown_seconds: int = 300 # 5 minutes default
@property
def last_scan_time(self) -> Optional[datetime]:
return self._last_scan_time
# ------------------------------------------------------------------
# Auto-download
# ------------------------------------------------------------------
async def run_auto_download(self) -> int:
"""Queue and start downloads for all series with missing episodes.
Returns:
Number of episodes queued.
"""
from datetime import timedelta
from src.server.models.download import EpisodeIdentifier
from src.server.utils.dependencies import (
get_anime_service,
get_download_service,
)
# Check cooldown to prevent rapid re-triggers
now = datetime.now(timezone.utc)
if self._last_auto_download_time is not None:
elapsed = now - self._last_auto_download_time
if elapsed < timedelta(seconds=self._auto_download_cooldown_seconds):
logger.debug(
"Auto-download skipped: cooldown active (elapsed=%.1fs cooldown=%ds)",
elapsed.total_seconds(),
self._auto_download_cooldown_seconds,
)
return 0
anime_service = get_anime_service()
download_service = get_download_service()
series_list = anime_service._cached_list_missing()
queued_count = 0
for series in series_list:
episode_dict: dict = series.get("episodeDict") or {}
if not episode_dict:
continue
episodes: List[EpisodeIdentifier] = []
for season_str, ep_numbers in episode_dict.items():
for ep_num in ep_numbers:
episodes.append(
EpisodeIdentifier(season=int(season_str), episode=int(ep_num))
)
if not episodes:
continue
await download_service.add_to_queue(
serie_id=series.get("key", ""),
serie_folder=series.get("folder", series.get("name", "")),
serie_name=series.get("name", ""),
episodes=episodes,
)
queued_count += len(episodes)
logger.info(
"Auto-download queued episodes for series=%s count=%d",
series.get("key"),
len(episodes),
)
if queued_count:
await download_service.start_queue_processing()
logger.info("Auto-download queue processing started: queued=%d", queued_count)
self._last_auto_download_time = datetime.now(timezone.utc)
logger.info("Auto-download completed: queued_count=%d", queued_count)
return queued_count
# ------------------------------------------------------------------
# Folder scan
# ------------------------------------------------------------------
async def run_folder_scan(self) -> None:
"""Run the folder scan maintenance task."""
from src.server.services.scheduler.folder_scan_service import FolderScanService
folder_scan_service = FolderScanService()
await folder_scan_service.run_folder_scan()
logger.info("Folder scan completed successfully")
# ------------------------------------------------------------------
# Key resolution
# ------------------------------------------------------------------
async def run_key_resolution(self) -> dict:
"""Run the orphaned folder key resolution scan.
Returns:
Dict with resolved/skipped/errors counts.
"""
from src.server.services.scheduler.key_resolution_service import (
perform_key_resolution_scan,
)
key_stats = await perform_key_resolution_scan()
logger.info(
"Key resolution scan completed: resolved=%d, skipped=%d, errors=%d",
key_stats["resolved"],
key_stats["skipped"],
key_stats["errors"],
)
return key_stats
# ------------------------------------------------------------------
# Main orchestrator entry point
# ------------------------------------------------------------------
async def execute(self) -> dict:
"""Execute the full rescan workflow.
Runs in order:
1. anime_service.rescan()
2. auto-download (if enabled)
3. folder scan (if enabled)
4. key resolution scan (always, if anime_directory configured)
Returns:
Dict with duration and counts for each step.
"""
scan_start = datetime.now(timezone.utc)
results = {
"started_at": scan_start.isoformat(),
"duration_seconds": 0.0,
"rescan_completed": False,
"auto_download_queued": 0,
"folder_scan_completed": False,
"key_resolution": {"resolved": 0, "skipped": 0, "errors": 0},
}
await self._broadcast(
"scheduled_rescan_started",
{"timestamp": scan_start.isoformat()},
)
try:
# 1. Main library rescan
await self._run_rescan()
results["rescan_completed"] = True
# 2. Auto-download
if self._config and self._config.auto_download_after_rescan:
try:
queued = await self.run_auto_download()
results["auto_download_queued"] = queued
await self._broadcast(
"auto_download_started", {"queued_count": queued}
)
except Exception as exc:
logger.error("Auto-download failed: %s", exc, exc_info=True)
await self._broadcast(
"auto_download_error", {"error": str(exc)}
)
# 3. Folder scan
if self._config and self._config.folder_scan_enabled:
try:
await self.run_folder_scan()
results["folder_scan_completed"] = True
except Exception as exc:
logger.error("Folder scan failed: %s", exc, exc_info=True)
await self._broadcast("folder_scan_error", {"error": str(exc)})
# 4. Key resolution scan (always runs if anime_directory configured)
try:
key_stats = await self.run_key_resolution()
results["key_resolution"] = key_stats
except Exception as exc:
logger.error("Key resolution scan failed: %s", exc, exc_info=True)
self._last_scan_time = datetime.now(timezone.utc)
results["duration_seconds"] = (
self._last_scan_time - scan_start
).total_seconds()
await self._broadcast(
"scheduled_rescan_completed",
{
"timestamp": self._last_scan_time.isoformat(),
"duration_seconds": results["duration_seconds"],
},
)
logger.info(
"Scheduled library rescan completed: duration=%.2fs",
results["duration_seconds"],
)
except Exception as exc:
logger.error("Scheduled rescan failed: %s", exc, exc_info=True)
await self._broadcast(
"scheduled_rescan_error",
{"error": str(exc), "timestamp": datetime.now(timezone.utc).isoformat()},
)
raise
return results
# ------------------------------------------------------------------
# Private helpers
# ------------------------------------------------------------------
async def _run_rescan(self) -> None:
"""Run the anime service rescan."""
from src.server.utils.dependencies import get_anime_service
anime_service = get_anime_service()
logger.info("Anime service obtained, calling anime_service.rescan()...")
await anime_service.rescan()
logger.info("anime_service.rescan() completed")
async def _broadcast(self, event_type: str, data: dict) -> None:
"""Broadcast a WebSocket event to all connected clients."""
try:
from src.server.services.websocket_service import get_websocket_service
ws_service = get_websocket_service()
await ws_service.manager.broadcast({"type": event_type, "data": data})
except Exception as exc:
logger.warning(
"WebSocket broadcast failed: event=%s error=%s", event_type, exc
)
# ---------------------------------------------------------------------------
# Module-level orchestrator
# ---------------------------------------------------------------------------
_orchestrator: Optional[RescanOrchestrator] = None
def get_rescan_orchestrator(
config: Optional[SchedulerConfig] = None,
) -> RescanOrchestrator:
"""Return a RescanOrchestrator singleton (or create with optional config)."""
global _orchestrator
if _orchestrator is None or config is not None:
_orchestrator = RescanOrchestrator(config=config)
logger.debug("Created new RescanOrchestrator singleton")
else:
logger.debug("Returning existing RescanOrchestrator singleton")
return _orchestrator
def reset_rescan_orchestrator() -> None:
"""Reset the orchestrator singleton (used in tests)."""
global _orchestrator
_orchestrator = None

View File

@@ -1,25 +1,34 @@
"""Scheduler service for automatic library rescans.
Uses APScheduler's AsyncIOScheduler with CronTrigger for precise
cron-based scheduling. The legacy interval-based loop has been removed
in favour of the cron approach.
cron-based scheduling.
Jobs are held in memory (no separate scheduler database). On startup,
if the last scan timestamp indicates a missed run (server was down at the
scheduled cron time), a rescan is triggered immediately.
Actual rescan logic is delegated to RescanService.
"""
from __future__ import annotations
from datetime import datetime, timezone
from typing import List, Optional
import logging
from datetime import datetime, timedelta, timezone
from typing import Optional
import structlog
from apscheduler.schedulers.asyncio import AsyncIOScheduler
from apscheduler.triggers.cron import CronTrigger
from src.server.models.config import SchedulerConfig
from src.server.services.config_service import ConfigServiceError, get_config_service
logger = structlog.get_logger(__name__)
logger = logging.getLogger(__name__)
_JOB_ID = "scheduled_rescan"
# Grace period for missed jobs (1 hour — handles server downtime between
# scheduled time and startup).
_MISFIRE_GRACE_SECONDS = 3600
class SchedulerServiceError(Exception):
"""Service-level exception for scheduler operations."""
@@ -34,7 +43,9 @@ class SchedulerService:
- Cron-based scheduling (time of day + days of week)
- Immediate manual trigger
- Live config reloading without app restart
- Auto-queueing downloads of missing episodes after rescan
Actual rescan/folder-scan/auto-download work is delegated to
RescanService.
"""
def __init__(self) -> None:
@@ -42,7 +53,6 @@ class SchedulerService:
self._is_running: bool = False
self._scheduler: Optional[AsyncIOScheduler] = None
self._config: Optional[SchedulerConfig] = None
self._last_scan_time: Optional[datetime] = None
self._scan_in_progress: bool = False
logger.info("SchedulerService initialised")
@@ -57,15 +67,18 @@ class SchedulerService:
SchedulerServiceError: If the scheduler is already running or
config cannot be loaded.
"""
logger.info("SchedulerService.start() called")
if self._is_running:
logger.warning("Scheduler start called but already running")
raise SchedulerServiceError("Scheduler is already running")
try:
config_service = get_config_service()
config = config_service.load_config()
self._config = config.scheduler
logger.info("Scheduler config loaded successfully")
except ConfigServiceError as exc:
logger.error("Failed to load scheduler configuration", error=str(exc))
logger.error("Failed to load scheduler configuration: %s", exc)
raise SchedulerServiceError(f"Failed to load config: {exc}") from exc
self._scheduler = AsyncIOScheduler()
@@ -75,6 +88,15 @@ class SchedulerService:
self._is_running = True
return
logger.info(
"Scheduler config loaded: enabled=%s time=%s days=%s auto_download=%s folder_scan=%s",
self._config.enabled,
self._config.schedule_time,
self._config.schedule_days,
self._config.auto_download_after_rescan,
self._config.folder_scan_enabled,
)
trigger = self._build_cron_trigger()
if trigger is None:
logger.warning(
@@ -82,23 +104,38 @@ class SchedulerService:
)
else:
self._scheduler.add_job(
self._perform_rescan,
_run_rescan_job,
trigger=trigger,
id=_JOB_ID,
replace_existing=True,
misfire_grace_time=300,
misfire_grace_time=_MISFIRE_GRACE_SECONDS,
coalesce=True,
)
logger.info(
"Scheduler started with cron trigger",
schedule_time=self._config.schedule_time,
schedule_days=self._config.schedule_days,
"Scheduler started with cron trigger: time=%s days=%s",
self._config.schedule_time,
self._config.schedule_days,
)
self._scheduler.start()
self._is_running = True
# Log next scheduled run for visibility.
job = self._scheduler.get_job(_JOB_ID)
if job:
next_run = job.next_run_time
logger.info(
"Scheduler next run: %s",
next_run.isoformat() if next_run else None,
)
# Startup misfire recovery: check if the last scan was missed while
# the server was down.
await self._check_missed_run()
async def stop(self) -> None:
"""Stop the APScheduler gracefully."""
logger.info("SchedulerService.stop() called")
if not self._is_running:
logger.debug("Scheduler stop called but not running")
return
@@ -106,8 +143,27 @@ class SchedulerService:
if self._scheduler and self._scheduler.running:
self._scheduler.shutdown(wait=False)
logger.info("Scheduler stopped")
else:
logger.info("Scheduler stop: scheduler was not running")
self._is_running = False
logger.info("SchedulerService stopped successfully")
async def ensure_started(self) -> None:
"""Ensure the scheduler is running (idempotent).
If already running, returns immediately. Otherwise, starts the scheduler.
This method is safe to call multiple times and from multiple callers.
Raises:
SchedulerServiceError: If startup fails (except for already running).
"""
if self._is_running:
logger.debug("Scheduler ensure_started called but already running")
return
logger.info("Scheduler ensure_started: starting scheduler")
await self.start()
async def trigger_rescan(self) -> bool:
"""Manually trigger a library rescan.
@@ -140,12 +196,12 @@ class SchedulerService:
"""
self._config = config
logger.info(
"Scheduler config reloaded",
enabled=config.enabled,
schedule_time=config.schedule_time,
schedule_days=config.schedule_days,
auto_download=config.auto_download_after_rescan,
folder_scan=config.folder_scan_enabled,
"Scheduler config reloaded: enabled=%s time=%s days=%s auto_download=%s folder_scan=%s",
config.enabled,
config.schedule_time,
config.schedule_days,
config.auto_download_after_rescan,
config.folder_scan_enabled,
)
if not self._scheduler or not self._scheduler.running:
@@ -166,22 +222,23 @@ class SchedulerService:
if self._scheduler.get_job(_JOB_ID):
self._scheduler.reschedule_job(_JOB_ID, trigger=trigger)
logger.info(
"Scheduler rescheduled with cron trigger",
schedule_time=config.schedule_time,
schedule_days=config.schedule_days,
"Scheduler rescheduled with cron trigger: time=%s days=%s",
config.schedule_time,
config.schedule_days,
)
else:
self._scheduler.add_job(
self._perform_rescan,
_run_rescan_job,
trigger=trigger,
id=_JOB_ID,
replace_existing=True,
misfire_grace_time=300,
misfire_grace_time=_MISFIRE_GRACE_SECONDS,
coalesce=True,
)
logger.info(
"Scheduler job added with cron trigger",
schedule_time=config.schedule_time,
schedule_days=config.schedule_days,
"Scheduler job added with cron trigger: time=%s days=%s",
config.schedule_time,
config.schedule_days,
)
def get_status(self) -> dict:
@@ -190,6 +247,10 @@ class SchedulerService:
Returns:
Dict containing scheduler state and config fields.
"""
from src.server.services.rescan_service import get_rescan_service
rescan_service = get_rescan_service()
next_run: Optional[str] = None
if self._scheduler and self._scheduler.running:
job = self._scheduler.get_job(_JOB_ID)
@@ -208,7 +269,11 @@ class SchedulerService:
"folder_scan_enabled": (
self._config.folder_scan_enabled if self._config else False
),
"last_run": self._last_scan_time.isoformat() if self._last_scan_time else None,
"last_run": (
rescan_service.last_scan_time.isoformat()
if rescan_service.last_scan_time
else None
),
"next_run": next_run,
"scan_in_progress": self._scan_in_progress,
}
@@ -235,158 +300,109 @@ class SchedulerService:
day_of_week=day_of_week,
)
logger.debug(
"CronTrigger built",
hour=hour_str,
minute=minute_str,
day_of_week=day_of_week,
"CronTrigger built: hour=%s minute=%s day_of_week=%s",
hour_str,
minute_str,
day_of_week,
)
return trigger
async def _broadcast(self, event_type: str, data: dict) -> None:
"""Broadcast a WebSocket event to all connected clients."""
async def _check_missed_run(self) -> None:
"""Check if a scheduled rescan was missed while the server was down.
Compares system_settings.last_scan_timestamp against the expected
schedule. If the last scan is overdue (more than 24h ago for a daily
schedule) but within the grace period, triggers an immediate rescan.
"""
if not self._config or not self._config.enabled:
return
if not self._config.schedule_days:
return
try:
from src.server.services.websocket_service import ( # noqa: PLC0415
get_websocket_service,
from src.server.database.connection import get_db_session
from src.server.database.system_settings_service import (
SystemSettingsService,
)
ws_service = get_websocket_service()
await ws_service.manager.broadcast({"type": event_type, "data": data})
async with get_db_session() as db:
settings = await SystemSettingsService.get_or_create(db)
last_scan = settings.last_scan_timestamp
if last_scan is None:
# Never scanned before — trigger immediately
logger.info("No previous scan recorded — triggering immediate rescan")
await self._perform_rescan()
return
# Ensure timezone-aware comparison
if last_scan.tzinfo is None:
last_scan = last_scan.replace(tzinfo=timezone.utc)
now = datetime.now(timezone.utc)
elapsed = now - last_scan
# If last scan was more than 24h + grace period ago, don't trigger
# (avoids surprise rescans after long downtime).
max_overdue = timedelta(hours=24, seconds=_MISFIRE_GRACE_SECONDS)
if elapsed > max_overdue:
logger.info(
"Last scan was %s ago (> %s) — skipping missed-run recovery",
elapsed,
max_overdue,
)
return
# Check if a run should have happened between last_scan and now.
if elapsed > timedelta(hours=23):
logger.info(
"Missed scheduled rescan detected (last scan %s ago) — triggering now",
elapsed,
)
await self._perform_rescan()
except Exception as exc: # pylint: disable=broad-exception-caught
logger.warning("WebSocket broadcast failed", event=event_type, error=str(exc))
async def _auto_download_missing(self) -> None:
"""Queue and start downloads for all series with missing episodes."""
from src.server.models.download import EpisodeIdentifier # noqa: PLC0415
from src.server.utils.dependencies import ( # noqa: PLC0415
get_anime_service,
get_download_service,
)
anime_service = get_anime_service()
download_service = get_download_service()
series_list = anime_service._cached_list_missing()
queued_count = 0
for series in series_list:
episode_dict: dict = series.get("episodeDict") or {}
if not episode_dict:
continue
episodes: List[EpisodeIdentifier] = []
for season_str, ep_numbers in episode_dict.items():
for ep_num in ep_numbers:
episodes.append(
EpisodeIdentifier(season=int(season_str), episode=int(ep_num))
)
if not episodes:
continue
await download_service.add_to_queue(
serie_id=series.get("key", ""),
serie_folder=series.get("folder", series.get("name", "")),
serie_name=series.get("name", ""),
episodes=episodes,
)
queued_count += len(episodes)
logger.info(
"Auto-download queued episodes",
series=series.get("key"),
count=len(episodes),
)
if queued_count:
await download_service.start_queue_processing()
logger.info("Auto-download queue processing started", queued=queued_count)
await self._broadcast("auto_download_started", {"queued_count": queued_count})
logger.info("Auto-download completed", queued_count=queued_count)
logger.warning("Missed-run check failed (non-fatal): %s", exc)
async def _perform_rescan(self) -> None:
"""Execute a library rescan and optionally trigger auto-download."""
"""Execute a library rescan via RescanService."""
from src.server.services.rescan_service import get_rescan_service
logger.info(
"Scheduler _perform_rescan entered: scan_in_progress=%s",
self._scan_in_progress,
)
if self._scan_in_progress:
logger.warning("Skipping rescan: previous scan still in progress")
return
self._scan_in_progress = True
scan_start = datetime.now(timezone.utc)
try:
logger.info("Starting scheduled library rescan")
from src.server.utils.dependencies import get_anime_service # noqa: PLC0415
anime_service = get_anime_service()
await self._broadcast(
"scheduled_rescan_started",
{"timestamp": scan_start.isoformat()},
)
await anime_service.rescan()
self._last_scan_time = datetime.now(timezone.utc)
duration = (self._last_scan_time - scan_start).total_seconds()
logger.info("Scheduled library rescan completed", duration_seconds=duration)
await self._broadcast(
"scheduled_rescan_completed",
{
"timestamp": self._last_scan_time.isoformat(),
"duration_seconds": duration,
},
)
# Auto-download after rescan
if self._config and self._config.auto_download_after_rescan:
logger.info("Auto-download after rescan is enabled — starting")
try:
await self._auto_download_missing()
except Exception as dl_exc: # pylint: disable=broad-exception-caught
logger.error(
"Auto-download after rescan failed",
error=str(dl_exc),
exc_info=True,
)
await self._broadcast(
"auto_download_error", {"error": str(dl_exc)}
)
else:
logger.debug("Auto-download after rescan is disabled — skipping")
# Folder scan (daily maintenance)
if self._config and self._config.folder_scan_enabled:
logger.info("Folder scan is enabled — starting")
try:
from src.server.services.folder_scan_service import ( # noqa: PLC0415
FolderScanService,
)
folder_scan_service = FolderScanService()
await folder_scan_service.run_folder_scan()
except Exception as fs_exc: # pylint: disable=broad-exception-caught
logger.error(
"Folder scan failed",
error=str(fs_exc),
exc_info=True,
)
await self._broadcast(
"folder_scan_error", {"error": str(fs_exc)}
)
else:
logger.debug("Folder scan is disabled — skipping")
except Exception as exc: # pylint: disable=broad-exception-caught
logger.error("Scheduled rescan failed", error=str(exc), exc_info=True)
await self._broadcast(
"scheduled_rescan_error",
{"error": str(exc), "timestamp": datetime.now(timezone.utc).isoformat()},
)
rescan_service = get_rescan_service(config=self._config)
await rescan_service.execute()
finally:
self._scan_in_progress = False
logger.info("Scheduled rescan finished: scan_in_progress reset to False")
# ---------------------------------------------------------------------------
# Module-level job runner
#
# APScheduler cannot serialize bound methods (SchedulerService instance
# contains a reference to the scheduler itself, creating a circular pickle
# error). Using a module-level function avoids this.
# ---------------------------------------------------------------------------
async def _run_rescan_job() -> None:
"""Module-level job entry point — delegates to the current service."""
logger.info("=" * 60)
logger.info("APScheduler triggered _run_rescan_job")
logger.info("Getting scheduler service singleton...")
svc = get_scheduler_service()
logger.info("Scheduler service obtained, calling _perform_rescan()")
await svc._perform_rescan()
logger.info("_run_rescan_job completed")
logger.info("=" * 60)
# ---------------------------------------------------------------------------
@@ -400,7 +416,10 @@ def get_scheduler_service() -> SchedulerService:
"""Return the singleton SchedulerService instance."""
global _scheduler_service
if _scheduler_service is None:
logger.info("Creating new SchedulerService singleton")
_scheduler_service = SchedulerService()
else:
logger.debug("Returning existing SchedulerService singleton")
return _scheduler_service

View File

@@ -21,6 +21,8 @@ from typing import Any, Dict, List, Optional
from fastapi import Request
from fastapi.templating import Jinja2Templates
from src.server.utils.version import APP_VERSION
logger = logging.getLogger(__name__)
# Configure templates directory
@@ -48,7 +50,7 @@ def get_base_context(
"request": request,
"title": title,
"app_name": "Aniworld Download Manager",
"version": "1.0.1",
"version": APP_VERSION,
"static_v": STATIC_VERSION,
}

View File

@@ -0,0 +1,26 @@
"""Version management utilities for Aniworld application."""
from pathlib import Path
from typing import Optional
def get_version() -> str:
"""
Get the current application version from Docker/VERSION file.
Returns:
Version string from the VERSION file, or "unknown" if not found.
"""
version_file = Path(__file__).parent.parent.parent.parent / "Docker" / "VERSION"
try:
if version_file.exists():
return version_file.read_text().strip()
except Exception:
pass
return "unknown"
# Module-level version constant (loaded once at import)
APP_VERSION: str = get_version()

View File

@@ -268,3 +268,205 @@
gap: 4px;
}
}
/* ============================================================================
Context Menu
============================================================================ */
.context-menu {
position: fixed;
z-index: 1500;
background: var(--color-surface);
border: 1px solid var(--color-border);
border-radius: var(--border-radius);
box-shadow: var(--shadow-elevated);
min-width: 180px;
padding: var(--spacing-xs) 0;
animation: contextMenuFadeIn 0.12s ease-out;
}
@keyframes contextMenuFadeIn {
from { opacity: 0; transform: scale(0.95); }
to { opacity: 1; transform: scale(1); }
}
.context-menu-item {
display: flex;
align-items: center;
gap: var(--spacing-sm);
padding: var(--spacing-sm) var(--spacing-md);
cursor: pointer;
color: var(--color-text-primary);
font-size: var(--font-size-body);
transition: background-color 0.1s ease;
}
.context-menu-item:hover {
background-color: var(--color-hover);
}
.context-menu-item i {
width: 16px;
text-align: center;
color: var(--color-text-secondary);
}
/* ============================================================================
Edit Metadata Modal
============================================================================ */
.edit-modal-content {
max-width: 520px;
}
.edit-section {
margin-bottom: var(--spacing-lg);
padding-bottom: var(--spacing-lg);
border-bottom: 1px solid var(--color-divider);
}
.edit-section:last-child {
border-bottom: none;
margin-bottom: 0;
padding-bottom: 0;
}
.edit-section h4 {
margin: 0 0 var(--spacing-md) 0;
font-size: var(--font-size-body);
font-weight: 600;
color: var(--color-text-primary);
display: flex;
align-items: center;
gap: var(--spacing-sm);
}
.edit-section h4 i {
color: var(--color-accent);
}
.form-group {
margin-bottom: var(--spacing-md);
}
.form-group label {
display: block;
margin-bottom: var(--spacing-xs);
font-size: var(--font-size-caption);
font-weight: 500;
color: var(--color-text-secondary);
}
.form-row {
display: grid;
grid-template-columns: 1fr 1fr;
gap: var(--spacing-md);
}
.field-error {
display: block;
margin-top: var(--spacing-xs);
font-size: var(--font-size-caption);
color: var(--color-error, #e74c3c);
}
.input-error {
border-color: var(--color-error, #e74c3c) !important;
}
.key-warning {
background: rgba(255, 193, 7, 0.1);
border: 1px solid rgba(255, 193, 7, 0.3);
border-radius: var(--border-radius);
padding: var(--spacing-sm) var(--spacing-md);
margin-top: var(--spacing-sm);
font-size: var(--font-size-caption);
color: var(--color-warning, #f39c12);
display: flex;
align-items: center;
gap: var(--spacing-sm);
}
/* NFO Diagnostics */
.nfo-diagnostics {
display: flex;
flex-direction: column;
gap: var(--spacing-sm);
}
.nfo-status-badge {
display: inline-block;
padding: 4px 12px;
border-radius: 12px;
font-size: var(--font-size-caption);
font-weight: 600;
width: fit-content;
}
.nfo-status-badge.nfo-complete {
background: rgba(46, 204, 113, 0.15);
color: var(--color-success, #2ecc71);
}
.nfo-status-badge.nfo-incomplete {
background: rgba(243, 156, 18, 0.15);
color: var(--color-warning, #f39c12);
}
.nfo-status-badge.nfo-missing {
background: rgba(231, 76, 60, 0.15);
color: var(--color-error, #e74c3c);
}
.missing-tags-list {
display: flex;
flex-wrap: wrap;
gap: var(--spacing-xs);
margin-top: var(--spacing-xs);
}
.missing-tag-chip {
display: inline-block;
padding: 2px 8px;
background: var(--color-background-subtle);
border: 1px solid var(--color-border);
border-radius: 4px;
font-size: var(--font-size-caption);
color: var(--color-text-secondary);
}
.nfo-all-good {
color: var(--color-success, #2ecc71);
font-size: var(--font-size-caption);
margin: 0;
}
.nfo-error {
color: var(--color-error, #e74c3c);
font-size: var(--font-size-caption);
margin: 0;
}
.repair-hint {
font-size: var(--font-size-caption);
color: var(--color-text-secondary);
margin: 0;
display: flex;
align-items: center;
gap: var(--spacing-xs);
}
.btn-repair {
align-self: flex-start;
margin-top: var(--spacing-sm);
}
.modal-footer {
display: flex;
justify-content: flex-end;
padding: var(--spacing-md) var(--spacing-lg);
border-top: 1px solid var(--color-border);
flex-shrink: 0;
}

View File

@@ -45,6 +45,7 @@ AniWorld.IndexApp = (function() {
AniWorld.Search.init();
AniWorld.ScanManager.init();
AniWorld.ConfigManager.init();
AniWorld.ContextMenu.init();
// Bind global events
bindGlobalEvents();

View File

@@ -0,0 +1,123 @@
/**
* AniWorld - Context Menu Component
*
* Right-click context menu for anime series cards.
* Provides quick access to edit metadata.
*
* Dependencies: ui-utils.js, edit-modal.js
*/
var AniWorld = window.AniWorld || {};
AniWorld.ContextMenu = (function() {
'use strict';
let menuElement = null;
let currentSeriesKey = null;
/**
* Initialize the context menu system.
* Attaches global dismissal listeners.
*/
function init() {
// Dismiss on click outside
document.addEventListener('click', function(e) {
if (menuElement && !menuElement.contains(e.target)) {
hide();
}
});
// Dismiss on Escape
document.addEventListener('keydown', function(e) {
if (e.key === 'Escape') {
hide();
}
});
// Dismiss on scroll or resize
window.addEventListener('scroll', hide, true);
window.addEventListener('resize', hide);
// Attach context menu via event delegation on the series grid
const grid = document.getElementById('series-grid');
if (grid) {
grid.addEventListener('contextmenu', function(e) {
const card = e.target.closest('.series-card');
if (card) {
e.preventDefault();
const key = card.getAttribute('data-key');
if (key) {
show(e, key);
}
}
});
}
}
/**
* Show context menu at cursor position.
* @param {MouseEvent} event - The contextmenu event
* @param {string} seriesKey - The series key to operate on
*/
function show(event, seriesKey) {
hide(); // Remove any existing menu first
currentSeriesKey = seriesKey;
menuElement = document.createElement('div');
menuElement.className = 'context-menu';
menuElement.innerHTML = `
<div class="context-menu-item" data-action="edit">
<i class="fa-solid fa-pen-to-square"></i>
<span>Edit Metadata</span>
</div>
`;
document.body.appendChild(menuElement);
// Position within viewport bounds
const x = event.clientX;
const y = event.clientY;
const menuRect = menuElement.getBoundingClientRect();
const viewportWidth = window.innerWidth;
const viewportHeight = window.innerHeight;
let posX = x;
let posY = y;
if (x + menuRect.width > viewportWidth) {
posX = viewportWidth - menuRect.width - 8;
}
if (y + menuRect.height > viewportHeight) {
posY = viewportHeight - menuRect.height - 8;
}
menuElement.style.left = posX + 'px';
menuElement.style.top = posY + 'px';
// Attach action handlers
menuElement.querySelector('[data-action="edit"]').addEventListener('click', function() {
hide();
if (AniWorld.EditModal) {
AniWorld.EditModal.open(currentSeriesKey);
}
});
}
/**
* Hide and remove the context menu from DOM.
*/
function hide() {
if (menuElement) {
menuElement.remove();
menuElement = null;
currentSeriesKey = null;
}
}
return {
init: init,
show: show,
hide: hide
};
})();

View File

@@ -0,0 +1,450 @@
/**
* AniWorld - Edit Modal Component
*
* Modal dialog for viewing/editing anime metadata (key, tmdb_id, tvdb_id)
* and NFO diagnostics with repair functionality.
*
* Dependencies: api-client.js, ui-utils.js
*/
var AniWorld = window.AniWorld || {};
AniWorld.EditModal = (function() {
'use strict';
let modalElement = null;
let originalData = null;
let currentKey = null;
/**
* Open the edit modal for a specific anime series.
* @param {string} seriesKey - The series key to edit
*/
async function open(seriesKey) {
currentKey = seriesKey;
modalElement = document.getElementById('edit-metadata-modal');
if (!modalElement) return;
// Show modal
modalElement.classList.remove('hidden');
// Reset form state
setLoading(true);
clearErrors();
hideKeyWarning();
try {
// Find series data from the local series list
const seriesData = findSeriesData(seriesKey);
originalData = {
key: seriesKey,
tmdb_id: seriesData ? seriesData.tmdb_id : null,
tvdb_id: seriesData ? seriesData.tvdb_id : null,
};
// Populate form fields
setFieldValue('edit-key', originalData.key);
setFieldValue('edit-tmdb-id', originalData.tmdb_id || '');
setFieldValue('edit-tvdb-id', originalData.tvdb_id || '');
// Load NFO diagnostics
await loadDiagnostics(seriesKey);
} catch (err) {
AniWorld.UI.showToast('Failed to load series data', 'error');
console.error('Edit modal load error:', err);
} finally {
setLoading(false);
}
// Attach event listeners
attachListeners();
}
/**
* Close the edit modal and reset state.
*/
function close() {
if (modalElement) {
modalElement.classList.add('hidden');
}
originalData = null;
currentKey = null;
detachListeners();
}
/**
* Save changed metadata to the backend.
*/
async function save() {
clearErrors();
const newKey = getFieldValue('edit-key').trim().toLowerCase();
const tmdbIdStr = getFieldValue('edit-tmdb-id').trim();
const tvdbIdStr = getFieldValue('edit-tvdb-id').trim();
// Validate key
if (!newKey) {
showFieldError('edit-key', 'Key cannot be empty');
return;
}
if (!/^[a-z0-9][a-z0-9-]*[a-z0-9]$|^[a-z0-9]$/.test(newKey)) {
showFieldError('edit-key', 'Key must contain only lowercase letters, numbers, and hyphens');
return;
}
// Validate IDs
const tmdbId = tmdbIdStr ? parseInt(tmdbIdStr, 10) : null;
const tvdbId = tvdbIdStr ? parseInt(tvdbIdStr, 10) : null;
if (tmdbIdStr && (isNaN(tmdbId) || tmdbId < 1)) {
showFieldError('edit-tmdb-id', 'TMDB ID must be a positive number');
return;
}
if (tvdbIdStr && (isNaN(tvdbId) || tvdbId < 1)) {
showFieldError('edit-tvdb-id', 'TVDB ID must be a positive number');
return;
}
// Check if key changed — show confirmation
if (newKey !== originalData.key) {
const confirmed = await AniWorld.UI.showConfirmModal(
'Rename Series Key',
`Changing the key from "${originalData.key}" to "${newKey}" will update the primary identifier. ` +
'This may affect provider linkage. Are you sure?'
);
if (!confirmed) return;
}
// Build update payload (only changed fields)
const payload = {};
if (newKey !== originalData.key) payload.key = newKey;
if (tmdbId !== originalData.tmdb_id) payload.tmdb_id = tmdbId;
if (tvdbId !== originalData.tvdb_id) payload.tvdb_id = tvdbId;
if (Object.keys(payload).length === 0) {
AniWorld.UI.showToast('No changes to save', 'info');
return;
}
// Send update
setSaveLoading(true);
try {
const response = await AniWorld.ApiClient.put(
'/api/anime/' + encodeURIComponent(currentKey),
payload
);
if (!response) return;
if (response.ok) {
const result = await response.json();
AniWorld.UI.showToast('Metadata updated successfully', 'success');
// Update local state
const oldKey = currentKey;
currentKey = result.key;
originalData = {
key: result.key,
tmdb_id: result.tmdb_id,
tvdb_id: result.tvdb_id,
};
// Update the card in the DOM
updateCardAfterSave(oldKey, result);
// Update repair button state
updateRepairButtonState();
} else if (response.status === 409) {
showFieldError('edit-key', 'A series with this key already exists');
} else if (response.status === 422) {
const err = await response.json();
AniWorld.UI.showToast('Validation error: ' + (err.detail || 'Invalid input'), 'error');
} else {
AniWorld.UI.showToast('Failed to update metadata', 'error');
}
} catch (err) {
AniWorld.UI.showToast('Connection error. Check your network.', 'error');
console.error('Save error:', err);
} finally {
setSaveLoading(false);
}
}
/**
* Trigger NFO repair for the current series.
*/
async function repairNfo() {
setRepairLoading(true);
try {
const response = await AniWorld.ApiClient.post(
'/api/nfo/' + encodeURIComponent(currentKey) + '/repair',
{}
);
if (!response) return;
if (response.ok) {
const result = await response.json();
AniWorld.UI.showToast(result.message, 'success');
// Refresh diagnostics
await loadDiagnostics(currentKey);
} else if (response.status === 400) {
const err = await response.json();
AniWorld.UI.showToast(err.detail || 'Cannot repair NFO', 'error');
} else {
AniWorld.UI.showToast('Failed to repair NFO', 'error');
}
} catch (err) {
AniWorld.UI.showToast('Connection error during repair', 'error');
console.error('Repair error:', err);
} finally {
setRepairLoading(false);
}
}
/**
* Load NFO diagnostics for the current series.
* @param {string} key - Series key
*/
async function loadDiagnostics(key) {
const container = document.getElementById('nfo-diagnostics-container');
if (!container) return;
try {
const response = await AniWorld.ApiClient.get(
'/api/nfo/' + encodeURIComponent(key) + '/diagnostics'
);
if (!response || !response.ok) {
container.innerHTML = '<p class="nfo-error">Failed to load NFO diagnostics</p>';
return;
}
const data = await response.json();
renderDiagnostics(data);
updateRepairButtonState();
} catch (err) {
container.innerHTML = '<p class="nfo-error">Error loading diagnostics</p>';
console.error('Diagnostics error:', err);
}
}
/**
* Render NFO diagnostics data into the modal.
* @param {Object} data - NfoDiagnosticsResponse
*/
function renderDiagnostics(data) {
const badge = document.getElementById('nfo-status-badge');
const tagsList = document.getElementById('nfo-missing-tags');
if (badge) {
if (!data.has_nfo) {
badge.className = 'nfo-status-badge nfo-missing';
badge.textContent = 'No NFO File';
} else if (data.missing_tags.length === 0) {
badge.className = 'nfo-status-badge nfo-complete';
badge.textContent = 'Complete';
} else {
badge.className = 'nfo-status-badge nfo-incomplete';
badge.textContent = data.missing_tags.length + ' Missing';
}
}
if (tagsList) {
if (data.missing_tags.length === 0) {
tagsList.innerHTML = '<p class="nfo-all-good">All required tags present</p>';
} else {
tagsList.innerHTML = data.missing_tags.map(function(tag) {
return '<span class="missing-tag-chip">' + escapeHtml(tag) + '</span>';
}).join('');
}
}
}
/**
* Update repair button disabled state based on tmdb_id field.
*/
function updateRepairButtonState() {
const btn = document.getElementById('btn-repair-nfo');
const hint = document.getElementById('repair-hint');
const tmdbValue = getFieldValue('edit-tmdb-id').trim();
if (btn) {
// Enable repair even without tmdb_id — the service can search by name
btn.disabled = false;
}
if (hint) {
hint.style.display = tmdbValue ? 'none' : 'block';
}
}
// ---- Helpers ----
function findSeriesData(key) {
// Access the series data from the series manager if available
if (AniWorld.SeriesManager && AniWorld.SeriesManager.getSeriesData) {
const allSeries = AniWorld.SeriesManager.getSeriesData();
if (allSeries) {
return allSeries.find(function(s) { return s.key === key; });
}
}
return null;
}
function updateCardAfterSave(oldKey, result) {
const card = document.querySelector('[data-series-id="' + oldKey + '"]');
if (card) {
card.setAttribute('data-key', result.key);
card.setAttribute('data-series-id', result.key);
// Update checkbox data-key
const checkbox = card.querySelector('.series-checkbox');
if (checkbox) {
checkbox.setAttribute('data-key', result.key);
}
}
// Update local series data array
if (AniWorld.SeriesManager && AniWorld.SeriesManager.updateSeriesKey) {
AniWorld.SeriesManager.updateSeriesKey(oldKey, result.key);
}
}
function setFieldValue(id, value) {
const el = document.getElementById(id);
if (el) el.value = value !== null && value !== undefined ? value : '';
}
function getFieldValue(id) {
const el = document.getElementById(id);
return el ? el.value : '';
}
function showFieldError(fieldId, message) {
const el = document.getElementById(fieldId);
if (el) {
const errorEl = el.parentElement.querySelector('.field-error');
if (errorEl) {
errorEl.textContent = message;
errorEl.style.display = 'block';
}
el.classList.add('input-error');
}
}
function clearErrors() {
if (!modalElement) return;
modalElement.querySelectorAll('.field-error').forEach(function(el) {
el.style.display = 'none';
el.textContent = '';
});
modalElement.querySelectorAll('.input-error').forEach(function(el) {
el.classList.remove('input-error');
});
}
function hideKeyWarning() {
const warning = document.getElementById('key-change-warning');
if (warning) warning.style.display = 'none';
}
function setLoading(loading) {
const form = document.getElementById('edit-metadata-form');
if (form) {
form.style.opacity = loading ? '0.5' : '1';
form.style.pointerEvents = loading ? 'none' : 'auto';
}
}
function setSaveLoading(loading) {
const btn = document.getElementById('btn-save-metadata');
if (btn) {
btn.disabled = loading;
btn.innerHTML = loading
? '<i class="fa-solid fa-spinner fa-spin"></i> Saving...'
: '<i class="fa-solid fa-floppy-disk"></i> Save';
}
}
function setRepairLoading(loading) {
const btn = document.getElementById('btn-repair-nfo');
if (btn) {
btn.disabled = loading;
btn.innerHTML = loading
? '<i class="fa-solid fa-spinner fa-spin"></i> Repairing...'
: '<i class="fa-solid fa-wrench"></i> Repair NFO';
}
}
function escapeHtml(str) {
var div = document.createElement('div');
div.textContent = str;
return div.innerHTML;
}
// Event listener management
let listeners = [];
function attachListeners() {
detachListeners();
const saveBtn = document.getElementById('btn-save-metadata');
const cancelBtn = document.getElementById('btn-cancel-metadata');
const repairBtn = document.getElementById('btn-repair-nfo');
const overlay = modalElement ? modalElement.querySelector('.modal-overlay') : null;
const keyInput = document.getElementById('edit-key');
if (saveBtn) {
var saveFn = function() { save(); };
saveBtn.addEventListener('click', saveFn);
listeners.push({ el: saveBtn, event: 'click', fn: saveFn });
}
if (cancelBtn) {
var cancelFn = function() { close(); };
cancelBtn.addEventListener('click', cancelFn);
listeners.push({ el: cancelBtn, event: 'click', fn: cancelFn });
}
if (repairBtn) {
var repairFn = function() { repairNfo(); };
repairBtn.addEventListener('click', repairFn);
listeners.push({ el: repairBtn, event: 'click', fn: repairFn });
}
if (overlay) {
var overlayFn = function() { close(); };
overlay.addEventListener('click', overlayFn);
listeners.push({ el: overlay, event: 'click', fn: overlayFn });
}
if (keyInput) {
var keyFn = function() {
var warning = document.getElementById('key-change-warning');
if (warning) {
warning.style.display = keyInput.value !== originalData.key ? 'block' : 'none';
}
};
keyInput.addEventListener('input', keyFn);
listeners.push({ el: keyInput, event: 'input', fn: keyFn });
}
}
function detachListeners() {
listeners.forEach(function(l) {
l.el.removeEventListener(l.event, l.fn);
});
listeners = [];
}
return {
open: open,
close: close,
save: save,
repairNfo: repairNfo
};
})();

View File

@@ -203,6 +203,17 @@ AniWorld.SeriesManager = (function() {
function applyFiltersAndSort() {
let filtered = seriesData.slice();
// Apply client-side filter so that real-time WebSocket updates
// (e.g. an episode being marked downloaded) are immediately
// reflected without a full server reload.
if (filterMode === 'missing_episodes') {
filtered = filtered.filter(function(s) {
return s.missing_episodes > 0;
});
}
// 'no_episodes' filter state is maintained server-side;
// don't try to replicate it client-side here.
// Sort based on the current sorting mode
filtered.sort(function(a, b) {
if (sortAlphabetical) {
@@ -233,8 +244,12 @@ AniWorld.SeriesManager = (function() {
*/
function renderSeries() {
const grid = document.getElementById('series-grid');
const dataToRender = filteredSeriesData.length > 0 ? filteredSeriesData :
(seriesData.length > 0 ? seriesData : []);
// Always use filteredSeriesData — applyFiltersAndSort() is always
// called before renderSeries(), so filteredSeriesData is current.
// The old fallback to seriesData was incorrect: when a filter is
// active and filteredSeriesData is empty it must show the empty-state
// message, not fall through to unfiltered seriesData.
const dataToRender = filteredSeriesData;
if (dataToRender.length === 0) {
let message;
@@ -377,6 +392,22 @@ AniWorld.SeriesManager = (function() {
return seriesData;
}
/**
* Update a series key in the local data arrays after rename.
* @param {string} oldKey - The previous key
* @param {string} newKey - The new key
*/
function updateSeriesKey(oldKey, newKey) {
if (seriesData) {
var s = seriesData.find(function(item) { return item.key === oldKey; });
if (s) s.key = newKey;
}
if (filteredSeriesData) {
var fs = filteredSeriesData.find(function(item) { return item.key === oldKey; });
if (fs) fs.key = newKey;
}
}
/**
* Get filtered series data
* @returns {Array} Filtered series data array
@@ -528,6 +559,7 @@ AniWorld.SeriesManager = (function() {
getFilteredSeriesData: getFilteredSeriesData,
findByKey: findByKey,
updateSeriesLoadingStatus: updateSeriesLoadingStatus,
updateSingleSeries: updateSingleSeries
updateSingleSeries: updateSingleSeries,
updateSeriesKey: updateSeriesKey
};
})();

View File

@@ -640,6 +640,80 @@
</div>
</div>
<!-- Edit Metadata Modal -->
<div id="edit-metadata-modal" class="modal hidden">
<div class="modal-overlay"></div>
<div class="modal-content edit-modal-content">
<div class="modal-header">
<h3>Edit Metadata</h3>
<button id="btn-cancel-metadata" class="btn btn-icon">
<i class="fas fa-times"></i>
</button>
</div>
<div class="modal-body">
<form id="edit-metadata-form" onsubmit="return false;">
<!-- Identity Section -->
<div class="edit-section">
<h4><i class="fa-solid fa-key"></i> Identity</h4>
<div class="form-group">
<label for="edit-key">Series Key</label>
<input type="text" id="edit-key" class="input-field"
placeholder="e.g. attack-on-titan"
pattern="[a-z0-9][a-z0-9-]*[a-z0-9]">
<span class="field-error" style="display:none;"></span>
</div>
<div id="key-change-warning" class="key-warning" style="display:none;">
<i class="fa-solid fa-triangle-exclamation"></i>
Changing the key will update the primary identifier. This may affect provider linkage.
</div>
</div>
<!-- External IDs Section -->
<div class="edit-section">
<h4><i class="fa-solid fa-database"></i> External IDs</h4>
<div class="form-row">
<div class="form-group">
<label for="edit-tmdb-id">TMDB ID</label>
<input type="number" id="edit-tmdb-id" class="input-field"
placeholder="e.g. 1429" min="1">
<span class="field-error" style="display:none;"></span>
</div>
<div class="form-group">
<label for="edit-tvdb-id">TVDB ID</label>
<input type="number" id="edit-tvdb-id" class="input-field"
placeholder="e.g. 267440" min="1">
<span class="field-error" style="display:none;"></span>
</div>
</div>
</div>
<!-- NFO Status Section -->
<div class="edit-section">
<h4><i class="fa-solid fa-file-lines"></i> NFO Status</h4>
<div class="nfo-diagnostics">
<div id="nfo-status-badge" class="nfo-status-badge">Loading...</div>
<div id="nfo-diagnostics-container">
<div id="nfo-missing-tags" class="missing-tags-list"></div>
</div>
<p id="repair-hint" class="repair-hint" style="display:none;">
<i class="fa-solid fa-circle-info"></i>
No TMDB ID set. Repair will search TMDB by series name.
</p>
<button type="button" id="btn-repair-nfo" class="btn btn-secondary btn-repair">
<i class="fa-solid fa-wrench"></i> Repair NFO
</button>
</div>
</div>
</form>
</div>
<div class="modal-footer">
<button type="button" id="btn-save-metadata" class="btn btn-primary">
<i class="fa-solid fa-floppy-disk"></i> Save
</button>
</div>
</div>
</div>
<!-- Toast notifications -->
<div id="toast-container" class="toast-container"></div>
</div>
@@ -665,6 +739,8 @@
<script src="/static/js/user_preferences.js?v={{ static_v }}"></script>
<!-- Index Page Modules -->
<script src="/static/js/index/context-menu.js?v={{ static_v }}"></script>
<script src="/static/js/index/edit-modal.js?v={{ static_v }}"></script>
<script src="/static/js/index/series-manager.js?v={{ static_v }}"></script>
<script src="/static/js/index/selection-manager.js?v={{ static_v }}"></script>
<script src="/static/js/index/search.js?v={{ static_v }}"></script>

View File

@@ -0,0 +1,255 @@
"""Tests for anime metadata edit (PUT /api/anime/{anime_key}) endpoint."""
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from httpx import ASGITransport, AsyncClient
from src.server.fastapi_app import app
from src.server.services.auth_service import auth_service
@pytest.fixture
def reset_auth():
"""Reset auth state before each test."""
auth_service._hash = None
auth_service._failed = {}
@pytest.fixture
async def client():
"""Create async test client."""
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as ac:
yield ac
@pytest.fixture
async def authenticated_client(client):
"""Get authenticated client with Bearer token."""
# Setup auth
await client.post("/api/auth/setup", json={"master_password": "TestPass123!"})
response = await client.post(
"/api/auth/login", json={"password": "TestPass123!"}
)
token = response.json()["access_token"]
client.headers["Authorization"] = f"Bearer {token}"
return client
@pytest.fixture
def mock_db_session():
"""Create a mock async database session."""
session = AsyncMock()
session.commit = AsyncMock()
session.flush = AsyncMock()
session.refresh = AsyncMock()
return session
@pytest.fixture
def mock_series_in_db():
"""Create a mock AnimeSeries DB record."""
series = MagicMock()
series.id = 1
series.key = "test-anime"
series.name = "Test Anime"
series.tmdb_id = 1234
series.tvdb_id = 5678
series.folder = "Test Anime (2023)"
return series
@pytest.fixture
def override_db_dependency(mock_db_session):
"""Override database session dependency."""
from src.server.utils.dependencies import get_database_session
app.dependency_overrides[get_database_session] = lambda: mock_db_session
yield mock_db_session
app.dependency_overrides.pop(get_database_session, None)
class TestUpdateAnimeMetadata:
"""Tests for PUT /api/anime/{anime_key}."""
@pytest.mark.asyncio
async def test_update_tmdb_id_success(
self, reset_auth, authenticated_client, override_db_dependency, mock_series_in_db
):
"""Test successful tmdb_id update."""
with patch(
"src.server.api.anime.AnimeSeriesService.get_by_key",
new_callable=AsyncMock,
return_value=mock_series_in_db,
), patch(
"src.server.api.anime.AnimeSeriesService.update",
new_callable=AsyncMock,
) as mock_update:
mock_series_in_db.tmdb_id = 9999
mock_update.return_value = mock_series_in_db
response = await authenticated_client.put(
"/api/anime/test-anime",
json={"tmdb_id": 9999},
)
assert response.status_code == 200
data = response.json()
assert data["tmdb_id"] == 9999
assert data["message"] == "Metadata updated successfully"
@pytest.mark.asyncio
async def test_update_tvdb_id_success(
self, reset_auth, authenticated_client, override_db_dependency, mock_series_in_db
):
"""Test successful tvdb_id update."""
with patch(
"src.server.api.anime.AnimeSeriesService.get_by_key",
new_callable=AsyncMock,
return_value=mock_series_in_db,
), patch(
"src.server.api.anime.AnimeSeriesService.update",
new_callable=AsyncMock,
) as mock_update:
mock_series_in_db.tvdb_id = 7777
mock_update.return_value = mock_series_in_db
response = await authenticated_client.put(
"/api/anime/test-anime",
json={"tvdb_id": 7777},
)
assert response.status_code == 200
data = response.json()
assert data["tvdb_id"] == 7777
@pytest.mark.asyncio
async def test_update_key_success(
self, reset_auth, authenticated_client, override_db_dependency, mock_series_in_db
):
"""Test successful key rename."""
with patch(
"src.server.api.anime.AnimeSeriesService.get_by_key",
new_callable=AsyncMock,
) as mock_get:
# First call finds the series, second call checks uniqueness (returns None)
mock_get.side_effect = [mock_series_in_db, None]
mock_series_in_db.key = "new-anime-key"
with patch(
"src.server.api.anime.AnimeSeriesService.update",
new_callable=AsyncMock,
return_value=mock_series_in_db,
):
response = await authenticated_client.put(
"/api/anime/test-anime",
json={"key": "new-anime-key"},
)
assert response.status_code == 200
data = response.json()
assert data["key"] == "new-anime-key"
@pytest.mark.asyncio
async def test_update_key_conflict_409(
self, reset_auth, authenticated_client, override_db_dependency, mock_series_in_db
):
"""Test key rename conflict returns 409."""
existing_series = MagicMock()
existing_series.key = "existing-key"
with patch(
"src.server.api.anime.AnimeSeriesService.get_by_key",
new_callable=AsyncMock,
) as mock_get:
# First call finds original series, second call finds conflict
mock_get.side_effect = [mock_series_in_db, existing_series]
response = await authenticated_client.put(
"/api/anime/test-anime",
json={"key": "existing-key"},
)
assert response.status_code == 409
assert "already exists" in response.json()["detail"]
@pytest.mark.asyncio
async def test_update_key_invalid_chars_422(
self, reset_auth, authenticated_client, override_db_dependency
):
"""Test key with invalid characters returns 422."""
response = await authenticated_client.put(
"/api/anime/test-anime",
json={"key": "Invalid Key With Spaces!"},
)
assert response.status_code == 422
@pytest.mark.asyncio
async def test_update_key_empty_422(
self, reset_auth, authenticated_client, override_db_dependency
):
"""Test empty key returns 422."""
response = await authenticated_client.put(
"/api/anime/test-anime",
json={"key": ""},
)
assert response.status_code == 422
@pytest.mark.asyncio
async def test_update_unauthenticated_401(self, reset_auth, client):
"""Test unauthenticated access returns 401."""
response = await client.put(
"/api/anime/test-anime",
json={"tmdb_id": 1234},
)
assert response.status_code == 401
@pytest.mark.asyncio
async def test_update_nonexistent_anime_404(
self, reset_auth, authenticated_client, override_db_dependency
):
"""Test update of non-existent anime returns 404."""
with patch(
"src.server.api.anime.AnimeSeriesService.get_by_key",
new_callable=AsyncMock,
return_value=None,
):
response = await authenticated_client.put(
"/api/anime/nonexistent-key",
json={"tmdb_id": 1234},
)
assert response.status_code == 404
@pytest.mark.asyncio
async def test_update_no_changes(
self, reset_auth, authenticated_client, override_db_dependency, mock_series_in_db
):
"""Test sending empty body returns no-op response."""
with patch(
"src.server.api.anime.AnimeSeriesService.get_by_key",
new_callable=AsyncMock,
return_value=mock_series_in_db,
):
response = await authenticated_client.put(
"/api/anime/test-anime",
json={},
)
assert response.status_code == 200
assert response.json()["message"] == "No changes"
@pytest.mark.asyncio
async def test_update_negative_tmdb_id_422(
self, reset_auth, authenticated_client, override_db_dependency
):
"""Test negative TMDB ID returns 422."""
response = await authenticated_client.put(
"/api/anime/test-anime",
json={"tmdb_id": -5},
)
assert response.status_code == 422

View File

@@ -334,6 +334,25 @@ async def test_add_series_sanitizes_folder_name(authenticated_client):
assert "?" not in folder
@pytest.mark.asyncio
async def test_add_series_does_not_duplicate_year(authenticated_client):
"""Test that add_series doesn't duplicate year when name already contains it."""
response = await authenticated_client.post(
"/api/anime/add",
json={
"link": "https://aniworld.to/anime/stream/eighty-six",
"name": "86 Eighty Six (2021)"
}
)
assert response.status_code == 202
data = response.json()
# Folder should contain year only once
folder = data["folder"]
assert folder.count("(2021)") == 1
@pytest.mark.asyncio
async def test_add_series_returns_missing_episodes(authenticated_client):
"""Test that add_series returns loading progress info."""

View File

@@ -2,7 +2,7 @@
import tempfile
from pathlib import Path
from unittest.mock import patch
from unittest.mock import AsyncMock, patch
import pytest
from httpx import ASGITransport, AsyncClient
@@ -207,3 +207,46 @@ async def test_tmdb_validation_endpoint_exists(authenticated_client):
assert "message" in data
assert data["valid"] is False # Empty key should be invalid
assert "required" in data["message"].lower()
@pytest.mark.asyncio
async def test_update_config_with_anime_directory_starts_scheduler(
authenticated_client, mock_config_service
):
"""PUT /api/config with anime_directory syncs and starts scheduler."""
mock_scheduler = AsyncMock()
with patch("src.server.services.scheduler.scheduler_service.get_scheduler_service") as mock_sched_fn:
mock_sched_fn.return_value = mock_scheduler
with patch("src.config.settings.settings") as mock_settings:
mock_settings.anime_directory = None
resp = await authenticated_client.put(
"/api/config",
json={"other": {"anime_directory": "/data/anime"}},
)
assert resp.status_code == 200
mock_scheduler.ensure_started.assert_called_once()
@pytest.mark.asyncio
async def test_update_config_without_anime_directory_does_not_start_scheduler(
authenticated_client, mock_config_service
):
"""PUT /api/config without new anime_directory does not call scheduler.ensure_started()."""
mock_scheduler = AsyncMock()
with patch("src.server.services.scheduler.scheduler_service.get_scheduler_service") as mock_sched_fn:
mock_sched_fn.return_value = mock_scheduler
with patch("src.config.settings.settings") as mock_settings:
mock_settings.anime_directory = "/already/set"
resp = await authenticated_client.put(
"/api/config", json={"other": {}}
)
assert resp.status_code == 200
mock_scheduler.ensure_started.assert_not_called()

View File

@@ -0,0 +1,317 @@
"""Tests for NFO diagnostics and repair API endpoints."""
from unittest.mock import AsyncMock, Mock, patch
import pytest
from httpx import ASGITransport, AsyncClient
from src.server.fastapi_app import app
from src.server.services.auth_service import auth_service
@pytest.fixture(autouse=True)
def reset_auth():
"""Reset authentication state before each test."""
original_hash = auth_service._hash
auth_service._hash = None
auth_service._failed.clear()
yield
auth_service._hash = original_hash
auth_service._failed.clear()
@pytest.fixture
async def client():
"""Create an async test client."""
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as ac:
yield ac
@pytest.fixture
async def authenticated_client(client):
"""Create an authenticated test client with token."""
await client.post(
"/api/auth/setup",
json={"master_password": "TestPassword123!"}
)
response = await client.post(
"/api/auth/login",
json={"password": "TestPassword123!"}
)
token = response.json()["access_token"]
client.headers.update({"Authorization": f"Bearer {token}"})
yield client
@pytest.fixture
def mock_series_app():
"""Create mock series app with one test series."""
app_mock = Mock()
serie = Mock()
serie.key = "test-anime"
serie.folder = "Test Anime (2024)"
serie.name = "Test Anime"
serie.ensure_folder_with_year = Mock(return_value="Test Anime (2024)")
list_manager = Mock()
list_manager.GetList = Mock(return_value=[serie])
app_mock.list = list_manager
return app_mock
@pytest.fixture
def mock_nfo_service():
"""Create mock NFO service."""
service = Mock()
service.check_nfo_exists = AsyncMock(return_value=False)
service.create_tvshow_nfo = AsyncMock(return_value="/path/to/tvshow.nfo")
service.update_tvshow_nfo = AsyncMock(return_value="/path/to/tvshow.nfo")
return service
@pytest.fixture
def override_dependencies(mock_series_app, mock_nfo_service):
"""Override dependencies for NFO tests."""
from src.server.api.nfo import get_nfo_service
from src.server.utils.dependencies import get_series_app
app.dependency_overrides[get_series_app] = lambda: mock_series_app
app.dependency_overrides[get_nfo_service] = lambda: mock_nfo_service
yield
if get_series_app in app.dependency_overrides:
del app.dependency_overrides[get_series_app]
if get_nfo_service in app.dependency_overrides:
del app.dependency_overrides[get_nfo_service]
class TestNfoDiagnostics:
"""Tests for GET /api/nfo/{serie_key}/diagnostics."""
@pytest.mark.asyncio
async def test_diagnostics_complete_nfo(
self, authenticated_client, override_dependencies
):
"""Test diagnostics with complete NFO returns no missing tags."""
with patch(
"src.server.api.nfo.Path.exists", return_value=True
), patch(
"src.server.api.nfo.find_missing_tags", return_value=[]
):
response = await authenticated_client.get(
"/api/nfo/test-anime/diagnostics"
)
assert response.status_code == 200
data = response.json()
assert data["has_nfo"] is True
assert data["missing_tags"] == []
assert len(data["required_tags"]) > 0
@pytest.mark.asyncio
async def test_diagnostics_missing_tags(
self, authenticated_client, override_dependencies
):
"""Test diagnostics with missing tags returns them."""
with patch(
"src.server.api.nfo.Path.exists", return_value=True
), patch(
"src.server.api.nfo.find_missing_tags",
return_value=["plot", "genre", "actor/name"],
):
response = await authenticated_client.get(
"/api/nfo/test-anime/diagnostics"
)
assert response.status_code == 200
data = response.json()
assert data["has_nfo"] is True
assert "plot" in data["missing_tags"]
assert "genre" in data["missing_tags"]
assert len(data["missing_tags"]) == 3
@pytest.mark.asyncio
async def test_diagnostics_no_nfo_file(
self, authenticated_client, override_dependencies
):
"""Test diagnostics when no NFO exists returns all tags as missing."""
with patch("src.server.api.nfo.Path") as MockPath:
# Make nfo_path.exists() return False
mock_path_instance = Mock()
mock_path_instance.exists.return_value = False
mock_path_instance.__truediv__ = Mock(return_value=mock_path_instance)
MockPath.return_value = mock_path_instance
response = await authenticated_client.get(
"/api/nfo/test-anime/diagnostics"
)
assert response.status_code == 200
data = response.json()
assert data["has_nfo"] is False
assert len(data["missing_tags"]) > 0
# All required tags should be listed as missing
assert data["missing_tags"] == data["required_tags"]
@pytest.mark.asyncio
async def test_diagnostics_nonexistent_series_404(
self, authenticated_client, override_dependencies, mock_series_app
):
"""Test diagnostics for non-existent series returns 404."""
# Override to return empty list
mock_series_app.list.GetList.return_value = []
response = await authenticated_client.get(
"/api/nfo/nonexistent-key/diagnostics"
)
assert response.status_code == 404
@pytest.mark.asyncio
async def test_diagnostics_unauthenticated_401(self, client):
"""Test diagnostics requires authentication."""
response = await client.get("/api/nfo/test-anime/diagnostics")
# May return 401 or 503 depending on NFO service availability
assert response.status_code in (401, 503)
class TestNfoRepair:
"""Tests for POST /api/nfo/{serie_key}/repair."""
@pytest.mark.asyncio
async def test_repair_success(
self, authenticated_client, override_dependencies
):
"""Test successful NFO repair."""
with patch("src.server.api.nfo.Path") as MockPath:
mock_path = Mock()
mock_path.exists.return_value = True
mock_path.__truediv__ = Mock(return_value=mock_path)
MockPath.return_value = mock_path
with patch(
"src.server.api.nfo.find_missing_tags",
return_value=["plot", "genre"],
), patch(
"src.server.api.nfo.NfoRepairService"
) as MockRepairService:
mock_instance = Mock()
mock_instance.repair_series = AsyncMock(return_value=True)
MockRepairService.return_value = mock_instance
response = await authenticated_client.post(
"/api/nfo/test-anime/repair", json={}
)
assert response.status_code == 200
data = response.json()
assert data["success"] is True
assert "2" in data["message"] # "Fixed 2 missing tags"
assert "plot" in data["repaired_tags"]
assert "genre" in data["repaired_tags"]
@pytest.mark.asyncio
async def test_repair_already_complete(
self, authenticated_client, override_dependencies
):
"""Test repair when NFO is already complete."""
with patch("src.server.api.nfo.Path") as MockPath:
mock_path = Mock()
mock_path.exists.return_value = True
mock_path.__truediv__ = Mock(return_value=mock_path)
MockPath.return_value = mock_path
with patch(
"src.server.api.nfo.find_missing_tags", return_value=[]
), patch(
"src.server.api.nfo.NfoRepairService"
) as MockRepairService:
mock_instance = Mock()
mock_instance.repair_series = AsyncMock(return_value=False)
MockRepairService.return_value = mock_instance
response = await authenticated_client.post(
"/api/nfo/test-anime/repair", json={}
)
assert response.status_code == 200
data = response.json()
assert data["success"] is True
assert "already complete" in data["message"]
@pytest.mark.asyncio
async def test_repair_creates_new_nfo(
self, authenticated_client, override_dependencies, mock_nfo_service
):
"""Test repair when no NFO exists creates a new one."""
with patch("src.server.api.nfo.Path") as MockPath:
mock_path = Mock()
mock_path.exists.return_value = False
mock_path.__truediv__ = Mock(return_value=mock_path)
MockPath.return_value = mock_path
with patch(
"src.server.api.nfo.REQUIRED_TAGS",
{"./title": "title", "./plot": "plot"},
):
response = await authenticated_client.post(
"/api/nfo/test-anime/repair", json={}
)
assert response.status_code == 200
data = response.json()
assert data["success"] is True
mock_nfo_service.create_tvshow_nfo.assert_awaited_once()
@pytest.mark.asyncio
async def test_repair_nonexistent_series_404(
self, authenticated_client, override_dependencies, mock_series_app
):
"""Test repair for non-existent series returns 404."""
mock_series_app.list.GetList.return_value = []
response = await authenticated_client.post(
"/api/nfo/nonexistent-key/repair", json={}
)
assert response.status_code == 404
@pytest.mark.asyncio
async def test_repair_unauthenticated_401(self, client):
"""Test repair requires authentication."""
response = await client.post("/api/nfo/test-anime/repair", json={})
assert response.status_code in (401, 503)
@pytest.mark.asyncio
async def test_repair_tmdb_api_failure(
self, authenticated_client, override_dependencies
):
"""Test repair handles TMDB API failure gracefully."""
from src.core.services.tmdb_client import TMDBAPIError
with patch("src.server.api.nfo.Path") as MockPath:
mock_path = Mock()
mock_path.exists.return_value = True
mock_path.__truediv__ = Mock(return_value=mock_path)
MockPath.return_value = mock_path
with patch(
"src.server.api.nfo.find_missing_tags",
return_value=["plot"],
), patch(
"src.server.api.nfo.NfoRepairService"
) as MockRepairService:
mock_instance = Mock()
mock_instance.repair_series = AsyncMock(
side_effect=TMDBAPIError("No TMDB ID found")
)
MockRepairService.return_value = mock_instance
response = await authenticated_client.post(
"/api/nfo/test-anime/repair", json={}
)
assert response.status_code == 400
assert "Cannot repair NFO" in response.json()["detail"]

View File

@@ -1,5 +1,6 @@
"""Pytest configuration and shared fixtures for all tests."""
import logging
from unittest.mock import Mock
import pytest
@@ -149,3 +150,44 @@ def mock_series_app_download(monkeypatch):
yield
@pytest.fixture(autouse=True)
def reset_logging_state():
"""Reset logging handlers and propagate flags before and after each test.
Tests that call setup_logging() or logging.config.dictConfig() may leave
FileHandlers and propagate=False on various loggers. This fixture clears
handlers and resets propagate for all relevant loggers before/after tests.
"""
# All loggers that might have handlers or propagate changes from test setup
logger_names = (
"aniworld", "uvicorn", "uvicorn.access", "uvicorn.error",
"watchfiles.main"
)
def clear_logger_state(logger_name):
logger = logging.getLogger(logger_name)
for handler in logger.handlers[:]:
logger.removeHandler(handler)
handler.close()
# Reset propagate to default (True) for child loggers
# Root logger propagate is always True by default
if logger_name != "root":
logger.propagate = True
# Clear state BEFORE test
for name in logger_names:
clear_logger_state(name)
yield
# Clear state AFTER test
for name in logger_names:
clear_logger_state(name)
# Also clear root handlers
root = logging.getLogger()
for handler in root.handlers[:]:
root.removeHandler(handler)
handler.close()

View File

@@ -0,0 +1,115 @@
"""Frontend tests for the edit metadata modal HTML structure."""
from unittest.mock import AsyncMock, Mock, patch
import pytest
from httpx import ASGITransport, AsyncClient
from src.server.fastapi_app import app
from src.server.services.auth_service import auth_service
@pytest.fixture(autouse=True)
def reset_auth():
"""Reset authentication state before each test."""
original_hash = auth_service._hash
auth_service._hash = None
auth_service._failed.clear()
yield
auth_service._hash = original_hash
auth_service._failed.clear()
@pytest.fixture
async def client():
"""Create an async test client."""
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as ac:
yield ac
@pytest.fixture
async def authenticated_client(client):
"""Create authenticated client to access index page."""
await client.post(
"/api/auth/setup",
json={"master_password": "TestPassword123!"}
)
response = await client.post(
"/api/auth/login",
json={"password": "TestPassword123!"}
)
token = response.json()["access_token"]
client.headers.update({"Authorization": f"Bearer {token}"})
# Set cookie for page access
client.cookies.set("access_token", token)
yield client
class TestEditModalHtmlPresence:
"""Tests verifying edit modal HTML elements exist in index page."""
@pytest.mark.asyncio
async def test_index_page_contains_edit_modal(self, authenticated_client):
"""Verify #edit-metadata-modal exists in rendered index page."""
response = await authenticated_client.get("/")
# Page may redirect or require different auth for HTML pages
if response.status_code == 200:
html = response.text
assert 'id="edit-metadata-modal"' in html
@pytest.mark.asyncio
async def test_index_page_loads_context_menu_script(self, authenticated_client):
"""Verify context-menu.js script tag is present."""
response = await authenticated_client.get("/")
if response.status_code == 200:
html = response.text
assert "context-menu.js" in html
@pytest.mark.asyncio
async def test_index_page_loads_edit_modal_script(self, authenticated_client):
"""Verify edit-modal.js script tag is present."""
response = await authenticated_client.get("/")
if response.status_code == 200:
html = response.text
assert "edit-modal.js" in html
@pytest.mark.asyncio
async def test_modal_form_fields_present(self, authenticated_client):
"""Verify key, tmdb_id, tvdb_id input fields exist in modal."""
response = await authenticated_client.get("/")
if response.status_code == 200:
html = response.text
assert 'id="edit-key"' in html
assert 'id="edit-tmdb-id"' in html
assert 'id="edit-tvdb-id"' in html
@pytest.mark.asyncio
async def test_nfo_repair_button_present(self, authenticated_client):
"""Verify repair NFO button exists in modal."""
response = await authenticated_client.get("/")
if response.status_code == 200:
html = response.text
assert 'id="btn-repair-nfo"' in html
@pytest.mark.asyncio
async def test_save_button_present(self, authenticated_client):
"""Verify save button exists in modal."""
response = await authenticated_client.get("/")
if response.status_code == 200:
html = response.text
assert 'id="btn-save-metadata"' in html
@pytest.mark.asyncio
async def test_modal_starts_hidden(self, authenticated_client):
"""Verify modal has hidden class by default."""
response = await authenticated_client.get("/")
if response.status_code == 200:
html = response.text
assert 'id="edit-metadata-modal" class="modal hidden"' in html

View File

@@ -0,0 +1,314 @@
"""Integration test: add an anime and verify NFO contains required information.
This test adds 'Sacrificial Princess And The King Of Beasts' and verifies
that the generated tvshow.nfo contains all required tags including plot,
outline, title, year, etc.
"""
from pathlib import Path
from unittest.mock import AsyncMock, patch
import pytest
from lxml import etree
from src.core.services.nfo_service import NFOService
# ---------------------------------------------------------------------------
# Mock TMDB data for "Sacrificial Princess And The King Of Beasts"
# ---------------------------------------------------------------------------
MOCK_TMDB_DATA = {
"id": 222093,
"name": "Sacrificial Princess and the King of Beasts",
"original_name": "贄姫と獣の王",
"overview": (
"A girl is offered as a sacrifice to a beastly king, "
"but instead of being eaten, she becomes his bride."
),
"tagline": "A tale of love between a sacrifice and a beast king.",
"first_air_date": "2023-04-20",
"vote_average": 7.5,
"vote_count": 150,
"status": "Ended",
"episode_run_time": [24],
"genres": [
{"id": 16, "name": "Animation"},
{"id": 10749, "name": "Romance"},
],
"networks": [{"id": 1, "name": "TBS"}],
"origin_country": ["JP"],
"poster_path": "/poster.jpg",
"backdrop_path": "/backdrop.jpg",
"external_ids": {"imdb_id": "tt19896734", "tvdb_id": 421737},
"credits": {
"cast": [
{
"id": 1,
"name": "Test Actor",
"character": "Sariphi",
"profile_path": "/actor.jpg",
}
]
},
"images": {"logos": [{"file_path": "/logo.png"}]},
"seasons": [{"season_number": 1, "name": "Season 1"}],
}
MOCK_CONTENT_RATINGS = {
"results": [
{"iso_3166_1": "DE", "rating": "12"},
{"iso_3166_1": "US", "rating": "TV-14"},
]
}
# ---------------------------------------------------------------------------
# Required XML tags that must exist and be non-empty after creation
# ---------------------------------------------------------------------------
REQUIRED_SINGLE_TAGS = [
"title",
"originaltitle",
"sorttitle",
"year",
"plot",
"outline",
"runtime",
"premiered",
"status",
"tmdbid",
"imdbid",
"tvdbid",
"dateadded",
"watched",
"mpaa",
"tagline",
]
REQUIRED_MULTI_TAGS = [
"genre",
"studio",
"country",
]
@pytest.fixture
def anime_dir(tmp_path: Path) -> Path:
"""Temporary anime root directory."""
d = tmp_path / "anime"
d.mkdir()
return d
@pytest.fixture
def nfo_service(anime_dir: Path) -> NFOService:
"""NFOService pointing at the temp directory."""
return NFOService(
tmdb_api_key="test_api_key",
anime_directory=str(anime_dir),
image_size="w500",
auto_create=True,
)
class TestAddAnimeNFOContent:
"""Test that adding an anime produces an NFO with required information."""
@pytest.mark.asyncio
async def test_add_anime_nfo_contains_required_tags(
self,
nfo_service: NFOService,
anime_dir: Path,
) -> None:
"""Add 'Sacrificial Princess And The King Of Beasts' and verify NFO.
Steps:
1. Create the series folder on disk.
2. Mock TMDB API responses.
3. Call create_tvshow_nfo to generate the NFO.
4. Parse the resulting XML and assert every required tag is present
and non-empty.
"""
series_key = "sacrificial-princess-and-the-king-of-beasts"
series_name = "Sacrificial Princess And The King Of Beasts"
series_folder = f"{series_name} (2023)"
# Step 1: Create series folder
series_path = anime_dir / series_folder
series_path.mkdir()
# Step 2: Mock TMDB API calls
with patch.object(
nfo_service.tmdb_client,
"search_tv_show",
new_callable=AsyncMock,
) as mock_search, patch.object(
nfo_service.tmdb_client,
"get_tv_show_details",
new_callable=AsyncMock,
) as mock_details, patch.object(
nfo_service.tmdb_client,
"get_tv_show_content_ratings",
new_callable=AsyncMock,
) as mock_ratings, patch.object(
nfo_service.image_downloader,
"download_all_media",
new_callable=AsyncMock,
) as mock_download:
mock_search.return_value = {
"results": [
{
"id": 222093,
"name": series_name,
"first_air_date": "2023-04-20",
"overview": (
"A girl is offered as a sacrifice to a beastly king..."
),
}
]
}
mock_details.return_value = MOCK_TMDB_DATA
mock_ratings.return_value = MOCK_CONTENT_RATINGS
mock_download.return_value = {
"poster": True,
"logo": True,
"fanart": True,
}
# Step 3: Create NFO
nfo_path = await nfo_service.create_tvshow_nfo(
serie_name=series_name,
serie_folder=series_folder,
year=2023,
download_poster=True,
download_logo=True,
download_fanart=True,
)
# Verify NFO was created
assert nfo_path.exists(), f"NFO file not created at {nfo_path}"
assert nfo_path.name == "tvshow.nfo"
# Step 4: Parse NFO XML and verify required tags
nfo_content = nfo_path.read_text(encoding="utf-8")
root = etree.fromstring(nfo_content.encode("utf-8"))
missing: list[str] = []
for tag in REQUIRED_SINGLE_TAGS:
elem = root.find(f".//{tag}")
if elem is None or not (elem.text or "").strip():
missing.append(tag)
for tag in REQUIRED_MULTI_TAGS:
elems = root.findall(f".//{tag}")
if not elems or not any((e.text or "").strip() for e in elems):
missing.append(tag)
# At least one actor must be present
actors = root.findall(".//actor/name")
if not actors or not any((a.text or "").strip() for a in actors):
missing.append("actor/name")
assert not missing, (
f"Missing or empty required tags in NFO for '{series_name}':\n "
+ "\n ".join(missing)
+ f"\n\nFull NFO content:\n{nfo_content}"
)
# Verify specific values for the requested anime
assert root.findtext(".//title") == "Sacrificial Princess and the King of Beasts"
assert root.findtext(".//year") == "2023"
assert root.findtext(".//status") == "Ended"
assert root.findtext(".//watched") == "false"
assert root.findtext(".//tmdbid") == "222093"
assert root.findtext(".//imdbid") == "tt19896734"
assert root.findtext(".//tvdbid") == "421737"
# Plot and outline must be non-trivial
plot = root.findtext(".//plot") or ""
outline = root.findtext(".//outline") or ""
assert len(plot) >= 10, f"plot too short: {plot!r}"
assert len(outline) >= 10, f"outline too short: {outline!r}"
# Verify multi-value fields
genres = [g.text for g in root.findall(".//genre") if g.text]
assert "Animation" in genres
assert "Romance" in genres
studios = [s.text for s in root.findall(".//studio") if s.text]
assert "TBS" in studios
countries = [c.text for c in root.findall(".//country") if c.text]
assert "JP" in countries
@pytest.mark.asyncio
async def test_add_anime_nfo_has_plot_and_outline(
self,
nfo_service: NFOService,
anime_dir: Path,
) -> None:
"""Specifically verify that plot and outline tags are populated.
This is a focused regression test ensuring the NFO always contains
meaningful plot and outline data.
"""
series_name = "Sacrificial Princess And The King Of Beasts"
series_folder = f"{series_name} (2023)"
series_path = anime_dir / series_folder
series_path.mkdir()
with patch.object(
nfo_service.tmdb_client,
"search_tv_show",
new_callable=AsyncMock,
) as mock_search, patch.object(
nfo_service.tmdb_client,
"get_tv_show_details",
new_callable=AsyncMock,
) as mock_details, patch.object(
nfo_service.tmdb_client,
"get_tv_show_content_ratings",
new_callable=AsyncMock,
) as mock_ratings, patch.object(
nfo_service.image_downloader,
"download_all_media",
new_callable=AsyncMock,
) as mock_download:
mock_search.return_value = {
"results": [
{
"id": 222093,
"name": series_name,
"first_air_date": "2023-04-20",
}
]
}
mock_details.return_value = MOCK_TMDB_DATA
mock_ratings.return_value = MOCK_CONTENT_RATINGS
mock_download.return_value = {"poster": False, "logo": False, "fanart": False}
nfo_path = await nfo_service.create_tvshow_nfo(
serie_name=series_name,
serie_folder=series_folder,
year=2023,
download_poster=False,
download_logo=False,
download_fanart=False,
)
assert nfo_path.exists()
root = etree.parse(str(nfo_path)).getroot()
plot_elem = root.find(".//plot")
outline_elem = root.find(".//outline")
assert plot_elem is not None, "<plot> tag missing from NFO"
assert outline_elem is not None, "<outline> tag missing from NFO"
plot_text = (plot_elem.text or "").strip()
outline_text = (outline_elem.text or "").strip()
assert plot_text, "<plot> tag is empty"
assert outline_text, "<outline> tag is empty"
assert "sacrifice" in plot_text.lower() or "beast" in plot_text.lower(), (
f"plot does not contain expected content: {plot_text!r}"
)

View File

@@ -111,17 +111,17 @@ class TestGetAllSeriesFromDataFiles:
class TestSyncSeriesToDatabase:
"""Test sync_series_from_data_files function from anime_service."""
"""Test sync_legacy_series_to_db function from anime_service."""
@pytest.mark.asyncio
async def test_sync_with_empty_directory(self):
"""Test sync with empty anime directory."""
from src.server.services.anime_service import sync_series_from_data_files
from src.server.services.anime_service import sync_legacy_series_to_db
with tempfile.TemporaryDirectory() as tmp_dir:
with patch('src.core.SeriesApp.Loaders'), \
patch('src.core.SeriesApp.SerieScanner'):
count = await sync_series_from_data_files(tmp_dir)
count = await sync_legacy_series_to_db(tmp_dir)
assert count == 0
# Function should complete successfully with no series
@@ -134,7 +134,7 @@ class TestSyncSeriesToDatabase:
from files and the sync function attempts to add them to the DB.
The actual DB interaction is tested in test_add_to_db_creates_record.
"""
from src.server.services.anime_service import sync_series_from_data_files
from src.server.services.anime_service import sync_legacy_series_to_db
with tempfile.TemporaryDirectory() as tmp_dir:
# Create test data files
@@ -160,7 +160,7 @@ class TestSyncSeriesToDatabase:
patch('src.core.SeriesApp.SerieScanner'):
# The function should return 0 because DB isn't available
# but should not crash
count = await sync_series_from_data_files(tmp_dir)
count = await sync_legacy_series_to_db(tmp_dir)
# Since no real DB, it will fail gracefully
# Function returns 0 when DB operations fail
@@ -170,7 +170,7 @@ class TestSyncSeriesToDatabase:
@pytest.mark.asyncio
async def test_sync_handles_exceptions_gracefully(self):
"""Test that sync handles exceptions without crashing."""
from src.server.services.anime_service import sync_series_from_data_files
from src.server.services.anime_service import sync_legacy_series_to_db
# Make SeriesApp raise an exception during initialization
with patch('src.core.SeriesApp.Loaders'), \
@@ -179,7 +179,7 @@ class TestSyncSeriesToDatabase:
'src.core.SeriesApp.SerieList',
side_effect=Exception("Test error")
):
count = await sync_series_from_data_files("/fake/path")
count = await sync_legacy_series_to_db("/fake/path")
assert count == 0
# Function should complete without crashing

View File

@@ -21,7 +21,7 @@ class TestInitializationWorkflow:
async def test_perform_initial_setup_with_mocked_dependencies(self):
"""Test initial setup completes with minimal mocking."""
# Mock only the external dependencies
with patch('src.server.services.anime_service.sync_series_from_data_files') as mock_sync:
with patch('src.server.services.anime_service.sync_legacy_series_to_db') as mock_sync:
mock_sync.return_value = 0 # No series to sync
# Call the actual function
@@ -241,9 +241,9 @@ class TestModuleStructure:
assert hasattr(initialization_service, 'settings')
def test_sync_series_function_imported(self):
"""Test sync_series_from_data_files is imported."""
assert hasattr(initialization_service, 'sync_series_from_data_files')
assert callable(initialization_service.sync_series_from_data_files)
"""Test sync_legacy_series_to_db is imported."""
assert hasattr(initialization_service, 'sync_legacy_series_to_db')
assert callable(initialization_service.sync_legacy_series_to_db)
# Simpler integration tests that don't require complex mocking
@@ -413,7 +413,7 @@ class TestInitialSetupWorkflow:
async def test_initial_setup_already_completed(self):
"""Test initial setup when already completed."""
with patch.object(initialization_service, '_check_initial_scan_status', return_value=True), \
patch('src.server.services.anime_service.sync_series_from_data_files'):
patch('src.server.services.anime_service.sync_legacy_series_to_db'):
result = await initialization_service.perform_initial_setup()
@@ -425,7 +425,7 @@ class TestInitialSetupWorkflow:
"""Test initial setup with no directory configured."""
with patch.object(initialization_service, '_check_initial_scan_status', return_value=False), \
patch.object(initialization_service, '_validate_anime_directory', return_value=False), \
patch('src.server.services.anime_service.sync_series_from_data_files'):
patch('src.server.services.anime_service.sync_legacy_series_to_db'):
result = await initialization_service.perform_initial_setup()
@@ -440,7 +440,7 @@ class TestInitialSetupWorkflow:
patch.object(initialization_service, '_sync_anime_folders', return_value=5), \
patch.object(initialization_service, '_mark_initial_scan_completed'), \
patch.object(initialization_service, '_load_series_into_memory'), \
patch('src.server.services.anime_service.sync_series_from_data_files'):
patch('src.server.services.anime_service.sync_legacy_series_to_db'):
mock_progress = AsyncMock()
result = await initialization_service.perform_initial_setup(mock_progress)
@@ -456,7 +456,7 @@ class TestInitialSetupWorkflow:
with patch.object(initialization_service, '_check_initial_scan_status', return_value=False), \
patch.object(initialization_service, '_validate_anime_directory', return_value=True), \
patch.object(initialization_service, '_sync_anime_folders', side_effect=OSError("Disk error")), \
patch('src.server.services.anime_service.sync_series_from_data_files'):
patch('src.server.services.anime_service.sync_legacy_series_to_db'):
result = await initialization_service.perform_initial_setup()
@@ -469,7 +469,7 @@ class TestInitialSetupWorkflow:
with patch.object(initialization_service, '_check_initial_scan_status', return_value=False), \
patch.object(initialization_service, '_validate_anime_directory', return_value=True), \
patch.object(initialization_service, '_sync_anime_folders', side_effect=RuntimeError("DB error")), \
patch('src.server.services.anime_service.sync_series_from_data_files'):
patch('src.server.services.anime_service.sync_legacy_series_to_db'):
result = await initialization_service.perform_initial_setup()

View File

@@ -0,0 +1,333 @@
"""Integration tests for episode download sync with data file updates.
Tests verify that when episodes are downloaded successfully:
- In-memory Serie.episodeDict is updated
- Deprecated data file is updated (if it exists)
- Missing episode list reflects the change immediately
"""
import asyncio
import json
import os
import tempfile
from pathlib import Path
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from src.core.entities.series import Serie
from src.core.SeriesApp import SeriesApp
from src.server.models.download import DownloadItem, DownloadPriority, DownloadStatus
from src.server.services.download_service import DownloadService
class TestEpisodeRemovedFromMissingListAfterDownload:
"""Verify episode no longer appears in missing list after download completes."""
@pytest.fixture
def temp_dir(self):
"""Create temp directory for test data files."""
with tempfile.TemporaryDirectory() as tmp:
yield Path(tmp)
@pytest.fixture
def mock_anime_service(self, temp_dir):
"""Create mock anime service with app."""
anime_service = MagicMock()
anime_service._directory = str(temp_dir)
# Create mock app withSerie with missing episodes
serie = Serie(
key="test-series",
name="Test Series",
site="https://example.com",
folder="Test Series",
episodeDict={1: [1, 2, 3]},
)
mock_app = MagicMock()
mock_app.list.keyDict = {"test-series": serie}
mock_app.list.GetMissingEpisode.return_value = [serie]
mock_app.series_list = [serie]
anime_service._app = mock_app
anime_service._cached_list_missing = MagicMock()
anime_service._broadcast_series_updated = AsyncMock()
return anime_service
@pytest.fixture
def mock_download_service(self, mock_anime_service):
"""Create download service with mocked dependencies."""
with tempfile.TemporaryDirectory() as tmp:
service = DownloadService(
anime_service=mock_anime_service,
queue_repository=MagicMock(),
max_retries=3,
)
service._directory = tmp
yield service
@pytest.mark.asyncio
async def test_episode_removed_from_missing_list_after_download(
self, mock_download_service, mock_anime_service
):
"""Verify episode no longer appears in missing list after download completes."""
serie = mock_anime_service._app.list.keyDict["test-series"]
# Verify episode starts in missing list
assert 2 in serie.episodeDict[1], "Episode should start in missing list"
# Simulate download completion by calling _remove_episode_from_memory
mock_download_service._remove_episode_from_memory("test-series", 1, 2)
# Episode should be removed from episodeDict
assert 2 not in serie.episodeDict[1], "Episode should be removed from missing list"
assert serie.episodeDict[1] == [1, 3]
# series_list should be refreshed
mock_anime_service._app.list.GetMissingEpisode.assert_called()
class TestDownloadUpdatesInMemoryCache:
"""Verify in-memory Serie.episodeDict is updated after download."""
@pytest.fixture
def mock_anime_service(self):
"""Create mock anime service with app."""
anime_service = MagicMock()
anime_service._directory = "/tmp/test"
# Create mock app with series having multiple seasons and episodes
serie = Serie(
key="multi-season-series",
name="Multi Season Series",
site="https://example.com",
folder="Multi Season Series",
episodeDict={
1: [1, 2, 3, 4, 5],
2: [1, 2, 3],
},
)
mock_app = MagicMock()
mock_app.list.keyDict = {"multi-season-series": serie}
mock_app.list.GetMissingEpisode.return_value = [serie]
mock_app.series_list = [serie]
anime_service._app = mock_app
anime_service._cached_list_missing = MagicMock()
anime_service._broadcast_series_updated = AsyncMock()
return anime_service
@pytest.fixture
def mock_download_service(self, mock_anime_service):
"""Create download service with mocked dependencies."""
with tempfile.TemporaryDirectory() as tmp:
service = DownloadService(
anime_service=mock_anime_service,
queue_repository=MagicMock(),
max_retries=3,
)
service._directory = tmp
yield service
@pytest.mark.asyncio
async def test_download_updates_in_memory_cache(
self, mock_download_service, mock_anime_service
):
"""Verify in-memory Serie.episodeDict is updated after download."""
# First reset to known state (remove the defaults first call might have set)
serie = mock_anime_service._app.list.keyDict["multi-season-series"]
# Put back episodes after the fixture setup
serie.episodeDict = {1: [1, 2, 3, 4, 5], 2: [1, 2, 3]}
# Verify preconditions
assert 1 in serie.episodeDict[1]
assert 3 in serie.episodeDict[2]
# Simulate downloading multiple episodes
mock_download_service._remove_episode_from_memory("multi-season-series", 1, 1)
mock_download_service._remove_episode_from_memory("multi-season-series", 1, 3)
mock_download_service._remove_episode_from_memory("multi-season-series", 2, 2)
# Verify episodes removed
assert 1 not in serie.episodeDict[1], "Episode 1 of season 1 should be removed"
assert 3 not in serie.episodeDict[1], "Episode 3 of season 1 should be removed"
assert 2 in serie.episodeDict[1], "Episode 2 of season 1 should remain"
assert 3 in serie.episodeDict[2], "Episode 3 of season 2 should remain"
assert 2 not in serie.episodeDict[2], "Episode 2 of season 2 should be removed"
# Verify seasons with no episodes are cleaned up
assert 2 in serie.episodeDict, "Season 2 should still exist (has episode 1, 3)"
@pytest.mark.asyncio
async def test_last_episode_removes_season(
self, mock_download_service, mock_anime_service
):
"""Verify that removing last episode in a season removes the season key."""
# Modify the series so season 1 only has episode 2 left
serie = mock_anime_service._app.list.keyDict["multi-season-series"]
# Reset and set to proper test state
serie.episodeDict = {1: [2], 2: [1, 2, 3]} # Season 1 only has episode 2
# Verify initial state
assert 2 in serie.episodeDict[1]
assert 2 in serie.episodeDict[2]
# Remove last episode of season 1 (episode 2)
mock_download_service._remove_episode_from_memory("multi-season-series", 1, 2)
# Season 1 should be completely removed
assert 1 not in serie.episodeDict, "Season 1 should be removed"
# Season 2 should still exist
assert 2 in serie.episodeDict, "Season 2 should still exist"
class TestDataFileUpdatedAfterDownload:
"""Verify data file is updated after download (when it exists)."""
@pytest.fixture
def temp_dir(self):
"""Create temp directory for test data files."""
with tempfile.TemporaryDirectory() as tmp:
yield Path(tmp)
@pytest.fixture
def mock_anime_service(self, temp_dir):
"""Create mock anime service with app."""
anime_service = MagicMock()
anime_service._directory = str(temp_dir)
# Create series folder with data file
series_folder = temp_dir / "Test Series"
series_folder.mkdir()
data_path = series_folder / "data"
serie = Serie(
key="test-series-with-data",
name="Test Series",
site="https://example.com",
folder="Test Series",
episodeDict={1: [1, 2, 3]},
)
# Save data file to disk
import warnings
with warnings.catch_warnings():
warnings.simplefilter("ignore", DeprecationWarning)
serie.save_to_file(str(data_path))
# Update episodeDict to simulate in-progress download state
# (episodeDict still has all episodes; will be updated after download)
mock_app = MagicMock()
mock_app.list.keyDict = {"test-series-with-data": serie}
mock_app.list.GetMissingEpisode.return_value = [serie]
mock_app.series_list = [serie]
anime_service._app = mock_app
anime_service._cached_list_missing = MagicMock()
anime_service._broadcast_series_updated = AsyncMock()
return anime_service
@pytest.fixture
def mock_download_service(self, mock_anime_service):
"""Create download service with mocked dependencies."""
service = DownloadService(
anime_service=mock_anime_service,
queue_repository=MagicMock(),
max_retries=3,
)
service._directory = str(mock_anime_service._directory)
yield service
@pytest.mark.asyncio
async def test_data_file_updated_after_download(
self, mock_download_service, mock_anime_service, temp_dir
):
"""Verify data file is updated after download when data file exists."""
serie = mock_anime_service._app.list.keyDict["test-series-with-data"]
data_path = temp_dir / "Test Series" / "data"
# Verify data file exists before test
assert data_path.exists(), "Data file should exist before test"
# Read original data file
with open(data_path) as f:
original_data = json.load(f)
assert 2 in original_data["episodeDict"]["1"], "Episode should be in original data"
# Simulate download completion
mock_download_service._remove_episode_from_memory("test-series-with-data", 1, 2)
# Read updated data file
with open(data_path) as f:
updated_data = json.load(f)
# Verify episode 2 was removed from data file
assert 2 not in updated_data["episodeDict"]["1"], "Episode should be removed from data file"
assert updated_data["episodeDict"]["1"] == [1, 3]
class TestDataFileNotRequiredForDownload:
"""Verify downloads work even when data file doesn't exist."""
@pytest.fixture
def temp_dir(self):
"""Create temp directory without data files."""
with tempfile.TemporaryDirectory() as tmp:
yield Path(tmp)
@pytest.fixture
def mock_anime_service(self, temp_dir):
"""Create mock anime service with app but no data file."""
anime_service = MagicMock()
anime_service._directory = str(temp_dir)
# Create series with NO data file on disk (only in memory)
serie = Serie(
key="memory-only-series",
name="Memory Only Series",
site="https://example.com",
folder="Memory Only Series",
episodeDict={1: [1, 2, 3]},
)
mock_app = MagicMock()
mock_app.list.keyDict = {"memory-only-series": serie}
mock_app.list.GetMissingEpisode.return_value = [serie]
mock_app.series_list = [serie]
anime_service._app = mock_app
anime_service._cached_list_missing = MagicMock()
anime_service._broadcast_series_updated = AsyncMock()
return anime_service
@pytest.fixture
def mock_download_service(self, mock_anime_service):
"""Create download service with mocked dependencies."""
service = DownloadService(
anime_service=mock_anime_service,
queue_repository=MagicMock(),
max_retries=3,
)
service._directory = str(mock_anime_service._directory)
yield service
@pytest.mark.asyncio
async def test_download_works_without_data_file(
self, mock_download_service, mock_anime_service
):
"""Verify downloads work even when no data file exists on disk."""
serie = mock_anime_service._app.list.keyDict["memory-only-series"]
data_path = Path(mock_anime_service._directory) / "Memory Only Series" / "data"
# Verify no data file exists
assert not data_path.exists(), "No data file should exist"
# Simulate download completion
# This should NOT raise even without data file
mock_download_service._remove_episode_from_memory("memory-only-series", 1, 2)
# Episode should be removed from in-memory state
assert 2 not in serie.episodeDict[1], "Episode should be removed from memory"
# Data file should still not exist (no file created)
assert not data_path.exists(), "No data file should be created"

View File

@@ -17,7 +17,7 @@ class TestFolderRenameScanCalledInFolderScan:
import importlib
source = importlib.util.find_spec(
"src.server.services.folder_scan_service"
"src.server.services.scheduler.folder_scan_service"
).origin
with open(source, "r", encoding="utf-8") as fh:
content = fh.read()
@@ -31,7 +31,7 @@ class TestFolderRenameScanCalledInFolderScan:
import importlib
source = importlib.util.find_spec(
"src.server.services.folder_scan_service"
"src.server.services.scheduler.folder_scan_service"
).origin
with open(source, "r", encoding="utf-8") as fh:
content = fh.read()
@@ -52,7 +52,7 @@ class TestFolderRenameIntegration:
@pytest.mark.asyncio
async def test_folder_rename_runs_during_scan(self, tmp_path):
"""When folder_scan_enabled is true, the scan renames mismatched folders."""
from src.server.services.folder_scan_service import FolderScanService
from src.server.services.scheduler.folder_scan_service import FolderScanService
anime_dir = tmp_path / "anime"
anime_dir.mkdir()
@@ -69,15 +69,15 @@ class TestFolderRenameIntegration:
with patch(
"src.config.settings.settings", mock_settings
), patch(
"src.server.services.folder_rename_service.settings", mock_settings
"src.server.services.scheduler.folder_rename_service.settings", mock_settings
), patch(
"src.server.services.folder_scan_service.perform_nfo_repair_scan",
"src.server.services.scheduler.folder_scan_service.perform_nfo_repair_scan",
new_callable=AsyncMock,
), patch(
"src.server.services.folder_rename_service._is_series_being_downloaded",
"src.server.services.scheduler.folder_rename_service._is_series_being_downloaded",
return_value=False,
), patch(
"src.server.services.folder_rename_service._update_database_paths",
"src.server.services.scheduler.folder_rename_service._update_database_paths",
new_callable=AsyncMock,
):
service = FolderScanService()
@@ -89,7 +89,7 @@ class TestFolderRenameIntegration:
@pytest.mark.asyncio
async def test_folder_rename_skipped_when_prerequisites_not_met(self, tmp_path):
"""If anime directory is missing, rename logic is skipped gracefully."""
from src.server.services.folder_scan_service import FolderScanService
from src.server.services.scheduler.folder_scan_service import FolderScanService
mock_settings = MagicMock()
mock_settings.tmdb_api_key = "test-key"
@@ -98,10 +98,10 @@ class TestFolderRenameIntegration:
with patch(
"src.config.settings.settings", mock_settings
), patch(
"src.server.services.folder_scan_service.perform_nfo_repair_scan",
"src.server.services.scheduler.folder_scan_service.perform_nfo_repair_scan",
new_callable=AsyncMock,
), patch(
"src.server.services.folder_rename_service.validate_and_rename_series_folders"
"src.server.services.scheduler.folder_rename_service.validate_and_rename_series_folders"
) as mock_rename:
service = FolderScanService()
await service.run_folder_scan()

View File

@@ -0,0 +1,335 @@
"""Integration tests for legacy key/data file migration.
Tests the one-time migration safety net that imports series from
legacy key and data files into the database.
"""
import json
import os
import tempfile
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from src.server.services.legacy_file_migration import (
_load_data_file,
_load_key_file,
migrate_series_from_files_to_db,
)
class TestLoadLegacyFiles:
"""Test helper functions for loading legacy files."""
def test_load_data_file_valid_json(self):
"""Test loading a valid JSON data file."""
with tempfile.TemporaryDirectory() as tmp_dir:
data_file = os.path.join(tmp_dir, "data")
test_data = {
"key": "test-anime",
"name": "Test Anime",
"site": "https://aniworld.to",
"folder": "Test Anime",
"episodeDict": {"1": [1, 2, 3]}
}
with open(data_file, "w", encoding="utf-8") as f:
json.dump(test_data, f)
result = _load_data_file(data_file)
assert result is not None
assert result["key"] == "test-anime"
assert result["name"] == "Test Anime"
# episodeDict keys should be converted to int
assert 1 in result["episodeDict"]
def test_load_data_file_invalid_json(self):
"""Test handling of corrupt JSON data file."""
with tempfile.TemporaryDirectory() as tmp_dir:
data_file = os.path.join(tmp_dir, "data")
with open(data_file, "w", encoding="utf-8") as f:
f.write("this is not valid json {{{")
result = _load_data_file(data_file)
assert result is None
def test_load_data_file_not_dict(self):
"""Test handling of JSON file that is not a dict."""
with tempfile.TemporaryDirectory() as tmp_dir:
data_file = os.path.join(tmp_dir, "data")
with open(data_file, "w", encoding="utf-8") as f:
json.dump(["not", "a", "dict"], f)
result = _load_data_file(data_file)
assert result is None
def test_load_key_file_valid(self):
"""Test loading a key file with valid content."""
with tempfile.TemporaryDirectory() as tmp_dir:
key_file = os.path.join(tmp_dir, "key")
with open(key_file, "w", encoding="utf-8") as f:
f.write("my-anime-key")
result = _load_key_file(key_file, "My Anime")
assert result is not None
assert result["key"] == "my-anime-key"
assert result["name"] == "My Anime"
assert result["site"] == "https://aniworld.to"
assert result["episodeDict"] == {}
def test_load_key_file_empty(self):
"""Test handling of empty key file."""
with tempfile.TemporaryDirectory() as tmp_dir:
key_file = os.path.join(tmp_dir, "key")
with open(key_file, "w", encoding="utf-8") as f:
f.write("")
result = _load_key_file(key_file, "My Anime")
assert result is None
class TestMigrateLegacyFiles:
"""Test the main migration function with database."""
@pytest.mark.asyncio
async def test_migrate_series_from_files_to_db_no_files(self):
"""Test migration with empty directory returns 0."""
mock_db = AsyncMock()
mock_db.execute = AsyncMock()
with tempfile.TemporaryDirectory() as tmp_dir:
count = await migrate_series_from_files_to_db(tmp_dir, mock_db)
assert count == 0
@pytest.mark.asyncio
async def test_migrate_data_file_to_db(self):
"""Test migration of a legacy data file."""
with tempfile.TemporaryDirectory() as tmp_dir:
# Create a folder with a data file
anime_folder = os.path.join(tmp_dir, "Test Anime")
os.makedirs(anime_folder, exist_ok=True)
data_file = os.path.join(anime_folder, "data")
test_data = {
"key": "migrate-test-anime",
"name": "Migrate Test Anime",
"site": "https://aniworld.to",
"folder": "Test Anime",
"episodeDict": {"1": [1, 2]}
}
with open(data_file, "w", encoding="utf-8") as f:
json.dump(test_data, f)
# Mock the DB session and services
mock_db = AsyncMock()
mock_series_service = AsyncMock()
mock_episode_service = AsyncMock()
# Mock get_by_key returning None (not in DB)
mock_series_service.get_by_key = AsyncMock(return_value=None)
# Mock AnimeSeriesService.create returning a mock with id=1
mock_created_series = MagicMock()
mock_created_series.id = 1
mock_series_service.create = AsyncMock(return_value=mock_created_series)
with patch.dict('sys.modules', {
'src.server.database.service': MagicMock(
AnimeSeriesService=mock_series_service,
EpisodeService=mock_episode_service
)
}):
count = await migrate_series_from_files_to_db(tmp_dir, mock_db)
assert count == 1
@pytest.mark.asyncio
async def test_migrate_key_file_to_db(self):
"""Test migration of a legacy key file."""
with tempfile.TemporaryDirectory() as tmp_dir:
# Create a folder with only a key file
anime_folder = os.path.join(tmp_dir, "Key Only Anime")
os.makedirs(anime_folder, exist_ok=True)
key_file = os.path.join(anime_folder, "key")
with open(key_file, "w", encoding="utf-8") as f:
f.write("key-only-anime")
# Mock the DB session and services
mock_db = AsyncMock()
mock_series_service = AsyncMock()
mock_episode_service = AsyncMock()
# Mock get_by_key returning None (not in DB)
mock_series_service.get_by_key = AsyncMock(return_value=None)
# Mock AnimeSeriesService.create returning a mock with id=1
mock_created_series = MagicMock()
mock_created_series.id = 1
mock_series_service.create = AsyncMock(return_value=mock_created_series)
with patch.dict('sys.modules', {
'src.server.database.service': MagicMock(
AnimeSeriesService=mock_series_service,
EpisodeService=mock_episode_service
)
}):
count = await migrate_series_from_files_to_db(tmp_dir, mock_db)
assert count == 1
@pytest.mark.asyncio
async def test_migration_skips_already_migrated(self):
"""Test that migration skips series already in DB."""
with tempfile.TemporaryDirectory() as tmp_dir:
# Create a folder with a data file
anime_folder = os.path.join(tmp_dir, "Already Migrated")
os.makedirs(anime_folder, exist_ok=True)
data_file = os.path.join(anime_folder, "data")
test_data = {
"key": "already-migrated",
"name": "Already Migrated",
"site": "https://aniworld.to",
"folder": "Already Migrated",
"episodeDict": {"1": [1]}
}
with open(data_file, "w", encoding="utf-8") as f:
json.dump(test_data, f)
# Mock the DB session and services
mock_db = AsyncMock()
mock_series_service = AsyncMock()
mock_episode_service = AsyncMock()
# Mock get_by_key returning existing series (already migrated)
mock_existing_series = MagicMock()
mock_existing_series.name = "Modified Name"
mock_series_service.get_by_key = AsyncMock(return_value=mock_existing_series)
with patch.dict('sys.modules', {
'src.server.database.service': MagicMock(
AnimeSeriesService=mock_series_service,
EpisodeService=mock_episode_service
)
}):
count = await migrate_series_from_files_to_db(tmp_dir, mock_db)
assert count == 0 # No new series migrated
@pytest.mark.asyncio
async def test_migration_handles_corrupt_data_file(self):
"""Test that corrupt data files don't crash migration."""
with tempfile.TemporaryDirectory() as tmp_dir:
# Create a folder with a corrupt data file
corrupt_folder = os.path.join(tmp_dir, "Corrupt Anime")
os.makedirs(corrupt_folder, exist_ok=True)
corrupt_file = os.path.join(corrupt_folder, "data")
with open(corrupt_file, "w", encoding="utf-8") as f:
f.write("not valid json {{{")
# Create a valid folder
valid_folder = os.path.join(tmp_dir, "Valid Anime")
os.makedirs(valid_folder, exist_ok=True)
valid_file = os.path.join(valid_folder, "data")
valid_data = {
"key": "valid-anime",
"name": "Valid Anime",
"site": "https://aniworld.to",
"folder": "Valid Anime",
"episodeDict": {"1": [1]}
}
with open(valid_file, "w", encoding="utf-8") as f:
json.dump(valid_data, f)
# Mock the DB session and services
mock_db = AsyncMock()
mock_series_service = AsyncMock()
mock_episode_service = AsyncMock()
# Mock get_by_key returning None (not in DB)
mock_series_service.get_by_key = AsyncMock(return_value=None)
# Mock AnimeSeriesService.create returning a mock with id=1
mock_created_series = MagicMock()
mock_created_series.id = 1
mock_series_service.create = AsyncMock(return_value=mock_created_series)
with patch.dict('sys.modules', {
'src.server.database.service': MagicMock(
AnimeSeriesService=mock_series_service,
EpisodeService=mock_episode_service
)
}):
# Migration should succeed despite corrupt file
count = await migrate_series_from_files_to_db(tmp_dir, mock_db)
assert count == 1 # Only the valid one
@pytest.mark.asyncio
async def test_migration_idempotent(self):
"""Test that running migration twice doesn't change DB state."""
with tempfile.TemporaryDirectory() as tmp_dir:
# Create a folder with a data file
anime_folder = os.path.join(tmp_dir, "Idempotent Test")
os.makedirs(anime_folder, exist_ok=True)
data_file = os.path.join(anime_folder, "data")
test_data = {
"key": "idempotent-test",
"name": "Idempotent Test",
"site": "https://aniworld.to",
"folder": "Idempotent Test",
"episodeDict": {"1": [1, 2]}
}
with open(data_file, "w", encoding="utf-8") as f:
json.dump(test_data, f)
# Mock the DB session and services
mock_db = AsyncMock()
mock_series_service = AsyncMock()
mock_episode_service = AsyncMock()
# First call returns None (not in DB), second call returns the series
mock_existing_series = MagicMock()
mock_existing_series.id = 1
mock_series_service.get_by_key = AsyncMock(side_effect=[None, mock_existing_series])
# Mock AnimeSeriesService.create returning a mock with id=1
mock_created_series = MagicMock()
mock_created_series.id = 1
mock_series_service.create = AsyncMock(return_value=mock_created_series)
with patch.dict('sys.modules', {
'src.server.database.service': MagicMock(
AnimeSeriesService=mock_series_service,
EpisodeService=mock_episode_service
)
}):
# First migration
count1 = await migrate_series_from_files_to_db(tmp_dir, mock_db)
assert count1 == 1
# Second migration
count2 = await migrate_series_from_files_to_db(tmp_dir, mock_db)
assert count2 == 0 # Already migrated
@pytest.mark.asyncio
async def test_migration_skips_folders_without_files(self):
"""Test that folders without key/data files are skipped."""
with tempfile.TemporaryDirectory() as tmp_dir:
# Create an empty folder (no key or data file)
empty_folder = os.path.join(tmp_dir, "Empty Folder")
os.makedirs(empty_folder, exist_ok=True)
# Create a folder with only a video file
video_folder = os.path.join(tmp_dir, "Video Folder")
os.makedirs(video_folder, exist_ok=True)
with open(os.path.join(video_folder, "episode1.mp4"), "w") as f:
f.write("fake video content")
mock_db = AsyncMock()
count = await migrate_series_from_files_to_db(tmp_dir, mock_db)
assert count == 0

View File

@@ -34,7 +34,7 @@ class TestNfoRepairScanCalledInFolderScan:
"""folder_scan_service.py imports perform_nfo_repair_scan."""
import importlib
source = importlib.util.find_spec("src.server.services.folder_scan_service").origin
source = importlib.util.find_spec("src.server.services.scheduler.folder_scan_service").origin
with open(source, "r", encoding="utf-8") as fh:
content = fh.read()
@@ -46,7 +46,7 @@ class TestNfoRepairScanCalledInFolderScan:
"""perform_nfo_repair_scan must be called inside run_folder_scan."""
import importlib
source = importlib.util.find_spec("src.server.services.folder_scan_service").origin
source = importlib.util.find_spec("src.server.services.scheduler.folder_scan_service").origin
with open(source, "r", encoding="utf-8") as fh:
content = fh.read()
@@ -67,7 +67,9 @@ class TestNfoRepairScanIntegrationWithBackgroundLoader:
@pytest.mark.asyncio
async def test_incomplete_nfo_series_scheduled_for_repair(self, tmp_path):
"""Series whose tvshow.nfo is missing required tags are scheduled via asyncio.create_task."""
from src.server.services.folder_scan_service import perform_nfo_repair_scan
from src.server.services.scheduler.folder_scan_service import (
perform_nfo_repair_scan,
)
series_dir = tmp_path / "IncompleteAnime"
series_dir.mkdir()
@@ -83,7 +85,7 @@ class TestNfoRepairScanIntegrationWithBackgroundLoader:
mock_repair_service.repair_series = AsyncMock(return_value=True)
with patch(
"src.server.services.folder_scan_service._settings", mock_settings
"src.server.services.scheduler.folder_scan_service._settings", mock_settings
), patch(
"src.core.services.nfo_repair_service.nfo_needs_repair",
return_value=True,
@@ -103,7 +105,9 @@ class TestNfoRepairScanIntegrationWithBackgroundLoader:
@pytest.mark.asyncio
async def test_complete_nfo_series_not_scheduled(self, tmp_path):
"""Series whose tvshow.nfo has all required tags are not scheduled for repair."""
from src.server.services.folder_scan_service import perform_nfo_repair_scan
from src.server.services.scheduler.folder_scan_service import (
perform_nfo_repair_scan,
)
series_dir = tmp_path / "CompleteAnime"
series_dir.mkdir()
@@ -116,7 +120,7 @@ class TestNfoRepairScanIntegrationWithBackgroundLoader:
mock_settings.anime_directory = str(tmp_path)
with patch(
"src.server.services.folder_scan_service._settings", mock_settings
"src.server.services.scheduler.folder_scan_service._settings", mock_settings
), patch(
"src.core.services.nfo_repair_service.nfo_needs_repair",
return_value=False,

View File

@@ -19,7 +19,7 @@ class TestPosterCheckScanCalledInFolderScan:
import importlib
source = importlib.util.find_spec(
"src.server.services.folder_scan_service"
"src.server.services.scheduler.folder_scan_service"
).origin
with open(source, "r", encoding="utf-8") as fh:
content = fh.read()
@@ -33,7 +33,7 @@ class TestPosterCheckScanCalledInFolderScan:
import importlib
source = importlib.util.find_spec(
"src.server.services.folder_scan_service"
"src.server.services.scheduler.folder_scan_service"
).origin
with open(source, "r", encoding="utf-8") as fh:
content = fh.read()
@@ -54,7 +54,7 @@ class TestPosterCheckIntegration:
@pytest.mark.asyncio
async def test_poster_check_downloads_missing_poster(self, tmp_path):
"""When poster.jpg is missing, the scan downloads it from the NFO thumb URL."""
from src.server.services.folder_scan_service import FolderScanService
from src.server.services.scheduler.folder_scan_service import FolderScanService
anime_dir = tmp_path / "anime"
anime_dir.mkdir()
@@ -91,14 +91,14 @@ class TestPosterCheckIntegration:
with patch(
"src.config.settings.settings", mock_settings
), patch(
"src.server.services.folder_scan_service.perform_nfo_repair_scan",
"src.server.services.scheduler.folder_scan_service.perform_nfo_repair_scan",
new_callable=AsyncMock,
), patch(
"src.server.services.folder_rename_service.validate_and_rename_series_folders",
"src.server.services.scheduler.folder_rename_service.validate_and_rename_series_folders",
new_callable=AsyncMock,
return_value={"scanned": 0, "renamed": 0, "skipped": 0, "errors": 0},
), patch(
"src.server.services.folder_scan_service.ImageDownloader",
"src.server.services.scheduler.folder_scan_service.ImageDownloader",
new=MockDownloader,
):
service = FolderScanService()
@@ -112,7 +112,7 @@ class TestPosterCheckIntegration:
@pytest.mark.asyncio
async def test_poster_check_skips_valid_poster(self, tmp_path):
"""When poster.jpg exists and is large enough, the scan skips it."""
from src.server.services.folder_scan_service import FolderScanService
from src.server.services.scheduler.folder_scan_service import FolderScanService
anime_dir = tmp_path / "anime"
anime_dir.mkdir()
@@ -136,14 +136,14 @@ class TestPosterCheckIntegration:
with patch(
"src.config.settings.settings", mock_settings
), patch(
"src.server.services.folder_scan_service.perform_nfo_repair_scan",
"src.server.services.scheduler.folder_scan_service.perform_nfo_repair_scan",
new_callable=AsyncMock,
), patch(
"src.server.services.folder_rename_service.validate_and_rename_series_folders",
"src.server.services.scheduler.folder_rename_service.validate_and_rename_series_folders",
new_callable=AsyncMock,
return_value={"scanned": 0, "renamed": 0, "skipped": 0, "errors": 0},
), patch(
"src.server.services.folder_scan_service.ImageDownloader"
"src.server.services.scheduler.folder_scan_service.ImageDownloader"
) as mock_downloader_cls:
service = FolderScanService()
await service.run_folder_scan()
@@ -153,7 +153,7 @@ class TestPosterCheckIntegration:
@pytest.mark.asyncio
async def test_poster_check_skips_when_no_thumb_url(self, tmp_path):
"""When NFO has no thumb URL, the scan skips the folder."""
from src.server.services.folder_scan_service import FolderScanService
from src.server.services.scheduler.folder_scan_service import FolderScanService
anime_dir = tmp_path / "anime"
anime_dir.mkdir()
@@ -173,14 +173,14 @@ class TestPosterCheckIntegration:
with patch(
"src.config.settings.settings", mock_settings
), patch(
"src.server.services.folder_scan_service.perform_nfo_repair_scan",
"src.server.services.scheduler.folder_scan_service.perform_nfo_repair_scan",
new_callable=AsyncMock,
), patch(
"src.server.services.folder_rename_service.validate_and_rename_series_folders",
"src.server.services.scheduler.folder_rename_service.validate_and_rename_series_folders",
new_callable=AsyncMock,
return_value={"scanned": 0, "renamed": 0, "skipped": 0, "errors": 0},
), patch(
"src.server.services.folder_scan_service.ImageDownloader"
"src.server.services.scheduler.folder_scan_service.ImageDownloader"
) as mock_downloader_cls:
service = FolderScanService()
await service.run_folder_scan()
@@ -190,7 +190,7 @@ class TestPosterCheckIntegration:
@pytest.mark.asyncio
async def test_poster_check_skipped_when_prerequisites_not_met(self, tmp_path):
"""If anime directory is missing, poster check logic is skipped gracefully."""
from src.server.services.folder_scan_service import FolderScanService
from src.server.services.scheduler.folder_scan_service import FolderScanService
mock_settings = MagicMock()
mock_settings.tmdb_api_key = "test-key"
@@ -199,12 +199,12 @@ class TestPosterCheckIntegration:
with patch(
"src.config.settings.settings", mock_settings
), patch(
"src.server.services.folder_scan_service.perform_nfo_repair_scan",
"src.server.services.scheduler.folder_scan_service.perform_nfo_repair_scan",
new_callable=AsyncMock,
), patch(
"src.server.services.folder_rename_service.validate_and_rename_series_folders"
"src.server.services.scheduler.folder_rename_service.validate_and_rename_series_folders"
) as mock_rename, patch(
"src.server.services.folder_scan_service.ImageDownloader"
"src.server.services.scheduler.folder_scan_service.ImageDownloader"
) as mock_downloader_cls:
service = FolderScanService()
await service.run_folder_scan()
@@ -220,7 +220,7 @@ class TestPosterCheckSemaphore:
import importlib
source = importlib.util.find_spec(
"src.server.services.folder_scan_service"
"src.server.services.scheduler.folder_scan_service"
).origin
with open(source, "r", encoding="utf-8") as fh:
content = fh.read()
@@ -232,7 +232,7 @@ class TestPosterCheckSemaphore:
@pytest.mark.asyncio
async def test_poster_download_uses_semaphore(self, tmp_path):
"""Poster downloads are gated by the semaphore."""
from src.server.services.folder_scan_service import (
from src.server.services.scheduler.folder_scan_service import (
_POSTER_DOWNLOAD_SEMAPHORE,
FolderScanService,
)
@@ -270,14 +270,14 @@ class TestPosterCheckSemaphore:
with patch(
"src.config.settings.settings", mock_settings
), patch(
"src.server.services.folder_scan_service.perform_nfo_repair_scan",
"src.server.services.scheduler.folder_scan_service.perform_nfo_repair_scan",
new_callable=AsyncMock,
), patch(
"src.server.services.folder_rename_service.validate_and_rename_series_folders",
"src.server.services.scheduler.folder_rename_service.validate_and_rename_series_folders",
new_callable=AsyncMock,
return_value={"scanned": 0, "renamed": 0, "skipped": 0, "errors": 0},
), patch(
"src.server.services.folder_scan_service.ImageDownloader"
"src.server.services.scheduler.folder_scan_service.ImageDownloader"
) as mock_downloader_cls:
mock_downloader = AsyncMock()
mock_downloader.download_poster = AsyncMock(side_effect=tracked_download)

View File

@@ -0,0 +1,429 @@
"""Integration test: add 'Sacrificial Princess And The King Of Beasts' and verify NFO completeness.
Simulates the production scenario where this anime is added and validates
that the generated tvshow.nfo contains plot, outline, and all other required
information. Also tests the repair path for an incomplete NFO.
"""
from pathlib import Path
from unittest.mock import AsyncMock, patch
import pytest
from lxml import etree
from src.core.services.nfo_repair_service import (
NfoRepairService,
_read_tmdb_id,
find_missing_tags,
nfo_needs_repair,
)
from src.core.services.nfo_service import NFOService
# ---------------------------------------------------------------------------
# TMDB mock data matching production responses for this anime
# ---------------------------------------------------------------------------
SERIES_KEY = "sacrificial-princess-and-the-king-of-beasts"
SERIES_NAME = "Sacrificial Princess And The King Of Beasts"
SERIES_FOLDER = "Sacrificial Princess And The King Of Beasts (2023)"
TMDB_ID = 222093
MOCK_TMDB_DETAILS = {
"id": TMDB_ID,
"name": "Sacrificial Princess and the King of Beasts",
"original_name": "贄姫と獣の王",
"overview": (
"On the outskirts of the Demon King's realm lies a small village of "
"humans who offer a sacrifice to the beast king every year. Sariphi, "
"the latest sacrificial girl, expects to be devoured — but instead "
"her fearless nature catches the king's attention and she becomes "
"his unlikely companion."
),
"tagline": "A tale of love between a sacrifice and a beast king.",
"first_air_date": "2023-04-20",
"last_air_date": "2023-09-28",
"vote_average": 7.5,
"vote_count": 150,
"status": "Ended",
"episode_run_time": [24],
"number_of_seasons": 1,
"number_of_episodes": 24,
"genres": [
{"id": 16, "name": "Animation"},
{"id": 10749, "name": "Romance"},
{"id": 10765, "name": "Sci-Fi & Fantasy"},
],
"networks": [{"id": 160, "name": "TBS"}],
"production_companies": [{"id": 291, "name": "J.C.Staff"}],
"origin_country": ["JP"],
"poster_path": "/sacrificial_poster.jpg",
"backdrop_path": "/sacrificial_backdrop.jpg",
"external_ids": {"imdb_id": "tt19896734", "tvdb_id": 421737},
"credits": {
"cast": [
{
"id": 2072089,
"name": "Kana Hanazawa",
"character": "Sariphi",
"profile_path": "/hanazawa.jpg",
"order": 0,
},
{
"id": 1254783,
"name": "Satoshi Hino",
"character": "Leonhart",
"profile_path": "/hino.jpg",
"order": 1,
},
]
},
"images": {"logos": [{"file_path": "/sacrificial_logo.png"}]},
"seasons": [
{"season_number": 0, "name": "Specials"},
{"season_number": 1, "name": "Season 1"},
],
}
MOCK_CONTENT_RATINGS = {
"results": [
{"iso_3166_1": "DE", "rating": "12"},
{"iso_3166_1": "US", "rating": "TV-14"},
]
}
MOCK_SEARCH_RESULTS = {
"results": [
{
"id": TMDB_ID,
"name": "Sacrificial Princess and the King of Beasts",
"first_air_date": "2023-04-20",
"overview": (
"On the outskirts of the Demon King's realm lies a small village "
"of humans who offer a sacrifice to the beast king every year."
),
}
]
}
# ---------------------------------------------------------------------------
# Tags that MUST be present and non-empty in a complete NFO
# ---------------------------------------------------------------------------
REQUIRED_TAGS = [
"title",
"originaltitle",
"year",
"plot",
"outline",
"runtime",
"premiered",
"status",
"tmdbid",
"imdbid",
"genre",
"studio",
"country",
"watched",
]
@pytest.fixture
def anime_dir(tmp_path: Path) -> Path:
"""Temporary anime directory."""
d = tmp_path / "anime"
d.mkdir()
return d
@pytest.fixture
def nfo_service(anime_dir: Path) -> NFOService:
"""NFOService configured for the temp directory."""
return NFOService(
tmdb_api_key="test_api_key",
anime_directory=str(anime_dir),
image_size="w500",
auto_create=True,
)
def _mock_tmdb_calls(nfo_service: NFOService):
"""Context manager that patches all TMDB calls with mock data."""
return _PatchContext(nfo_service)
class _PatchContext:
"""Helper to patch TMDB calls on an NFOService instance."""
def __init__(self, svc: NFOService):
self._svc = svc
self._patches = []
def __enter__(self):
p1 = patch.object(
self._svc.tmdb_client, "search_tv_show", new_callable=AsyncMock
)
p2 = patch.object(
self._svc.tmdb_client, "get_tv_show_details", new_callable=AsyncMock
)
p3 = patch.object(
self._svc.tmdb_client, "get_tv_show_content_ratings", new_callable=AsyncMock
)
p4 = patch.object(
self._svc.image_downloader, "download_all_media", new_callable=AsyncMock
)
p5 = patch.object(
self._svc.tmdb_client, "_ensure_session", new_callable=AsyncMock
)
p6 = patch.object(
self._svc.tmdb_client, "close", new_callable=AsyncMock
)
self._patches = [p1, p2, p3, p4, p5, p6]
mocks = [p.start() for p in self._patches]
mocks[0].return_value = MOCK_SEARCH_RESULTS
mocks[1].return_value = MOCK_TMDB_DETAILS
mocks[2].return_value = MOCK_CONTENT_RATINGS
mocks[3].return_value = {"poster": True, "logo": True, "fanart": True}
return self
def __exit__(self, *args):
for p in self._patches:
p.stop()
class TestSacrificialPrincessNFO:
"""Tests for 'Sacrificial Princess And The King Of Beasts' NFO generation."""
@pytest.mark.asyncio
async def test_add_anime_creates_complete_nfo(
self, nfo_service: NFOService, anime_dir: Path
) -> None:
"""Adding the anime produces an NFO with all required tags filled."""
series_path = anime_dir / SERIES_FOLDER
series_path.mkdir()
with _PatchContext(nfo_service):
nfo_path = await nfo_service.create_tvshow_nfo(
serie_name=SERIES_NAME,
serie_folder=SERIES_FOLDER,
year=2023,
download_poster=True,
download_logo=True,
download_fanart=True,
)
assert nfo_path.exists(), f"NFO not created at {nfo_path}"
root = etree.parse(str(nfo_path)).getroot()
missing = []
for tag in REQUIRED_TAGS:
elems = root.findall(f".//{tag}")
if not elems or not any((e.text or "").strip() for e in elems):
missing.append(tag)
# Actor check
actors = root.findall(".//actor/name")
if not actors or not any((a.text or "").strip() for a in actors):
missing.append("actor/name")
assert not missing, (
f"Missing or empty tags in NFO for '{SERIES_NAME}':\n"
f" {', '.join(missing)}\n\n"
f"NFO content:\n{nfo_path.read_text(encoding='utf-8')}"
)
@pytest.mark.asyncio
async def test_nfo_plot_and_outline_are_meaningful(
self, nfo_service: NFOService, anime_dir: Path
) -> None:
"""Plot and outline must contain substantial descriptive text."""
series_path = anime_dir / SERIES_FOLDER
series_path.mkdir()
with _PatchContext(nfo_service):
nfo_path = await nfo_service.create_tvshow_nfo(
serie_name=SERIES_NAME,
serie_folder=SERIES_FOLDER,
year=2023,
download_poster=False,
download_logo=False,
download_fanart=False,
)
root = etree.parse(str(nfo_path)).getroot()
plot = (root.findtext(".//plot") or "").strip()
outline = (root.findtext(".//outline") or "").strip()
assert len(plot) >= 20, f"Plot too short ({len(plot)} chars): {plot!r}"
assert len(outline) >= 20, f"Outline too short ({len(outline)} chars): {outline!r}"
# Should mention relevant keywords from the series
combined = (plot + outline).lower()
assert any(
kw in combined for kw in ("sacrifice", "beast", "king", "sariphi")
), f"Plot/outline missing expected content:\n plot={plot!r}\n outline={outline!r}"
@pytest.mark.asyncio
async def test_nfo_specific_values(
self, nfo_service: NFOService, anime_dir: Path
) -> None:
"""Verify specific metadata values match the anime."""
series_path = anime_dir / SERIES_FOLDER
series_path.mkdir()
with _PatchContext(nfo_service):
nfo_path = await nfo_service.create_tvshow_nfo(
serie_name=SERIES_NAME,
serie_folder=SERIES_FOLDER,
year=2023,
download_poster=False,
download_logo=False,
download_fanart=False,
)
root = etree.parse(str(nfo_path)).getroot()
assert root.findtext(".//year") == "2023"
assert root.findtext(".//status") == "Ended"
assert root.findtext(".//tmdbid") == str(TMDB_ID)
assert root.findtext(".//imdbid") == "tt19896734"
assert root.findtext(".//watched") == "false"
assert root.findtext(".//premiered") == "2023-04-20"
genres = [g.text for g in root.findall(".//genre") if g.text]
assert "Animation" in genres
@pytest.mark.asyncio
async def test_incomplete_nfo_detected_as_needing_repair(
self, anime_dir: Path
) -> None:
"""An NFO with only a <title> tag is detected as incomplete."""
series_path = anime_dir / SERIES_FOLDER
series_path.mkdir()
nfo_path = series_path / "tvshow.nfo"
# Simulate production state: minimal NFO with only title
nfo_path.write_text(
'<?xml version="1.0" encoding="UTF-8"?>\n'
"<tvshow>\n"
f" <title>{SERIES_NAME}</title>\n"
"</tvshow>\n",
encoding="utf-8",
)
assert nfo_needs_repair(nfo_path) is True
missing = find_missing_tags(nfo_path)
# All these should be detected as missing
for tag_label in ["plot", "year", "runtime", "premiered", "genre", "studio"]:
assert tag_label in missing, f"'{tag_label}' not detected as missing"
@pytest.mark.asyncio
async def test_repair_fixes_incomplete_nfo(
self, nfo_service: NFOService, anime_dir: Path
) -> None:
"""NfoRepairService re-fetches and creates a complete NFO from an incomplete one."""
series_path = anime_dir / SERIES_FOLDER
series_path.mkdir()
nfo_path = series_path / "tvshow.nfo"
# Write an incomplete NFO with a tmdbid so update_tvshow_nfo can work
nfo_path.write_text(
'<?xml version="1.0" encoding="UTF-8"?>\n'
"<tvshow>\n"
f" <title>{SERIES_NAME}</title>\n"
f" <tmdbid>{TMDB_ID}</tmdbid>\n"
"</tvshow>\n",
encoding="utf-8",
)
assert nfo_needs_repair(nfo_path) is True
# Patch TMDB calls for the update path
with patch.object(
nfo_service.tmdb_client, "_ensure_session", new_callable=AsyncMock
), patch.object(
nfo_service.tmdb_client, "get_tv_show_details", new_callable=AsyncMock
) as mock_details, patch.object(
nfo_service.tmdb_client, "get_tv_show_content_ratings", new_callable=AsyncMock
) as mock_ratings, patch.object(
nfo_service.tmdb_client, "close", new_callable=AsyncMock
):
mock_details.return_value = MOCK_TMDB_DETAILS
mock_ratings.return_value = MOCK_CONTENT_RATINGS
repair_service = NfoRepairService(nfo_service)
repaired = await repair_service.repair_series(series_path, SERIES_FOLDER)
assert repaired is True
# After repair, NFO should be complete
assert nfo_needs_repair(nfo_path) is False
# Verify content
root = etree.parse(str(nfo_path)).getroot()
plot = (root.findtext(".//plot") or "").strip()
assert len(plot) >= 20, f"Plot still incomplete after repair: {plot!r}"
@pytest.mark.asyncio
async def test_repair_recreates_nfo_without_tmdb_id(
self, nfo_service: NFOService, anime_dir: Path
) -> None:
"""If the NFO has no <tmdbid>, repair falls back to create_tvshow_nfo."""
series_path = anime_dir / SERIES_FOLDER
series_path.mkdir()
nfo_path = series_path / "tvshow.nfo"
# Simulate the production worst-case: only a title, no TMDB ID
nfo_path.write_text(
'<?xml version="1.0" encoding="UTF-8"?>\n'
"<tvshow>\n"
f" <title>{SERIES_NAME}</title>\n"
"</tvshow>\n",
encoding="utf-8",
)
assert _read_tmdb_id(nfo_path) is None
assert nfo_needs_repair(nfo_path) is True
with _PatchContext(nfo_service):
repair_service = NfoRepairService(nfo_service)
repaired = await repair_service.repair_series(series_path, SERIES_FOLDER)
assert repaired is True
assert nfo_path.exists()
assert nfo_needs_repair(nfo_path) is False
root = etree.parse(str(nfo_path)).getroot()
plot = (root.findtext(".//plot") or "").strip()
assert len(plot) >= 20, f"Plot incomplete after recreate: {plot!r}"
assert root.findtext(".//tmdbid") == str(TMDB_ID)
@pytest.mark.asyncio
async def test_complete_nfo_not_repaired(
self, nfo_service: NFOService, anime_dir: Path
) -> None:
"""A complete NFO should not trigger a repair."""
series_path = anime_dir / SERIES_FOLDER
series_path.mkdir()
# First create a complete NFO
with _PatchContext(nfo_service):
await nfo_service.create_tvshow_nfo(
serie_name=SERIES_NAME,
serie_folder=SERIES_FOLDER,
year=2023,
download_poster=False,
download_logo=False,
download_fanart=False,
)
nfo_path = series_path / "tvshow.nfo"
assert nfo_path.exists()
assert nfo_needs_repair(nfo_path) is False
# Repair should be skipped
repair_service = NfoRepairService(nfo_service)
repaired = await repair_service.repair_series(series_path, SERIES_FOLDER)
assert repaired is False

View File

@@ -11,15 +11,14 @@ from unittest.mock import AsyncMock, Mock, patch
import pytest
from src.server.models.config import AppConfig, SchedulerConfig
from src.server.services.scheduler_service import (
from src.server.services.scheduler.scheduler_service import (
_JOB_ID,
SchedulerService,
SchedulerServiceError,
_JOB_ID,
get_scheduler_service,
reset_scheduler_service,
)
# ---------------------------------------------------------------------------
# Shared fixtures
# ---------------------------------------------------------------------------
@@ -27,7 +26,7 @@ from src.server.services.scheduler_service import (
@pytest.fixture
def mock_config_service():
"""Patch get_config_service used by SchedulerService.start()."""
with patch("src.server.services.scheduler_service.get_config_service") as mock:
with patch("src.server.services.scheduler.scheduler_service.get_config_service") as mock:
config_service = Mock()
app_config = AppConfig(
scheduler=SchedulerConfig(

View File

@@ -495,6 +495,20 @@ class TestNameWithYearProperty:
assert "(2013)" in sanitized
assert "Attack on Titan" in sanitized
def test_name_with_year_does_not_duplicate(self):
"""Test that name_with_year doesn't duplicate year."""
serie = Serie(
key="eighty-six",
name="86 Eighty Six (2021)",
site="aniworld.to",
folder="86 Eighty Six (2021)",
episodeDict={},
year=2021
)
assert serie.name_with_year == "86 Eighty Six (2021)"
assert serie.name_with_year.count("(2021)") == 1
class TestEnsureFolderWithYear:
"""Test Serie.ensure_folder_with_year method."""

View File

@@ -23,6 +23,7 @@ class TestDownloadQueueStress:
def mock_anime_service(self):
"""Create mock AnimeService."""
service = MagicMock(spec=AnimeService)
service._directory = "/tmp/test_anime"
service.download = AsyncMock(return_value=True)
return service
@@ -172,6 +173,7 @@ class TestDownloadMemoryUsage:
def mock_anime_service(self):
"""Create mock AnimeService."""
service = MagicMock(spec=AnimeService)
service._directory = "/tmp/test_anime"
service.download = AsyncMock(return_value=True)
return service
@@ -180,6 +182,7 @@ class TestDownloadMemoryUsage:
"""Create download service with mock repository."""
from tests.unit.test_download_service import MockQueueRepository
mock_repo = MockQueueRepository()
mock_anime_service._directory = "/tmp/test_anime"
service = DownloadService(
anime_service=mock_anime_service,
max_retries=3,
@@ -223,6 +226,7 @@ class TestDownloadConcurrency:
def mock_anime_service(self):
"""Create mock AnimeService with slow downloads."""
service = MagicMock(spec=AnimeService)
service._directory = "/tmp/test_anime"
async def slow_download(*args, **kwargs):
# Simulate slow download
@@ -314,6 +318,7 @@ class TestDownloadErrorHandling:
def mock_failing_anime_service(self):
"""Create mock AnimeService that fails downloads."""
service = MagicMock(spec=AnimeService)
service._directory = "/tmp/test_anime"
service.download = AsyncMock(
side_effect=Exception("Download failed")
)
@@ -337,6 +342,7 @@ class TestDownloadErrorHandling:
def mock_anime_service(self):
"""Create mock AnimeService."""
service = MagicMock(spec=AnimeService)
service._directory = "/tmp/test_anime"
service.download = AsyncMock(return_value=True)
return service
@@ -345,6 +351,7 @@ class TestDownloadErrorHandling:
"""Create download service with mock repository."""
from tests.unit.test_download_service import MockQueueRepository
mock_repo = MockQueueRepository()
mock_anime_service._directory = "/tmp/test_anime"
service = DownloadService(
anime_service=mock_anime_service,
max_retries=3,

View File

@@ -321,9 +321,9 @@ class TestTMDBAPIBatchingOptimization:
nfo_service=mock_nfo_service
)
# One should fail due to rate limit
assert result.successful == num_series - 1
assert result.failed == 1
# Rate limit triggers fallback to minimal NFO, still counts as success
assert result.successful == num_series
assert result.failed == 0
print(f"\nRate limit test: {result.successful} success, {result.failed} failed")

View File

@@ -392,23 +392,33 @@ class TestAddSeriesWithEpisodes:
nfo_created_at=datetime(2024, 1, 1, 12, 0, 0),
nfo_updated_at=datetime(2024, 1, 2, 12, 0, 0)
)
mock_db_series.id = 1
# Create service with mocked WebSocket
anime_service = AnimeService(mock_series_app)
mock_websocket = AsyncMock()
anime_service._websocket_service = mock_websocket
# Mock database session and service
mock_db_session = AsyncMock()
mock_db_session.__aenter__ = AsyncMock(return_value=mock_db_session)
mock_db_session.__aexit__ = AsyncMock()
# Mock episodes that match the in-memory episodeDict
mock_episodes = [
MagicMock(season=1, episode_number=1),
MagicMock(season=1, episode_number=2),
MagicMock(season=1, episode_number=3),
]
with patch('src.server.database.connection.get_db_session', return_value=mock_db_session):
with patch('src.server.database.service.AnimeSeriesService') as MockAnimeSeriesService:
MockAnimeSeriesService.get_by_key = AsyncMock(return_value=mock_db_series)
# Act
await anime_service._broadcast_series_updated(key)
with patch('src.server.database.service.EpisodeService') as MockEpisodeService:
MockEpisodeService.get_by_series = AsyncMock(return_value=mock_episodes)
# Act
await anime_service._broadcast_series_updated(key)
# Assert
mock_websocket.broadcast.assert_called_once()

View File

@@ -0,0 +1,161 @@
"""Unit tests for anime key rename logic and validation."""
import pytest
from pydantic import ValidationError
from src.server.models.anime import AnimeMetadataUpdate, KEY_PATTERN
class TestKeyValidation:
"""Tests for AnimeMetadataUpdate key validation."""
def test_valid_key_simple(self):
"""Test simple valid key."""
model = AnimeMetadataUpdate(key="attack-on-titan")
assert model.key == "attack-on-titan"
def test_valid_key_single_char(self):
"""Test single character key is valid."""
model = AnimeMetadataUpdate(key="a")
assert model.key == "a"
def test_valid_key_numbers(self):
"""Test key with numbers."""
model = AnimeMetadataUpdate(key="86-eighty-six")
assert model.key == "86-eighty-six"
def test_valid_key_allows_hyphens(self):
"""Test hyphens in key are allowed."""
model = AnimeMetadataUpdate(key="my-anime-key")
assert model.key == "my-anime-key"
def test_valid_key_normalizes_to_lowercase(self):
"""Test key is normalized to lowercase."""
model = AnimeMetadataUpdate(key="Attack-On-Titan")
assert model.key == "attack-on-titan"
def test_valid_key_strips_whitespace(self):
"""Test key strips leading/trailing whitespace."""
model = AnimeMetadataUpdate(key=" my-key ")
assert model.key == "my-key"
def test_invalid_key_spaces(self):
"""Test key with spaces is rejected."""
with pytest.raises(ValidationError) as exc_info:
AnimeMetadataUpdate(key="my anime key")
assert "Key must contain only" in str(exc_info.value)
def test_invalid_key_uppercase_special(self):
"""Test key with special characters is rejected."""
with pytest.raises(ValidationError) as exc_info:
AnimeMetadataUpdate(key="anime!@#key")
assert "Key must contain only" in str(exc_info.value)
def test_invalid_key_empty(self):
"""Test empty key is rejected."""
with pytest.raises(ValidationError) as exc_info:
AnimeMetadataUpdate(key="")
assert "cannot be empty" in str(exc_info.value)
def test_invalid_key_only_whitespace(self):
"""Test whitespace-only key is rejected."""
with pytest.raises(ValidationError) as exc_info:
AnimeMetadataUpdate(key=" ")
assert "cannot be empty" in str(exc_info.value)
def test_invalid_key_starts_with_hyphen(self):
"""Test key starting with hyphen is rejected."""
with pytest.raises(ValidationError):
AnimeMetadataUpdate(key="-my-key")
def test_invalid_key_ends_with_hyphen(self):
"""Test key ending with hyphen is rejected."""
with pytest.raises(ValidationError):
AnimeMetadataUpdate(key="my-key-")
def test_key_none_is_allowed(self):
"""Test None key (no change requested) is allowed."""
model = AnimeMetadataUpdate(key=None)
assert model.key is None
def test_key_omitted_is_allowed(self):
"""Test omitting key entirely is allowed."""
model = AnimeMetadataUpdate(tmdb_id=1234)
assert model.key is None
class TestTmdbIdValidation:
"""Tests for tmdb_id validation."""
def test_valid_tmdb_id(self):
"""Test valid positive TMDB ID."""
model = AnimeMetadataUpdate(tmdb_id=1429)
assert model.tmdb_id == 1429
def test_tmdb_id_none(self):
"""Test None tmdb_id is allowed."""
model = AnimeMetadataUpdate(tmdb_id=None)
assert model.tmdb_id is None
def test_tmdb_id_negative_rejected(self):
"""Test negative tmdb_id is rejected."""
with pytest.raises(ValidationError):
AnimeMetadataUpdate(tmdb_id=-1)
def test_tmdb_id_zero_rejected(self):
"""Test zero tmdb_id is rejected."""
with pytest.raises(ValidationError):
AnimeMetadataUpdate(tmdb_id=0)
class TestTvdbIdValidation:
"""Tests for tvdb_id validation."""
def test_valid_tvdb_id(self):
"""Test valid positive TVDB ID."""
model = AnimeMetadataUpdate(tvdb_id=267440)
assert model.tvdb_id == 267440
def test_tvdb_id_none(self):
"""Test None tvdb_id is allowed."""
model = AnimeMetadataUpdate(tvdb_id=None)
assert model.tvdb_id is None
def test_tvdb_id_negative_rejected(self):
"""Test negative tvdb_id is rejected."""
with pytest.raises(ValidationError):
AnimeMetadataUpdate(tvdb_id=-5)
def test_tvdb_id_zero_rejected(self):
"""Test zero tvdb_id is rejected."""
with pytest.raises(ValidationError):
AnimeMetadataUpdate(tvdb_id=0)
class TestKeyPattern:
"""Tests for the KEY_PATTERN regex directly."""
@pytest.mark.parametrize("key", [
"a",
"abc",
"attack-on-titan",
"86-eighty-six",
"a1b2c3",
"x",
"1",
])
def test_valid_patterns(self, key):
"""Test keys that should match the pattern."""
assert KEY_PATTERN.match(key) is not None
@pytest.mark.parametrize("key", [
"-start",
"end-",
"has space",
"UPPER",
"special!char",
"under_score",
"",
])
def test_invalid_patterns(self, key):
"""Test keys that should not match the pattern."""
assert KEY_PATTERN.match(key) is None

View File

@@ -16,7 +16,7 @@ import pytest
from src.server.services.anime_service import (
AnimeService,
AnimeServiceError,
sync_series_from_data_files,
sync_legacy_series_to_db,
)
from src.server.services.progress_service import ProgressService
@@ -1303,7 +1303,7 @@ class TestGetNFOStatisticsSelfManaged:
class TestSyncSeriesFromDataFiles:
"""Test module-level sync_series_from_data_files function."""
"""Test module-level sync_legacy_series_to_db function."""
@pytest.mark.asyncio
async def test_sync_adds_new_series(self, tmp_path):
@@ -1343,7 +1343,7 @@ class TestSyncSeriesFromDataFiles:
]
MockApp.return_value = mock_app_instance
count = await sync_series_from_data_files(str(tmp_path))
count = await sync_legacy_series_to_db(str(tmp_path))
assert count == 1
mock_create.assert_called_once()
@@ -1382,7 +1382,7 @@ class TestSyncSeriesFromDataFiles:
]
MockApp.return_value = mock_app_instance
count = await sync_series_from_data_files(str(tmp_path))
count = await sync_legacy_series_to_db(str(tmp_path))
assert count == 0
mock_create.assert_not_called()
@@ -1397,7 +1397,7 @@ class TestSyncSeriesFromDataFiles:
mock_app_instance.get_all_series_from_data_files.return_value = []
MockApp.return_value = mock_app_instance
count = await sync_series_from_data_files(str(tmp_path))
count = await sync_legacy_series_to_db(str(tmp_path))
assert count == 0
@@ -1436,7 +1436,7 @@ class TestSyncSeriesFromDataFiles:
]
MockApp.return_value = mock_app_instance
count = await sync_series_from_data_files(str(tmp_path))
count = await sync_legacy_series_to_db(str(tmp_path))
assert count == 1
# The name should have been set to folder

Some files were not shown because too many files have changed in this diff Show More