When the year-suffixed target folder already exists on disk, both
FolderNamingService and AnimeService.rename_folder_if_needed used to
silently bail out. The bare folder (e.g. 'Ultraman') was left next to
the year-suffixed one ('Ultraman (2019)'), producing the symptom
'reports series like Ultraman as added twice' — the DB has one row but
the filesystem has two folders holding the same content.
Fix: when the target already exists, merge the source's contents into
the target (target version wins on file conflicts; source copies are
removed so cleanup succeeds), remove the now-empty source directory,
update DB row + in-memory cache. Plain rename path is unchanged.
Also fixes a latent TypeError in rename_folder_if_needed where
self._directory (a str) was used with the '/' operator. Production
behavior was that any rename through that method raised and was
swallowed by the caller's try/except, leaving the bare folder
untouched. The new path builds Path objects from the string base.
Tests:
- Replaced test_skips_when_target_folder_already_exists (which
codified the bug) with three tests that cover the new merge
contract: clean merge, no-overwrite, and empty-source removal.
- Added tests/unit/test_rename_folder_if_needed.py covering the
same scenarios plus the str-directory regression. All seven go
red on the unfixed code and green with the fix.
Fixes the 'Ultraman' / 'Ultraman (2019)' duplicate-folder bug.
Two follow-ups to the previous fix that resolved the CWD-relative-path
bug in delete_series:
1. Reorder deletion: filesystem first, database second.
When both delete_database=True and delete_folder=True were set and
the folder delete failed (any reason — path math bug, permission
error, missing volume, etc.), the database row was already gone by
the time the folder delete was attempted. This left an orphan
folder on disk that the user had no normal way to clean up — the
delete UI returns 'Series not found' when the row is gone.
Doing folder delete first means a folder-side failure preserves
the DB row, so the user can retry the delete once the underlying
issue is resolved.
2. Orphan-folder recovery: when the user requests delete_folder=True
for a key whose DB row no longer exists, scan the configured
anime directory for a folder that uniquely matches the key
(normalized: strip trailing (YYYY), lowercase, remove
hyphens/underscores/non-alphanumerics) and delete it. This
recovers the Beyblade Burst scenario: a previous delete attempt
removed the row but the folder stayed on disk; retrying the
delete now cleans up the orphan.
Safety: recovery only triggers when exactly one folder matches
the normalized key. Zero matches → clear 'no folder matching'
error. Multiple matches (ambiguous) → refuses to delete anything.
Adds four regression tests:
- test_delete_series_db_preserved_when_folder_fails
- test_delete_series_orphan_folder_recovery (the Beyblade Burst case)
- test_delete_series_orphan_folder_no_match
- test_delete_series_orphan_folder_ambiguous
The folder column in anime_series stores the relative folder name
(e.g. 'Beyblade Burst (2016)'), not an absolute path. The old
delete_series code called os.path.abspath(folder_path) directly,
which joins a relative path against the process current working
directory. Inside the container the FastAPI app runs with CWD=/app
while the anime directory is /data, so 'Beyblade Burst (2016)'
resolved to '/app/Beyblade Burst (2016)' and the is_safe_path check
correctly (but unhelpfully) flagged it as outside the /data base,
skipping the folder delete while still removing the row.
Fix: resolve relative folder paths against the configured anime
directory before validating against the base. Absolute paths still
work unchanged.
Also harden is_safe_path() the same way so a relative target is
treated as relative to base_path, not to the process CWD. Path
traversal ('../etc/passwd') is still rejected.
Adds two regression tests:
- is_safe_path with chdir to '/' and relative target resolves
against the base
- delete_series with chdir to '/', relative folder in DB,
succeeds and removes the folder
structlog fails with BrokenPipeError when stdout is redirected (e.g., background
processes, Docker logs). Replace all structlog.get_logger() calls with
logging.getLogger() and convert keyword-style log calls to %-format strings.
Also removes stale Docs/tasks.md (2028 lines) and updates Robot Framework
tests to match current API behavior.
Series now loaded directly from database. Removed:
- sync_legacy_series_to_db() from anime_service.py
- Corresponding sync call after directory update in config.py
- Safety nets in initialization_service.py for missing progress IDs
- Add _compute_folder_name helper that deduplicates year (handles cases like 'Name (2023)' not becoming 'Name (2023) (2023)')
- Create anime folder on disk when adding series (not just DB + memory)
- Add rename_folder_if_needed to auto-rename existing folders without year
- Fetch year from aniworld_provider and include in folder as 'Name (YYYY)'
Closes: anime folders now include release year when available from provider
- Move src/core/ → src/server/
- Split SerieList.py (531 lines) and series.py (414 lines) into src/server/database/
- Add database/models.py for SQLAlchemy models
- Update all test imports to reflect new structure
- Remove deprecated test files (test_serie_class.py, test_serie_folder_with_year.py)
- SerieScanner: generate key from folder when no key/data files exist
- Handle edge cases: non-Latin characters, special symbols in folder names
- anime_service: expose loading_status and loading_error fields
- Update tests to match new fallback behavior
- Rename function to reflect its legacy status
- Add deprecation warning log on execution
- Update all callers (initialization_service, api/config, fastapi_app)
- Update tests to use new name
- Add deprecation notice to DEVELOPMENT.md
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use the database as the authoritative source for missing-episode lists so
that episodes marked is_downloaded=True are never shown as missing, even
when the in-memory state is stale.
Key changes:
- EpisodeService.get_by_series() gains only_missing flag
- AnimeService uses DB-backed episodeDict and preserves downloaded episodes
during sync, skipping them when adding/removing missing episodes
- DownloadService broadcasts series_updated after marking an episode downloaded
so the frontend reflects the change immediately
- Frontend filters out series with zero missing episodes client-side and
fixes renderSeries to respect the active filter
- Unit tests updated to assert the broadcast is sent
- Add missing year field when building series list in anime_service
- Add _create_missing_nfo to generate minimal NFO for series without one
- Update perform_nfo_repair_scan to detect and create missing NFOs
- Add semaphore-protected async creation with TMDB rate limiting
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- New migration script: populate year from folder (YYYY) pattern
- SerieScanner: refactor year extraction logic
- anime_service: pass year when syncing from data files
- Implement sync_single_series_after_scan to persist scanned series to database
- Enhanced _broadcast_series_updated to include full NFO metadata (nfo_created_at, nfo_updated_at, tmdb_id, tvdb_id)
- Add immediate episode scanning in add_series endpoint when background loader isn't running
- Implement updateSingleSeries in frontend to handle series_updated WebSocket events
- Add SERIES_UPDATED event constant to WebSocket event definitions
- Update background loader to use sync_single_series_after_scan method
- Simplified background loader initialization in FastAPI app
- Add comprehensive tests for series update WebSocket payload and episode counting logic
- Import reorganization: move get_background_loader_service to dependencies module
- Added 5 new service methods for complete database coverage:
* get_series_without_nfo()
* count_all()
* count_with_nfo()
* count_with_tmdb_id()
* count_with_tvdb_id()
- Eliminated all direct database queries from business logic:
* series_manager_service.py - now uses AnimeSeriesService
* anime_service.py - now uses service layer methods
- Documented architecture decision in ARCHITECTURE.md:
* Service layer IS the repository layer
* No direct SQLAlchemy queries allowed outside service layer
- All database access must go through service methods
- 1449 tests passing, repository pattern enforced
- Add async method list_series_with_filters() to AnimeService
- Refactor list_anime to use service layer instead of direct DB access
- Convert sync database queries to async patterns
- Remove unused series_app parameter from endpoint
- Update test to skip direct unit test (covered by integration tests)
- Mark Issue 1 as resolved in documentation
- Fixed async context manager issue in anime.py (use get_sync_session)
- Fixed async methods in anime_service.py to use async with
- Fixed folder_name attribute error (should be folder)
- All three methods now properly handle database sessions
- Extended AnimeSummary model with NFO fields (has_nfo, nfo_created_at, nfo_updated_at, tmdb_id, tvdb_id)
- Updated list_anime endpoint to fetch and return NFO data from database
- Added NFO status badges to series cards (green=exists, gray=missing)
- Created nfo-manager.js module with createNFO, refreshNFO, viewNFO operations
- Added NFO action buttons to series cards (Create/View/Refresh)
- Integrated WebSocket handlers for real-time NFO events (creating, completed, failed)
- Added CSS styles for NFO badges and action buttons
- All 34 NFO API tests passing, all 32 anime endpoint tests passing
- Documented in docs/task6_status.md (90% complete, NFO status page deferred)
- Add year property to Serie entity with name_with_year
- Add year column to AnimeSeries database model
- Add get_year() method to AniworldLoader provider
- Extract year from folder names before fetching from API
- Update SerieScanner to populate year during rescan
- Update add_series endpoint to fetch and store year
- Optimize: check folder name for year before API call
- Import and use DownloadCancelled exception which YT-DLP properly handles
- Add InterruptedError handling throughout the call chain
- Fire 'cancelled' status event when download is cancelled
- Handle InterruptedError in DownloadService to set CANCELLED status
- Add cancellation flag to AniworldLoader with request_cancel/reset_cancel/is_cancelled methods
- Update base_provider.Loader interface with cancellation abstract methods
- Integrate cancellation check in YT-DLP progress hooks
- Add request_download_cancel method to SeriesApp and AnimeService
- Update DownloadService.stop() to request cancellation before shutdown
- Clean up temp files on cancellation
- Add debug logging to checkActiveScanStatus() for better tracing
- Update status indicator before showing overlay for faster feedback
- Add warning logs when DOM elements are not found
- Ensure idle state is explicitly set when no scan is running
- Add debug logging to AnimeService.get_scan_status()
- Add _scan_lock asyncio.Lock to AnimeService
- Check if lock is held before starting rescan
- Use async with to ensure lock is released on completion or exception
- All 1024 tests passing
- Add is_scanning state tracking in AnimeService
- Add get_scan_status method to AnimeService
- Add /api/anime/scan/status endpoint to check scan state
- Add checkActiveScanStatus in JS to restore overlay on reconnect
- All 1024 tests passing
- Add total_items parameter to broadcast_scan_started and broadcast_scan_progress
- Pass total from SeriesApp to WebSocket broadcasts in AnimeService
- Update JS overlay to show progress bar and current/total count
- Add CSS for progress bar styling
- Add unit tests for new total_items parameter
- All 1024 tests passing
- Add broadcast_scan_started, broadcast_scan_progress, broadcast_scan_completed to WebSocketService
- Inject WebSocketService into AnimeService for real-time scan progress broadcasts
- Add CSS styles for scan progress overlay with spinner, stats, and completion state
- Update app.js to handle scan events and display progress overlay
- Add unit tests for new WebSocket broadcast methods
- All 1022 tests passing
- Update _update_series_in_db to sync missing episodes bidirectionally
- Add delete_by_series_and_episode method to EpisodeService
- Remove downloaded episodes from DB after successful download
- Clear anime service cache when episodes are removed
- Fix tests to use 'message' instead of 'detail' in API responses
- Mock DB operations in rescan tests
- Remove db_session parameter from SeriesApp, SerieList, SerieScanner
- Move all database operations to AnimeService (service layer)
- Add add_series_to_db, contains_in_db methods to AnimeService
- Update sync_series_from_data_files to use inline DB operations
- Remove obsolete test classes for removed DB methods
- Fix pylint issues: add broad-except comments, fix line lengths
- Core layer (src/core/) now has zero database imports
722 unit tests pass
- Use folder name as fallback when series name is empty
- Skip series with both empty name and folder
- Add try/catch for individual series to prevent one failure
from stopping the entire sync
- Moved _sync_series_to_database from fastapi_app.py to anime_service.py
- Renamed to sync_series_from_data_files for better clarity
- Updated all imports and test references
- Removed completed TODO tasks from instructions.md
- Enhanced class and method docstrings to clarify 'key' as primary identifier
- Documented that 'folder' is metadata only (display and filesystem operations)
- Updated event handler documentation to show both key and folder are received
- Modernized type hints to Python 3.9+ style (list[dict] vs List[dict])
- Fixed PEP 8 line length violations
- All 18 anime service tests passing
Implementation follows identifier standardization initiative:
- key: Primary series identifier (provider-assigned, URL-safe)
- folder: Metadata for display and filesystem paths only
Task 3.2 completed November 23, 2025
Documented in infrastructure.md and instructions.md
