Compare commits
17 Commits
a336733ea9
...
v1.1.18
| Author | SHA1 | Date | |
|---|---|---|---|
| 7ded5a6e4d | |||
| d596902ca3 | |||
| d358a07290 | |||
| b9c55f9e7a | |||
| fc4e52f1a2 | |||
| 6d30747f25 | |||
| ceb6a2aeb4 | |||
| 53d6da5dac | |||
| 102d83e947 | |||
| 841368bf85 | |||
| cbd53ef2a0 | |||
| 50a77976d5 | |||
| dfc28b8e66 | |||
| 6c9605e896 | |||
| 3947f6d266 | |||
| a3176f5ac1 | |||
| 9a81b04b65 |
@@ -1 +1 @@
|
|||||||
v1.1.13
|
v1.1.18
|
||||||
|
|||||||
@@ -139,6 +139,10 @@ This changelog follows [Keep a Changelog](https://keepachangelog.com/) principle
|
|||||||
- Modified `src/server/api/anime.py` to save scanned episodes to database
|
- Modified `src/server/api/anime.py` to save scanned episodes to database
|
||||||
- Episodes table properly tracks missing episodes with automatic cleanup
|
- Episodes table properly tracks missing episodes with automatic cleanup
|
||||||
|
|
||||||
|
### Deprecated
|
||||||
|
|
||||||
|
- **Legacy Series Files (key/data)**: File-based series storage is deprecated. `key` and `data` files in anime folders will be removed in v3.0.0. Database storage is now the primary method. See [docs/MIGRATION_GUIDE.md](docs/MIGRATION_GUIDE.md) for details.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Sections for Each Release
|
## Sections for Each Release
|
||||||
|
|||||||
216
docs/DATABASE.md
216
docs/DATABASE.md
@@ -83,17 +83,23 @@ Source: [src/server/database/models.py](../src/server/database/models.py), [src/
|
|||||||
|
|
||||||
### 3.2 anime_series
|
### 3.2 anime_series
|
||||||
|
|
||||||
Stores anime series metadata.
|
Stores anime series metadata. Corresponds to the core `Serie` class.
|
||||||
|
|
||||||
| Column | Type | Constraints | Description |
|
| Column | Type | Constraints | Description |
|
||||||
| ------------ | ------------- | -------------------------- | ------------------------------------------------------- |
|
| ---------------- | ------------- | -------------------------- | ------------------------------------------------------- |
|
||||||
| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Internal database ID |
|
| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Internal database ID |
|
||||||
| `key` | VARCHAR(255) | UNIQUE, NOT NULL, INDEX | **Primary identifier** - provider-assigned URL-safe key |
|
| `key` | VARCHAR(255) | UNIQUE, NOT NULL, INDEX | **Primary identifier** - provider-assigned URL-safe key |
|
||||||
| `name` | VARCHAR(500) | NOT NULL, INDEX | Display name of the series |
|
| `name` | VARCHAR(500) | NOT NULL, INDEX | Display name of the series |
|
||||||
| `site` | VARCHAR(500) | NOT NULL | Provider site URL |
|
| `site` | VARCHAR(500) | NOT NULL | Provider site URL |
|
||||||
| `folder` | VARCHAR(1000) | NOT NULL | Filesystem folder name (metadata only) |
|
| `folder` | VARCHAR(1000) | NOT NULL | Filesystem folder name (metadata only) |
|
||||||
| `created_at` | DATETIME | NOT NULL, DEFAULT NOW | Record creation timestamp |
|
| `year` | INTEGER | NULLABLE | Release year of the series |
|
||||||
| `updated_at` | DATETIME | NOT NULL, ON UPDATE NOW | Last update timestamp |
|
| `nfo_path` | VARCHAR(1000) | NULLABLE | Path to tvshow.nfo metadata file |
|
||||||
|
| `tmdb_id` | INTEGER | NULLABLE, INDEX | TMDB (The Movie Database) ID for metadata |
|
||||||
|
| `tvdb_id` | INTEGER | NULLABLE, INDEX | TVDB (TheTVDB) ID for metadata |
|
||||||
|
| `has_nfo` | BOOLEAN | NOT NULL, DEFAULT FALSE | Whether tvshow.nfo exists |
|
||||||
|
| `loading_status` | VARCHAR(50) | NOT NULL, DEFAULT 'completed' | Status: pending, loading_episodes, loading_nfo, completed, failed |
|
||||||
|
| `created_at` | DATETIME | NOT NULL, DEFAULT NOW | Record creation timestamp |
|
||||||
|
| `updated_at` | DATETIME | NOT NULL, ON UPDATE NOW | Last update timestamp |
|
||||||
|
|
||||||
**Identifier Convention:**
|
**Identifier Convention:**
|
||||||
|
|
||||||
@@ -101,7 +107,13 @@ Stores anime series metadata.
|
|||||||
- `folder` is **metadata only** for filesystem operations (e.g., `"Attack on Titan (2013)"`)
|
- `folder` is **metadata only** for filesystem operations (e.g., `"Attack on Titan (2013)"`)
|
||||||
- `id` is used only for database relationships
|
- `id` is used only for database relationships
|
||||||
|
|
||||||
Source: [src/server/database/models.py](../src/server/database/models.py#L23-L87)
|
**EpisodeDict Mapping:**
|
||||||
|
|
||||||
|
The `episodeDict` (season → episode numbers mapping) is stored as individual `Episode` records:
|
||||||
|
- Each `Episode` has `season` and `episode_number` columns
|
||||||
|
- Relationship: `AnimeSeries.episodes` returns all Episode records for that series
|
||||||
|
|
||||||
|
Source: [src/server/database/models.py](../src/server/database/models.py#L23-L150)
|
||||||
|
|
||||||
### 3.3 episodes
|
### 3.3 episodes
|
||||||
|
|
||||||
@@ -441,7 +453,187 @@ items = await db.execute(
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 12. Database Location
|
## 12. Series Storage: Database vs Files (Deprecated)
|
||||||
|
|
||||||
|
### File-Based Storage (Removed in v2.0)
|
||||||
|
|
||||||
|
Prior to v2.0, series metadata was stored in two files per anime folder:
|
||||||
|
|
||||||
|
| File | Contents |
|
||||||
|
| -------- | ------------------------------------------------------- |
|
||||||
|
| `key` | Series provider key (e.g., `"attack-on-titan"`) |
|
||||||
|
| `data` | JSON serialization of `Serie` object |
|
||||||
|
|
||||||
|
File structure example:
|
||||||
|
```
|
||||||
|
/anime/Attack on Titan (2013)/
|
||||||
|
├── key # Contains: attack-on-titan
|
||||||
|
├── data # Contains: {"key": "...", "name": "...", "episodeDict": {...}}
|
||||||
|
├── Season 1/
|
||||||
|
│ └── ...
|
||||||
|
```
|
||||||
|
|
||||||
|
### Database Storage (Current)
|
||||||
|
|
||||||
|
Since v2.0, all series metadata is stored in the `anime_series` table with `Episode` records for episode tracking. This provides:
|
||||||
|
|
||||||
|
- **ACID transactions** for data consistency
|
||||||
|
- **Foreign key constraints** (cascade delete)
|
||||||
|
- **Indexed queries** for fast lookups
|
||||||
|
- **No filesystem dependency** for metadata
|
||||||
|
|
||||||
|
### Migration from Files to Database
|
||||||
|
|
||||||
|
The `Serie.save_to_file()` and `Serie.load_from_file()` methods are deprecated but still functional for backward compatibility during migration:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from src.core.entities.series import Serie
|
||||||
|
|
||||||
|
# Old file-based loading (deprecated)
|
||||||
|
serie = Serie.load_from_file("/anime/Attack on Titan (2013)/data")
|
||||||
|
|
||||||
|
# New database-based loading
|
||||||
|
from src.server.database.service import AnimeSeriesService
|
||||||
|
serie = await AnimeSeriesService.get_by_key(db, "attack-on-titan")
|
||||||
|
```
|
||||||
|
|
||||||
|
### Removing File Dependencies
|
||||||
|
|
||||||
|
After verifying database schema supports all fields, file-based storage can be removed:
|
||||||
|
|
||||||
|
1. ✅ Schema verified: All `Serie` fields have corresponding DB columns
|
||||||
|
2. ✅ Migration complete: All existing series migrated to database
|
||||||
|
3. ❌ File cleanup: Remove `key` and `data` files (pending)
|
||||||
|
|
||||||
|
**Note:** The `save_to_file()` and `load_from_file()` methods will be removed in v3.0.0.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 12. Series Persistence Flow
|
||||||
|
|
||||||
|
When a directory scan discovers or updates series, the scanner persists data to the database instead of writing to disk files.
|
||||||
|
|
||||||
|
### Scan Flow
|
||||||
|
|
||||||
|
```
|
||||||
|
Scan Directory
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Find MP4 Files → Extract Serie Key
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Check DB for Existing Series (by key)
|
||||||
|
│
|
||||||
|
├─── EXISTS ──────────────────────► Update Series Metadata
|
||||||
|
│ │
|
||||||
|
│ ▼
|
||||||
|
│ Sync Episodes to DB
|
||||||
|
│ │
|
||||||
|
│◄──────────────────────────────────────┘
|
||||||
|
│
|
||||||
|
└─── NEW ───────────────────────────► Create New Series Record
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Create Episode Records
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Return to Scan Loop
|
||||||
|
```
|
||||||
|
|
||||||
|
### Key Methods
|
||||||
|
|
||||||
|
**SerieScanner._persist_serie_to_db()**
|
||||||
|
- Called after `get_missing_episodes_and_season()` computes episodeDict
|
||||||
|
- Uses `AnimeSeriesService.get_by_key()` to check if series exists
|
||||||
|
- If exists: calls `AnimeSeriesService.update()` + `_sync_episodes_to_db()`
|
||||||
|
- If new: calls `AnimeSeriesService.create()` + creates episodes
|
||||||
|
|
||||||
|
**SerieScanner._sync_episodes_to_db()**
|
||||||
|
- Gets existing episodes from DB via `EpisodeService.get_by_series()`
|
||||||
|
- Compares with new episodeDict
|
||||||
|
- Removes episodes no longer missing (unless `is_downloaded=True`)
|
||||||
|
- Adds new missing episodes
|
||||||
|
- Preserves `is_downloaded=True` episodes when removing missing ones
|
||||||
|
|
||||||
|
**SerieList.add_to_db()**
|
||||||
|
- Used when adding a new discovered series via API
|
||||||
|
- Creates filesystem folder + database record + episode records
|
||||||
|
|
||||||
|
### Episode Sync Logic
|
||||||
|
|
||||||
|
```python
|
||||||
|
# For each episode in DB but not in new episodeDict:
|
||||||
|
if episode.is_downloaded:
|
||||||
|
# Keep - file exists, don't remove
|
||||||
|
pass
|
||||||
|
else:
|
||||||
|
# Remove - no longer missing
|
||||||
|
EpisodeService.delete()
|
||||||
|
|
||||||
|
# For each episode in new episodeDict but not in DB:
|
||||||
|
# Add as new missing episode
|
||||||
|
EpisodeService.create(is_downloaded=False)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Transaction Handling
|
||||||
|
|
||||||
|
- DB operations use their own session with commit/rollback
|
||||||
|
- If DB write fails, error is logged and scan continues
|
||||||
|
- File-based `save_to_file()` no longer called during scan
|
||||||
|
|
||||||
|
### Migration Path
|
||||||
|
|
||||||
|
1. v2.x: Scanner writes to both DB (primary) and files (fallback)
|
||||||
|
2. v3.0: Scanner writes only to DB, file methods removed
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 13. Series Persistence
|
||||||
|
|
||||||
|
### Schema
|
||||||
|
|
||||||
|
**AnimeSeries Table**: Stores series metadata (key, name, site, folder, year)
|
||||||
|
|
||||||
|
| Column | Type | Constraints | Description |
|
||||||
|
|-----------|--------------|---------------------------|----------------------|
|
||||||
|
| `id` | INTEGER | PRIMARY KEY | Auto-increment |
|
||||||
|
| `key` | VARCHAR(255) | UNIQUE, NOT NULL | Series provider key |
|
||||||
|
| `name` | VARCHAR(500) | NOT NULL | Display name |
|
||||||
|
| `site` | VARCHAR(500) | | Provider site URL |
|
||||||
|
| `folder` | VARCHAR(1000)| | Filesystem folder |
|
||||||
|
|
||||||
|
**Episode Table**: Stores per-episode metadata (season, episode_number, is_downloaded)
|
||||||
|
|
||||||
|
| Column | Type | Constraints | Description |
|
||||||
|
|-----------------|--------------|---------------------------|----------------------|
|
||||||
|
| `id` | INTEGER | PRIMARY KEY | Auto-increment |
|
||||||
|
| `series_id` | INTEGER | FOREIGN KEY → anime_series| Parent series |
|
||||||
|
| `season` | INTEGER | NOT NULL | Season number |
|
||||||
|
| `episode_number`| INTEGER | NOT NULL | Episode number |
|
||||||
|
| `is_downloaded` | BOOLEAN | DEFAULT FALSE | Download status |
|
||||||
|
|
||||||
|
### Relationships
|
||||||
|
|
||||||
|
- `AnimeSeries.episodes` → List of Episode objects (one-to-many)
|
||||||
|
- `Episode.series` → Parent AnimeSeries (many-to-one)
|
||||||
|
- Cascade delete: Deleting a series removes all its episodes
|
||||||
|
|
||||||
|
### Queries
|
||||||
|
|
||||||
|
```python
|
||||||
|
# Get all series with episodes
|
||||||
|
AnimeSeriesService.get_all(db, with_episodes=True)
|
||||||
|
|
||||||
|
# Get by provider key
|
||||||
|
AnimeSeriesService.get_by_key(db, key)
|
||||||
|
|
||||||
|
# Get by folder path
|
||||||
|
AnimeSeriesService.get_by_folder(db, folder)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 14. Database Location
|
||||||
|
|
||||||
| Environment | Default Location |
|
| Environment | Default Location |
|
||||||
| ----------- | ------------------------------------------------- |
|
| ----------- | ------------------------------------------------- |
|
||||||
|
|||||||
@@ -181,6 +181,22 @@ scheduler = AsyncIOScheduler(jobstores=jobstores)
|
|||||||
|
|
||||||
**If server is down >1 hour:** No automatic recovery. Manual trigger via `POST /api/scheduler/trigger-rescan` or wait for next scheduled run.
|
**If server is down >1 hour:** No automatic recovery. Manual trigger via `POST /api/scheduler/trigger-rescan` or wait for next scheduled run.
|
||||||
|
|
||||||
|
### Database Session Management
|
||||||
|
|
||||||
|
`get_async_session_factory()` returns a **new AsyncSession instance** directly (not a factory). The function name is historical — callers receive the session immediately:
|
||||||
|
|
||||||
|
```python
|
||||||
|
# Correct usage:
|
||||||
|
db = get_async_session_factory() # db IS the session
|
||||||
|
await db.execute(...)
|
||||||
|
await db.commit()
|
||||||
|
await db.close()
|
||||||
|
```
|
||||||
|
|
||||||
|
Do NOT call the result again with `()` — that tries to call an `AsyncSession` object, causing `'AsyncSession' object is not callable`.
|
||||||
|
|
||||||
|
For context manager usage, prefer `get_db_session()` (auto-commits) or `get_transactional_session()` (manual commit).
|
||||||
|
|
||||||
### Health Check Endpoints
|
### Health Check Endpoints
|
||||||
|
|
||||||
The application provides health check endpoints for monitoring and container orchestration:
|
The application provides health check endpoints for monitoring and container orchestration:
|
||||||
@@ -398,3 +414,29 @@ forward this to `notification_service.notify_download_failed()` so users see
|
|||||||
a HIGH-priority alert. The loader keeps the failure detail in
|
a HIGH-priority alert. The loader keeps the failure detail in
|
||||||
`logs/download_errors.log` for post-mortem.
|
`logs/download_errors.log` for post-mortem.
|
||||||
|
|
||||||
|
## Series Storage
|
||||||
|
|
||||||
|
### Overview
|
||||||
|
|
||||||
|
Series metadata now stored in the database (SQLAlchemy ORM).
|
||||||
|
Legacy files (`key` and `data` per folder) are deprecated but preserved
|
||||||
|
for backward compatibility.
|
||||||
|
|
||||||
|
### Architecture
|
||||||
|
|
||||||
|
- **Database**: Single source of truth for all series metadata
|
||||||
|
- **In-Memory Cache**: SeriesApp maintains a cache for performance
|
||||||
|
- **Filesystem**: Only used for episode files themselves, not metadata
|
||||||
|
|
||||||
|
### Migration
|
||||||
|
|
||||||
|
First startup after upgrade automatically imports any legacy
|
||||||
|
series files into the database.
|
||||||
|
|
||||||
|
### Legacy Files
|
||||||
|
|
||||||
|
- `key` file: Contains series provider key (deprecated)
|
||||||
|
- `data` file: Contains Serie JSON object (deprecated)
|
||||||
|
|
||||||
|
Both are safe to delete after migration; not needed for normal operation.
|
||||||
|
|
||||||
|
|||||||
111
docs/MIGRATION_GUIDE.md
Normal file
111
docs/MIGRATION_GUIDE.md
Normal file
@@ -0,0 +1,111 @@
|
|||||||
|
# Migration Guide: File-Based to Database Storage
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
This guide covers the transition from file-based series metadata storage to the new database-backed system introduced in v2.0.
|
||||||
|
|
||||||
|
## What Changed
|
||||||
|
|
||||||
|
**Before v2.0**: Series metadata stored in `key` and `data` files alongside anime folders.
|
||||||
|
|
||||||
|
**After v2.0**: All metadata stored in SQLite database (`aniworld.db`). Files are deprecated but still supported for backward compatibility during migration.
|
||||||
|
|
||||||
|
## Automated Migration
|
||||||
|
|
||||||
|
The application automatically migrates on first startup:
|
||||||
|
|
||||||
|
1. Scans anime directory for `key` and `data` files
|
||||||
|
2. Parses legacy files into `AnimeSeries` and `Episode` records
|
||||||
|
3. Loads series into in-memory cache
|
||||||
|
4. Logs migration results
|
||||||
|
|
||||||
|
**No manual action required.**
|
||||||
|
|
||||||
|
## Manual Verification
|
||||||
|
|
||||||
|
After first startup with the new version:
|
||||||
|
|
||||||
|
1. **Check logs** for: `"Migrated X series from files to DB"`
|
||||||
|
2. **Verify series count**: UI shows same number of series as before
|
||||||
|
3. **Confirm episodes**: Episode counts match expected totals
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Check migration log
|
||||||
|
grep "Migrated" logs/app.log
|
||||||
|
|
||||||
|
# Verify series via API
|
||||||
|
curl http://localhost:8000/api/anime | jq '.total'
|
||||||
|
```
|
||||||
|
|
||||||
|
## After Migration
|
||||||
|
|
||||||
|
### Safe to Delete
|
||||||
|
|
||||||
|
Once verified, these files can be removed:
|
||||||
|
|
||||||
|
```
|
||||||
|
<anime_folder>/
|
||||||
|
├── Attack on Titan (2013)/
|
||||||
|
│ ├── key # ❌ Can delete
|
||||||
|
│ ├── data # ❌ Can delete
|
||||||
|
│ └── Season 1/
|
||||||
|
│ └── ...
|
||||||
|
```
|
||||||
|
|
||||||
|
**Deleting these files does not affect the database.** The metadata now lives in `aniworld.db`.
|
||||||
|
|
||||||
|
### Backup (Recommended)
|
||||||
|
|
||||||
|
Before deleting, backup the files:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Create backup directory
|
||||||
|
mkdir -p backup/legacy_series_files
|
||||||
|
|
||||||
|
# Copy all key and data files
|
||||||
|
find /path/to/anime -name "key" -o -name "data" | while read f; do
|
||||||
|
cp "$f" "backup/legacy_series_files/"
|
||||||
|
done
|
||||||
|
```
|
||||||
|
|
||||||
|
## Reverting (Not Recommended)
|
||||||
|
|
||||||
|
If you must revert to file-based storage:
|
||||||
|
|
||||||
|
1. **Restore from database backup** (if available)
|
||||||
|
2. **Export manually** (no export script exists)
|
||||||
|
|
||||||
|
**Warning**: File-based storage is deprecated and will be removed in v3.0.0.
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Series Not Appearing After Migration
|
||||||
|
|
||||||
|
1. Check logs for migration errors: `grep -i error logs/app.log`
|
||||||
|
2. Verify `key` and `data` files exist and are readable
|
||||||
|
3. Manually trigger rescan: `POST /api/scheduler/trigger-rescan`
|
||||||
|
|
||||||
|
### Duplicate Series
|
||||||
|
|
||||||
|
1. Check for duplicate `key` files (same series in multiple folders)
|
||||||
|
2. Verify series key uniqueness in database:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sqlite3 aniworld.db "SELECT key, COUNT(*) FROM anime_series GROUP BY key HAVING COUNT(*) > 1;"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Missing Episodes
|
||||||
|
|
||||||
|
1. Trigger targeted scan for affected series
|
||||||
|
2. Check episode sync logs
|
||||||
|
3. Verify file permissions on anime directory
|
||||||
|
|
||||||
|
## Deprecation Timeline
|
||||||
|
|
||||||
|
| Version | Status |
|
||||||
|
|---------|--------|
|
||||||
|
| v2.0.x | Legacy files supported, migration automated |
|
||||||
|
| v2.1.x | Legacy files still supported, warnings in logs |
|
||||||
|
| v3.0.0 | **Legacy files removed** - database only |
|
||||||
|
|
||||||
|
Upgrade to v3.0.0 before legacy file support ends.
|
||||||
@@ -246,6 +246,7 @@ NFO files are created in the anime directory:
|
|||||||
<genre>Action</genre>
|
<genre>Action</genre>
|
||||||
<genre>Sci-Fi & Fantasy</genre>
|
<genre>Sci-Fi & Fantasy</genre>
|
||||||
<uniqueid type="tmdb">1429</uniqueid>
|
<uniqueid type="tmdb">1429</uniqueid>
|
||||||
|
<tmdbid>1429</tmdbid>
|
||||||
<thumb aspect="poster">https://image.tmdb.org/t/p/w500/...</thumb>
|
<thumb aspect="poster">https://image.tmdb.org/t/p/w500/...</thumb>
|
||||||
<fanart>
|
<fanart>
|
||||||
<thumb>https://image.tmdb.org/t/p/original/...</thumb>
|
<thumb>https://image.tmdb.org/t/p/original/...</thumb>
|
||||||
@@ -253,6 +254,13 @@ NFO files are created in the anime directory:
|
|||||||
</tvshow>
|
</tvshow>
|
||||||
```
|
```
|
||||||
|
|
||||||
|
**Manual TMDB ID Override**: To skip TMDB search and use a specific ID directly, include `<tmdbid>YOUR_ID</tmdbid>` in the NFO. This is useful when:
|
||||||
|
- TMDB search fails for your series (e.g., new or obscure anime)
|
||||||
|
- You already know the correct TMDB ID
|
||||||
|
- You want to avoid rate limiting from repeated searches
|
||||||
|
|
||||||
|
Aniworld reads `<tmdbid>` element and `<uniqueid type="tmdb">` first. If found, it uses the ID directly instead of searching.
|
||||||
|
|
||||||
### 4.3 Episode NFO Format
|
### 4.3 Episode NFO Format
|
||||||
|
|
||||||
```xml
|
```xml
|
||||||
@@ -629,6 +637,36 @@ Every poster check action is logged:
|
|||||||
4. Check network speed to TMDB servers
|
4. Check network speed to TMDB servers
|
||||||
5. Verify disk I/O performance
|
5. Verify disk I/O performance
|
||||||
|
|
||||||
|
### 6.7 TMDB Lookup Fails for My Series
|
||||||
|
|
||||||
|
**Problem**: TMDB search fails with "No results found" for a valid series.
|
||||||
|
|
||||||
|
**Solutions**:
|
||||||
|
|
||||||
|
1. **Check if series exists on TMDB**: Visit https://www.themoviedb.org and search for your series
|
||||||
|
2. **Use manual ID override**: Add TMDB ID directly to `tvshow.nfo`:
|
||||||
|
```xml
|
||||||
|
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
|
||||||
|
<tvshow>
|
||||||
|
<title>Your Series Name</title>
|
||||||
|
<tmdbid>12345</tmdbid>
|
||||||
|
<uniqueid type="tmdb">12345</uniqueid>
|
||||||
|
</tvshow>
|
||||||
|
```
|
||||||
|
Aniworld will use this ID directly instead of searching.
|
||||||
|
|
||||||
|
3. **Try alternative titles**: Some anime have different titles (Japanese, romaji, English). If you have access to the folder, rename it to match the TMDB title.
|
||||||
|
|
||||||
|
4. **Add to existing NFO**: If `tvshow.nfo` exists but has no TMDB ID, edit it to add:
|
||||||
|
```xml
|
||||||
|
<tmdbid>YOUR_TMDB_ID</tmdbid>
|
||||||
|
```
|
||||||
|
Then use the Update endpoint to refresh metadata.
|
||||||
|
|
||||||
|
5. **Check for rate limiting**: If many lookups fail at once, you may be hitting TMDB rate limits. Wait and retry later.
|
||||||
|
|
||||||
|
6. **Verify API key**: Ensure your TMDB API key is valid and has not exceeded usage limits.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 7. Best Practices
|
## 7. Best Practices
|
||||||
|
|||||||
@@ -31,14 +31,16 @@ flowchart TB
|
|||||||
|
|
||||||
subgraph Core["Core Layer"]
|
subgraph Core["Core Layer"]
|
||||||
SeriesApp["SeriesApp"]
|
SeriesApp["SeriesApp"]
|
||||||
|
SeriesCache["SeriesCache<br/>(In-Memory)"]
|
||||||
SerieScanner["SerieScanner"]
|
SerieScanner["SerieScanner"]
|
||||||
SerieList["SerieList"]
|
SerieList["SerieList"]
|
||||||
end
|
end
|
||||||
|
|
||||||
subgraph Data["Data Layer"]
|
subgraph Data["Data Layer"]
|
||||||
SQLite[(SQLite<br/>aniworld.db)]
|
SQLite[("SQLite<br/>aniworld.db")]
|
||||||
ConfigJSON[(config.json)]
|
ConfigJSON[(config.json)]
|
||||||
FileSystem[(File System<br/>Anime Directory)]
|
FileSystem[(File System<br/>Anime Episodes)]
|
||||||
|
LegacyFiles[("Legacy Files<br/>key/data<br/>(Deprecated)")]
|
||||||
end
|
end
|
||||||
|
|
||||||
subgraph External["External"]
|
subgraph External["External"]
|
||||||
@@ -71,9 +73,13 @@ flowchart TB
|
|||||||
AnimeService --> SQLite
|
AnimeService --> SQLite
|
||||||
|
|
||||||
%% Core to Data
|
%% Core to Data
|
||||||
|
SeriesApp --> SeriesCache
|
||||||
|
SeriesCache -.->|Cached Series| SQLite
|
||||||
SeriesApp --> SerieScanner
|
SeriesApp --> SerieScanner
|
||||||
SeriesApp --> SerieList
|
SeriesApp --> SerieList
|
||||||
SerieScanner --> FileSystem
|
SerieScanner -->|Scan Episodes| FileSystem
|
||||||
|
SerieScanner -->|Detect Series| SQLite
|
||||||
|
SerieScanner -->|Migrate Legacy| LegacyFiles
|
||||||
SerieScanner --> Provider
|
SerieScanner --> Provider
|
||||||
|
|
||||||
%% Event flow
|
%% Event flow
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "aniworld-web",
|
"name": "aniworld-web",
|
||||||
"version": "1.1.13",
|
"version": "1.1.18",
|
||||||
"description": "Aniworld Anime Download Manager - Web Frontend",
|
"description": "Aniworld Anime Download Manager - Web Frontend",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"scripts": {
|
"scripts": {
|
||||||
|
|||||||
@@ -10,6 +10,7 @@ Note:
|
|||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
import logging
|
import logging
|
||||||
import os
|
import os
|
||||||
import re
|
import re
|
||||||
@@ -23,6 +24,9 @@ from src.core.entities.series import Serie
|
|||||||
from src.core.exceptions.Exceptions import MatchNotFoundError, NoKeyFoundException
|
from src.core.exceptions.Exceptions import MatchNotFoundError, NoKeyFoundException
|
||||||
from src.core.providers.base_provider import Loader
|
from src.core.providers.base_provider import Loader
|
||||||
|
|
||||||
|
from src.server.database.connection import get_sync_session
|
||||||
|
from src.server.database.service import AnimeSeriesService, EpisodeService
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
error_logger = logging.getLogger("error")
|
error_logger = logging.getLogger("error")
|
||||||
no_key_found_logger = logging.getLogger("series.nokey")
|
no_key_found_logger = logging.getLogger("series.nokey")
|
||||||
@@ -40,8 +44,13 @@ class SerieScanner:
|
|||||||
in keyDict and can be retrieved after scanning.
|
in keyDict and can be retrieved after scanning.
|
||||||
|
|
||||||
Example:
|
Example:
|
||||||
|
# Synchronous context (CLI):
|
||||||
scanner = SerieScanner("/path/to/anime", loader)
|
scanner = SerieScanner("/path/to/anime", loader)
|
||||||
scanner.scan()
|
scanner.scan() # asyncio.run() used internally when no event loop
|
||||||
|
|
||||||
|
# Asynchronous context (server/scheduler):
|
||||||
|
# scan() detects running event loop and uses create_task()
|
||||||
|
# internally, so no special handling needed by caller.
|
||||||
# Results are in scanner.keyDict
|
# Results are in scanner.keyDict
|
||||||
|
|
||||||
# With DB lookup fallback:
|
# With DB lookup fallback:
|
||||||
@@ -205,6 +214,105 @@ class SerieScanner:
|
|||||||
"""Reinitialize the series dictionary (keyed by serie.key)."""
|
"""Reinitialize the series dictionary (keyed by serie.key)."""
|
||||||
self.keyDict: dict[str, Serie] = {}
|
self.keyDict: dict[str, Serie] = {}
|
||||||
|
|
||||||
|
async def _persist_serie_to_db(self, serie: Serie) -> None:
|
||||||
|
"""Persist serie to database (create or update).
|
||||||
|
|
||||||
|
Args:
|
||||||
|
serie: Serie domain object to persist
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
from src.server.database.connection import get_async_session_factory
|
||||||
|
|
||||||
|
db = get_async_session_factory()
|
||||||
|
try:
|
||||||
|
existing = await AnimeSeriesService.get_by_key(db, serie.key)
|
||||||
|
if existing:
|
||||||
|
await AnimeSeriesService.update(
|
||||||
|
db, existing.id,
|
||||||
|
name=serie.name,
|
||||||
|
folder=serie.folder,
|
||||||
|
year=serie.year
|
||||||
|
)
|
||||||
|
await self._sync_episodes_to_db(db, existing.id, serie.episodeDict)
|
||||||
|
else:
|
||||||
|
anime_series = await AnimeSeriesService.create(
|
||||||
|
db=db,
|
||||||
|
key=serie.key,
|
||||||
|
name=serie.name,
|
||||||
|
site=serie.site,
|
||||||
|
folder=serie.folder,
|
||||||
|
year=serie.year
|
||||||
|
)
|
||||||
|
for season, eps in serie.episodeDict.items():
|
||||||
|
for ep in eps:
|
||||||
|
await EpisodeService.create(
|
||||||
|
db=db,
|
||||||
|
series_id=anime_series.id,
|
||||||
|
season=season,
|
||||||
|
episode_number=ep
|
||||||
|
)
|
||||||
|
await db.commit()
|
||||||
|
logger.debug(
|
||||||
|
"Persisted serie '%s' (key=%s) to database",
|
||||||
|
serie.name, serie.key
|
||||||
|
)
|
||||||
|
except Exception as e:
|
||||||
|
await db.rollback()
|
||||||
|
logger.error(
|
||||||
|
"Failed to persist serie '%s' to DB: %s",
|
||||||
|
serie.key, e, exc_info=True
|
||||||
|
)
|
||||||
|
raise
|
||||||
|
finally:
|
||||||
|
await db.close()
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(
|
||||||
|
"Could not persist serie '%s' to DB (DB unavailable?): %s",
|
||||||
|
serie.key, e
|
||||||
|
)
|
||||||
|
|
||||||
|
async def _sync_episodes_to_db(
|
||||||
|
self, db, series_id: int, episode_dict: dict[int, list[int]]
|
||||||
|
) -> None:
|
||||||
|
"""Sync episodes to database, preserving downloaded flags.
|
||||||
|
|
||||||
|
Adds missing episodes, removes episodes no longer missing,
|
||||||
|
and preserves is_downloaded=True episodes.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
db: Async database session
|
||||||
|
series_id: Database ID of the series
|
||||||
|
episode_dict: Dict mapping season -> list of episode numbers
|
||||||
|
"""
|
||||||
|
existing_episodes = await EpisodeService.get_by_series(db, series_id)
|
||||||
|
existing_map = {
|
||||||
|
(ep.season, ep.episode_number): ep for ep in existing_episodes
|
||||||
|
}
|
||||||
|
new_keys = set()
|
||||||
|
for season, eps in episode_dict.items():
|
||||||
|
for ep_num in eps:
|
||||||
|
new_keys.add((season, ep_num))
|
||||||
|
for (season, ep_num), ep in existing_map.items():
|
||||||
|
if (season, ep_num) not in new_keys:
|
||||||
|
if ep.is_downloaded:
|
||||||
|
logger.debug(
|
||||||
|
"Preserving downloaded episode S%02dE%02d for series_id=%d",
|
||||||
|
season, ep_num, series_id
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
await EpisodeService.delete_by_series(
|
||||||
|
db, series_id, season, ep_num
|
||||||
|
)
|
||||||
|
for season, eps in episode_dict.items():
|
||||||
|
for ep_num in eps:
|
||||||
|
if (season, ep_num) not in existing_map:
|
||||||
|
await EpisodeService.create(
|
||||||
|
db=db,
|
||||||
|
series_id=series_id,
|
||||||
|
season=season,
|
||||||
|
episode_number=ep_num
|
||||||
|
)
|
||||||
|
|
||||||
def get_total_to_scan(self) -> int:
|
def get_total_to_scan(self) -> int:
|
||||||
"""Get the total number of folders to scan.
|
"""Get the total number of folders to scan.
|
||||||
|
|
||||||
@@ -278,25 +386,6 @@ class SerieScanner:
|
|||||||
)
|
)
|
||||||
|
|
||||||
serie = self.__read_data_from_file(folder)
|
serie = self.__read_data_from_file(folder)
|
||||||
if serie is None or not serie.key or not serie.key.strip():
|
|
||||||
# Fallback: ask the database for a matching series
|
|
||||||
if self._db_lookup is not None:
|
|
||||||
try:
|
|
||||||
serie = self._db_lookup(folder)
|
|
||||||
if serie:
|
|
||||||
logger.info(
|
|
||||||
"DB lookup resolved folder '%s' -> key='%s'",
|
|
||||||
folder,
|
|
||||||
serie.key,
|
|
||||||
)
|
|
||||||
except Exception as exc:
|
|
||||||
logger.warning(
|
|
||||||
"DB lookup failed for folder '%s': %s",
|
|
||||||
folder,
|
|
||||||
exc,
|
|
||||||
)
|
|
||||||
serie = None
|
|
||||||
|
|
||||||
if serie is None or not serie.key or not serie.key.strip():
|
if serie is None or not serie.key or not serie.key.strip():
|
||||||
logger.warning(
|
logger.warning(
|
||||||
"No key or data file found for folder '%s', skipping",
|
"No key or data file found for folder '%s', skipping",
|
||||||
@@ -345,10 +434,23 @@ class SerieScanner:
|
|||||||
)
|
)
|
||||||
serie.episodeDict = missing_episodes
|
serie.episodeDict = missing_episodes
|
||||||
serie.folder = folder
|
serie.folder = folder
|
||||||
data_path = os.path.join(
|
|
||||||
self.directory, folder, 'data'
|
# Persist to database (async)
|
||||||
)
|
try:
|
||||||
serie.save_to_file(data_path)
|
try:
|
||||||
|
loop = asyncio.get_running_loop()
|
||||||
|
except RuntimeError:
|
||||||
|
# No running loop — safe to use asyncio.run()
|
||||||
|
asyncio.run(self._persist_serie_to_db(serie))
|
||||||
|
else:
|
||||||
|
# Already in async context — schedule as task
|
||||||
|
asyncio.create_task(self._persist_serie_to_db(serie))
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(
|
||||||
|
"DB persistence failed for '%s', "
|
||||||
|
"continuing without DB: %s",
|
||||||
|
serie.key, e
|
||||||
|
)
|
||||||
|
|
||||||
# Store by key (primary identifier), not folder
|
# Store by key (primary identifier), not folder
|
||||||
if serie.key in self.keyDict:
|
if serie.key in self.keyDict:
|
||||||
@@ -470,25 +572,83 @@ class SerieScanner:
|
|||||||
yield anime_name, mp4_files if has_files else []
|
yield anime_name, mp4_files if has_files else []
|
||||||
|
|
||||||
def __read_data_from_file(self, folder_name: str) -> Optional[Serie]:
|
def __read_data_from_file(self, folder_name: str) -> Optional[Serie]:
|
||||||
"""Read serie data from file or key file.
|
"""Load or discover a Serie for the given folder.
|
||||||
|
|
||||||
|
Strategy:
|
||||||
|
1. Query DB by folder name
|
||||||
|
2. If found, return cached Serie object
|
||||||
|
3. If not in DB, fall back to provider search via _db_lookup callback
|
||||||
|
4. (Legacy) If still not found, try reading 'key' file as last resort
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
folder_name: Filesystem folder name
|
folder_name: Filesystem folder name
|
||||||
(used only to locate data files)
|
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Serie object with valid key if found, None otherwise
|
Serie object with valid key if found, None otherwise
|
||||||
|
|
||||||
Note:
|
|
||||||
The returned Serie will have its 'key' as the primary identifier.
|
|
||||||
The 'folder' field is metadata only.
|
|
||||||
"""
|
|
||||||
folder_path = os.path.join(self.directory, folder_name)
|
|
||||||
key = None
|
|
||||||
key_file = os.path.join(folder_path, 'key')
|
|
||||||
serie_file = os.path.join(folder_path, 'data')
|
|
||||||
|
|
||||||
|
Note:
|
||||||
|
DB is the source of truth. File-based lookups (key/data files)
|
||||||
|
are temporary backward compatibility for deployments with old data.
|
||||||
|
Will be removed in v3.0.0.
|
||||||
|
"""
|
||||||
|
# Step 1: Try DB lookup by folder name
|
||||||
|
try:
|
||||||
|
session = get_sync_session()
|
||||||
|
try:
|
||||||
|
anime_series = AnimeSeriesService.get_by_folder_sync(session, folder_name)
|
||||||
|
if anime_series:
|
||||||
|
# Reconstruct Serie from DB record
|
||||||
|
episode_dict: dict[int, list[int]] = {}
|
||||||
|
if anime_series.episodes:
|
||||||
|
for ep in anime_series.episodes:
|
||||||
|
season = ep.season or 1
|
||||||
|
if season not in episode_dict:
|
||||||
|
episode_dict[season] = []
|
||||||
|
episode_dict[season].append(ep.episode_number or ep.number or 0)
|
||||||
|
return Serie(
|
||||||
|
key=anime_series.key,
|
||||||
|
name=anime_series.name,
|
||||||
|
site=anime_series.site,
|
||||||
|
folder=anime_series.folder,
|
||||||
|
episodeDict=episode_dict,
|
||||||
|
year=anime_series.year
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
session.close()
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning(
|
||||||
|
"DB lookup failed for folder '%s': %s",
|
||||||
|
folder_name,
|
||||||
|
exc
|
||||||
|
)
|
||||||
|
|
||||||
|
# Step 2: Fall back to provider search callback
|
||||||
|
if self._db_lookup is not None:
|
||||||
|
try:
|
||||||
|
serie = self._db_lookup(folder_name)
|
||||||
|
if serie and serie.key and serie.key.strip():
|
||||||
|
logger.info(
|
||||||
|
"Provider lookup resolved folder '%s' -> key='%s'",
|
||||||
|
folder_name,
|
||||||
|
serie.key
|
||||||
|
)
|
||||||
|
return serie
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning(
|
||||||
|
"Provider lookup failed for folder '%s': %s",
|
||||||
|
folder_name,
|
||||||
|
exc
|
||||||
|
)
|
||||||
|
|
||||||
|
# Step 3: Legacy fallback - TEMPORARY (remove in v3.0.0)
|
||||||
|
folder_path = os.path.join(self.directory, folder_name)
|
||||||
|
key_file = os.path.join(folder_path, 'key')
|
||||||
if os.path.exists(key_file):
|
if os.path.exists(key_file):
|
||||||
|
logger.warning(
|
||||||
|
"Using legacy 'key' file for '%s' - this fallback is deprecated "
|
||||||
|
"and will be removed in v3.0.0",
|
||||||
|
folder_name
|
||||||
|
)
|
||||||
with open(key_file, 'r', encoding='utf-8') as file:
|
with open(key_file, 'r', encoding='utf-8') as file:
|
||||||
key = file.read().strip()
|
key = file.read().strip()
|
||||||
logger.info(
|
logger.info(
|
||||||
@@ -499,6 +659,7 @@ class SerieScanner:
|
|||||||
year_from_folder = self._extract_year_from_folder_name(folder_name)
|
year_from_folder = self._extract_year_from_folder_name(folder_name)
|
||||||
return Serie(key, "", "aniworld.to", folder_name, dict(), year=year_from_folder)
|
return Serie(key, "", "aniworld.to", folder_name, dict(), year=year_from_folder)
|
||||||
|
|
||||||
|
serie_file = os.path.join(folder_path, 'data')
|
||||||
if os.path.exists(serie_file):
|
if os.path.exists(serie_file):
|
||||||
with open(serie_file, "rb") as file:
|
with open(serie_file, "rb") as file:
|
||||||
logger.info(
|
logger.info(
|
||||||
|
|||||||
@@ -1,320 +1,527 @@
|
|||||||
"""Utilities for loading and managing stored anime series metadata.
|
"""Utilities for loading and managing stored anime series metadata.
|
||||||
|
|
||||||
This module provides the SerieList class for managing collections of anime
|
This module provides the SerieList class for managing collections of anime
|
||||||
series metadata. It uses file-based storage only.
|
series metadata. It supports loading from both filesystem (legacy) and
|
||||||
|
database (primary).
|
||||||
Note:
|
|
||||||
This module is part of the core domain layer and has no database
|
Note:
|
||||||
dependencies. All database operations are handled by the service layer.
|
This module is part of the core domain layer. Database operations
|
||||||
"""
|
are handled by the service layer via add_to_db().
|
||||||
|
"""
|
||||||
from __future__ import annotations
|
|
||||||
|
from __future__ import annotations
|
||||||
import logging
|
|
||||||
import os
|
import logging
|
||||||
import warnings
|
import os
|
||||||
from json import JSONDecodeError
|
import warnings
|
||||||
from typing import Dict, Iterable, List, Optional
|
from json import JSONDecodeError
|
||||||
|
from typing import Dict, Iterable, List, Optional
|
||||||
from src.core.entities.series import Serie
|
|
||||||
|
from src.core.entities.series import Serie
|
||||||
logger = logging.getLogger(__name__)
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
class SerieList:
|
|
||||||
"""
|
class SerieList:
|
||||||
Represents the collection of cached series stored on disk.
|
"""
|
||||||
|
Represents the collection of cached series stored on disk.
|
||||||
Series are identified by their unique 'key' (provider identifier).
|
|
||||||
The 'folder' is metadata only and not used for lookups.
|
Series are identified by their unique 'key' (provider identifier).
|
||||||
|
The 'folder' is metadata only and not used for lookups.
|
||||||
This class manages in-memory series data loaded from filesystem.
|
|
||||||
It has no database dependencies - all persistence is handled by
|
This class manages in-memory series data loaded from filesystem.
|
||||||
the service layer.
|
It has no database dependencies - all persistence is handled by
|
||||||
|
the service layer.
|
||||||
Example:
|
|
||||||
# File-based mode
|
Example:
|
||||||
serie_list = SerieList("/path/to/anime")
|
# File-based mode
|
||||||
series = serie_list.get_all()
|
serie_list = SerieList("/path/to/anime")
|
||||||
|
series = serie_list.get_all()
|
||||||
Attributes:
|
|
||||||
directory: Path to the anime directory
|
Attributes:
|
||||||
keyDict: Internal dictionary mapping serie.key to Serie objects
|
directory: Path to the anime directory
|
||||||
"""
|
keyDict: Internal dictionary mapping serie.key to Serie objects
|
||||||
|
"""
|
||||||
def __init__(
|
|
||||||
self,
|
def __init__(
|
||||||
base_path: str,
|
self,
|
||||||
skip_load: bool = False
|
base_path: str,
|
||||||
) -> None:
|
skip_load: bool = False
|
||||||
"""Initialize the SerieList.
|
) -> None:
|
||||||
|
"""Initialize the SerieList.
|
||||||
Args:
|
|
||||||
base_path: Path to the anime directory
|
Args:
|
||||||
skip_load: If True, skip automatic loading of series from files.
|
base_path: Path to the anime directory
|
||||||
Useful when planning to load from database instead.
|
skip_load: If True, skip automatic loading of series from files.
|
||||||
"""
|
Useful when planning to load from database instead.
|
||||||
self.directory: str = base_path
|
"""
|
||||||
# Internal storage using serie.key as the dictionary key
|
self.directory: str = base_path
|
||||||
self.keyDict: Dict[str, Serie] = {}
|
# Internal storage using serie.key as the dictionary key
|
||||||
|
self.keyDict: Dict[str, Serie] = {}
|
||||||
# Only auto-load from files if not skipping
|
|
||||||
if not skip_load:
|
# Only auto-load from files if not skipping
|
||||||
self.load_series()
|
if not skip_load:
|
||||||
|
self.load_series()
|
||||||
def add(self, serie: Serie, use_sanitized_folder: bool = True) -> str:
|
|
||||||
"""
|
def add(self, serie: Serie, use_sanitized_folder: bool = True) -> str:
|
||||||
Persist a new series if it is not already present (file-based mode).
|
"""
|
||||||
|
Persist a new series if it is not already present (file-based mode).
|
||||||
Uses serie.key for identification. Creates the filesystem folder
|
|
||||||
using either the sanitized display name (default) or the existing
|
Uses serie.key for identification. Creates the filesystem folder
|
||||||
folder property.
|
using either the sanitized display name (default) or the existing
|
||||||
|
folder property.
|
||||||
Args:
|
|
||||||
serie: The Serie instance to add
|
Args:
|
||||||
use_sanitized_folder: If True (default), use serie.sanitized_folder
|
serie: The Serie instance to add
|
||||||
for the filesystem folder name based on display name.
|
use_sanitized_folder: If True (default), use serie.sanitized_folder
|
||||||
If False, use serie.folder as-is for backward compatibility.
|
for the filesystem folder name based on display name.
|
||||||
|
If False, use serie.folder as-is for backward compatibility.
|
||||||
Returns:
|
|
||||||
str: The folder path that was created/used
|
Returns:
|
||||||
|
str: The folder path that was created/used
|
||||||
Note:
|
|
||||||
This method creates data files on disk. For database storage,
|
Note:
|
||||||
use add_to_db() instead.
|
This method creates data files on disk. For database storage,
|
||||||
"""
|
use add_to_db() instead.
|
||||||
if self.contains(serie.key):
|
"""
|
||||||
# Return existing folder path
|
if self.contains(serie.key):
|
||||||
existing = self.keyDict[serie.key]
|
# Return existing folder path
|
||||||
return os.path.join(self.directory, existing.folder)
|
existing = self.keyDict[serie.key]
|
||||||
|
return os.path.join(self.directory, existing.folder)
|
||||||
# Determine folder name to use
|
|
||||||
if use_sanitized_folder:
|
# Determine folder name to use
|
||||||
folder_name = serie.sanitized_folder
|
if use_sanitized_folder:
|
||||||
# Update the serie's folder property to match what we create
|
folder_name = serie.sanitized_folder
|
||||||
serie.folder = folder_name
|
# Update the serie's folder property to match what we create
|
||||||
else:
|
serie.folder = folder_name
|
||||||
folder_name = serie.folder
|
else:
|
||||||
|
folder_name = serie.folder
|
||||||
data_path = os.path.join(self.directory, folder_name, "data")
|
|
||||||
anime_path = os.path.join(self.directory, folder_name)
|
data_path = os.path.join(self.directory, folder_name, "data")
|
||||||
os.makedirs(anime_path, exist_ok=True)
|
anime_path = os.path.join(self.directory, folder_name)
|
||||||
if not os.path.isfile(data_path):
|
os.makedirs(anime_path, exist_ok=True)
|
||||||
serie.save_to_file(data_path)
|
if not os.path.isfile(data_path):
|
||||||
# Store by key, not folder
|
serie.save_to_file(data_path)
|
||||||
self.keyDict[serie.key] = serie
|
# Store by key, not folder
|
||||||
|
self.keyDict[serie.key] = serie
|
||||||
return anime_path
|
|
||||||
|
return anime_path
|
||||||
def contains(self, key: str) -> bool:
|
|
||||||
"""
|
async def add_to_db(self, serie: Serie) -> bool:
|
||||||
Return True when a series identified by ``key`` already exists.
|
"""Persist a new series to the database.
|
||||||
|
|
||||||
Args:
|
Creates the filesystem folder using serie.folder, then persists
|
||||||
key: The unique provider identifier for the series
|
the series metadata to the database.
|
||||||
|
|
||||||
Returns:
|
Args:
|
||||||
True if the series exists in the collection
|
serie: The Serie instance to add
|
||||||
"""
|
|
||||||
return key in self.keyDict
|
Returns:
|
||||||
|
True if successful, False otherwise
|
||||||
def load_series(self) -> None:
|
"""
|
||||||
"""Populate the in-memory map with metadata discovered on disk."""
|
try:
|
||||||
|
from src.server.database.connection import get_async_session_factory
|
||||||
logger.info("Scanning anime folders in %s", self.directory)
|
from src.server.database.service import AnimeSeriesService, EpisodeService
|
||||||
try:
|
|
||||||
entries: Iterable[str] = os.listdir(self.directory)
|
folder_name = serie.folder
|
||||||
except OSError as error:
|
anime_path = os.path.join(self.directory, folder_name)
|
||||||
logger.error(
|
os.makedirs(anime_path, exist_ok=True)
|
||||||
"Unable to scan directory %s: %s",
|
|
||||||
self.directory,
|
session_factory = get_async_session_factory()
|
||||||
error,
|
db = session_factory()
|
||||||
)
|
try:
|
||||||
return
|
existing = await AnimeSeriesService.get_by_key(db, serie.key)
|
||||||
|
if existing:
|
||||||
nfo_stats = {"total": 0, "with_nfo": 0, "without_nfo": 0}
|
logger.debug(
|
||||||
media_stats = {
|
"Series '%s' (key=%s) already exists in DB, skipping",
|
||||||
"with_poster": 0,
|
serie.name, serie.key
|
||||||
"without_poster": 0,
|
)
|
||||||
"with_logo": 0,
|
return True
|
||||||
"without_logo": 0,
|
|
||||||
"with_fanart": 0,
|
anime_series = await AnimeSeriesService.create(
|
||||||
"without_fanart": 0
|
db=db,
|
||||||
}
|
key=serie.key,
|
||||||
|
name=serie.name,
|
||||||
for anime_folder in entries:
|
site=serie.site,
|
||||||
anime_path = os.path.join(self.directory, anime_folder, "data")
|
folder=folder_name,
|
||||||
if os.path.isfile(anime_path):
|
year=serie.year
|
||||||
logger.debug("Found data file for folder %s", anime_folder)
|
)
|
||||||
serie = self._load_data(anime_folder, anime_path)
|
for season, eps in serie.episodeDict.items():
|
||||||
|
for ep in eps:
|
||||||
if serie:
|
await EpisodeService.create(
|
||||||
nfo_stats["total"] += 1
|
db=db,
|
||||||
# Check for NFO file
|
series_id=anime_series.id,
|
||||||
nfo_file_path = os.path.join(
|
season=season,
|
||||||
self.directory, anime_folder, "tvshow.nfo"
|
episode_number=ep
|
||||||
)
|
)
|
||||||
if os.path.isfile(nfo_file_path):
|
await db.commit()
|
||||||
serie.nfo_path = nfo_file_path
|
self.keyDict[serie.key] = serie
|
||||||
nfo_stats["with_nfo"] += 1
|
logger.info(
|
||||||
else:
|
"Persisted series '%s' to database",
|
||||||
nfo_stats["without_nfo"] += 1
|
serie.name
|
||||||
logger.debug(
|
)
|
||||||
"Series '%s' (key: %s) is missing tvshow.nfo",
|
return True
|
||||||
serie.name,
|
except Exception as e:
|
||||||
serie.key
|
await db.rollback()
|
||||||
)
|
logger.error(
|
||||||
|
"Failed to persist series '%s' to DB: %s",
|
||||||
# Check for media files
|
serie.key, e, exc_info=True
|
||||||
folder_path = os.path.join(self.directory, anime_folder)
|
)
|
||||||
|
return False
|
||||||
poster_path = os.path.join(folder_path, "poster.jpg")
|
finally:
|
||||||
if os.path.isfile(poster_path):
|
await db.close()
|
||||||
media_stats["with_poster"] += 1
|
except Exception as e:
|
||||||
else:
|
logger.error(
|
||||||
media_stats["without_poster"] += 1
|
"Could not add series '%s' to DB (DB unavailable?): %s",
|
||||||
logger.debug(
|
serie.key, e
|
||||||
"Series '%s' (key: %s) is missing poster.jpg",
|
)
|
||||||
serie.name,
|
return False
|
||||||
serie.key
|
|
||||||
)
|
def contains(self, key: str) -> bool:
|
||||||
|
"""
|
||||||
logo_path = os.path.join(folder_path, "logo.png")
|
Return True when a series identified by ``key`` already exists.
|
||||||
if os.path.isfile(logo_path):
|
|
||||||
media_stats["with_logo"] += 1
|
Args:
|
||||||
else:
|
key: The unique provider identifier for the series
|
||||||
media_stats["without_logo"] += 1
|
|
||||||
logger.debug(
|
Returns:
|
||||||
"Series '%s' (key: %s) is missing logo.png",
|
True if the series exists in the collection
|
||||||
serie.name,
|
"""
|
||||||
serie.key
|
return key in self.keyDict
|
||||||
)
|
|
||||||
|
def load_series(self) -> None:
|
||||||
fanart_path = os.path.join(folder_path, "fanart.jpg")
|
"""Populate the in-memory map with metadata discovered on disk."""
|
||||||
if os.path.isfile(fanart_path):
|
|
||||||
media_stats["with_fanart"] += 1
|
logger.info("Scanning anime folders in %s", self.directory)
|
||||||
else:
|
try:
|
||||||
media_stats["without_fanart"] += 1
|
entries: Iterable[str] = os.listdir(self.directory)
|
||||||
logger.debug(
|
except OSError as error:
|
||||||
"Series '%s' (key: %s) is missing fanart.jpg",
|
logger.error(
|
||||||
serie.name,
|
"Unable to scan directory %s: %s",
|
||||||
serie.key
|
self.directory,
|
||||||
)
|
error,
|
||||||
|
)
|
||||||
continue
|
return
|
||||||
|
|
||||||
logger.warning(
|
nfo_stats = {"total": 0, "with_nfo": 0, "without_nfo": 0}
|
||||||
"Skipping folder %s because no metadata file was found",
|
media_stats = {
|
||||||
anime_folder,
|
"with_poster": 0,
|
||||||
)
|
"without_poster": 0,
|
||||||
|
"with_logo": 0,
|
||||||
# Log summary statistics
|
"without_logo": 0,
|
||||||
if nfo_stats["total"] > 0:
|
"with_fanart": 0,
|
||||||
logger.info(
|
"without_fanart": 0
|
||||||
"NFO scan complete: %d series total, %d with NFO, %d without NFO",
|
}
|
||||||
nfo_stats["total"],
|
|
||||||
nfo_stats["with_nfo"],
|
for anime_folder in entries:
|
||||||
nfo_stats["without_nfo"]
|
anime_path = os.path.join(self.directory, anime_folder, "data")
|
||||||
)
|
if os.path.isfile(anime_path):
|
||||||
logger.info(
|
logger.debug("Found data file for folder %s", anime_folder)
|
||||||
"Media scan complete: Poster (%d/%d), Logo (%d/%d), Fanart (%d/%d)",
|
serie = self._load_data(anime_folder, anime_path)
|
||||||
media_stats["with_poster"],
|
|
||||||
nfo_stats["total"],
|
if serie:
|
||||||
media_stats["with_logo"],
|
nfo_stats["total"] += 1
|
||||||
nfo_stats["total"],
|
# Check for NFO file
|
||||||
media_stats["with_fanart"],
|
nfo_file_path = os.path.join(
|
||||||
nfo_stats["total"]
|
self.directory, anime_folder, "tvshow.nfo"
|
||||||
)
|
)
|
||||||
|
if os.path.isfile(nfo_file_path):
|
||||||
def _load_data(self, anime_folder: str, data_path: str) -> Optional[Serie]:
|
serie.nfo_path = nfo_file_path
|
||||||
"""
|
nfo_stats["with_nfo"] += 1
|
||||||
Load a single series metadata file into the in-memory collection.
|
else:
|
||||||
|
nfo_stats["without_nfo"] += 1
|
||||||
Args:
|
logger.debug(
|
||||||
anime_folder: The folder name (for logging only)
|
"Series '%s' (key: %s) is missing tvshow.nfo",
|
||||||
data_path: Path to the metadata file
|
serie.name,
|
||||||
|
serie.key
|
||||||
Returns:
|
)
|
||||||
Serie: The loaded Serie object, or None if loading failed
|
|
||||||
"""
|
# Check for media files
|
||||||
try:
|
folder_path = os.path.join(self.directory, anime_folder)
|
||||||
serie = Serie.load_from_file(data_path)
|
|
||||||
# Store by key, not folder
|
poster_path = os.path.join(folder_path, "poster.jpg")
|
||||||
self.keyDict[serie.key] = serie
|
if os.path.isfile(poster_path):
|
||||||
logger.debug(
|
media_stats["with_poster"] += 1
|
||||||
"Successfully loaded metadata for %s (key: %s)",
|
else:
|
||||||
anime_folder,
|
media_stats["without_poster"] += 1
|
||||||
serie.key
|
logger.debug(
|
||||||
)
|
"Series '%s' (key: %s) is missing poster.jpg",
|
||||||
return serie
|
serie.name,
|
||||||
except (OSError, JSONDecodeError, KeyError, ValueError) as error:
|
serie.key
|
||||||
logger.error(
|
)
|
||||||
"Failed to load metadata for folder %s from %s: %s",
|
|
||||||
anime_folder,
|
logo_path = os.path.join(folder_path, "logo.png")
|
||||||
data_path,
|
if os.path.isfile(logo_path):
|
||||||
error,
|
media_stats["with_logo"] += 1
|
||||||
)
|
else:
|
||||||
return None
|
media_stats["without_logo"] += 1
|
||||||
|
logger.debug(
|
||||||
def GetMissingEpisode(self) -> List[Serie]:
|
"Series '%s' (key: %s) is missing logo.png",
|
||||||
"""Return all series that still contain missing episodes."""
|
serie.name,
|
||||||
return [
|
serie.key
|
||||||
serie
|
)
|
||||||
for serie in self.keyDict.values()
|
|
||||||
if serie.episodeDict
|
fanart_path = os.path.join(folder_path, "fanart.jpg")
|
||||||
]
|
if os.path.isfile(fanart_path):
|
||||||
|
media_stats["with_fanart"] += 1
|
||||||
def get_missing_episodes(self) -> List[Serie]:
|
else:
|
||||||
"""PEP8-friendly alias for :meth:`GetMissingEpisode`."""
|
media_stats["without_fanart"] += 1
|
||||||
return self.GetMissingEpisode()
|
logger.debug(
|
||||||
|
"Series '%s' (key: %s) is missing fanart.jpg",
|
||||||
def GetList(self) -> List[Serie]:
|
serie.name,
|
||||||
"""Return all series instances stored in the list."""
|
serie.key
|
||||||
return list(self.keyDict.values())
|
)
|
||||||
|
|
||||||
def get_all(self) -> List[Serie]:
|
continue
|
||||||
"""PEP8-friendly alias for :meth:`GetList`."""
|
|
||||||
return self.GetList()
|
logger.warning(
|
||||||
|
"Skipping folder %s because no metadata file was found",
|
||||||
def get_by_key(self, key: str) -> Optional[Serie]:
|
anime_folder,
|
||||||
"""
|
)
|
||||||
Get a series by its unique provider key.
|
|
||||||
|
# Log summary statistics
|
||||||
This is the primary method for series lookup.
|
if nfo_stats["total"] > 0:
|
||||||
|
logger.info(
|
||||||
Args:
|
"NFO scan complete: %d series total, %d with NFO, %d without NFO",
|
||||||
key: The unique provider identifier (e.g., "attack-on-titan")
|
nfo_stats["total"],
|
||||||
|
nfo_stats["with_nfo"],
|
||||||
Returns:
|
nfo_stats["without_nfo"]
|
||||||
The Serie instance if found, None otherwise
|
)
|
||||||
"""
|
logger.info(
|
||||||
return self.keyDict.get(key)
|
"Media scan complete: Poster (%d/%d), Logo (%d/%d), Fanart (%d/%d)",
|
||||||
|
media_stats["with_poster"],
|
||||||
def get_by_folder(self, folder: str) -> Optional[Serie]:
|
nfo_stats["total"],
|
||||||
"""
|
media_stats["with_logo"],
|
||||||
Get a series by its folder name.
|
nfo_stats["total"],
|
||||||
|
media_stats["with_fanart"],
|
||||||
.. deprecated:: 2.0.0
|
nfo_stats["total"]
|
||||||
Use :meth:`get_by_key` instead. Folder-based lookups will be
|
)
|
||||||
removed in version 3.0.0. The `folder` field is metadata only
|
|
||||||
and should not be used for identification.
|
def _load_data(self, anime_folder: str, data_path: str) -> Optional[Serie]:
|
||||||
|
"""
|
||||||
This method is provided for backward compatibility only.
|
Load a single series metadata file into the in-memory collection.
|
||||||
Prefer using get_by_key() for new code.
|
|
||||||
|
Args:
|
||||||
Args:
|
anime_folder: The folder name (for logging only)
|
||||||
folder: The filesystem folder name (e.g., "Attack on Titan (2013)")
|
data_path: Path to the metadata file
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
The Serie instance if found, None otherwise
|
Serie: The loaded Serie object, or None if loading failed
|
||||||
"""
|
"""
|
||||||
warnings.warn(
|
try:
|
||||||
"get_by_folder() is deprecated and will be removed in v3.0.0. "
|
serie = Serie.load_from_file(data_path)
|
||||||
"Use get_by_key() instead. The 'folder' field is metadata only.",
|
# Store by key, not folder
|
||||||
DeprecationWarning,
|
self.keyDict[serie.key] = serie
|
||||||
stacklevel=2
|
logger.debug(
|
||||||
)
|
"Successfully loaded metadata for %s (key: %s)",
|
||||||
for serie in self.keyDict.values():
|
anime_folder,
|
||||||
if serie.folder == folder:
|
serie.key
|
||||||
return serie
|
)
|
||||||
return None
|
return serie
|
||||||
|
except (OSError, JSONDecodeError, KeyError, ValueError) as error:
|
||||||
|
logger.error(
|
||||||
|
"Failed to load metadata for folder %s from %s: %s",
|
||||||
|
anime_folder,
|
||||||
|
data_path,
|
||||||
|
error,
|
||||||
|
)
|
||||||
|
return None
|
||||||
|
|
||||||
|
def GetMissingEpisode(self) -> List[Serie]:
|
||||||
|
"""Return all series that still contain missing episodes."""
|
||||||
|
return [
|
||||||
|
serie
|
||||||
|
for serie in self.keyDict.values()
|
||||||
|
if serie.episodeDict
|
||||||
|
]
|
||||||
|
|
||||||
|
def get_missing_episodes(self) -> List[Serie]:
|
||||||
|
"""PEP8-friendly alias for :meth:`GetMissingEpisode`."""
|
||||||
|
return self.GetMissingEpisode()
|
||||||
|
|
||||||
|
def GetList(self) -> List[Serie]:
|
||||||
|
"""Return all series instances stored in the list."""
|
||||||
|
return list(self.keyDict.values())
|
||||||
|
|
||||||
|
def get_all(self) -> List[Serie]:
|
||||||
|
"""PEP8-friendly alias for :meth:`GetList`."""
|
||||||
|
return self.GetList()
|
||||||
|
|
||||||
|
def get_by_key(self, key: str) -> Optional[Serie]:
|
||||||
|
"""
|
||||||
|
Get a series by its unique provider key.
|
||||||
|
|
||||||
|
This is the primary method for series lookup.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
key: The unique provider identifier (e.g., "attack-on-titan")
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
The Serie instance if found, None otherwise
|
||||||
|
"""
|
||||||
|
return self.keyDict.get(key)
|
||||||
|
|
||||||
|
def get_by_folder(self, folder: str) -> Optional[Serie]:
|
||||||
|
"""
|
||||||
|
Get a series by its folder name.
|
||||||
|
|
||||||
|
.. deprecated:: 2.0.0
|
||||||
|
Use :meth:`get_by_key` instead. Folder-based lookups will be
|
||||||
|
removed in version 3.0.0. The `folder` field is metadata only
|
||||||
|
and should not be used for identification.
|
||||||
|
|
||||||
|
This method is provided for backward compatibility only.
|
||||||
|
Prefer using get_by_key() for new code.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
folder: The filesystem folder name (e.g., "Attack on Titan (2013)")
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
The Serie instance if found, None otherwise
|
||||||
|
"""
|
||||||
|
warnings.warn(
|
||||||
|
"get_by_folder() is deprecated and will be removed in v3.0.0. "
|
||||||
|
"Use get_by_key() instead. The 'folder' field is metadata only.",
|
||||||
|
DeprecationWarning,
|
||||||
|
stacklevel=2
|
||||||
|
)
|
||||||
|
for serie in self.keyDict.values():
|
||||||
|
if serie.folder == folder:
|
||||||
|
return serie
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def load_all_from_db(self) -> int:
|
||||||
|
"""Load all series from database into in-memory cache.
|
||||||
|
|
||||||
|
Retrieves all anime series from the database with their episodes
|
||||||
|
and populates the in-memory keyDict for fast access.
|
||||||
|
|
||||||
|
This method replaces file-based loading. Use after initialization
|
||||||
|
when database is ready.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
int: Number of series loaded into cache
|
||||||
|
"""
|
||||||
|
from src.server.database.connection import get_async_session_factory
|
||||||
|
from src.server.database.service import AnimeSeriesService
|
||||||
|
|
||||||
|
try:
|
||||||
|
session_factory = get_async_session_factory()
|
||||||
|
db = session_factory()
|
||||||
|
try:
|
||||||
|
anime_series_list = await AnimeSeriesService.get_all(
|
||||||
|
db, with_episodes=True
|
||||||
|
)
|
||||||
|
|
||||||
|
count = 0
|
||||||
|
for anime_series in anime_series_list:
|
||||||
|
episode_dict: Dict[int, List[int]] = {}
|
||||||
|
if anime_series.episodes:
|
||||||
|
for ep in anime_series.episodes:
|
||||||
|
if ep.season not in episode_dict:
|
||||||
|
episode_dict[ep.season] = []
|
||||||
|
episode_dict[ep.season].append(ep.episode_number)
|
||||||
|
|
||||||
|
serie = Serie(
|
||||||
|
key=anime_series.key,
|
||||||
|
name=anime_series.name,
|
||||||
|
site=anime_series.site,
|
||||||
|
folder=anime_series.folder,
|
||||||
|
episodeDict=episode_dict,
|
||||||
|
year=anime_series.year
|
||||||
|
)
|
||||||
|
self.keyDict[serie.key] = serie
|
||||||
|
count += 1
|
||||||
|
|
||||||
|
logger.info(
|
||||||
|
"Loaded %d series from database into in-memory cache",
|
||||||
|
count
|
||||||
|
)
|
||||||
|
return count
|
||||||
|
finally:
|
||||||
|
await db.close()
|
||||||
|
except RuntimeError:
|
||||||
|
logger.warning(
|
||||||
|
"Database not available, skipping DB load"
|
||||||
|
)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
async def _load_single_series_from_db(
|
||||||
|
self,
|
||||||
|
anime_folder: str
|
||||||
|
) -> Optional[Serie]:
|
||||||
|
"""Load a single series from database by folder name.
|
||||||
|
|
||||||
|
Looks up a series in the database by its folder name and adds
|
||||||
|
it to the in-memory cache.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
anime_folder: The filesystem folder name to look up
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Serie if found and loaded, None otherwise
|
||||||
|
"""
|
||||||
|
from src.server.database.connection import get_async_session_factory
|
||||||
|
from src.server.database.service import AnimeSeriesService
|
||||||
|
|
||||||
|
try:
|
||||||
|
session_factory = get_async_session_factory()
|
||||||
|
db = session_factory()
|
||||||
|
try:
|
||||||
|
anime_series = await AnimeSeriesService.get_by_folder(
|
||||||
|
db, anime_folder
|
||||||
|
)
|
||||||
|
if not anime_series:
|
||||||
|
logger.debug(
|
||||||
|
"Series with folder '%s' not found in DB",
|
||||||
|
anime_folder
|
||||||
|
)
|
||||||
|
return None
|
||||||
|
|
||||||
|
episode_dict: Dict[int, List[int]] = {}
|
||||||
|
if anime_series.episodes:
|
||||||
|
for ep in anime_series.episodes:
|
||||||
|
if ep.season not in episode_dict:
|
||||||
|
episode_dict[ep.season] = []
|
||||||
|
episode_dict[ep.season].append(ep.episode_number)
|
||||||
|
|
||||||
|
serie = Serie(
|
||||||
|
key=anime_series.key,
|
||||||
|
name=anime_series.name,
|
||||||
|
site=anime_series.site,
|
||||||
|
folder=anime_series.folder,
|
||||||
|
episodeDict=episode_dict,
|
||||||
|
year=anime_series.year
|
||||||
|
)
|
||||||
|
self.keyDict[serie.key] = serie
|
||||||
|
logger.debug(
|
||||||
|
"Loaded series '%s' (key=%s) from DB",
|
||||||
|
serie.name, serie.key
|
||||||
|
)
|
||||||
|
return serie
|
||||||
|
finally:
|
||||||
|
await db.close()
|
||||||
|
except RuntimeError:
|
||||||
|
logger.warning(
|
||||||
|
"Database not available, cannot load series '%s'",
|
||||||
|
anime_folder
|
||||||
|
)
|
||||||
|
return None
|
||||||
|
|
||||||
|
def invalidate_cache(self) -> None:
|
||||||
|
"""Clear the in-memory cache.
|
||||||
|
|
||||||
|
Use after database modifications to force reload from DB
|
||||||
|
on next access.
|
||||||
|
"""
|
||||||
|
self.keyDict.clear()
|
||||||
|
logger.debug("SerieList in-memory cache invalidated")
|
||||||
|
|
||||||
|
def reload(self) -> None:
|
||||||
|
"""Reload series from filesystem (legacy mode).
|
||||||
|
|
||||||
|
Warning:
|
||||||
|
This method uses file-based loading and should only be
|
||||||
|
used as fallback when database is not available.
|
||||||
|
"""
|
||||||
|
self.load_series()
|
||||||
|
|||||||
@@ -16,6 +16,7 @@ from typing import Dict, List
|
|||||||
from lxml import etree
|
from lxml import etree
|
||||||
|
|
||||||
from src.core.services.nfo_service import NFOService
|
from src.core.services.nfo_service import NFOService
|
||||||
|
from src.core.services.tmdb_client import TMDBAPIError
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
@@ -202,10 +203,26 @@ class NfoRepairService:
|
|||||||
", ".join(missing),
|
", ".join(missing),
|
||||||
)
|
)
|
||||||
|
|
||||||
await self._nfo_service.update_tvshow_nfo(
|
try:
|
||||||
series_name,
|
await self._nfo_service.update_tvshow_nfo(
|
||||||
download_media=False,
|
series_name,
|
||||||
)
|
download_media=False,
|
||||||
|
)
|
||||||
|
except TMDBAPIError as e:
|
||||||
|
if "No TMDB ID found" in str(e):
|
||||||
|
# No TMDB ID in existing NFO — create new one via search
|
||||||
|
logger.info(
|
||||||
|
"NFO has no TMDB ID, creating new NFO via TMDB search"
|
||||||
|
)
|
||||||
|
await self._nfo_service.create_tvshow_nfo(
|
||||||
|
serie_name=series_name,
|
||||||
|
serie_folder=series_name,
|
||||||
|
download_poster=False,
|
||||||
|
download_logo=False,
|
||||||
|
download_fanart=False,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
raise
|
||||||
|
|
||||||
logger.info("NFO repair completed: %s", series_name)
|
logger.info("NFO repair completed: %s", series_name)
|
||||||
return True
|
return True
|
||||||
|
|||||||
@@ -163,58 +163,89 @@ class NFOService:
|
|||||||
logger.info("Creating series folder: %s", folder_path)
|
logger.info("Creating series folder: %s", folder_path)
|
||||||
folder_path.mkdir(parents=True, exist_ok=True)
|
folder_path.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
# Check for existing NFO with TMDB ID to skip search
|
||||||
|
nfo_path = folder_path / "tvshow.nfo"
|
||||||
|
existing_ids = None
|
||||||
|
if nfo_path.exists():
|
||||||
|
try:
|
||||||
|
existing_ids = self.parse_nfo_ids(nfo_path)
|
||||||
|
if existing_ids.get("tmdb_id"):
|
||||||
|
logger.info(
|
||||||
|
"Found existing TMDB ID %s in NFO, using directly",
|
||||||
|
existing_ids["tmdb_id"]
|
||||||
|
)
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug("Could not parse existing NFO IDs: %s", e)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
await self.tmdb_client._ensure_session()
|
await self.tmdb_client._ensure_session()
|
||||||
|
|
||||||
# Search for TV show - try multiple strategies
|
# Use existing TMDB ID if found, otherwise search
|
||||||
tv_show, search_source = await self._search_with_fallback(
|
if existing_ids and existing_ids.get("tmdb_id"):
|
||||||
search_name, year, alt_titles
|
tv_id = existing_ids["tmdb_id"]
|
||||||
)
|
logger.info("Fetching details directly for TMDB ID: %s", tv_id)
|
||||||
tv_id = tv_show["id"]
|
details = await self.tmdb_client.get_tv_show_details(
|
||||||
|
tv_id,
|
||||||
|
append_to_response="credits,external_ids,images"
|
||||||
|
)
|
||||||
|
content_ratings = await self.tmdb_client.get_tv_show_content_ratings(tv_id)
|
||||||
|
tv_show = {"id": tv_id, "name": details.get("name", serie_name)}
|
||||||
|
search_source = "nfo_override"
|
||||||
|
else:
|
||||||
|
# Search for TV show - try multiple strategies
|
||||||
|
tv_show, search_source = await self._search_with_fallback(
|
||||||
|
search_name, year, alt_titles
|
||||||
|
)
|
||||||
|
tv_id = tv_show["id"]
|
||||||
|
|
||||||
logger.info("Found match: %s (ID: %s)", tv_show['name'], tv_id)
|
logger.info("Found match: %s (ID: %s)", tv_show['name'], tv_id)
|
||||||
|
|
||||||
# Get detailed information with multi-language image support
|
# Get detailed information with multi-language image support
|
||||||
details = await self.tmdb_client.get_tv_show_details(
|
# Skip if we already fetched details via nfo_override
|
||||||
tv_id,
|
if search_source != "nfo_override":
|
||||||
append_to_response="credits,external_ids,images"
|
details = await self.tmdb_client.get_tv_show_details(
|
||||||
)
|
tv_id,
|
||||||
|
append_to_response="credits,external_ids,images"
|
||||||
|
)
|
||||||
|
|
||||||
# Get content ratings for FSK
|
# Get content ratings for FSK
|
||||||
content_ratings = await self.tmdb_client.get_tv_show_content_ratings(tv_id)
|
content_ratings = await self.tmdb_client.get_tv_show_content_ratings(tv_id)
|
||||||
|
|
||||||
# Enrich with fallback languages for empty overview/tagline
|
# Enrich with fallback languages for empty overview/tagline
|
||||||
# Pass search result overview as last resort fallback
|
# Pass search result overview as last resort fallback
|
||||||
search_overview = tv_show.get("overview") or None
|
search_overview = tv_show.get("overview") or None
|
||||||
if not search_overview:
|
if not search_overview:
|
||||||
try:
|
try:
|
||||||
logger.debug(
|
logger.debug(
|
||||||
"No overview in German search result, trying en-US search fallback for: %s",
|
"No overview in German search result, trying en-US search fallback for: %s",
|
||||||
search_name,
|
search_name,
|
||||||
)
|
|
||||||
en_search_results = await self.tmdb_client.search_tv_show(
|
|
||||||
search_name,
|
|
||||||
language="en-US",
|
|
||||||
)
|
|
||||||
if en_search_results.get("results"):
|
|
||||||
en_match = self._find_best_match(
|
|
||||||
en_search_results["results"], search_name, year
|
|
||||||
)
|
)
|
||||||
search_overview = en_match.get("overview") or None
|
en_search_results = await self.tmdb_client.search_tv_show(
|
||||||
if search_overview:
|
search_name,
|
||||||
logger.info(
|
language="en-US",
|
||||||
"Using en-US search overview fallback for %s",
|
)
|
||||||
search_name,
|
if en_search_results.get("results"):
|
||||||
|
en_match = self._find_best_match(
|
||||||
|
en_search_results["results"], search_name, year
|
||||||
)
|
)
|
||||||
except (TMDBAPIError, Exception) as exc:
|
search_overview = en_match.get("overview") or None
|
||||||
logger.warning(
|
if search_overview:
|
||||||
"Failed en-US search fallback for overview: %s",
|
logger.info(
|
||||||
exc,
|
"Using en-US search overview fallback for %s",
|
||||||
)
|
search_name,
|
||||||
|
)
|
||||||
|
except (TMDBAPIError, Exception) as exc:
|
||||||
|
logger.warning(
|
||||||
|
"Failed en-US search fallback for overview: %s",
|
||||||
|
exc,
|
||||||
|
)
|
||||||
|
|
||||||
details = await self._enrich_details_with_fallback(
|
details = await self._enrich_details_with_fallback(
|
||||||
details, search_overview=search_overview
|
details, search_overview=search_overview
|
||||||
)
|
)
|
||||||
|
else:
|
||||||
|
# When using nfo_override, content_ratings already fetched
|
||||||
|
pass
|
||||||
|
|
||||||
# Convert TMDB data to TVShowNFO model
|
# Convert TMDB data to TVShowNFO model
|
||||||
nfo_model = tmdb_to_nfo_model(
|
nfo_model = tmdb_to_nfo_model(
|
||||||
@@ -646,21 +677,45 @@ class NFOService:
|
|||||||
{"query": normalized, "year": year, "lang": "de-DE", "desc": f"normalized:{normalized}"}
|
{"query": normalized, "year": year, "lang": "de-DE", "desc": f"normalized:{normalized}"}
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# Strategy 6: Try search/multi for series indexed as movies
|
||||||
|
search_strategies.append(
|
||||||
|
{"query": primary_query, "year": year, "lang": "en-US", "desc": "multi_search", "use_multi": True}
|
||||||
|
)
|
||||||
|
|
||||||
last_error = None
|
last_error = None
|
||||||
for strategy in search_strategies:
|
for strategy in search_strategies:
|
||||||
query = strategy["query"]
|
query = strategy["query"]
|
||||||
lang = strategy["lang"]
|
lang = strategy["lang"]
|
||||||
desc = strategy["desc"]
|
desc = strategy["desc"]
|
||||||
|
use_multi = strategy.get("use_multi", False)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
logger.debug(
|
logger.debug(
|
||||||
"TMDB search attempt: query='%s', lang=%s, year=%s, strategy=%s",
|
"TMDB search attempt: query='%s', lang=%s, year=%s, strategy=%s",
|
||||||
query, lang, strategy["year"], desc
|
query, lang, strategy["year"], desc
|
||||||
)
|
)
|
||||||
search_results = await self.tmdb_client.search_tv_show(
|
|
||||||
query,
|
# Use search/multi for multi_search strategy
|
||||||
language=lang
|
if use_multi:
|
||||||
)
|
search_results = await self.tmdb_client.search_multi(
|
||||||
|
query,
|
||||||
|
language=lang
|
||||||
|
)
|
||||||
|
# Filter for TV shows only
|
||||||
|
if search_results.get("results"):
|
||||||
|
tv_results = [
|
||||||
|
r for r in search_results["results"]
|
||||||
|
if r.get("media_type") == "tv"
|
||||||
|
]
|
||||||
|
if tv_results:
|
||||||
|
search_results["results"] = tv_results
|
||||||
|
else:
|
||||||
|
search_results["results"] = []
|
||||||
|
else:
|
||||||
|
search_results = await self.tmdb_client.search_tv_show(
|
||||||
|
query,
|
||||||
|
language=lang
|
||||||
|
)
|
||||||
|
|
||||||
if search_results.get("results"):
|
if search_results.get("results"):
|
||||||
# Apply year filter if we have one
|
# Apply year filter if we have one
|
||||||
@@ -784,6 +839,7 @@ class NFOService:
|
|||||||
async def close(self):
|
async def close(self):
|
||||||
"""Clean up resources."""
|
"""Clean up resources."""
|
||||||
await self.tmdb_client.close()
|
await self.tmdb_client.close()
|
||||||
|
await self.image_downloader.close()
|
||||||
|
|
||||||
async def create_minimal_nfo(
|
async def create_minimal_nfo(
|
||||||
self,
|
self,
|
||||||
|
|||||||
@@ -129,6 +129,9 @@ class SeriesManagerService:
|
|||||||
if not self.nfo_service:
|
if not self.nfo_service:
|
||||||
return
|
return
|
||||||
|
|
||||||
|
nfo_exists = False
|
||||||
|
ids = {}
|
||||||
|
|
||||||
try:
|
try:
|
||||||
folder_path = Path(self.anime_directory) / serie_folder
|
folder_path = Path(self.anime_directory) / serie_folder
|
||||||
nfo_path = folder_path / "tvshow.nfo"
|
nfo_path = folder_path / "tvshow.nfo"
|
||||||
@@ -195,22 +198,49 @@ class SeriesManagerService:
|
|||||||
logger.info(
|
logger.info(
|
||||||
f"Creating NFO for '{serie_name}' ({serie_folder})"
|
f"Creating NFO for '{serie_name}' ({serie_folder})"
|
||||||
)
|
)
|
||||||
await self.nfo_service.create_tvshow_nfo(
|
try:
|
||||||
serie_name=serie_name,
|
await self.nfo_service.create_tvshow_nfo(
|
||||||
serie_folder=serie_folder,
|
serie_name=serie_name,
|
||||||
year=year,
|
serie_folder=serie_folder,
|
||||||
download_poster=self.download_poster,
|
year=year,
|
||||||
download_logo=self.download_logo,
|
download_poster=self.download_poster,
|
||||||
download_fanart=self.download_fanart
|
download_logo=self.download_logo,
|
||||||
)
|
download_fanart=self.download_fanart
|
||||||
logger.info("Successfully created NFO for '%s'", serie_name)
|
)
|
||||||
|
logger.info("Successfully created NFO for '%s'", serie_name)
|
||||||
|
except TMDBAPIError as create_error:
|
||||||
|
# TMDB lookup failed, create minimal NFO to track the series
|
||||||
|
logger.warning(
|
||||||
|
"TMDB lookup failed for '%s', creating minimal NFO: %s",
|
||||||
|
serie_name, create_error
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
await self.nfo_service.create_minimal_nfo(
|
||||||
|
serie_name=serie_name,
|
||||||
|
serie_folder=serie_folder,
|
||||||
|
year=year
|
||||||
|
)
|
||||||
|
logger.info("Created minimal NFO for '%s'", serie_name)
|
||||||
|
except Exception as minimal_error:
|
||||||
|
logger.error(
|
||||||
|
"Failed to create minimal NFO for '%s': %s",
|
||||||
|
serie_name, minimal_error
|
||||||
|
)
|
||||||
elif nfo_exists:
|
elif nfo_exists:
|
||||||
logger.debug(
|
logger.debug(
|
||||||
f"NFO exists for '{serie_name}', skipping download"
|
f"NFO exists for '{serie_name}', skipping download"
|
||||||
)
|
)
|
||||||
|
|
||||||
except TMDBAPIError as e:
|
except TMDBAPIError as e:
|
||||||
logger.error("TMDB API error processing '%s': %s", serie_name, e)
|
# Only log at ERROR if no NFO exists and we have no IDs
|
||||||
|
# If NFO exists with IDs, this is just a lookup failure, log at DEBUG
|
||||||
|
if nfo_exists and (ids.get("tmdb_id") or ids.get("tvdb_id")):
|
||||||
|
logger.debug(
|
||||||
|
"TMDB API lookup failed for '%s' (has NFO with IDs): %s",
|
||||||
|
serie_name, e
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
logger.error("TMDB API error processing '%s': %s", serie_name, e)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.error(
|
logger.error(
|
||||||
f"Unexpected error processing NFO for '{serie_name}': {e}",
|
f"Unexpected error processing NFO for '{serie_name}': {e}",
|
||||||
|
|||||||
@@ -128,7 +128,7 @@ class TMDBClient:
|
|||||||
# Expired negative cache entry
|
# Expired negative cache entry
|
||||||
del self._negative_cache[negative_cache_key]
|
del self._negative_cache[negative_cache_key]
|
||||||
|
|
||||||
delay = 2
|
delay = 1
|
||||||
last_error = None
|
last_error = None
|
||||||
|
|
||||||
# Rate limiting: ensure we don't exceed ~35 requests/second
|
# Rate limiting: ensure we don't exceed ~35 requests/second
|
||||||
@@ -162,7 +162,7 @@ class TMDBClient:
|
|||||||
raise TMDBAPIError(f"Resource not found: {endpoint}")
|
raise TMDBAPIError(f"Resource not found: {endpoint}")
|
||||||
elif resp.status == 429:
|
elif resp.status == 429:
|
||||||
# Rate limit - wait longer with exponential backoff
|
# Rate limit - wait longer with exponential backoff
|
||||||
retry_after = int(resp.headers.get('Retry-After', max(delay * 2, 10)))
|
retry_after = int(resp.headers.get('Retry-After', max(delay * 2, 2)))
|
||||||
logger.warning("Rate limited, waiting %ss", retry_after)
|
logger.warning("Rate limited, waiting %ss", retry_after)
|
||||||
await asyncio.sleep(retry_after)
|
await asyncio.sleep(retry_after)
|
||||||
continue
|
continue
|
||||||
@@ -181,7 +181,7 @@ class TMDBClient:
|
|||||||
if attempt < max_retries - 1:
|
if attempt < max_retries - 1:
|
||||||
logger.warning("Request timeout (attempt %s), retrying in %ss", attempt + 1, delay)
|
logger.warning("Request timeout (attempt %s), retrying in %ss", attempt + 1, delay)
|
||||||
await asyncio.sleep(delay)
|
await asyncio.sleep(delay)
|
||||||
delay = min(delay * 2, 30)
|
delay *= 2
|
||||||
else:
|
else:
|
||||||
logger.error("Request timed out after %s attempts", max_retries)
|
logger.error("Request timed out after %s attempts", max_retries)
|
||||||
|
|
||||||
@@ -209,7 +209,7 @@ class TMDBClient:
|
|||||||
if attempt < max_retries - 1:
|
if attempt < max_retries - 1:
|
||||||
logger.warning("Request failed (attempt %s): %s, retrying in %ss", attempt + 1, e, delay)
|
logger.warning("Request failed (attempt %s): %s, retrying in %ss", attempt + 1, e, delay)
|
||||||
await asyncio.sleep(delay)
|
await asyncio.sleep(delay)
|
||||||
delay = min(delay * 2, 30)
|
delay *= 2
|
||||||
else:
|
else:
|
||||||
logger.error("Request failed after %s attempts: %s", max_retries, e)
|
logger.error("Request failed after %s attempts: %s", max_retries, e)
|
||||||
|
|
||||||
|
|||||||
@@ -163,6 +163,22 @@ async def setup_auth(req: SetupRequest):
|
|||||||
# Perform NFO scan if configured
|
# Perform NFO scan if configured
|
||||||
await perform_nfo_scan_if_needed(progress_service)
|
await perform_nfo_scan_if_needed(progress_service)
|
||||||
|
|
||||||
|
# Start scheduler if anime_directory is now set
|
||||||
|
try:
|
||||||
|
from src.server.services.scheduler_service import (
|
||||||
|
get_scheduler_service,
|
||||||
|
)
|
||||||
|
|
||||||
|
scheduler_svc = get_scheduler_service()
|
||||||
|
logger.info("Starting scheduler after initialization")
|
||||||
|
await scheduler_svc.ensure_started()
|
||||||
|
logger.info("Scheduler started successfully during setup")
|
||||||
|
except Exception as sched_exc:
|
||||||
|
logger.warning(
|
||||||
|
"Failed to start scheduler during setup: %s", sched_exc
|
||||||
|
)
|
||||||
|
# Continue — scheduler failure should not break initialization
|
||||||
|
|
||||||
# Send completion event
|
# Send completion event
|
||||||
from src.server.services.progress_service import ProgressType
|
from src.server.services.progress_service import ProgressType
|
||||||
await progress_service.start_progress(
|
await progress_service.start_progress(
|
||||||
|
|||||||
@@ -1,7 +1,10 @@
|
|||||||
|
import logging
|
||||||
from typing import Any, Dict, List, Optional
|
from typing import Any, Dict, List, Optional
|
||||||
|
|
||||||
from fastapi import APIRouter, Depends, HTTPException, status
|
from fastapi import APIRouter, Depends, HTTPException, status
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
from src.server.models.config import AppConfig, ConfigUpdate, ValidationResult
|
from src.server.models.config import AppConfig, ConfigUpdate, ValidationResult
|
||||||
from src.server.services.config_service import (
|
from src.server.services.config_service import (
|
||||||
ConfigBackupError,
|
ConfigBackupError,
|
||||||
@@ -28,16 +31,53 @@ def get_config(auth: Optional[dict] = Depends(require_auth)) -> AppConfig:
|
|||||||
|
|
||||||
|
|
||||||
@router.put("", response_model=AppConfig)
|
@router.put("", response_model=AppConfig)
|
||||||
def update_config(
|
async def update_config(
|
||||||
update: ConfigUpdate, auth: dict = Depends(require_auth)
|
update: ConfigUpdate, auth: dict = Depends(require_auth)
|
||||||
) -> AppConfig:
|
) -> AppConfig:
|
||||||
"""Apply an update to the configuration and persist it.
|
"""Apply an update to the configuration and persist it.
|
||||||
|
|
||||||
Creates automatic backup before applying changes.
|
Creates automatic backup before applying changes. If anime_directory
|
||||||
|
is configured, starts the scheduler service.
|
||||||
"""
|
"""
|
||||||
try:
|
try:
|
||||||
config_service = get_config_service()
|
config_service = get_config_service()
|
||||||
return config_service.update_config(update)
|
updated_config = config_service.update_config(update)
|
||||||
|
|
||||||
|
# Sync anime_directory to settings if it was updated
|
||||||
|
from src.config.settings import settings as app_settings
|
||||||
|
|
||||||
|
anime_dir_changed = False
|
||||||
|
if update.other and update.other.get("anime_directory"):
|
||||||
|
anime_dir = update.other.get("anime_directory")
|
||||||
|
if anime_dir and not app_settings.anime_directory:
|
||||||
|
app_settings.anime_directory = str(anime_dir)
|
||||||
|
anime_dir_changed = True
|
||||||
|
logger.info("Synced anime_directory from config: %s", anime_dir)
|
||||||
|
|
||||||
|
# Start scheduler if anime_directory was just configured
|
||||||
|
if anime_dir_changed:
|
||||||
|
try:
|
||||||
|
from src.server.services.scheduler_service import (
|
||||||
|
get_scheduler_service,
|
||||||
|
)
|
||||||
|
|
||||||
|
scheduler_svc = get_scheduler_service()
|
||||||
|
logger.info(
|
||||||
|
"Starting scheduler after anime_directory configuration"
|
||||||
|
)
|
||||||
|
await scheduler_svc.ensure_started()
|
||||||
|
logger.info(
|
||||||
|
"Scheduler started successfully after config update"
|
||||||
|
)
|
||||||
|
except Exception as sched_exc:
|
||||||
|
logger.warning(
|
||||||
|
"Failed to start scheduler after config update: %s",
|
||||||
|
sched_exc,
|
||||||
|
)
|
||||||
|
# Config was already saved, don't fail the request
|
||||||
|
|
||||||
|
return updated_config
|
||||||
|
|
||||||
except ConfigValidationError as e:
|
except ConfigValidationError as e:
|
||||||
raise HTTPException(
|
raise HTTPException(
|
||||||
status_code=status.HTTP_400_BAD_REQUEST,
|
status_code=status.HTTP_400_BAD_REQUEST,
|
||||||
@@ -244,9 +284,9 @@ async def update_directory(
|
|||||||
try:
|
try:
|
||||||
import structlog
|
import structlog
|
||||||
|
|
||||||
from src.server.services.anime_service import sync_series_from_data_files
|
from src.server.services.anime_service import sync_legacy_series_to_db
|
||||||
logger = structlog.get_logger(__name__)
|
logger = structlog.get_logger(__name__)
|
||||||
sync_count = await sync_series_from_data_files(directory, logger)
|
sync_count = await sync_legacy_series_to_db(directory, logger)
|
||||||
logger.info(
|
logger.info(
|
||||||
"Directory updated: synced series from data files",
|
"Directory updated: synced series from data files",
|
||||||
directory=directory,
|
directory=directory,
|
||||||
|
|||||||
@@ -83,6 +83,10 @@ class AnimeSeries(Base, TimestampMixin):
|
|||||||
Boolean, nullable=False, default=False, server_default="0",
|
Boolean, nullable=False, default=False, server_default="0",
|
||||||
doc="Whether tvshow.nfo file exists for this series"
|
doc="Whether tvshow.nfo file exists for this series"
|
||||||
)
|
)
|
||||||
|
nfo_path: Mapped[Optional[str]] = mapped_column(
|
||||||
|
String(1000), nullable=True,
|
||||||
|
doc="Path to the tvshow.nfo metadata file"
|
||||||
|
)
|
||||||
nfo_created_at: Mapped[Optional[datetime]] = mapped_column(
|
nfo_created_at: Mapped[Optional[datetime]] = mapped_column(
|
||||||
DateTime(timezone=True), nullable=True,
|
DateTime(timezone=True), nullable=True,
|
||||||
doc="Timestamp when NFO was first created"
|
doc="Timestamp when NFO was first created"
|
||||||
@@ -91,6 +95,7 @@ class AnimeSeries(Base, TimestampMixin):
|
|||||||
DateTime(timezone=True), nullable=True,
|
DateTime(timezone=True), nullable=True,
|
||||||
doc="Timestamp when NFO was last updated"
|
doc="Timestamp when NFO was last updated"
|
||||||
)
|
)
|
||||||
|
# TMDB (The Movie Database) ID for series metadata
|
||||||
tmdb_id: Mapped[Optional[int]] = mapped_column(
|
tmdb_id: Mapped[Optional[int]] = mapped_column(
|
||||||
Integer, nullable=True, index=True,
|
Integer, nullable=True, index=True,
|
||||||
doc="TMDB (The Movie Database) ID for series metadata"
|
doc="TMDB (The Movie Database) ID for series metadata"
|
||||||
@@ -608,6 +613,10 @@ class SystemSettings(Base, TimestampMixin):
|
|||||||
Boolean, nullable=False, default=False, server_default="0",
|
Boolean, nullable=False, default=False, server_default="0",
|
||||||
doc="Whether the initial media scan has been completed"
|
doc="Whether the initial media scan has been completed"
|
||||||
)
|
)
|
||||||
|
migration_legacy_files_completed: Mapped[bool] = mapped_column(
|
||||||
|
Boolean, nullable=False, default=False, server_default="0",
|
||||||
|
doc="Whether legacy key/data file migration has been completed"
|
||||||
|
)
|
||||||
last_scan_timestamp: Mapped[Optional[datetime]] = mapped_column(
|
last_scan_timestamp: Mapped[Optional[datetime]] = mapped_column(
|
||||||
DateTime(timezone=True), nullable=True,
|
DateTime(timezone=True), nullable=True,
|
||||||
doc="Timestamp of the last completed scan"
|
doc="Timestamp of the last completed scan"
|
||||||
|
|||||||
@@ -169,6 +169,26 @@ class AnimeSeriesService:
|
|||||||
)
|
)
|
||||||
return result.scalar_one_or_none()
|
return result.scalar_one_or_none()
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
async def get_by_folder(db: AsyncSession, folder: str) -> Optional[AnimeSeries]:
|
||||||
|
"""Look up an anime series by its filesystem folder name (async).
|
||||||
|
|
||||||
|
Intended as primary lookup for ``SerieScanner`` when scanning
|
||||||
|
directories, replacing the legacy file-based lookups (key/data files).
|
||||||
|
|
||||||
|
Args:
|
||||||
|
db: Async database session.
|
||||||
|
folder: Filesystem folder name to match (e.g.
|
||||||
|
``"Rooster Fighter (2026)"``).
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
``AnimeSeries`` instance or ``None`` if not found.
|
||||||
|
"""
|
||||||
|
result = await db.execute(
|
||||||
|
select(AnimeSeries).where(AnimeSeries.folder == folder)
|
||||||
|
)
|
||||||
|
return result.scalar_one_or_none()
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
async def get_all(
|
async def get_all(
|
||||||
db: AsyncSession,
|
db: AsyncSession,
|
||||||
@@ -541,6 +561,7 @@ class EpisodeService:
|
|||||||
db: AsyncSession,
|
db: AsyncSession,
|
||||||
series_id: int,
|
series_id: int,
|
||||||
season: Optional[int] = None,
|
season: Optional[int] = None,
|
||||||
|
only_missing: bool = False,
|
||||||
) -> List[Episode]:
|
) -> List[Episode]:
|
||||||
"""Get episodes for a series.
|
"""Get episodes for a series.
|
||||||
|
|
||||||
@@ -548,6 +569,9 @@ class EpisodeService:
|
|||||||
db: Database session
|
db: Database session
|
||||||
series_id: Foreign key to AnimeSeries
|
series_id: Foreign key to AnimeSeries
|
||||||
season: Optional season filter
|
season: Optional season filter
|
||||||
|
only_missing: If True, only return episodes where
|
||||||
|
is_downloaded is False (i.e., missing episodes).
|
||||||
|
Default False returns all episodes.
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
List of Episode instances
|
List of Episode instances
|
||||||
@@ -557,6 +581,9 @@ class EpisodeService:
|
|||||||
if season is not None:
|
if season is not None:
|
||||||
query = query.where(Episode.season == season)
|
query = query.where(Episode.season == season)
|
||||||
|
|
||||||
|
if only_missing:
|
||||||
|
query = query.where(Episode.is_downloaded == False)
|
||||||
|
|
||||||
query = query.order_by(Episode.season, Episode.episode_number)
|
query = query.order_by(Episode.season, Episode.episode_number)
|
||||||
result = await db.execute(query)
|
result = await db.execute(query)
|
||||||
return list(result.scalars().all())
|
return list(result.scalars().all())
|
||||||
@@ -622,11 +649,11 @@ class EpisodeService:
|
|||||||
@staticmethod
|
@staticmethod
|
||||||
async def delete(db: AsyncSession, episode_id: int) -> bool:
|
async def delete(db: AsyncSession, episode_id: int) -> bool:
|
||||||
"""Delete episode.
|
"""Delete episode.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
db: Database session
|
db: Database session
|
||||||
episode_id: Episode primary key
|
episode_id: Episode primary key
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
True if deleted, False if not found
|
True if deleted, False if not found
|
||||||
"""
|
"""
|
||||||
@@ -635,6 +662,33 @@ class EpisodeService:
|
|||||||
)
|
)
|
||||||
return result.rowcount > 0
|
return result.rowcount > 0
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
async def delete_by_series(
|
||||||
|
db: AsyncSession,
|
||||||
|
series_id: int,
|
||||||
|
season: int,
|
||||||
|
episode_number: int,
|
||||||
|
) -> bool:
|
||||||
|
"""Delete episode by series ID, season, and episode number.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
db: Database session
|
||||||
|
series_id: Foreign key to AnimeSeries
|
||||||
|
season: Season number
|
||||||
|
episode_number: Episode number within season
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
True if deleted, False if not found
|
||||||
|
"""
|
||||||
|
result = await db.execute(
|
||||||
|
delete(Episode).where(
|
||||||
|
Episode.series_id == series_id,
|
||||||
|
Episode.season == season,
|
||||||
|
Episode.episode_number == episode_number,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return result.rowcount > 0
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
async def delete_by_series_and_episode(
|
async def delete_by_series_and_episode(
|
||||||
db: AsyncSession,
|
db: AsyncSession,
|
||||||
|
|||||||
@@ -125,6 +125,36 @@ class SystemSettingsService:
|
|||||||
settings = await SystemSettingsService.get_or_create(db)
|
settings = await SystemSettingsService.get_or_create(db)
|
||||||
return settings.initial_media_scan_completed
|
return settings.initial_media_scan_completed
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
async def is_migration_legacy_files_completed(db: AsyncSession) -> bool:
|
||||||
|
"""Check if legacy key/data file migration has been completed.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
db: Database session
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
True if legacy migration is completed, False otherwise
|
||||||
|
"""
|
||||||
|
settings = await SystemSettingsService.get_or_create(db)
|
||||||
|
return settings.migration_legacy_files_completed
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
async def mark_migration_legacy_files_completed(
|
||||||
|
db: AsyncSession,
|
||||||
|
timestamp: Optional[datetime] = None
|
||||||
|
) -> None:
|
||||||
|
"""Mark the legacy key/data file migration as completed.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
db: Database session
|
||||||
|
timestamp: Optional timestamp to set, defaults to current time
|
||||||
|
"""
|
||||||
|
settings = await SystemSettingsService.get_or_create(db)
|
||||||
|
settings.migration_legacy_files_completed = True
|
||||||
|
settings.last_scan_timestamp = timestamp or datetime.now(timezone.utc)
|
||||||
|
await db.commit()
|
||||||
|
logger.info("Marked legacy files migration as completed")
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
async def mark_initial_media_scan_completed(
|
async def mark_initial_media_scan_completed(
|
||||||
db: AsyncSession,
|
db: AsyncSession,
|
||||||
|
|||||||
@@ -38,7 +38,6 @@ from src.server.controllers.page_controller import router as page_router
|
|||||||
from src.server.middleware.auth import AuthMiddleware
|
from src.server.middleware.auth import AuthMiddleware
|
||||||
from src.server.middleware.error_handler import register_exception_handlers
|
from src.server.middleware.error_handler import register_exception_handlers
|
||||||
from src.server.middleware.setup_redirect import SetupRedirectMiddleware
|
from src.server.middleware.setup_redirect import SetupRedirectMiddleware
|
||||||
from src.server.services.anime_service import sync_series_from_data_files
|
|
||||||
from src.server.services.progress_service import get_progress_service
|
from src.server.services.progress_service import get_progress_service
|
||||||
from src.server.services.websocket_service import get_websocket_service
|
from src.server.services.websocket_service import get_websocket_service
|
||||||
|
|
||||||
@@ -121,7 +120,7 @@ async def _run_startup_health_checks(logger) -> dict:
|
|||||||
import asyncio
|
import asyncio
|
||||||
import shutil
|
import shutil
|
||||||
import socket
|
import socket
|
||||||
from typing import Dict, Any
|
from typing import Any, Dict
|
||||||
|
|
||||||
checks: Dict[str, Any] = {
|
checks: Dict[str, Any] = {
|
||||||
"ffmpeg": {"status": "unknown", "message": None},
|
"ffmpeg": {"status": "unknown", "message": None},
|
||||||
@@ -398,19 +397,6 @@ async def lifespan(_application: FastAPI):
|
|||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.warning("Failed to start background loader service: %s", e)
|
logger.warning("Failed to start background loader service: %s", e)
|
||||||
|
|
||||||
# Initialize and start scheduler service
|
|
||||||
try:
|
|
||||||
from src.server.services.scheduler_service import (
|
|
||||||
get_scheduler_service,
|
|
||||||
)
|
|
||||||
scheduler_service = get_scheduler_service()
|
|
||||||
await scheduler_service.start()
|
|
||||||
initialized['scheduler'] = True
|
|
||||||
logger.info("Scheduler service started")
|
|
||||||
except Exception as e:
|
|
||||||
logger.warning("Failed to start scheduler service: %s", e)
|
|
||||||
# Continue - scheduler is optional
|
|
||||||
|
|
||||||
# Run media scan only on first run
|
# Run media scan only on first run
|
||||||
await perform_media_scan_if_needed(background_loader)
|
await perform_media_scan_if_needed(background_loader)
|
||||||
else:
|
else:
|
||||||
@@ -418,6 +404,22 @@ async def lifespan(_application: FastAPI):
|
|||||||
"Download service initialization skipped - "
|
"Download service initialization skipped - "
|
||||||
"anime directory not configured"
|
"anime directory not configured"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# Initialize and start scheduler service (independent of anime_directory)
|
||||||
|
# The scheduler loads its own config from config.json and the
|
||||||
|
# anime_directory may be configured there even if the env var is empty.
|
||||||
|
try:
|
||||||
|
logger.info("Initializing scheduler service...")
|
||||||
|
from src.server.services.scheduler_service import (
|
||||||
|
get_scheduler_service,
|
||||||
|
)
|
||||||
|
scheduler_service = get_scheduler_service()
|
||||||
|
logger.info("Scheduler service instance obtained, starting...")
|
||||||
|
await scheduler_service.start()
|
||||||
|
initialized['scheduler'] = True
|
||||||
|
logger.info("Scheduler service started successfully")
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning("Failed to start scheduler service: %s", e)
|
||||||
except (OSError, RuntimeError, ValueError) as e:
|
except (OSError, RuntimeError, ValueError) as e:
|
||||||
logger.warning("Failed to initialize services: %s", e)
|
logger.warning("Failed to initialize services: %s", e)
|
||||||
# Continue startup - services can be initialized later
|
# Continue startup - services can be initialized later
|
||||||
|
|||||||
@@ -498,13 +498,19 @@ class AnimeService:
|
|||||||
logger.info("No series found in SeriesApp")
|
logger.info("No series found in SeriesApp")
|
||||||
return []
|
return []
|
||||||
|
|
||||||
# Build NFO metadata map and filter data from database
|
# Build NFO metadata map, episode dict, and filter data from database.
|
||||||
nfo_map = {}
|
# Using DB as authoritative source for episodeDict ensures that
|
||||||
series_with_no_episodes = set()
|
# episodes marked is_downloaded=True are never shown as missing,
|
||||||
|
# even if the in-memory state is stale.
|
||||||
|
nfo_map: dict = {}
|
||||||
|
db_episode_dict_map: dict[str, dict[int, list[int]]] = {}
|
||||||
|
series_with_no_episodes: set = set()
|
||||||
|
|
||||||
async with get_db_session() as db:
|
async with get_db_session() as db:
|
||||||
# Get all series NFO metadata using service layer
|
# Single query: load all series with their episodes eagerly
|
||||||
db_series_list = await AnimeSeriesService.get_all(db)
|
db_series_list = await AnimeSeriesService.get_all(
|
||||||
|
db, with_episodes=True
|
||||||
|
)
|
||||||
|
|
||||||
for db_series in db_series_list:
|
for db_series in db_series_list:
|
||||||
nfo_created = (
|
nfo_created = (
|
||||||
@@ -523,6 +529,20 @@ class AnimeService:
|
|||||||
"tvdb_id": db_series.tvdb_id,
|
"tvdb_id": db_series.tvdb_id,
|
||||||
"series_id": db_series.id,
|
"series_id": db_series.id,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# Build episodeDict from DB, skipping is_downloaded=True
|
||||||
|
# episodes so they are never shown as missing in the UI.
|
||||||
|
ep_dict: dict[int, list[int]] = {}
|
||||||
|
if db_series.episodes:
|
||||||
|
for ep in db_series.episodes:
|
||||||
|
if ep.is_downloaded:
|
||||||
|
continue
|
||||||
|
if ep.season not in ep_dict:
|
||||||
|
ep_dict[ep.season] = []
|
||||||
|
ep_dict[ep.season].append(ep.episode_number)
|
||||||
|
for s in ep_dict:
|
||||||
|
ep_dict[s].sort()
|
||||||
|
db_episode_dict_map[db_series.folder] = ep_dict
|
||||||
|
|
||||||
# If filter is "missing_episodes", get series with any missing episodes
|
# If filter is "missing_episodes", get series with any missing episodes
|
||||||
if filter_type == "missing_episodes":
|
if filter_type == "missing_episodes":
|
||||||
@@ -545,7 +565,12 @@ class AnimeService:
|
|||||||
name = getattr(serie, "name", "")
|
name = getattr(serie, "name", "")
|
||||||
site = getattr(serie, "site", "")
|
site = getattr(serie, "site", "")
|
||||||
folder = getattr(serie, "folder", "")
|
folder = getattr(serie, "folder", "")
|
||||||
episode_dict = getattr(serie, "episodeDict", {}) or {}
|
# Use DB-backed episodeDict (is_downloaded=True already filtered out)
|
||||||
|
# with in-memory episodeDict as fallback if the series isn't in DB yet.
|
||||||
|
episode_dict = db_episode_dict_map.get(
|
||||||
|
folder,
|
||||||
|
getattr(serie, "episodeDict", {}) or {}
|
||||||
|
)
|
||||||
|
|
||||||
# Apply filter if specified
|
# Apply filter if specified
|
||||||
if filter_type == "missing_episodes":
|
if filter_type == "missing_episodes":
|
||||||
@@ -815,18 +840,24 @@ class AnimeService:
|
|||||||
- Adds new missing episodes that are not in the database
|
- Adds new missing episodes that are not in the database
|
||||||
- Removes episodes from database that are no longer missing
|
- Removes episodes from database that are no longer missing
|
||||||
(i.e., the file has been added to the filesystem)
|
(i.e., the file has been added to the filesystem)
|
||||||
|
- Preserves episodes marked as downloaded (is_downloaded=True)
|
||||||
|
so download history is not lost
|
||||||
"""
|
"""
|
||||||
from src.server.database.service import AnimeSeriesService, EpisodeService
|
from src.server.database.service import AnimeSeriesService, EpisodeService
|
||||||
|
|
||||||
# Get existing episodes from database
|
# Get existing episodes from database (all episodes, including downloaded)
|
||||||
existing_episodes = await EpisodeService.get_by_series(db, existing.id)
|
existing_episodes = await EpisodeService.get_by_series(db, existing.id)
|
||||||
|
|
||||||
# Build dict of existing episodes: {season: {ep_num: episode_id}}
|
# Build dict of existing episodes: {season: {ep_num: episode_id}}
|
||||||
|
# and track which ones are already downloaded
|
||||||
existing_dict: dict[int, dict[int, int]] = {}
|
existing_dict: dict[int, dict[int, int]] = {}
|
||||||
|
downloaded_set: set[tuple[int, int]] = set()
|
||||||
for ep in existing_episodes:
|
for ep in existing_episodes:
|
||||||
if ep.season not in existing_dict:
|
if ep.season not in existing_dict:
|
||||||
existing_dict[ep.season] = {}
|
existing_dict[ep.season] = {}
|
||||||
existing_dict[ep.season][ep.episode_number] = ep.id
|
existing_dict[ep.season][ep.episode_number] = ep.id
|
||||||
|
if ep.is_downloaded:
|
||||||
|
downloaded_set.add((ep.season, ep.episode_number))
|
||||||
|
|
||||||
# Get new missing episodes from scan
|
# Get new missing episodes from scan
|
||||||
new_dict = serie.episodeDict or {}
|
new_dict = serie.episodeDict or {}
|
||||||
@@ -857,9 +888,22 @@ class AnimeService:
|
|||||||
|
|
||||||
# Remove episodes from database that are no longer missing
|
# Remove episodes from database that are no longer missing
|
||||||
# (i.e., the episode file now exists on the filesystem)
|
# (i.e., the episode file now exists on the filesystem)
|
||||||
|
# BUT: preserve episodes that are already downloaded (is_downloaded=True)
|
||||||
|
# so we don't lose download history
|
||||||
for season, eps_dict in existing_dict.items():
|
for season, eps_dict in existing_dict.items():
|
||||||
for ep_num, episode_id in eps_dict.items():
|
for ep_num, episode_id in eps_dict.items():
|
||||||
if (season, ep_num) not in new_missing_set:
|
if (season, ep_num) not in new_missing_set:
|
||||||
|
# Skip already-downloaded episodes — they should stay in DB
|
||||||
|
# with is_downloaded=True to preserve download history
|
||||||
|
if (season, ep_num) in downloaded_set:
|
||||||
|
logger.debug(
|
||||||
|
"Preserving downloaded episode in database: "
|
||||||
|
"%s S%02dE%02d",
|
||||||
|
serie.key,
|
||||||
|
season,
|
||||||
|
ep_num
|
||||||
|
)
|
||||||
|
continue
|
||||||
await EpisodeService.delete(db, episode_id)
|
await EpisodeService.delete(db, episode_id)
|
||||||
logger.info(
|
logger.info(
|
||||||
"Removed episode from database (no longer missing): "
|
"Removed episode from database (no longer missing): "
|
||||||
@@ -889,6 +933,10 @@ class AnimeService:
|
|||||||
|
|
||||||
This method is called during initialization and after rescans
|
This method is called during initialization and after rescans
|
||||||
to ensure the in-memory series list is in sync with the database.
|
to ensure the in-memory series list is in sync with the database.
|
||||||
|
|
||||||
|
Only episodes where is_downloaded=False are loaded into the
|
||||||
|
in-memory episodeDict, so downloaded episodes are not shown
|
||||||
|
as missing.
|
||||||
"""
|
"""
|
||||||
from src.core.entities.series import Serie
|
from src.core.entities.series import Serie
|
||||||
from src.server.database.connection import get_db_session
|
from src.server.database.connection import get_db_session
|
||||||
@@ -903,9 +951,14 @@ class AnimeService:
|
|||||||
series_list = []
|
series_list = []
|
||||||
for anime_series in anime_series_list:
|
for anime_series in anime_series_list:
|
||||||
# Build episode_dict from episodes relationship
|
# Build episode_dict from episodes relationship
|
||||||
|
# Only include episodes that are NOT downloaded (is_downloaded=False)
|
||||||
|
# so the missing-episode list stays accurate
|
||||||
episode_dict: dict[int, list[int]] = {}
|
episode_dict: dict[int, list[int]] = {}
|
||||||
if anime_series.episodes:
|
if anime_series.episodes:
|
||||||
for episode in anime_series.episodes:
|
for episode in anime_series.episodes:
|
||||||
|
# Skip downloaded episodes — they are not missing
|
||||||
|
if episode.is_downloaded:
|
||||||
|
continue
|
||||||
season = episode.season
|
season = episode.season
|
||||||
if season not in episode_dict:
|
if season not in episode_dict:
|
||||||
episode_dict[season] = []
|
episode_dict[season] = []
|
||||||
@@ -963,23 +1016,39 @@ class AnimeService:
|
|||||||
logger.warning("Series not found in database: %s", series_key)
|
logger.warning("Series not found in database: %s", series_key)
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
# Get existing episodes from database
|
# Get existing episodes from database (all, including downloaded)
|
||||||
existing_episodes = await EpisodeService.get_by_series(db, series_db.id)
|
existing_episodes = await EpisodeService.get_by_series(db, series_db.id)
|
||||||
|
|
||||||
# Build dict of existing episodes: {season: {ep_num: episode_id}}
|
# Build dict of existing episodes: {season: {ep_num: episode_id}}
|
||||||
|
# and track which ones are already downloaded
|
||||||
existing_dict: dict[int, dict[int, int]] = {}
|
existing_dict: dict[int, dict[int, int]] = {}
|
||||||
|
downloaded_set: set[tuple[int, int]] = set()
|
||||||
for ep in existing_episodes:
|
for ep in existing_episodes:
|
||||||
if ep.season not in existing_dict:
|
if ep.season not in existing_dict:
|
||||||
existing_dict[ep.season] = {}
|
existing_dict[ep.season] = {}
|
||||||
existing_dict[ep.season][ep.episode_number] = ep.id
|
existing_dict[ep.season][ep.episode_number] = ep.id
|
||||||
|
if ep.is_downloaded:
|
||||||
|
downloaded_set.add((ep.season, ep.episode_number))
|
||||||
|
|
||||||
# Get new missing episodes from in-memory serie
|
# Get new missing episodes from in-memory serie
|
||||||
new_dict = serie.episodeDict or {}
|
new_dict = serie.episodeDict or {}
|
||||||
|
|
||||||
# Add new missing episodes that are not in the database
|
# Add new missing episodes that are not in the database
|
||||||
|
# Skip episodes that are already downloaded (is_downloaded=True)
|
||||||
|
# so we don't re-add them as missing after they've been downloaded
|
||||||
for season, episode_numbers in new_dict.items():
|
for season, episode_numbers in new_dict.items():
|
||||||
existing_season_eps = existing_dict.get(season, {})
|
existing_season_eps = existing_dict.get(season, {})
|
||||||
for ep_num in episode_numbers:
|
for ep_num in episode_numbers:
|
||||||
|
# Skip if already downloaded — don't re-add as missing
|
||||||
|
if (season, ep_num) in downloaded_set:
|
||||||
|
logger.debug(
|
||||||
|
"Skipping already-downloaded episode: "
|
||||||
|
"%s S%02dE%02d",
|
||||||
|
series_key,
|
||||||
|
season,
|
||||||
|
ep_num,
|
||||||
|
)
|
||||||
|
continue
|
||||||
if ep_num not in existing_season_eps:
|
if ep_num not in existing_season_eps:
|
||||||
await EpisodeService.create(
|
await EpisodeService.create(
|
||||||
db=db,
|
db=db,
|
||||||
@@ -1015,20 +1084,23 @@ class AnimeService:
|
|||||||
if hasattr(self._app, 'list') and hasattr(self._app.list, 'keyDict'):
|
if hasattr(self._app, 'list') and hasattr(self._app.list, 'keyDict'):
|
||||||
serie = self._app.list.keyDict.get(series_key)
|
serie = self._app.list.keyDict.get(series_key)
|
||||||
if serie:
|
if serie:
|
||||||
# Convert episode dict keys to strings for JSON
|
# Fetch NFO metadata and episodes from database.
|
||||||
missing_episodes = {str(k): v for k, v in (serie.episodeDict or {}).items()}
|
# Using DB as the authoritative source for missing_episodes
|
||||||
total_missing = sum(len(eps) for eps in missing_episodes.values())
|
# ensures that episodes marked is_downloaded=True are never
|
||||||
|
# broadcast as missing, even if in-memory state is stale.
|
||||||
# Fetch NFO metadata from database
|
|
||||||
has_nfo = False
|
has_nfo = False
|
||||||
nfo_created_at = None
|
nfo_created_at = None
|
||||||
nfo_updated_at = None
|
nfo_updated_at = None
|
||||||
tmdb_id = None
|
tmdb_id = None
|
||||||
tvdb_id = None
|
tvdb_id = None
|
||||||
|
missing_episodes: dict[str, list] = {}
|
||||||
|
|
||||||
try:
|
try:
|
||||||
from src.server.database.connection import get_db_session
|
from src.server.database.connection import get_db_session
|
||||||
from src.server.database.service import AnimeSeriesService
|
from src.server.database.service import (
|
||||||
|
AnimeSeriesService,
|
||||||
|
EpisodeService,
|
||||||
|
)
|
||||||
|
|
||||||
async with get_db_session() as db:
|
async with get_db_session() as db:
|
||||||
db_series = await AnimeSeriesService.get_by_key(db, series_key)
|
db_series = await AnimeSeriesService.get_by_key(db, series_key)
|
||||||
@@ -1044,12 +1116,31 @@ class AnimeService:
|
|||||||
)
|
)
|
||||||
tmdb_id = db_series.tmdb_id
|
tmdb_id = db_series.tmdb_id
|
||||||
tvdb_id = db_series.tvdb_id
|
tvdb_id = db_series.tvdb_id
|
||||||
|
|
||||||
|
# Build missing_episodes from DB, skipping is_downloaded=True
|
||||||
|
db_episodes = await EpisodeService.get_by_series(
|
||||||
|
db, db_series.id, only_missing=True
|
||||||
|
)
|
||||||
|
for ep in db_episodes:
|
||||||
|
key_str = str(ep.season)
|
||||||
|
if key_str not in missing_episodes:
|
||||||
|
missing_episodes[key_str] = []
|
||||||
|
missing_episodes[key_str].append(ep.episode_number)
|
||||||
|
for s in missing_episodes:
|
||||||
|
missing_episodes[s].sort()
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.warning(
|
logger.warning(
|
||||||
"Could not fetch NFO data for %s: %s",
|
"Could not fetch series data for %s from DB: %s",
|
||||||
series_key,
|
series_key,
|
||||||
str(e)
|
str(e)
|
||||||
)
|
)
|
||||||
|
# Fallback to in-memory state
|
||||||
|
missing_episodes = {
|
||||||
|
str(k): v
|
||||||
|
for k, v in (serie.episodeDict or {}).items()
|
||||||
|
}
|
||||||
|
|
||||||
|
total_missing = sum(len(eps) for eps in missing_episodes.values())
|
||||||
|
|
||||||
series_data = {
|
series_data = {
|
||||||
"key": serie.key,
|
"key": serie.key,
|
||||||
@@ -1463,19 +1554,17 @@ def get_anime_service(series_app: SeriesApp) -> AnimeService:
|
|||||||
return AnimeService(series_app)
|
return AnimeService(series_app)
|
||||||
|
|
||||||
|
|
||||||
async def sync_series_from_data_files(
|
async def sync_legacy_series_to_db(
|
||||||
anime_directory: str,
|
anime_directory: str,
|
||||||
log_instance=None # pylint: disable=unused-argument
|
log_instance=None # pylint: disable=unused-argument
|
||||||
) -> int:
|
) -> int:
|
||||||
"""
|
"""
|
||||||
Sync series from data files to the database.
|
One-time legacy sync: import any series from 'data' files
|
||||||
|
not already in the database.
|
||||||
|
|
||||||
Scans the anime directory for data files and adds any new series
|
Deprecated: Series are now loaded directly from the database.
|
||||||
to the database. Existing series are skipped (no duplicates).
|
This function remains for backwards compatibility with legacy
|
||||||
|
file-based data during migration.
|
||||||
This function is typically called during application startup to ensure
|
|
||||||
series metadata stored in filesystem data files is available in the
|
|
||||||
database.
|
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
anime_directory: Path to the anime directory with data files
|
anime_directory: Path to the anime directory with data files
|
||||||
@@ -1487,6 +1576,11 @@ async def sync_series_from_data_files(
|
|||||||
"""
|
"""
|
||||||
# Always use structlog for structured logging with keyword arguments
|
# Always use structlog for structured logging with keyword arguments
|
||||||
log = structlog.get_logger(__name__)
|
log = structlog.get_logger(__name__)
|
||||||
|
|
||||||
|
log.warning(
|
||||||
|
"sync_legacy_series_to_db is deprecated. "
|
||||||
|
"Series are now loaded directly from database."
|
||||||
|
)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
from src.server.database.connection import get_db_session
|
from src.server.database.connection import get_db_session
|
||||||
|
|||||||
@@ -275,7 +275,7 @@ class DownloadService:
|
|||||||
"""
|
"""
|
||||||
try:
|
try:
|
||||||
from src.server.database.connection import get_db_session
|
from src.server.database.connection import get_db_session
|
||||||
from src.server.database.service import EpisodeService, AnimeSeriesService
|
from src.server.database.service import AnimeSeriesService, EpisodeService
|
||||||
|
|
||||||
logger.info(
|
logger.info(
|
||||||
"Attempting to mark episode as downloaded in DB: "
|
"Attempting to mark episode as downloaded in DB: "
|
||||||
@@ -362,6 +362,31 @@ class DownloadService:
|
|||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
|
# Broadcast real-time update to frontend so the series card
|
||||||
|
# immediately reflects the new downloaded state (no longer
|
||||||
|
# shows the episode as missing) without waiting for a full
|
||||||
|
# reload on DOWNLOAD_COMPLETED.
|
||||||
|
try:
|
||||||
|
await self._anime_service._broadcast_series_updated(
|
||||||
|
series_key
|
||||||
|
)
|
||||||
|
logger.debug(
|
||||||
|
"Broadcast series_updated after marking "
|
||||||
|
"%s S%02dE%02d as downloaded",
|
||||||
|
series_key,
|
||||||
|
season,
|
||||||
|
episode,
|
||||||
|
)
|
||||||
|
except Exception as broadcast_exc:
|
||||||
|
logger.warning(
|
||||||
|
"Failed to broadcast series update after marking "
|
||||||
|
"%s S%02dE%02d as downloaded: %s",
|
||||||
|
series_key,
|
||||||
|
season,
|
||||||
|
episode,
|
||||||
|
broadcast_exc,
|
||||||
|
)
|
||||||
|
|
||||||
return True
|
return True
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.error(
|
logger.error(
|
||||||
@@ -441,6 +466,27 @@ class DownloadService:
|
|||||||
"missing episodes remaining",
|
"missing episodes remaining",
|
||||||
len(app.series_list),
|
len(app.series_list),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# Update deprecated data file if it exists
|
||||||
|
# DB is authoritative; data file is optional backup
|
||||||
|
serie_folder = serie.folder
|
||||||
|
data_path = Path(self._directory) / serie_folder / "data"
|
||||||
|
if data_path.exists():
|
||||||
|
try:
|
||||||
|
import warnings
|
||||||
|
with warnings.catch_warnings():
|
||||||
|
warnings.simplefilter("ignore", DeprecationWarning)
|
||||||
|
serie.save_to_file(str(data_path))
|
||||||
|
logger.debug(
|
||||||
|
"Updated data file after download: %s",
|
||||||
|
data_path,
|
||||||
|
)
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(
|
||||||
|
"Failed to update data file %s: %s",
|
||||||
|
data_path,
|
||||||
|
e,
|
||||||
|
)
|
||||||
else:
|
else:
|
||||||
logger.debug(
|
logger.debug(
|
||||||
"Episode %d not in season %d for %s, "
|
"Episode %d not in season %d for %s, "
|
||||||
@@ -1097,6 +1143,7 @@ class DownloadService:
|
|||||||
item.status = DownloadStatus.PENDING
|
item.status = DownloadStatus.PENDING
|
||||||
item.error = None
|
item.error = None
|
||||||
item.progress = None
|
item.progress = None
|
||||||
|
item.retry_count += 1
|
||||||
self._add_to_pending_queue(item)
|
self._add_to_pending_queue(item)
|
||||||
retried_ids.append(item.id)
|
retried_ids.append(item.id)
|
||||||
|
|
||||||
|
|||||||
@@ -6,7 +6,8 @@ from typing import Callable, Optional
|
|||||||
import structlog
|
import structlog
|
||||||
|
|
||||||
from src.config.settings import settings
|
from src.config.settings import settings
|
||||||
from src.server.services.anime_service import sync_series_from_data_files
|
from src.server.services.anime_service import sync_legacy_series_to_db
|
||||||
|
from src.server.services.legacy_file_migration import migrate_series_from_files_to_db
|
||||||
|
|
||||||
logger = structlog.get_logger(__name__)
|
logger = structlog.get_logger(__name__)
|
||||||
|
|
||||||
@@ -99,6 +100,57 @@ async def _mark_initial_scan_completed() -> None:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def _check_legacy_migration_status() -> bool:
|
||||||
|
"""Check if legacy key/data file migration has been completed.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
bool: True if migration was completed, False otherwise
|
||||||
|
"""
|
||||||
|
return await _check_scan_status(
|
||||||
|
check_method=lambda svc, db: svc.is_migration_legacy_files_completed(db),
|
||||||
|
scan_type="legacy_migration",
|
||||||
|
log_completed_msg="Legacy file migration already completed, skipping",
|
||||||
|
log_not_completed_msg="Legacy file migration not yet run, will check for files"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def _mark_legacy_migration_completed() -> None:
|
||||||
|
"""Mark the legacy file migration as completed in system settings."""
|
||||||
|
await _mark_scan_completed(
|
||||||
|
mark_method=lambda svc, db: svc.mark_migration_legacy_files_completed(db),
|
||||||
|
scan_type="legacy_migration"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def _migrate_legacy_files() -> int:
|
||||||
|
"""Migrate series from legacy key/data files to database.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
int: Number of series migrated
|
||||||
|
"""
|
||||||
|
from src.server.database.connection import get_db_session
|
||||||
|
|
||||||
|
logger.info("Checking for legacy key/data files to migrate...")
|
||||||
|
|
||||||
|
try:
|
||||||
|
async with get_db_session() as db:
|
||||||
|
migrated_count = await migrate_series_from_files_to_db(
|
||||||
|
settings.anime_directory,
|
||||||
|
db
|
||||||
|
)
|
||||||
|
|
||||||
|
if migrated_count > 0:
|
||||||
|
logger.info("Migrated %d series from legacy files", migrated_count)
|
||||||
|
else:
|
||||||
|
logger.info("No series found in legacy files to migrate")
|
||||||
|
|
||||||
|
return migrated_count
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning("Failed to migrate legacy files: %s", e)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
async def _sync_anime_folders(progress_service=None) -> int:
|
async def _sync_anime_folders(progress_service=None) -> int:
|
||||||
"""Scan anime folders and sync series to database.
|
"""Scan anime folders and sync series to database.
|
||||||
|
|
||||||
@@ -118,7 +170,7 @@ async def _sync_anime_folders(progress_service=None) -> int:
|
|||||||
metadata={"step_id": "series_sync"}
|
metadata={"step_id": "series_sync"}
|
||||||
)
|
)
|
||||||
|
|
||||||
sync_count = await sync_series_from_data_files(settings.anime_directory)
|
sync_count = await sync_legacy_series_to_db(settings.anime_directory)
|
||||||
logger.info("Data file sync complete. Added %d series.", sync_count)
|
logger.info("Data file sync complete. Added %d series.", sync_count)
|
||||||
|
|
||||||
if progress_service:
|
if progress_service:
|
||||||
@@ -181,18 +233,19 @@ async def _validate_anime_directory(progress_service=None) -> bool:
|
|||||||
|
|
||||||
async def perform_initial_setup(progress_service=None):
|
async def perform_initial_setup(progress_service=None):
|
||||||
"""Perform initial setup including series sync and scan completion marking.
|
"""Perform initial setup including series sync and scan completion marking.
|
||||||
|
|
||||||
This function is called both during application lifespan startup
|
This function is called both during application lifespan startup
|
||||||
and when the setup endpoint is completed. It ensures that:
|
and when the setup endpoint is completed. It ensures that:
|
||||||
1. Series are synced from data files to database
|
1. Legacy key/data files are migrated to database (one-time)
|
||||||
2. Initial scan is marked as completed
|
2. Series are synced from data files to database
|
||||||
3. Series are loaded into memory
|
3. Initial scan is marked as completed
|
||||||
4. NFO scan is performed if configured
|
4. Series are loaded into memory
|
||||||
5. Media scan is performed
|
5. NFO scan is performed if configured
|
||||||
|
6. Media scan is performed
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
progress_service: Optional ProgressService for emitting updates
|
progress_service: Optional ProgressService for emitting updates
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
bool: True if initialization was performed, False if skipped
|
bool: True if initialization was performed, False if skipped
|
||||||
"""
|
"""
|
||||||
@@ -225,17 +278,23 @@ async def perform_initial_setup(progress_service=None):
|
|||||||
|
|
||||||
# Perform the actual initialization
|
# Perform the actual initialization
|
||||||
try:
|
try:
|
||||||
|
# First, run legacy file migration if needed (independent of initial scan)
|
||||||
|
is_legacy_migration_done = await _check_legacy_migration_status()
|
||||||
|
if not is_legacy_migration_done:
|
||||||
|
await _migrate_legacy_files()
|
||||||
|
await _mark_legacy_migration_completed()
|
||||||
|
|
||||||
# Sync series from anime folders to database
|
# Sync series from anime folders to database
|
||||||
await _sync_anime_folders(progress_service)
|
await _sync_anime_folders(progress_service)
|
||||||
|
|
||||||
# Mark the initial scan as completed
|
# Mark the initial scan as completed
|
||||||
await _mark_initial_scan_completed()
|
await _mark_initial_scan_completed()
|
||||||
|
|
||||||
# Load series into memory from database
|
# Load series into memory from database
|
||||||
await _load_series_into_memory(progress_service)
|
await _load_series_into_memory(progress_service)
|
||||||
|
|
||||||
return True
|
return True
|
||||||
|
|
||||||
except (OSError, RuntimeError, ValueError) as e:
|
except (OSError, RuntimeError, ValueError) as e:
|
||||||
logger.warning("Failed to perform initial setup: %s", e)
|
logger.warning("Failed to perform initial setup: %s", e)
|
||||||
return False
|
return False
|
||||||
|
|||||||
233
src/server/services/legacy_file_migration.py
Normal file
233
src/server/services/legacy_file_migration.py
Normal file
@@ -0,0 +1,233 @@
|
|||||||
|
"""One-time migration service for legacy key and data files.
|
||||||
|
|
||||||
|
This module provides functionality to migrate series data from legacy
|
||||||
|
file-based storage (key/data files) to the database. The migration is
|
||||||
|
designed to be idempotent and run only once per environment.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
import structlog
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncSession
|
||||||
|
|
||||||
|
logger = structlog.get_logger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
async def migrate_series_from_files_to_db(
|
||||||
|
anime_dir: str,
|
||||||
|
db: AsyncSession,
|
||||||
|
) -> int:
|
||||||
|
"""Migrate series from legacy key/data files to database.
|
||||||
|
|
||||||
|
Scans for folders containing legacy 'key' or 'data' files and imports
|
||||||
|
any series not already in the database. The DB version wins if a series
|
||||||
|
exists in both places.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
anime_dir: Path to the anime directory
|
||||||
|
db: Database session
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Number of series imported
|
||||||
|
"""
|
||||||
|
from src.server.database.service import AnimeSeriesService, EpisodeService
|
||||||
|
|
||||||
|
if not anime_dir or not os.path.isdir(anime_dir):
|
||||||
|
logger.warning(
|
||||||
|
"Anime directory does not exist, skipping legacy migration",
|
||||||
|
anime_dir=anime_dir
|
||||||
|
)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
migrated_count = 0
|
||||||
|
scanned_count = 0
|
||||||
|
|
||||||
|
try:
|
||||||
|
for folder_name in os.listdir(anime_dir):
|
||||||
|
folder_path = os.path.join(anime_dir, folder_name)
|
||||||
|
|
||||||
|
if not os.path.isdir(folder_path):
|
||||||
|
continue
|
||||||
|
|
||||||
|
scanned_count += 1
|
||||||
|
|
||||||
|
# Check for 'key' file (single line with series key)
|
||||||
|
key_file = os.path.join(folder_path, "key")
|
||||||
|
# Check for 'data' file (JSON with series metadata)
|
||||||
|
data_file = os.path.join(folder_path, "data")
|
||||||
|
|
||||||
|
series_data: Optional[dict] = None
|
||||||
|
|
||||||
|
# Try to load from 'data' file first (more complete)
|
||||||
|
if os.path.isfile(data_file):
|
||||||
|
series_data = _load_data_file(data_file)
|
||||||
|
elif os.path.isfile(key_file):
|
||||||
|
# Fall back to 'key' file - just the key, need to infer other data
|
||||||
|
series_data = _load_key_file(key_file, folder_name)
|
||||||
|
|
||||||
|
if series_data is None:
|
||||||
|
continue
|
||||||
|
|
||||||
|
key = series_data.get("key")
|
||||||
|
if not key:
|
||||||
|
logger.warning(
|
||||||
|
"Skipping folder with no valid key",
|
||||||
|
folder=folder_name
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Check if already in DB
|
||||||
|
existing = await AnimeSeriesService.get_by_key(db, key)
|
||||||
|
if existing:
|
||||||
|
logger.debug(
|
||||||
|
"Series already in database, skipping",
|
||||||
|
key=key,
|
||||||
|
folder=folder_name
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Create the series in DB
|
||||||
|
try:
|
||||||
|
name = series_data.get("name") or folder_name
|
||||||
|
site = series_data.get("site", "https://aniworld.to")
|
||||||
|
folder = series_data.get("folder", folder_name)
|
||||||
|
year = series_data.get("year")
|
||||||
|
|
||||||
|
anime_series = await AnimeSeriesService.create(
|
||||||
|
db=db,
|
||||||
|
key=key,
|
||||||
|
name=name,
|
||||||
|
site=site,
|
||||||
|
folder=folder,
|
||||||
|
year=year,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Create episodes if present
|
||||||
|
episode_dict = series_data.get("episodeDict", {})
|
||||||
|
if episode_dict:
|
||||||
|
for season, episode_numbers in episode_dict.items():
|
||||||
|
for episode_number in episode_numbers:
|
||||||
|
await EpisodeService.create(
|
||||||
|
db=db,
|
||||||
|
series_id=anime_series.id,
|
||||||
|
season=season,
|
||||||
|
episode_number=episode_number,
|
||||||
|
)
|
||||||
|
|
||||||
|
migrated_count += 1
|
||||||
|
logger.info(
|
||||||
|
"Migrated series from legacy file",
|
||||||
|
key=key,
|
||||||
|
name=name,
|
||||||
|
folder=folder_name
|
||||||
|
)
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(
|
||||||
|
"Failed to migrate series from legacy file",
|
||||||
|
key=key,
|
||||||
|
folder=folder_name,
|
||||||
|
error=str(e)
|
||||||
|
)
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(
|
||||||
|
"Legacy migration failed",
|
||||||
|
anime_dir=anime_dir,
|
||||||
|
error=str(e),
|
||||||
|
exc_info=True
|
||||||
|
)
|
||||||
|
|
||||||
|
logger.info(
|
||||||
|
"Legacy file migration complete",
|
||||||
|
scanned_folders=scanned_count,
|
||||||
|
migrated=migrated_count
|
||||||
|
)
|
||||||
|
return migrated_count
|
||||||
|
|
||||||
|
|
||||||
|
def _load_data_file(data_file_path: str) -> Optional[dict]:
|
||||||
|
"""Load and parse a legacy 'data' file (JSON).
|
||||||
|
|
||||||
|
Args:
|
||||||
|
data_file_path: Path to the data file
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Parsed data dict or None if parsing fails
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
with open(data_file_path, "r", encoding="utf-8") as f:
|
||||||
|
data = json.load(f)
|
||||||
|
|
||||||
|
if not isinstance(data, dict):
|
||||||
|
logger.warning(
|
||||||
|
"Data file is not a dictionary",
|
||||||
|
file=data_file_path
|
||||||
|
)
|
||||||
|
return None
|
||||||
|
|
||||||
|
# Ensure episodeDict has int keys
|
||||||
|
if "episodeDict" in data and isinstance(data["episodeDict"], dict):
|
||||||
|
data["episodeDict"] = {
|
||||||
|
int(k): v for k, v in data["episodeDict"].items()
|
||||||
|
}
|
||||||
|
|
||||||
|
return data
|
||||||
|
|
||||||
|
except json.JSONDecodeError as e:
|
||||||
|
logger.warning(
|
||||||
|
"Failed to parse legacy data file (JSON error)",
|
||||||
|
file=data_file_path,
|
||||||
|
error=str(e)
|
||||||
|
)
|
||||||
|
return None
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(
|
||||||
|
"Failed to read legacy data file",
|
||||||
|
file=data_file_path,
|
||||||
|
error=str(e)
|
||||||
|
)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _load_key_file(key_file_path: str, folder_name: str) -> Optional[dict]:
|
||||||
|
"""Load a legacy 'key' file (single line with series key).
|
||||||
|
|
||||||
|
Args:
|
||||||
|
key_file_path: Path to the key file
|
||||||
|
folder_name: Folder name to use as fallback name
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Data dict with key and inferred fields, or None if loading fails
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
with open(key_file_path, "r", encoding="utf-8") as f:
|
||||||
|
key = f.read().strip()
|
||||||
|
|
||||||
|
if not key:
|
||||||
|
logger.warning(
|
||||||
|
"Key file is empty",
|
||||||
|
file=key_file_path
|
||||||
|
)
|
||||||
|
return None
|
||||||
|
|
||||||
|
# Infer basic data from key file
|
||||||
|
return {
|
||||||
|
"key": key,
|
||||||
|
"name": folder_name,
|
||||||
|
"site": "https://aniworld.to",
|
||||||
|
"folder": folder_name,
|
||||||
|
"episodeDict": {},
|
||||||
|
}
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(
|
||||||
|
"Failed to read legacy key file",
|
||||||
|
file=key_file_path,
|
||||||
|
error=str(e)
|
||||||
|
)
|
||||||
|
return None
|
||||||
@@ -10,10 +10,10 @@ cron time), the job is triggered immediately within a grace period.
|
|||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
from datetime import datetime, timezone
|
from datetime import datetime, timezone
|
||||||
from typing import List, Optional
|
from typing import List, Optional
|
||||||
|
|
||||||
import structlog
|
|
||||||
from apscheduler.jobstores.sqlalchemy import SQLAlchemyJobStore
|
from apscheduler.jobstores.sqlalchemy import SQLAlchemyJobStore
|
||||||
from apscheduler.schedulers.asyncio import AsyncIOScheduler
|
from apscheduler.schedulers.asyncio import AsyncIOScheduler
|
||||||
from apscheduler.triggers.cron import CronTrigger
|
from apscheduler.triggers.cron import CronTrigger
|
||||||
@@ -21,7 +21,7 @@ from apscheduler.triggers.cron import CronTrigger
|
|||||||
from src.server.models.config import SchedulerConfig
|
from src.server.models.config import SchedulerConfig
|
||||||
from src.server.services.config_service import ConfigServiceError, get_config_service
|
from src.server.services.config_service import ConfigServiceError, get_config_service
|
||||||
|
|
||||||
logger = structlog.get_logger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
_JOB_ID = "scheduled_rescan"
|
_JOB_ID = "scheduled_rescan"
|
||||||
|
|
||||||
@@ -69,15 +69,18 @@ class SchedulerService:
|
|||||||
SchedulerServiceError: If the scheduler is already running or
|
SchedulerServiceError: If the scheduler is already running or
|
||||||
config cannot be loaded.
|
config cannot be loaded.
|
||||||
"""
|
"""
|
||||||
|
logger.info("SchedulerService.start() called")
|
||||||
if self._is_running:
|
if self._is_running:
|
||||||
|
logger.warning("Scheduler start called but already running")
|
||||||
raise SchedulerServiceError("Scheduler is already running")
|
raise SchedulerServiceError("Scheduler is already running")
|
||||||
|
|
||||||
try:
|
try:
|
||||||
config_service = get_config_service()
|
config_service = get_config_service()
|
||||||
config = config_service.load_config()
|
config = config_service.load_config()
|
||||||
self._config = config.scheduler
|
self._config = config.scheduler
|
||||||
|
logger.info("Scheduler config loaded successfully")
|
||||||
except ConfigServiceError as exc:
|
except ConfigServiceError as exc:
|
||||||
logger.error("Failed to load scheduler configuration", error=str(exc))
|
logger.error("Failed to load scheduler configuration: %s", exc)
|
||||||
raise SchedulerServiceError(f"Failed to load config: {exc}") from exc
|
raise SchedulerServiceError(f"Failed to load config: {exc}") from exc
|
||||||
|
|
||||||
jobstores = {
|
jobstores = {
|
||||||
@@ -90,6 +93,15 @@ class SchedulerService:
|
|||||||
self._is_running = True
|
self._is_running = True
|
||||||
return
|
return
|
||||||
|
|
||||||
|
logger.info(
|
||||||
|
"Scheduler config loaded: enabled=%s time=%s days=%s auto_download=%s folder_scan=%s",
|
||||||
|
self._config.enabled,
|
||||||
|
self._config.schedule_time,
|
||||||
|
self._config.schedule_days,
|
||||||
|
self._config.auto_download_after_rescan,
|
||||||
|
self._config.folder_scan_enabled,
|
||||||
|
)
|
||||||
|
|
||||||
trigger = self._build_cron_trigger()
|
trigger = self._build_cron_trigger()
|
||||||
if trigger is None:
|
if trigger is None:
|
||||||
logger.warning(
|
logger.warning(
|
||||||
@@ -105,9 +117,9 @@ class SchedulerService:
|
|||||||
coalesce=True,
|
coalesce=True,
|
||||||
)
|
)
|
||||||
logger.info(
|
logger.info(
|
||||||
"Scheduler started with cron trigger",
|
"Scheduler started with cron trigger: time=%s days=%s",
|
||||||
schedule_time=self._config.schedule_time,
|
self._config.schedule_time,
|
||||||
schedule_days=self._config.schedule_days,
|
self._config.schedule_days,
|
||||||
)
|
)
|
||||||
|
|
||||||
self._scheduler.start()
|
self._scheduler.start()
|
||||||
@@ -121,12 +133,13 @@ class SchedulerService:
|
|||||||
if job:
|
if job:
|
||||||
next_run = job.next_run_time
|
next_run = job.next_run_time
|
||||||
logger.info(
|
logger.info(
|
||||||
"Scheduler next run",
|
"Scheduler next run: %s",
|
||||||
next_run=next_run.isoformat() if next_run else None,
|
next_run.isoformat() if next_run else None,
|
||||||
)
|
)
|
||||||
|
|
||||||
async def stop(self) -> None:
|
async def stop(self) -> None:
|
||||||
"""Stop the APScheduler gracefully."""
|
"""Stop the APScheduler gracefully."""
|
||||||
|
logger.info("SchedulerService.stop() called")
|
||||||
if not self._is_running:
|
if not self._is_running:
|
||||||
logger.debug("Scheduler stop called but not running")
|
logger.debug("Scheduler stop called but not running")
|
||||||
return
|
return
|
||||||
@@ -134,8 +147,27 @@ class SchedulerService:
|
|||||||
if self._scheduler and self._scheduler.running:
|
if self._scheduler and self._scheduler.running:
|
||||||
self._scheduler.shutdown(wait=False)
|
self._scheduler.shutdown(wait=False)
|
||||||
logger.info("Scheduler stopped")
|
logger.info("Scheduler stopped")
|
||||||
|
else:
|
||||||
|
logger.info("Scheduler stop: scheduler was not running")
|
||||||
|
|
||||||
self._is_running = False
|
self._is_running = False
|
||||||
|
logger.info("SchedulerService stopped successfully")
|
||||||
|
|
||||||
|
async def ensure_started(self) -> None:
|
||||||
|
"""Ensure the scheduler is running (idempotent).
|
||||||
|
|
||||||
|
If already running, returns immediately. Otherwise, starts the scheduler.
|
||||||
|
This method is safe to call multiple times and from multiple callers.
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
SchedulerServiceError: If startup fails (except for already running).
|
||||||
|
"""
|
||||||
|
if self._is_running:
|
||||||
|
logger.debug("Scheduler ensure_started called but already running")
|
||||||
|
return
|
||||||
|
|
||||||
|
logger.info("Scheduler ensure_started: starting scheduler")
|
||||||
|
await self.start()
|
||||||
|
|
||||||
async def trigger_rescan(self) -> bool:
|
async def trigger_rescan(self) -> bool:
|
||||||
"""Manually trigger a library rescan.
|
"""Manually trigger a library rescan.
|
||||||
@@ -168,12 +200,12 @@ class SchedulerService:
|
|||||||
"""
|
"""
|
||||||
self._config = config
|
self._config = config
|
||||||
logger.info(
|
logger.info(
|
||||||
"Scheduler config reloaded",
|
"Scheduler config reloaded: enabled=%s time=%s days=%s auto_download=%s folder_scan=%s",
|
||||||
enabled=config.enabled,
|
config.enabled,
|
||||||
schedule_time=config.schedule_time,
|
config.schedule_time,
|
||||||
schedule_days=config.schedule_days,
|
config.schedule_days,
|
||||||
auto_download=config.auto_download_after_rescan,
|
config.auto_download_after_rescan,
|
||||||
folder_scan=config.folder_scan_enabled,
|
config.folder_scan_enabled,
|
||||||
)
|
)
|
||||||
|
|
||||||
if not self._scheduler or not self._scheduler.running:
|
if not self._scheduler or not self._scheduler.running:
|
||||||
@@ -194,9 +226,9 @@ class SchedulerService:
|
|||||||
if self._scheduler.get_job(_JOB_ID):
|
if self._scheduler.get_job(_JOB_ID):
|
||||||
self._scheduler.reschedule_job(_JOB_ID, trigger=trigger)
|
self._scheduler.reschedule_job(_JOB_ID, trigger=trigger)
|
||||||
logger.info(
|
logger.info(
|
||||||
"Scheduler rescheduled with cron trigger",
|
"Scheduler rescheduled with cron trigger: time=%s days=%s",
|
||||||
schedule_time=config.schedule_time,
|
config.schedule_time,
|
||||||
schedule_days=config.schedule_days,
|
config.schedule_days,
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
self._scheduler.add_job(
|
self._scheduler.add_job(
|
||||||
@@ -208,9 +240,9 @@ class SchedulerService:
|
|||||||
coalesce=True,
|
coalesce=True,
|
||||||
)
|
)
|
||||||
logger.info(
|
logger.info(
|
||||||
"Scheduler job added with cron trigger",
|
"Scheduler job added with cron trigger: time=%s days=%s",
|
||||||
schedule_time=config.schedule_time,
|
config.schedule_time,
|
||||||
schedule_days=config.schedule_days,
|
config.schedule_days,
|
||||||
)
|
)
|
||||||
|
|
||||||
def get_status(self) -> dict:
|
def get_status(self) -> dict:
|
||||||
@@ -264,10 +296,10 @@ class SchedulerService:
|
|||||||
day_of_week=day_of_week,
|
day_of_week=day_of_week,
|
||||||
)
|
)
|
||||||
logger.debug(
|
logger.debug(
|
||||||
"CronTrigger built",
|
"CronTrigger built: hour=%s minute=%s day_of_week=%s",
|
||||||
hour=hour_str,
|
hour_str,
|
||||||
minute=minute_str,
|
minute_str,
|
||||||
day_of_week=day_of_week,
|
day_of_week,
|
||||||
)
|
)
|
||||||
return trigger
|
return trigger
|
||||||
|
|
||||||
@@ -281,7 +313,7 @@ class SchedulerService:
|
|||||||
ws_service = get_websocket_service()
|
ws_service = get_websocket_service()
|
||||||
await ws_service.manager.broadcast({"type": event_type, "data": data})
|
await ws_service.manager.broadcast({"type": event_type, "data": data})
|
||||||
except Exception as exc: # pylint: disable=broad-exception-caught
|
except Exception as exc: # pylint: disable=broad-exception-caught
|
||||||
logger.warning("WebSocket broadcast failed", event=event_type, error=str(exc))
|
logger.warning("WebSocket broadcast failed: event=%s error=%s", event_type, exc)
|
||||||
|
|
||||||
async def _auto_download_missing(self) -> None:
|
async def _auto_download_missing(self) -> None:
|
||||||
"""Queue and start downloads for all series with missing episodes."""
|
"""Queue and start downloads for all series with missing episodes."""
|
||||||
@@ -299,9 +331,9 @@ class SchedulerService:
|
|||||||
elapsed = now - self._last_auto_download_time
|
elapsed = now - self._last_auto_download_time
|
||||||
if elapsed < timedelta(seconds=self._auto_download_cooldown_seconds):
|
if elapsed < timedelta(seconds=self._auto_download_cooldown_seconds):
|
||||||
logger.debug(
|
logger.debug(
|
||||||
"Auto-download skipped: cooldown active",
|
"Auto-download skipped: cooldown active (elapsed=%.1fs cooldown=%ds)",
|
||||||
elapsed_seconds=elapsed.total_seconds(),
|
elapsed.total_seconds(),
|
||||||
cooldown_seconds=self._auto_download_cooldown_seconds,
|
self._auto_download_cooldown_seconds,
|
||||||
)
|
)
|
||||||
return
|
return
|
||||||
|
|
||||||
@@ -334,30 +366,31 @@ class SchedulerService:
|
|||||||
)
|
)
|
||||||
queued_count += len(episodes)
|
queued_count += len(episodes)
|
||||||
logger.info(
|
logger.info(
|
||||||
"Auto-download queued episodes",
|
"Auto-download queued episodes for series=%s count=%d",
|
||||||
series=series.get("key"),
|
series.get("key"),
|
||||||
count=len(episodes),
|
len(episodes),
|
||||||
)
|
)
|
||||||
|
|
||||||
if queued_count:
|
if queued_count:
|
||||||
await download_service.start_queue_processing()
|
await download_service.start_queue_processing()
|
||||||
logger.info("Auto-download queue processing started", queued=queued_count)
|
logger.info("Auto-download queue processing started: queued=%d", queued_count)
|
||||||
|
|
||||||
await self._broadcast("auto_download_started", {"queued_count": queued_count})
|
await self._broadcast("auto_download_started", {"queued_count": queued_count})
|
||||||
logger.info("Auto-download completed", queued_count=queued_count)
|
logger.info("Auto-download completed: queued_count=%d", queued_count)
|
||||||
|
|
||||||
# Update cooldown timestamp after successful auto-download
|
# Update cooldown timestamp after successful auto-download
|
||||||
self._last_auto_download_time = datetime.now(timezone.utc)
|
self._last_auto_download_time = datetime.now(timezone.utc)
|
||||||
|
|
||||||
async def _perform_rescan(self) -> None:
|
async def _perform_rescan(self) -> None:
|
||||||
"""Execute a library rescan and optionally trigger auto-download."""
|
"""Execute a library rescan and optionally trigger auto-download."""
|
||||||
logger.info("Scheduler _perform_rescan entered", scan_in_progress=self._scan_in_progress)
|
logger.info("Scheduler _perform_rescan entered: scan_in_progress=%s", self._scan_in_progress)
|
||||||
if self._scan_in_progress:
|
if self._scan_in_progress:
|
||||||
logger.warning("Skipping rescan: previous scan still in progress")
|
logger.warning("Skipping rescan: previous scan still in progress")
|
||||||
return
|
return
|
||||||
|
|
||||||
self._scan_in_progress = True
|
self._scan_in_progress = True
|
||||||
scan_start = datetime.now(timezone.utc)
|
scan_start = datetime.now(timezone.utc)
|
||||||
|
logger.info("Scheduled rescan started at %s", scan_start.isoformat())
|
||||||
|
|
||||||
try:
|
try:
|
||||||
logger.info("Starting scheduled library rescan")
|
logger.info("Starting scheduled library rescan")
|
||||||
@@ -365,18 +398,20 @@ class SchedulerService:
|
|||||||
from src.server.utils.dependencies import get_anime_service # noqa: PLC0415
|
from src.server.utils.dependencies import get_anime_service # noqa: PLC0415
|
||||||
|
|
||||||
anime_service = get_anime_service()
|
anime_service = get_anime_service()
|
||||||
|
logger.info("Anime service obtained for rescan")
|
||||||
|
|
||||||
await self._broadcast(
|
await self._broadcast(
|
||||||
"scheduled_rescan_started",
|
"scheduled_rescan_started",
|
||||||
{"timestamp": scan_start.isoformat()},
|
{"timestamp": scan_start.isoformat()},
|
||||||
)
|
)
|
||||||
|
|
||||||
|
logger.info("Calling anime_service.rescan()...")
|
||||||
await anime_service.rescan()
|
await anime_service.rescan()
|
||||||
|
|
||||||
self._last_scan_time = datetime.now(timezone.utc)
|
self._last_scan_time = datetime.now(timezone.utc)
|
||||||
duration = (self._last_scan_time - scan_start).total_seconds()
|
duration = (self._last_scan_time - scan_start).total_seconds()
|
||||||
|
|
||||||
logger.info("Scheduled library rescan completed", duration_seconds=duration)
|
logger.info("Scheduled library rescan completed: duration=%.2fs", duration)
|
||||||
|
|
||||||
await self._broadcast(
|
await self._broadcast(
|
||||||
"scheduled_rescan_completed",
|
"scheduled_rescan_completed",
|
||||||
@@ -393,8 +428,8 @@ class SchedulerService:
|
|||||||
await self._auto_download_missing()
|
await self._auto_download_missing()
|
||||||
except Exception as dl_exc: # pylint: disable=broad-exception-caught
|
except Exception as dl_exc: # pylint: disable=broad-exception-caught
|
||||||
logger.error(
|
logger.error(
|
||||||
"Auto-download after rescan failed",
|
"Auto-download after rescan failed: %s",
|
||||||
error=str(dl_exc),
|
dl_exc,
|
||||||
exc_info=True,
|
exc_info=True,
|
||||||
)
|
)
|
||||||
await self._broadcast(
|
await self._broadcast(
|
||||||
@@ -413,10 +448,11 @@ class SchedulerService:
|
|||||||
|
|
||||||
folder_scan_service = FolderScanService()
|
folder_scan_service = FolderScanService()
|
||||||
await folder_scan_service.run_folder_scan()
|
await folder_scan_service.run_folder_scan()
|
||||||
|
logger.info("Folder scan completed successfully")
|
||||||
except Exception as fs_exc: # pylint: disable=broad-exception-caught
|
except Exception as fs_exc: # pylint: disable=broad-exception-caught
|
||||||
logger.error(
|
logger.error(
|
||||||
"Folder scan failed",
|
"Folder scan failed: %s",
|
||||||
error=str(fs_exc),
|
fs_exc,
|
||||||
exc_info=True,
|
exc_info=True,
|
||||||
)
|
)
|
||||||
await self._broadcast(
|
await self._broadcast(
|
||||||
@@ -426,7 +462,7 @@ class SchedulerService:
|
|||||||
logger.debug("Folder scan is disabled — skipping")
|
logger.debug("Folder scan is disabled — skipping")
|
||||||
|
|
||||||
except Exception as exc: # pylint: disable=broad-exception-caught
|
except Exception as exc: # pylint: disable=broad-exception-caught
|
||||||
logger.error("Scheduled rescan failed", error=str(exc), exc_info=True)
|
logger.error("Scheduled rescan failed: %s", exc, exc_info=True)
|
||||||
await self._broadcast(
|
await self._broadcast(
|
||||||
"scheduled_rescan_error",
|
"scheduled_rescan_error",
|
||||||
{"error": str(exc), "timestamp": datetime.now(timezone.utc).isoformat()},
|
{"error": str(exc), "timestamp": datetime.now(timezone.utc).isoformat()},
|
||||||
@@ -434,6 +470,7 @@ class SchedulerService:
|
|||||||
|
|
||||||
finally:
|
finally:
|
||||||
self._scan_in_progress = False
|
self._scan_in_progress = False
|
||||||
|
logger.info("Scheduled rescan finished: scan_in_progress reset to False")
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -446,9 +483,14 @@ class SchedulerService:
|
|||||||
|
|
||||||
async def _run_rescan_job() -> None:
|
async def _run_rescan_job() -> None:
|
||||||
"""Module-level job entry point — delegates to the current service."""
|
"""Module-level job entry point — delegates to the current service."""
|
||||||
|
logger.info("=" * 60)
|
||||||
logger.info("APScheduler triggered _run_rescan_job")
|
logger.info("APScheduler triggered _run_rescan_job")
|
||||||
|
logger.info("Getting scheduler service singleton...")
|
||||||
svc = get_scheduler_service()
|
svc = get_scheduler_service()
|
||||||
|
logger.info("Scheduler service obtained, calling _perform_rescan()")
|
||||||
await svc._perform_rescan()
|
await svc._perform_rescan()
|
||||||
|
logger.info("_run_rescan_job completed")
|
||||||
|
logger.info("=" * 60)
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -462,7 +504,10 @@ def get_scheduler_service() -> SchedulerService:
|
|||||||
"""Return the singleton SchedulerService instance."""
|
"""Return the singleton SchedulerService instance."""
|
||||||
global _scheduler_service
|
global _scheduler_service
|
||||||
if _scheduler_service is None:
|
if _scheduler_service is None:
|
||||||
|
logger.info("Creating new SchedulerService singleton")
|
||||||
_scheduler_service = SchedulerService()
|
_scheduler_service = SchedulerService()
|
||||||
|
else:
|
||||||
|
logger.debug("Returning existing SchedulerService singleton")
|
||||||
return _scheduler_service
|
return _scheduler_service
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -203,6 +203,17 @@ AniWorld.SeriesManager = (function() {
|
|||||||
function applyFiltersAndSort() {
|
function applyFiltersAndSort() {
|
||||||
let filtered = seriesData.slice();
|
let filtered = seriesData.slice();
|
||||||
|
|
||||||
|
// Apply client-side filter so that real-time WebSocket updates
|
||||||
|
// (e.g. an episode being marked downloaded) are immediately
|
||||||
|
// reflected without a full server reload.
|
||||||
|
if (filterMode === 'missing_episodes') {
|
||||||
|
filtered = filtered.filter(function(s) {
|
||||||
|
return s.missing_episodes > 0;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
// 'no_episodes' filter state is maintained server-side;
|
||||||
|
// don't try to replicate it client-side here.
|
||||||
|
|
||||||
// Sort based on the current sorting mode
|
// Sort based on the current sorting mode
|
||||||
filtered.sort(function(a, b) {
|
filtered.sort(function(a, b) {
|
||||||
if (sortAlphabetical) {
|
if (sortAlphabetical) {
|
||||||
@@ -233,8 +244,12 @@ AniWorld.SeriesManager = (function() {
|
|||||||
*/
|
*/
|
||||||
function renderSeries() {
|
function renderSeries() {
|
||||||
const grid = document.getElementById('series-grid');
|
const grid = document.getElementById('series-grid');
|
||||||
const dataToRender = filteredSeriesData.length > 0 ? filteredSeriesData :
|
// Always use filteredSeriesData — applyFiltersAndSort() is always
|
||||||
(seriesData.length > 0 ? seriesData : []);
|
// called before renderSeries(), so filteredSeriesData is current.
|
||||||
|
// The old fallback to seriesData was incorrect: when a filter is
|
||||||
|
// active and filteredSeriesData is empty it must show the empty-state
|
||||||
|
// message, not fall through to unfiltered seriesData.
|
||||||
|
const dataToRender = filteredSeriesData;
|
||||||
|
|
||||||
if (dataToRender.length === 0) {
|
if (dataToRender.length === 0) {
|
||||||
let message;
|
let message;
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
import tempfile
|
import tempfile
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from unittest.mock import patch
|
from unittest.mock import AsyncMock, patch
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
from httpx import ASGITransport, AsyncClient
|
from httpx import ASGITransport, AsyncClient
|
||||||
@@ -207,3 +207,46 @@ async def test_tmdb_validation_endpoint_exists(authenticated_client):
|
|||||||
assert "message" in data
|
assert "message" in data
|
||||||
assert data["valid"] is False # Empty key should be invalid
|
assert data["valid"] is False # Empty key should be invalid
|
||||||
assert "required" in data["message"].lower()
|
assert "required" in data["message"].lower()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_update_config_with_anime_directory_starts_scheduler(
|
||||||
|
authenticated_client, mock_config_service
|
||||||
|
):
|
||||||
|
"""PUT /api/config with anime_directory syncs and starts scheduler."""
|
||||||
|
mock_scheduler = AsyncMock()
|
||||||
|
|
||||||
|
with patch("src.server.services.scheduler_service.get_scheduler_service") as mock_sched_fn:
|
||||||
|
mock_sched_fn.return_value = mock_scheduler
|
||||||
|
|
||||||
|
with patch("src.config.settings.settings") as mock_settings:
|
||||||
|
mock_settings.anime_directory = None
|
||||||
|
|
||||||
|
resp = await authenticated_client.put(
|
||||||
|
"/api/config",
|
||||||
|
json={"other": {"anime_directory": "/data/anime"}},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert resp.status_code == 200
|
||||||
|
mock_scheduler.ensure_started.assert_called_once()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_update_config_without_anime_directory_does_not_start_scheduler(
|
||||||
|
authenticated_client, mock_config_service
|
||||||
|
):
|
||||||
|
"""PUT /api/config without new anime_directory does not call scheduler.ensure_started()."""
|
||||||
|
mock_scheduler = AsyncMock()
|
||||||
|
|
||||||
|
with patch("src.server.services.scheduler_service.get_scheduler_service") as mock_sched_fn:
|
||||||
|
mock_sched_fn.return_value = mock_scheduler
|
||||||
|
|
||||||
|
with patch("src.config.settings.settings") as mock_settings:
|
||||||
|
mock_settings.anime_directory = "/already/set"
|
||||||
|
|
||||||
|
resp = await authenticated_client.put(
|
||||||
|
"/api/config", json={"other": {}}
|
||||||
|
)
|
||||||
|
|
||||||
|
assert resp.status_code == 200
|
||||||
|
mock_scheduler.ensure_started.assert_not_called()
|
||||||
|
|||||||
@@ -111,17 +111,17 @@ class TestGetAllSeriesFromDataFiles:
|
|||||||
|
|
||||||
|
|
||||||
class TestSyncSeriesToDatabase:
|
class TestSyncSeriesToDatabase:
|
||||||
"""Test sync_series_from_data_files function from anime_service."""
|
"""Test sync_legacy_series_to_db function from anime_service."""
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_sync_with_empty_directory(self):
|
async def test_sync_with_empty_directory(self):
|
||||||
"""Test sync with empty anime directory."""
|
"""Test sync with empty anime directory."""
|
||||||
from src.server.services.anime_service import sync_series_from_data_files
|
from src.server.services.anime_service import sync_legacy_series_to_db
|
||||||
|
|
||||||
with tempfile.TemporaryDirectory() as tmp_dir:
|
with tempfile.TemporaryDirectory() as tmp_dir:
|
||||||
with patch('src.core.SeriesApp.Loaders'), \
|
with patch('src.core.SeriesApp.Loaders'), \
|
||||||
patch('src.core.SeriesApp.SerieScanner'):
|
patch('src.core.SeriesApp.SerieScanner'):
|
||||||
count = await sync_series_from_data_files(tmp_dir)
|
count = await sync_legacy_series_to_db(tmp_dir)
|
||||||
|
|
||||||
assert count == 0
|
assert count == 0
|
||||||
# Function should complete successfully with no series
|
# Function should complete successfully with no series
|
||||||
@@ -134,7 +134,7 @@ class TestSyncSeriesToDatabase:
|
|||||||
from files and the sync function attempts to add them to the DB.
|
from files and the sync function attempts to add them to the DB.
|
||||||
The actual DB interaction is tested in test_add_to_db_creates_record.
|
The actual DB interaction is tested in test_add_to_db_creates_record.
|
||||||
"""
|
"""
|
||||||
from src.server.services.anime_service import sync_series_from_data_files
|
from src.server.services.anime_service import sync_legacy_series_to_db
|
||||||
|
|
||||||
with tempfile.TemporaryDirectory() as tmp_dir:
|
with tempfile.TemporaryDirectory() as tmp_dir:
|
||||||
# Create test data files
|
# Create test data files
|
||||||
@@ -160,7 +160,7 @@ class TestSyncSeriesToDatabase:
|
|||||||
patch('src.core.SeriesApp.SerieScanner'):
|
patch('src.core.SeriesApp.SerieScanner'):
|
||||||
# The function should return 0 because DB isn't available
|
# The function should return 0 because DB isn't available
|
||||||
# but should not crash
|
# but should not crash
|
||||||
count = await sync_series_from_data_files(tmp_dir)
|
count = await sync_legacy_series_to_db(tmp_dir)
|
||||||
|
|
||||||
# Since no real DB, it will fail gracefully
|
# Since no real DB, it will fail gracefully
|
||||||
# Function returns 0 when DB operations fail
|
# Function returns 0 when DB operations fail
|
||||||
@@ -170,7 +170,7 @@ class TestSyncSeriesToDatabase:
|
|||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_sync_handles_exceptions_gracefully(self):
|
async def test_sync_handles_exceptions_gracefully(self):
|
||||||
"""Test that sync handles exceptions without crashing."""
|
"""Test that sync handles exceptions without crashing."""
|
||||||
from src.server.services.anime_service import sync_series_from_data_files
|
from src.server.services.anime_service import sync_legacy_series_to_db
|
||||||
|
|
||||||
# Make SeriesApp raise an exception during initialization
|
# Make SeriesApp raise an exception during initialization
|
||||||
with patch('src.core.SeriesApp.Loaders'), \
|
with patch('src.core.SeriesApp.Loaders'), \
|
||||||
@@ -179,7 +179,7 @@ class TestSyncSeriesToDatabase:
|
|||||||
'src.core.SeriesApp.SerieList',
|
'src.core.SeriesApp.SerieList',
|
||||||
side_effect=Exception("Test error")
|
side_effect=Exception("Test error")
|
||||||
):
|
):
|
||||||
count = await sync_series_from_data_files("/fake/path")
|
count = await sync_legacy_series_to_db("/fake/path")
|
||||||
|
|
||||||
assert count == 0
|
assert count == 0
|
||||||
# Function should complete without crashing
|
# Function should complete without crashing
|
||||||
|
|||||||
@@ -21,7 +21,7 @@ class TestInitializationWorkflow:
|
|||||||
async def test_perform_initial_setup_with_mocked_dependencies(self):
|
async def test_perform_initial_setup_with_mocked_dependencies(self):
|
||||||
"""Test initial setup completes with minimal mocking."""
|
"""Test initial setup completes with minimal mocking."""
|
||||||
# Mock only the external dependencies
|
# Mock only the external dependencies
|
||||||
with patch('src.server.services.anime_service.sync_series_from_data_files') as mock_sync:
|
with patch('src.server.services.anime_service.sync_legacy_series_to_db') as mock_sync:
|
||||||
mock_sync.return_value = 0 # No series to sync
|
mock_sync.return_value = 0 # No series to sync
|
||||||
|
|
||||||
# Call the actual function
|
# Call the actual function
|
||||||
@@ -241,9 +241,9 @@ class TestModuleStructure:
|
|||||||
assert hasattr(initialization_service, 'settings')
|
assert hasattr(initialization_service, 'settings')
|
||||||
|
|
||||||
def test_sync_series_function_imported(self):
|
def test_sync_series_function_imported(self):
|
||||||
"""Test sync_series_from_data_files is imported."""
|
"""Test sync_legacy_series_to_db is imported."""
|
||||||
assert hasattr(initialization_service, 'sync_series_from_data_files')
|
assert hasattr(initialization_service, 'sync_legacy_series_to_db')
|
||||||
assert callable(initialization_service.sync_series_from_data_files)
|
assert callable(initialization_service.sync_legacy_series_to_db)
|
||||||
|
|
||||||
|
|
||||||
# Simpler integration tests that don't require complex mocking
|
# Simpler integration tests that don't require complex mocking
|
||||||
@@ -413,7 +413,7 @@ class TestInitialSetupWorkflow:
|
|||||||
async def test_initial_setup_already_completed(self):
|
async def test_initial_setup_already_completed(self):
|
||||||
"""Test initial setup when already completed."""
|
"""Test initial setup when already completed."""
|
||||||
with patch.object(initialization_service, '_check_initial_scan_status', return_value=True), \
|
with patch.object(initialization_service, '_check_initial_scan_status', return_value=True), \
|
||||||
patch('src.server.services.anime_service.sync_series_from_data_files'):
|
patch('src.server.services.anime_service.sync_legacy_series_to_db'):
|
||||||
|
|
||||||
result = await initialization_service.perform_initial_setup()
|
result = await initialization_service.perform_initial_setup()
|
||||||
|
|
||||||
@@ -425,7 +425,7 @@ class TestInitialSetupWorkflow:
|
|||||||
"""Test initial setup with no directory configured."""
|
"""Test initial setup with no directory configured."""
|
||||||
with patch.object(initialization_service, '_check_initial_scan_status', return_value=False), \
|
with patch.object(initialization_service, '_check_initial_scan_status', return_value=False), \
|
||||||
patch.object(initialization_service, '_validate_anime_directory', return_value=False), \
|
patch.object(initialization_service, '_validate_anime_directory', return_value=False), \
|
||||||
patch('src.server.services.anime_service.sync_series_from_data_files'):
|
patch('src.server.services.anime_service.sync_legacy_series_to_db'):
|
||||||
|
|
||||||
result = await initialization_service.perform_initial_setup()
|
result = await initialization_service.perform_initial_setup()
|
||||||
|
|
||||||
@@ -440,7 +440,7 @@ class TestInitialSetupWorkflow:
|
|||||||
patch.object(initialization_service, '_sync_anime_folders', return_value=5), \
|
patch.object(initialization_service, '_sync_anime_folders', return_value=5), \
|
||||||
patch.object(initialization_service, '_mark_initial_scan_completed'), \
|
patch.object(initialization_service, '_mark_initial_scan_completed'), \
|
||||||
patch.object(initialization_service, '_load_series_into_memory'), \
|
patch.object(initialization_service, '_load_series_into_memory'), \
|
||||||
patch('src.server.services.anime_service.sync_series_from_data_files'):
|
patch('src.server.services.anime_service.sync_legacy_series_to_db'):
|
||||||
|
|
||||||
mock_progress = AsyncMock()
|
mock_progress = AsyncMock()
|
||||||
result = await initialization_service.perform_initial_setup(mock_progress)
|
result = await initialization_service.perform_initial_setup(mock_progress)
|
||||||
@@ -456,7 +456,7 @@ class TestInitialSetupWorkflow:
|
|||||||
with patch.object(initialization_service, '_check_initial_scan_status', return_value=False), \
|
with patch.object(initialization_service, '_check_initial_scan_status', return_value=False), \
|
||||||
patch.object(initialization_service, '_validate_anime_directory', return_value=True), \
|
patch.object(initialization_service, '_validate_anime_directory', return_value=True), \
|
||||||
patch.object(initialization_service, '_sync_anime_folders', side_effect=OSError("Disk error")), \
|
patch.object(initialization_service, '_sync_anime_folders', side_effect=OSError("Disk error")), \
|
||||||
patch('src.server.services.anime_service.sync_series_from_data_files'):
|
patch('src.server.services.anime_service.sync_legacy_series_to_db'):
|
||||||
|
|
||||||
result = await initialization_service.perform_initial_setup()
|
result = await initialization_service.perform_initial_setup()
|
||||||
|
|
||||||
@@ -469,7 +469,7 @@ class TestInitialSetupWorkflow:
|
|||||||
with patch.object(initialization_service, '_check_initial_scan_status', return_value=False), \
|
with patch.object(initialization_service, '_check_initial_scan_status', return_value=False), \
|
||||||
patch.object(initialization_service, '_validate_anime_directory', return_value=True), \
|
patch.object(initialization_service, '_validate_anime_directory', return_value=True), \
|
||||||
patch.object(initialization_service, '_sync_anime_folders', side_effect=RuntimeError("DB error")), \
|
patch.object(initialization_service, '_sync_anime_folders', side_effect=RuntimeError("DB error")), \
|
||||||
patch('src.server.services.anime_service.sync_series_from_data_files'):
|
patch('src.server.services.anime_service.sync_legacy_series_to_db'):
|
||||||
|
|
||||||
result = await initialization_service.perform_initial_setup()
|
result = await initialization_service.perform_initial_setup()
|
||||||
|
|
||||||
|
|||||||
333
tests/integration/test_episode_download_sync.py
Normal file
333
tests/integration/test_episode_download_sync.py
Normal file
@@ -0,0 +1,333 @@
|
|||||||
|
"""Integration tests for episode download sync with data file updates.
|
||||||
|
|
||||||
|
Tests verify that when episodes are downloaded successfully:
|
||||||
|
- In-memory Serie.episodeDict is updated
|
||||||
|
- Deprecated data file is updated (if it exists)
|
||||||
|
- Missing episode list reflects the change immediately
|
||||||
|
"""
|
||||||
|
import asyncio
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import tempfile
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest.mock import AsyncMock, MagicMock, patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from src.core.entities.series import Serie
|
||||||
|
from src.core.SeriesApp import SeriesApp
|
||||||
|
from src.server.models.download import DownloadItem, DownloadPriority, DownloadStatus
|
||||||
|
from src.server.services.download_service import DownloadService
|
||||||
|
|
||||||
|
|
||||||
|
class TestEpisodeRemovedFromMissingListAfterDownload:
|
||||||
|
"""Verify episode no longer appears in missing list after download completes."""
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def temp_dir(self):
|
||||||
|
"""Create temp directory for test data files."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
yield Path(tmp)
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def mock_anime_service(self, temp_dir):
|
||||||
|
"""Create mock anime service with app."""
|
||||||
|
anime_service = MagicMock()
|
||||||
|
anime_service._directory = str(temp_dir)
|
||||||
|
|
||||||
|
# Create mock app withSerie with missing episodes
|
||||||
|
serie = Serie(
|
||||||
|
key="test-series",
|
||||||
|
name="Test Series",
|
||||||
|
site="https://example.com",
|
||||||
|
folder="Test Series",
|
||||||
|
episodeDict={1: [1, 2, 3]},
|
||||||
|
)
|
||||||
|
mock_app = MagicMock()
|
||||||
|
mock_app.list.keyDict = {"test-series": serie}
|
||||||
|
mock_app.list.GetMissingEpisode.return_value = [serie]
|
||||||
|
mock_app.series_list = [serie]
|
||||||
|
anime_service._app = mock_app
|
||||||
|
anime_service._cached_list_missing = MagicMock()
|
||||||
|
anime_service._broadcast_series_updated = AsyncMock()
|
||||||
|
|
||||||
|
return anime_service
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def mock_download_service(self, mock_anime_service):
|
||||||
|
"""Create download service with mocked dependencies."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
service = DownloadService(
|
||||||
|
anime_service=mock_anime_service,
|
||||||
|
queue_repository=MagicMock(),
|
||||||
|
max_retries=3,
|
||||||
|
)
|
||||||
|
service._directory = tmp
|
||||||
|
yield service
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_episode_removed_from_missing_list_after_download(
|
||||||
|
self, mock_download_service, mock_anime_service
|
||||||
|
):
|
||||||
|
"""Verify episode no longer appears in missing list after download completes."""
|
||||||
|
serie = mock_anime_service._app.list.keyDict["test-series"]
|
||||||
|
|
||||||
|
# Verify episode starts in missing list
|
||||||
|
assert 2 in serie.episodeDict[1], "Episode should start in missing list"
|
||||||
|
|
||||||
|
# Simulate download completion by calling _remove_episode_from_memory
|
||||||
|
mock_download_service._remove_episode_from_memory("test-series", 1, 2)
|
||||||
|
|
||||||
|
# Episode should be removed from episodeDict
|
||||||
|
assert 2 not in serie.episodeDict[1], "Episode should be removed from missing list"
|
||||||
|
assert serie.episodeDict[1] == [1, 3]
|
||||||
|
|
||||||
|
# series_list should be refreshed
|
||||||
|
mock_anime_service._app.list.GetMissingEpisode.assert_called()
|
||||||
|
|
||||||
|
|
||||||
|
class TestDownloadUpdatesInMemoryCache:
|
||||||
|
"""Verify in-memory Serie.episodeDict is updated after download."""
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def mock_anime_service(self):
|
||||||
|
"""Create mock anime service with app."""
|
||||||
|
anime_service = MagicMock()
|
||||||
|
anime_service._directory = "/tmp/test"
|
||||||
|
|
||||||
|
# Create mock app with series having multiple seasons and episodes
|
||||||
|
serie = Serie(
|
||||||
|
key="multi-season-series",
|
||||||
|
name="Multi Season Series",
|
||||||
|
site="https://example.com",
|
||||||
|
folder="Multi Season Series",
|
||||||
|
episodeDict={
|
||||||
|
1: [1, 2, 3, 4, 5],
|
||||||
|
2: [1, 2, 3],
|
||||||
|
},
|
||||||
|
)
|
||||||
|
mock_app = MagicMock()
|
||||||
|
mock_app.list.keyDict = {"multi-season-series": serie}
|
||||||
|
mock_app.list.GetMissingEpisode.return_value = [serie]
|
||||||
|
mock_app.series_list = [serie]
|
||||||
|
anime_service._app = mock_app
|
||||||
|
anime_service._cached_list_missing = MagicMock()
|
||||||
|
anime_service._broadcast_series_updated = AsyncMock()
|
||||||
|
|
||||||
|
return anime_service
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def mock_download_service(self, mock_anime_service):
|
||||||
|
"""Create download service with mocked dependencies."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
service = DownloadService(
|
||||||
|
anime_service=mock_anime_service,
|
||||||
|
queue_repository=MagicMock(),
|
||||||
|
max_retries=3,
|
||||||
|
)
|
||||||
|
service._directory = tmp
|
||||||
|
yield service
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_download_updates_in_memory_cache(
|
||||||
|
self, mock_download_service, mock_anime_service
|
||||||
|
):
|
||||||
|
"""Verify in-memory Serie.episodeDict is updated after download."""
|
||||||
|
# First reset to known state (remove the defaults first call might have set)
|
||||||
|
serie = mock_anime_service._app.list.keyDict["multi-season-series"]
|
||||||
|
|
||||||
|
# Put back episodes after the fixture setup
|
||||||
|
serie.episodeDict = {1: [1, 2, 3, 4, 5], 2: [1, 2, 3]}
|
||||||
|
|
||||||
|
# Verify preconditions
|
||||||
|
assert 1 in serie.episodeDict[1]
|
||||||
|
assert 3 in serie.episodeDict[2]
|
||||||
|
|
||||||
|
# Simulate downloading multiple episodes
|
||||||
|
mock_download_service._remove_episode_from_memory("multi-season-series", 1, 1)
|
||||||
|
mock_download_service._remove_episode_from_memory("multi-season-series", 1, 3)
|
||||||
|
mock_download_service._remove_episode_from_memory("multi-season-series", 2, 2)
|
||||||
|
|
||||||
|
# Verify episodes removed
|
||||||
|
assert 1 not in serie.episodeDict[1], "Episode 1 of season 1 should be removed"
|
||||||
|
assert 3 not in serie.episodeDict[1], "Episode 3 of season 1 should be removed"
|
||||||
|
assert 2 in serie.episodeDict[1], "Episode 2 of season 1 should remain"
|
||||||
|
assert 3 in serie.episodeDict[2], "Episode 3 of season 2 should remain"
|
||||||
|
assert 2 not in serie.episodeDict[2], "Episode 2 of season 2 should be removed"
|
||||||
|
|
||||||
|
# Verify seasons with no episodes are cleaned up
|
||||||
|
assert 2 in serie.episodeDict, "Season 2 should still exist (has episode 1, 3)"
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_last_episode_removes_season(
|
||||||
|
self, mock_download_service, mock_anime_service
|
||||||
|
):
|
||||||
|
"""Verify that removing last episode in a season removes the season key."""
|
||||||
|
# Modify the series so season 1 only has episode 2 left
|
||||||
|
serie = mock_anime_service._app.list.keyDict["multi-season-series"]
|
||||||
|
# Reset and set to proper test state
|
||||||
|
serie.episodeDict = {1: [2], 2: [1, 2, 3]} # Season 1 only has episode 2
|
||||||
|
|
||||||
|
# Verify initial state
|
||||||
|
assert 2 in serie.episodeDict[1]
|
||||||
|
assert 2 in serie.episodeDict[2]
|
||||||
|
|
||||||
|
# Remove last episode of season 1 (episode 2)
|
||||||
|
mock_download_service._remove_episode_from_memory("multi-season-series", 1, 2)
|
||||||
|
|
||||||
|
# Season 1 should be completely removed
|
||||||
|
assert 1 not in serie.episodeDict, "Season 1 should be removed"
|
||||||
|
# Season 2 should still exist
|
||||||
|
assert 2 in serie.episodeDict, "Season 2 should still exist"
|
||||||
|
|
||||||
|
|
||||||
|
class TestDataFileUpdatedAfterDownload:
|
||||||
|
"""Verify data file is updated after download (when it exists)."""
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def temp_dir(self):
|
||||||
|
"""Create temp directory for test data files."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
yield Path(tmp)
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def mock_anime_service(self, temp_dir):
|
||||||
|
"""Create mock anime service with app."""
|
||||||
|
anime_service = MagicMock()
|
||||||
|
anime_service._directory = str(temp_dir)
|
||||||
|
|
||||||
|
# Create series folder with data file
|
||||||
|
series_folder = temp_dir / "Test Series"
|
||||||
|
series_folder.mkdir()
|
||||||
|
data_path = series_folder / "data"
|
||||||
|
|
||||||
|
serie = Serie(
|
||||||
|
key="test-series-with-data",
|
||||||
|
name="Test Series",
|
||||||
|
site="https://example.com",
|
||||||
|
folder="Test Series",
|
||||||
|
episodeDict={1: [1, 2, 3]},
|
||||||
|
)
|
||||||
|
|
||||||
|
# Save data file to disk
|
||||||
|
import warnings
|
||||||
|
with warnings.catch_warnings():
|
||||||
|
warnings.simplefilter("ignore", DeprecationWarning)
|
||||||
|
serie.save_to_file(str(data_path))
|
||||||
|
|
||||||
|
# Update episodeDict to simulate in-progress download state
|
||||||
|
# (episodeDict still has all episodes; will be updated after download)
|
||||||
|
mock_app = MagicMock()
|
||||||
|
mock_app.list.keyDict = {"test-series-with-data": serie}
|
||||||
|
mock_app.list.GetMissingEpisode.return_value = [serie]
|
||||||
|
mock_app.series_list = [serie]
|
||||||
|
anime_service._app = mock_app
|
||||||
|
anime_service._cached_list_missing = MagicMock()
|
||||||
|
anime_service._broadcast_series_updated = AsyncMock()
|
||||||
|
|
||||||
|
return anime_service
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def mock_download_service(self, mock_anime_service):
|
||||||
|
"""Create download service with mocked dependencies."""
|
||||||
|
service = DownloadService(
|
||||||
|
anime_service=mock_anime_service,
|
||||||
|
queue_repository=MagicMock(),
|
||||||
|
max_retries=3,
|
||||||
|
)
|
||||||
|
service._directory = str(mock_anime_service._directory)
|
||||||
|
yield service
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_data_file_updated_after_download(
|
||||||
|
self, mock_download_service, mock_anime_service, temp_dir
|
||||||
|
):
|
||||||
|
"""Verify data file is updated after download when data file exists."""
|
||||||
|
serie = mock_anime_service._app.list.keyDict["test-series-with-data"]
|
||||||
|
data_path = temp_dir / "Test Series" / "data"
|
||||||
|
|
||||||
|
# Verify data file exists before test
|
||||||
|
assert data_path.exists(), "Data file should exist before test"
|
||||||
|
|
||||||
|
# Read original data file
|
||||||
|
with open(data_path) as f:
|
||||||
|
original_data = json.load(f)
|
||||||
|
assert 2 in original_data["episodeDict"]["1"], "Episode should be in original data"
|
||||||
|
|
||||||
|
# Simulate download completion
|
||||||
|
mock_download_service._remove_episode_from_memory("test-series-with-data", 1, 2)
|
||||||
|
|
||||||
|
# Read updated data file
|
||||||
|
with open(data_path) as f:
|
||||||
|
updated_data = json.load(f)
|
||||||
|
|
||||||
|
# Verify episode 2 was removed from data file
|
||||||
|
assert 2 not in updated_data["episodeDict"]["1"], "Episode should be removed from data file"
|
||||||
|
assert updated_data["episodeDict"]["1"] == [1, 3]
|
||||||
|
|
||||||
|
|
||||||
|
class TestDataFileNotRequiredForDownload:
|
||||||
|
"""Verify downloads work even when data file doesn't exist."""
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def temp_dir(self):
|
||||||
|
"""Create temp directory without data files."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
yield Path(tmp)
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def mock_anime_service(self, temp_dir):
|
||||||
|
"""Create mock anime service with app but no data file."""
|
||||||
|
anime_service = MagicMock()
|
||||||
|
anime_service._directory = str(temp_dir)
|
||||||
|
|
||||||
|
# Create series with NO data file on disk (only in memory)
|
||||||
|
serie = Serie(
|
||||||
|
key="memory-only-series",
|
||||||
|
name="Memory Only Series",
|
||||||
|
site="https://example.com",
|
||||||
|
folder="Memory Only Series",
|
||||||
|
episodeDict={1: [1, 2, 3]},
|
||||||
|
)
|
||||||
|
|
||||||
|
mock_app = MagicMock()
|
||||||
|
mock_app.list.keyDict = {"memory-only-series": serie}
|
||||||
|
mock_app.list.GetMissingEpisode.return_value = [serie]
|
||||||
|
mock_app.series_list = [serie]
|
||||||
|
anime_service._app = mock_app
|
||||||
|
anime_service._cached_list_missing = MagicMock()
|
||||||
|
anime_service._broadcast_series_updated = AsyncMock()
|
||||||
|
|
||||||
|
return anime_service
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def mock_download_service(self, mock_anime_service):
|
||||||
|
"""Create download service with mocked dependencies."""
|
||||||
|
service = DownloadService(
|
||||||
|
anime_service=mock_anime_service,
|
||||||
|
queue_repository=MagicMock(),
|
||||||
|
max_retries=3,
|
||||||
|
)
|
||||||
|
service._directory = str(mock_anime_service._directory)
|
||||||
|
yield service
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_download_works_without_data_file(
|
||||||
|
self, mock_download_service, mock_anime_service
|
||||||
|
):
|
||||||
|
"""Verify downloads work even when no data file exists on disk."""
|
||||||
|
serie = mock_anime_service._app.list.keyDict["memory-only-series"]
|
||||||
|
data_path = Path(mock_anime_service._directory) / "Memory Only Series" / "data"
|
||||||
|
|
||||||
|
# Verify no data file exists
|
||||||
|
assert not data_path.exists(), "No data file should exist"
|
||||||
|
|
||||||
|
# Simulate download completion
|
||||||
|
# This should NOT raise even without data file
|
||||||
|
mock_download_service._remove_episode_from_memory("memory-only-series", 1, 2)
|
||||||
|
|
||||||
|
# Episode should be removed from in-memory state
|
||||||
|
assert 2 not in serie.episodeDict[1], "Episode should be removed from memory"
|
||||||
|
|
||||||
|
# Data file should still not exist (no file created)
|
||||||
|
assert not data_path.exists(), "No data file should be created"
|
||||||
335
tests/integration/test_legacy_migration.py
Normal file
335
tests/integration/test_legacy_migration.py
Normal file
@@ -0,0 +1,335 @@
|
|||||||
|
"""Integration tests for legacy key/data file migration.
|
||||||
|
|
||||||
|
Tests the one-time migration safety net that imports series from
|
||||||
|
legacy key and data files into the database.
|
||||||
|
"""
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import tempfile
|
||||||
|
from unittest.mock import AsyncMock, MagicMock, patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from src.server.services.legacy_file_migration import (
|
||||||
|
_load_data_file,
|
||||||
|
_load_key_file,
|
||||||
|
migrate_series_from_files_to_db,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class TestLoadLegacyFiles:
|
||||||
|
"""Test helper functions for loading legacy files."""
|
||||||
|
|
||||||
|
def test_load_data_file_valid_json(self):
|
||||||
|
"""Test loading a valid JSON data file."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp_dir:
|
||||||
|
data_file = os.path.join(tmp_dir, "data")
|
||||||
|
test_data = {
|
||||||
|
"key": "test-anime",
|
||||||
|
"name": "Test Anime",
|
||||||
|
"site": "https://aniworld.to",
|
||||||
|
"folder": "Test Anime",
|
||||||
|
"episodeDict": {"1": [1, 2, 3]}
|
||||||
|
}
|
||||||
|
with open(data_file, "w", encoding="utf-8") as f:
|
||||||
|
json.dump(test_data, f)
|
||||||
|
|
||||||
|
result = _load_data_file(data_file)
|
||||||
|
|
||||||
|
assert result is not None
|
||||||
|
assert result["key"] == "test-anime"
|
||||||
|
assert result["name"] == "Test Anime"
|
||||||
|
# episodeDict keys should be converted to int
|
||||||
|
assert 1 in result["episodeDict"]
|
||||||
|
|
||||||
|
def test_load_data_file_invalid_json(self):
|
||||||
|
"""Test handling of corrupt JSON data file."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp_dir:
|
||||||
|
data_file = os.path.join(tmp_dir, "data")
|
||||||
|
with open(data_file, "w", encoding="utf-8") as f:
|
||||||
|
f.write("this is not valid json {{{")
|
||||||
|
|
||||||
|
result = _load_data_file(data_file)
|
||||||
|
|
||||||
|
assert result is None
|
||||||
|
|
||||||
|
def test_load_data_file_not_dict(self):
|
||||||
|
"""Test handling of JSON file that is not a dict."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp_dir:
|
||||||
|
data_file = os.path.join(tmp_dir, "data")
|
||||||
|
with open(data_file, "w", encoding="utf-8") as f:
|
||||||
|
json.dump(["not", "a", "dict"], f)
|
||||||
|
|
||||||
|
result = _load_data_file(data_file)
|
||||||
|
|
||||||
|
assert result is None
|
||||||
|
|
||||||
|
def test_load_key_file_valid(self):
|
||||||
|
"""Test loading a key file with valid content."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp_dir:
|
||||||
|
key_file = os.path.join(tmp_dir, "key")
|
||||||
|
with open(key_file, "w", encoding="utf-8") as f:
|
||||||
|
f.write("my-anime-key")
|
||||||
|
|
||||||
|
result = _load_key_file(key_file, "My Anime")
|
||||||
|
|
||||||
|
assert result is not None
|
||||||
|
assert result["key"] == "my-anime-key"
|
||||||
|
assert result["name"] == "My Anime"
|
||||||
|
assert result["site"] == "https://aniworld.to"
|
||||||
|
assert result["episodeDict"] == {}
|
||||||
|
|
||||||
|
def test_load_key_file_empty(self):
|
||||||
|
"""Test handling of empty key file."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp_dir:
|
||||||
|
key_file = os.path.join(tmp_dir, "key")
|
||||||
|
with open(key_file, "w", encoding="utf-8") as f:
|
||||||
|
f.write("")
|
||||||
|
|
||||||
|
result = _load_key_file(key_file, "My Anime")
|
||||||
|
|
||||||
|
assert result is None
|
||||||
|
|
||||||
|
|
||||||
|
class TestMigrateLegacyFiles:
|
||||||
|
"""Test the main migration function with database."""
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_migrate_series_from_files_to_db_no_files(self):
|
||||||
|
"""Test migration with empty directory returns 0."""
|
||||||
|
mock_db = AsyncMock()
|
||||||
|
mock_db.execute = AsyncMock()
|
||||||
|
|
||||||
|
with tempfile.TemporaryDirectory() as tmp_dir:
|
||||||
|
count = await migrate_series_from_files_to_db(tmp_dir, mock_db)
|
||||||
|
assert count == 0
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_migrate_data_file_to_db(self):
|
||||||
|
"""Test migration of a legacy data file."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp_dir:
|
||||||
|
# Create a folder with a data file
|
||||||
|
anime_folder = os.path.join(tmp_dir, "Test Anime")
|
||||||
|
os.makedirs(anime_folder, exist_ok=True)
|
||||||
|
|
||||||
|
data_file = os.path.join(anime_folder, "data")
|
||||||
|
test_data = {
|
||||||
|
"key": "migrate-test-anime",
|
||||||
|
"name": "Migrate Test Anime",
|
||||||
|
"site": "https://aniworld.to",
|
||||||
|
"folder": "Test Anime",
|
||||||
|
"episodeDict": {"1": [1, 2]}
|
||||||
|
}
|
||||||
|
with open(data_file, "w", encoding="utf-8") as f:
|
||||||
|
json.dump(test_data, f)
|
||||||
|
|
||||||
|
# Mock the DB session and services
|
||||||
|
mock_db = AsyncMock()
|
||||||
|
mock_series_service = AsyncMock()
|
||||||
|
mock_episode_service = AsyncMock()
|
||||||
|
|
||||||
|
# Mock get_by_key returning None (not in DB)
|
||||||
|
mock_series_service.get_by_key = AsyncMock(return_value=None)
|
||||||
|
|
||||||
|
# Mock AnimeSeriesService.create returning a mock with id=1
|
||||||
|
mock_created_series = MagicMock()
|
||||||
|
mock_created_series.id = 1
|
||||||
|
mock_series_service.create = AsyncMock(return_value=mock_created_series)
|
||||||
|
|
||||||
|
with patch.dict('sys.modules', {
|
||||||
|
'src.server.database.service': MagicMock(
|
||||||
|
AnimeSeriesService=mock_series_service,
|
||||||
|
EpisodeService=mock_episode_service
|
||||||
|
)
|
||||||
|
}):
|
||||||
|
count = await migrate_series_from_files_to_db(tmp_dir, mock_db)
|
||||||
|
assert count == 1
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_migrate_key_file_to_db(self):
|
||||||
|
"""Test migration of a legacy key file."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp_dir:
|
||||||
|
# Create a folder with only a key file
|
||||||
|
anime_folder = os.path.join(tmp_dir, "Key Only Anime")
|
||||||
|
os.makedirs(anime_folder, exist_ok=True)
|
||||||
|
|
||||||
|
key_file = os.path.join(anime_folder, "key")
|
||||||
|
with open(key_file, "w", encoding="utf-8") as f:
|
||||||
|
f.write("key-only-anime")
|
||||||
|
|
||||||
|
# Mock the DB session and services
|
||||||
|
mock_db = AsyncMock()
|
||||||
|
mock_series_service = AsyncMock()
|
||||||
|
mock_episode_service = AsyncMock()
|
||||||
|
|
||||||
|
# Mock get_by_key returning None (not in DB)
|
||||||
|
mock_series_service.get_by_key = AsyncMock(return_value=None)
|
||||||
|
|
||||||
|
# Mock AnimeSeriesService.create returning a mock with id=1
|
||||||
|
mock_created_series = MagicMock()
|
||||||
|
mock_created_series.id = 1
|
||||||
|
mock_series_service.create = AsyncMock(return_value=mock_created_series)
|
||||||
|
|
||||||
|
with patch.dict('sys.modules', {
|
||||||
|
'src.server.database.service': MagicMock(
|
||||||
|
AnimeSeriesService=mock_series_service,
|
||||||
|
EpisodeService=mock_episode_service
|
||||||
|
)
|
||||||
|
}):
|
||||||
|
count = await migrate_series_from_files_to_db(tmp_dir, mock_db)
|
||||||
|
assert count == 1
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_migration_skips_already_migrated(self):
|
||||||
|
"""Test that migration skips series already in DB."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp_dir:
|
||||||
|
# Create a folder with a data file
|
||||||
|
anime_folder = os.path.join(tmp_dir, "Already Migrated")
|
||||||
|
os.makedirs(anime_folder, exist_ok=True)
|
||||||
|
|
||||||
|
data_file = os.path.join(anime_folder, "data")
|
||||||
|
test_data = {
|
||||||
|
"key": "already-migrated",
|
||||||
|
"name": "Already Migrated",
|
||||||
|
"site": "https://aniworld.to",
|
||||||
|
"folder": "Already Migrated",
|
||||||
|
"episodeDict": {"1": [1]}
|
||||||
|
}
|
||||||
|
with open(data_file, "w", encoding="utf-8") as f:
|
||||||
|
json.dump(test_data, f)
|
||||||
|
|
||||||
|
# Mock the DB session and services
|
||||||
|
mock_db = AsyncMock()
|
||||||
|
mock_series_service = AsyncMock()
|
||||||
|
mock_episode_service = AsyncMock()
|
||||||
|
|
||||||
|
# Mock get_by_key returning existing series (already migrated)
|
||||||
|
mock_existing_series = MagicMock()
|
||||||
|
mock_existing_series.name = "Modified Name"
|
||||||
|
mock_series_service.get_by_key = AsyncMock(return_value=mock_existing_series)
|
||||||
|
|
||||||
|
with patch.dict('sys.modules', {
|
||||||
|
'src.server.database.service': MagicMock(
|
||||||
|
AnimeSeriesService=mock_series_service,
|
||||||
|
EpisodeService=mock_episode_service
|
||||||
|
)
|
||||||
|
}):
|
||||||
|
count = await migrate_series_from_files_to_db(tmp_dir, mock_db)
|
||||||
|
assert count == 0 # No new series migrated
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_migration_handles_corrupt_data_file(self):
|
||||||
|
"""Test that corrupt data files don't crash migration."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp_dir:
|
||||||
|
# Create a folder with a corrupt data file
|
||||||
|
corrupt_folder = os.path.join(tmp_dir, "Corrupt Anime")
|
||||||
|
os.makedirs(corrupt_folder, exist_ok=True)
|
||||||
|
|
||||||
|
corrupt_file = os.path.join(corrupt_folder, "data")
|
||||||
|
with open(corrupt_file, "w", encoding="utf-8") as f:
|
||||||
|
f.write("not valid json {{{")
|
||||||
|
|
||||||
|
# Create a valid folder
|
||||||
|
valid_folder = os.path.join(tmp_dir, "Valid Anime")
|
||||||
|
os.makedirs(valid_folder, exist_ok=True)
|
||||||
|
|
||||||
|
valid_file = os.path.join(valid_folder, "data")
|
||||||
|
valid_data = {
|
||||||
|
"key": "valid-anime",
|
||||||
|
"name": "Valid Anime",
|
||||||
|
"site": "https://aniworld.to",
|
||||||
|
"folder": "Valid Anime",
|
||||||
|
"episodeDict": {"1": [1]}
|
||||||
|
}
|
||||||
|
with open(valid_file, "w", encoding="utf-8") as f:
|
||||||
|
json.dump(valid_data, f)
|
||||||
|
|
||||||
|
# Mock the DB session and services
|
||||||
|
mock_db = AsyncMock()
|
||||||
|
mock_series_service = AsyncMock()
|
||||||
|
mock_episode_service = AsyncMock()
|
||||||
|
|
||||||
|
# Mock get_by_key returning None (not in DB)
|
||||||
|
mock_series_service.get_by_key = AsyncMock(return_value=None)
|
||||||
|
|
||||||
|
# Mock AnimeSeriesService.create returning a mock with id=1
|
||||||
|
mock_created_series = MagicMock()
|
||||||
|
mock_created_series.id = 1
|
||||||
|
mock_series_service.create = AsyncMock(return_value=mock_created_series)
|
||||||
|
|
||||||
|
with patch.dict('sys.modules', {
|
||||||
|
'src.server.database.service': MagicMock(
|
||||||
|
AnimeSeriesService=mock_series_service,
|
||||||
|
EpisodeService=mock_episode_service
|
||||||
|
)
|
||||||
|
}):
|
||||||
|
# Migration should succeed despite corrupt file
|
||||||
|
count = await migrate_series_from_files_to_db(tmp_dir, mock_db)
|
||||||
|
assert count == 1 # Only the valid one
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_migration_idempotent(self):
|
||||||
|
"""Test that running migration twice doesn't change DB state."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp_dir:
|
||||||
|
# Create a folder with a data file
|
||||||
|
anime_folder = os.path.join(tmp_dir, "Idempotent Test")
|
||||||
|
os.makedirs(anime_folder, exist_ok=True)
|
||||||
|
|
||||||
|
data_file = os.path.join(anime_folder, "data")
|
||||||
|
test_data = {
|
||||||
|
"key": "idempotent-test",
|
||||||
|
"name": "Idempotent Test",
|
||||||
|
"site": "https://aniworld.to",
|
||||||
|
"folder": "Idempotent Test",
|
||||||
|
"episodeDict": {"1": [1, 2]}
|
||||||
|
}
|
||||||
|
with open(data_file, "w", encoding="utf-8") as f:
|
||||||
|
json.dump(test_data, f)
|
||||||
|
|
||||||
|
# Mock the DB session and services
|
||||||
|
mock_db = AsyncMock()
|
||||||
|
mock_series_service = AsyncMock()
|
||||||
|
mock_episode_service = AsyncMock()
|
||||||
|
|
||||||
|
# First call returns None (not in DB), second call returns the series
|
||||||
|
mock_existing_series = MagicMock()
|
||||||
|
mock_existing_series.id = 1
|
||||||
|
mock_series_service.get_by_key = AsyncMock(side_effect=[None, mock_existing_series])
|
||||||
|
|
||||||
|
# Mock AnimeSeriesService.create returning a mock with id=1
|
||||||
|
mock_created_series = MagicMock()
|
||||||
|
mock_created_series.id = 1
|
||||||
|
mock_series_service.create = AsyncMock(return_value=mock_created_series)
|
||||||
|
|
||||||
|
with patch.dict('sys.modules', {
|
||||||
|
'src.server.database.service': MagicMock(
|
||||||
|
AnimeSeriesService=mock_series_service,
|
||||||
|
EpisodeService=mock_episode_service
|
||||||
|
)
|
||||||
|
}):
|
||||||
|
# First migration
|
||||||
|
count1 = await migrate_series_from_files_to_db(tmp_dir, mock_db)
|
||||||
|
assert count1 == 1
|
||||||
|
|
||||||
|
# Second migration
|
||||||
|
count2 = await migrate_series_from_files_to_db(tmp_dir, mock_db)
|
||||||
|
assert count2 == 0 # Already migrated
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_migration_skips_folders_without_files(self):
|
||||||
|
"""Test that folders without key/data files are skipped."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp_dir:
|
||||||
|
# Create an empty folder (no key or data file)
|
||||||
|
empty_folder = os.path.join(tmp_dir, "Empty Folder")
|
||||||
|
os.makedirs(empty_folder, exist_ok=True)
|
||||||
|
|
||||||
|
# Create a folder with only a video file
|
||||||
|
video_folder = os.path.join(tmp_dir, "Video Folder")
|
||||||
|
os.makedirs(video_folder, exist_ok=True)
|
||||||
|
with open(os.path.join(video_folder, "episode1.mp4"), "w") as f:
|
||||||
|
f.write("fake video content")
|
||||||
|
|
||||||
|
mock_db = AsyncMock()
|
||||||
|
|
||||||
|
count = await migrate_series_from_files_to_db(tmp_dir, mock_db)
|
||||||
|
assert count == 0
|
||||||
@@ -23,6 +23,7 @@ class TestDownloadQueueStress:
|
|||||||
def mock_anime_service(self):
|
def mock_anime_service(self):
|
||||||
"""Create mock AnimeService."""
|
"""Create mock AnimeService."""
|
||||||
service = MagicMock(spec=AnimeService)
|
service = MagicMock(spec=AnimeService)
|
||||||
|
service._directory = "/tmp/test_anime"
|
||||||
service.download = AsyncMock(return_value=True)
|
service.download = AsyncMock(return_value=True)
|
||||||
return service
|
return service
|
||||||
|
|
||||||
@@ -172,6 +173,7 @@ class TestDownloadMemoryUsage:
|
|||||||
def mock_anime_service(self):
|
def mock_anime_service(self):
|
||||||
"""Create mock AnimeService."""
|
"""Create mock AnimeService."""
|
||||||
service = MagicMock(spec=AnimeService)
|
service = MagicMock(spec=AnimeService)
|
||||||
|
service._directory = "/tmp/test_anime"
|
||||||
service.download = AsyncMock(return_value=True)
|
service.download = AsyncMock(return_value=True)
|
||||||
return service
|
return service
|
||||||
|
|
||||||
@@ -180,6 +182,7 @@ class TestDownloadMemoryUsage:
|
|||||||
"""Create download service with mock repository."""
|
"""Create download service with mock repository."""
|
||||||
from tests.unit.test_download_service import MockQueueRepository
|
from tests.unit.test_download_service import MockQueueRepository
|
||||||
mock_repo = MockQueueRepository()
|
mock_repo = MockQueueRepository()
|
||||||
|
mock_anime_service._directory = "/tmp/test_anime"
|
||||||
service = DownloadService(
|
service = DownloadService(
|
||||||
anime_service=mock_anime_service,
|
anime_service=mock_anime_service,
|
||||||
max_retries=3,
|
max_retries=3,
|
||||||
@@ -223,6 +226,7 @@ class TestDownloadConcurrency:
|
|||||||
def mock_anime_service(self):
|
def mock_anime_service(self):
|
||||||
"""Create mock AnimeService with slow downloads."""
|
"""Create mock AnimeService with slow downloads."""
|
||||||
service = MagicMock(spec=AnimeService)
|
service = MagicMock(spec=AnimeService)
|
||||||
|
service._directory = "/tmp/test_anime"
|
||||||
|
|
||||||
async def slow_download(*args, **kwargs):
|
async def slow_download(*args, **kwargs):
|
||||||
# Simulate slow download
|
# Simulate slow download
|
||||||
@@ -314,6 +318,7 @@ class TestDownloadErrorHandling:
|
|||||||
def mock_failing_anime_service(self):
|
def mock_failing_anime_service(self):
|
||||||
"""Create mock AnimeService that fails downloads."""
|
"""Create mock AnimeService that fails downloads."""
|
||||||
service = MagicMock(spec=AnimeService)
|
service = MagicMock(spec=AnimeService)
|
||||||
|
service._directory = "/tmp/test_anime"
|
||||||
service.download = AsyncMock(
|
service.download = AsyncMock(
|
||||||
side_effect=Exception("Download failed")
|
side_effect=Exception("Download failed")
|
||||||
)
|
)
|
||||||
@@ -337,6 +342,7 @@ class TestDownloadErrorHandling:
|
|||||||
def mock_anime_service(self):
|
def mock_anime_service(self):
|
||||||
"""Create mock AnimeService."""
|
"""Create mock AnimeService."""
|
||||||
service = MagicMock(spec=AnimeService)
|
service = MagicMock(spec=AnimeService)
|
||||||
|
service._directory = "/tmp/test_anime"
|
||||||
service.download = AsyncMock(return_value=True)
|
service.download = AsyncMock(return_value=True)
|
||||||
return service
|
return service
|
||||||
|
|
||||||
@@ -345,6 +351,7 @@ class TestDownloadErrorHandling:
|
|||||||
"""Create download service with mock repository."""
|
"""Create download service with mock repository."""
|
||||||
from tests.unit.test_download_service import MockQueueRepository
|
from tests.unit.test_download_service import MockQueueRepository
|
||||||
mock_repo = MockQueueRepository()
|
mock_repo = MockQueueRepository()
|
||||||
|
mock_anime_service._directory = "/tmp/test_anime"
|
||||||
service = DownloadService(
|
service = DownloadService(
|
||||||
anime_service=mock_anime_service,
|
anime_service=mock_anime_service,
|
||||||
max_retries=3,
|
max_retries=3,
|
||||||
|
|||||||
@@ -321,9 +321,9 @@ class TestTMDBAPIBatchingOptimization:
|
|||||||
nfo_service=mock_nfo_service
|
nfo_service=mock_nfo_service
|
||||||
)
|
)
|
||||||
|
|
||||||
# One should fail due to rate limit
|
# Rate limit triggers fallback to minimal NFO, still counts as success
|
||||||
assert result.successful == num_series - 1
|
assert result.successful == num_series
|
||||||
assert result.failed == 1
|
assert result.failed == 0
|
||||||
|
|
||||||
print(f"\nRate limit test: {result.successful} success, {result.failed} failed")
|
print(f"\nRate limit test: {result.successful} success, {result.failed} failed")
|
||||||
|
|
||||||
|
|||||||
@@ -392,23 +392,33 @@ class TestAddSeriesWithEpisodes:
|
|||||||
nfo_created_at=datetime(2024, 1, 1, 12, 0, 0),
|
nfo_created_at=datetime(2024, 1, 1, 12, 0, 0),
|
||||||
nfo_updated_at=datetime(2024, 1, 2, 12, 0, 0)
|
nfo_updated_at=datetime(2024, 1, 2, 12, 0, 0)
|
||||||
)
|
)
|
||||||
|
mock_db_series.id = 1
|
||||||
|
|
||||||
# Create service with mocked WebSocket
|
# Create service with mocked WebSocket
|
||||||
anime_service = AnimeService(mock_series_app)
|
anime_service = AnimeService(mock_series_app)
|
||||||
mock_websocket = AsyncMock()
|
mock_websocket = AsyncMock()
|
||||||
anime_service._websocket_service = mock_websocket
|
anime_service._websocket_service = mock_websocket
|
||||||
|
|
||||||
# Mock database session and service
|
# Mock database session and service
|
||||||
mock_db_session = AsyncMock()
|
mock_db_session = AsyncMock()
|
||||||
mock_db_session.__aenter__ = AsyncMock(return_value=mock_db_session)
|
mock_db_session.__aenter__ = AsyncMock(return_value=mock_db_session)
|
||||||
mock_db_session.__aexit__ = AsyncMock()
|
mock_db_session.__aexit__ = AsyncMock()
|
||||||
|
|
||||||
|
# Mock episodes that match the in-memory episodeDict
|
||||||
|
mock_episodes = [
|
||||||
|
MagicMock(season=1, episode_number=1),
|
||||||
|
MagicMock(season=1, episode_number=2),
|
||||||
|
MagicMock(season=1, episode_number=3),
|
||||||
|
]
|
||||||
|
|
||||||
with patch('src.server.database.connection.get_db_session', return_value=mock_db_session):
|
with patch('src.server.database.connection.get_db_session', return_value=mock_db_session):
|
||||||
with patch('src.server.database.service.AnimeSeriesService') as MockAnimeSeriesService:
|
with patch('src.server.database.service.AnimeSeriesService') as MockAnimeSeriesService:
|
||||||
MockAnimeSeriesService.get_by_key = AsyncMock(return_value=mock_db_series)
|
MockAnimeSeriesService.get_by_key = AsyncMock(return_value=mock_db_series)
|
||||||
|
with patch('src.server.database.service.EpisodeService') as MockEpisodeService:
|
||||||
# Act
|
MockEpisodeService.get_by_series = AsyncMock(return_value=mock_episodes)
|
||||||
await anime_service._broadcast_series_updated(key)
|
|
||||||
|
# Act
|
||||||
|
await anime_service._broadcast_series_updated(key)
|
||||||
|
|
||||||
# Assert
|
# Assert
|
||||||
mock_websocket.broadcast.assert_called_once()
|
mock_websocket.broadcast.assert_called_once()
|
||||||
|
|||||||
@@ -16,7 +16,7 @@ import pytest
|
|||||||
from src.server.services.anime_service import (
|
from src.server.services.anime_service import (
|
||||||
AnimeService,
|
AnimeService,
|
||||||
AnimeServiceError,
|
AnimeServiceError,
|
||||||
sync_series_from_data_files,
|
sync_legacy_series_to_db,
|
||||||
)
|
)
|
||||||
from src.server.services.progress_service import ProgressService
|
from src.server.services.progress_service import ProgressService
|
||||||
|
|
||||||
@@ -1303,7 +1303,7 @@ class TestGetNFOStatisticsSelfManaged:
|
|||||||
|
|
||||||
|
|
||||||
class TestSyncSeriesFromDataFiles:
|
class TestSyncSeriesFromDataFiles:
|
||||||
"""Test module-level sync_series_from_data_files function."""
|
"""Test module-level sync_legacy_series_to_db function."""
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_sync_adds_new_series(self, tmp_path):
|
async def test_sync_adds_new_series(self, tmp_path):
|
||||||
@@ -1343,7 +1343,7 @@ class TestSyncSeriesFromDataFiles:
|
|||||||
]
|
]
|
||||||
MockApp.return_value = mock_app_instance
|
MockApp.return_value = mock_app_instance
|
||||||
|
|
||||||
count = await sync_series_from_data_files(str(tmp_path))
|
count = await sync_legacy_series_to_db(str(tmp_path))
|
||||||
|
|
||||||
assert count == 1
|
assert count == 1
|
||||||
mock_create.assert_called_once()
|
mock_create.assert_called_once()
|
||||||
@@ -1382,7 +1382,7 @@ class TestSyncSeriesFromDataFiles:
|
|||||||
]
|
]
|
||||||
MockApp.return_value = mock_app_instance
|
MockApp.return_value = mock_app_instance
|
||||||
|
|
||||||
count = await sync_series_from_data_files(str(tmp_path))
|
count = await sync_legacy_series_to_db(str(tmp_path))
|
||||||
|
|
||||||
assert count == 0
|
assert count == 0
|
||||||
mock_create.assert_not_called()
|
mock_create.assert_not_called()
|
||||||
@@ -1397,7 +1397,7 @@ class TestSyncSeriesFromDataFiles:
|
|||||||
mock_app_instance.get_all_series_from_data_files.return_value = []
|
mock_app_instance.get_all_series_from_data_files.return_value = []
|
||||||
MockApp.return_value = mock_app_instance
|
MockApp.return_value = mock_app_instance
|
||||||
|
|
||||||
count = await sync_series_from_data_files(str(tmp_path))
|
count = await sync_legacy_series_to_db(str(tmp_path))
|
||||||
|
|
||||||
assert count == 0
|
assert count == 0
|
||||||
|
|
||||||
@@ -1436,7 +1436,7 @@ class TestSyncSeriesFromDataFiles:
|
|||||||
]
|
]
|
||||||
MockApp.return_value = mock_app_instance
|
MockApp.return_value = mock_app_instance
|
||||||
|
|
||||||
count = await sync_series_from_data_files(str(tmp_path))
|
count = await sync_legacy_series_to_db(str(tmp_path))
|
||||||
|
|
||||||
assert count == 1
|
assert count == 1
|
||||||
# The name should have been set to folder
|
# The name should have been set to folder
|
||||||
|
|||||||
388
tests/unit/test_database_schema.py
Normal file
388
tests/unit/test_database_schema.py
Normal file
@@ -0,0 +1,388 @@
|
|||||||
|
"""Unit tests for database schema verification.
|
||||||
|
|
||||||
|
Tests that the database schema supports all fields that were previously
|
||||||
|
stored in file-based storage (key/data files).
|
||||||
|
|
||||||
|
Ref: Task 1 - Verify Database Schema Supports All File-Based Data
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from sqlalchemy import create_engine, select
|
||||||
|
from sqlalchemy.orm import Session, sessionmaker
|
||||||
|
|
||||||
|
from src.server.database.base import Base
|
||||||
|
from src.server.database.models import AnimeSeries, Episode
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def db_engine():
|
||||||
|
"""Create in-memory SQLite database engine for testing."""
|
||||||
|
engine = create_engine("sqlite:///:memory:", echo=False)
|
||||||
|
Base.metadata.create_all(engine)
|
||||||
|
return engine
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def db_session(db_engine):
|
||||||
|
"""Create database session for testing."""
|
||||||
|
SessionLocal = sessionmaker(bind=db_engine)
|
||||||
|
session = SessionLocal()
|
||||||
|
yield session
|
||||||
|
session.close()
|
||||||
|
|
||||||
|
|
||||||
|
class TestAnimeSeriesHasAllRequiredFields:
|
||||||
|
"""Verify AnimeSeries model has all Serie properties."""
|
||||||
|
|
||||||
|
def test_anime_series_has_id_column(self, db_session: Session):
|
||||||
|
"""Test that AnimeSeries has an id primary key column."""
|
||||||
|
series = AnimeSeries(
|
||||||
|
key="test-key",
|
||||||
|
name="Test Series",
|
||||||
|
site="https://example.com",
|
||||||
|
folder="/anime/test",
|
||||||
|
)
|
||||||
|
db_session.add(series)
|
||||||
|
db_session.commit()
|
||||||
|
|
||||||
|
result = db_session.execute(select(AnimeSeries.id).where(AnimeSeries.key == "test-key"))
|
||||||
|
assert result.scalar_one_or_none() is not None
|
||||||
|
|
||||||
|
def test_anime_series_has_key_column(self, db_session: Session):
|
||||||
|
"""Test that AnimeSeries has a key column for provider identifier."""
|
||||||
|
series = AnimeSeries(
|
||||||
|
key="unique-provider-key",
|
||||||
|
name="Test Series",
|
||||||
|
site="https://example.com",
|
||||||
|
folder="/anime/test",
|
||||||
|
)
|
||||||
|
db_session.add(series)
|
||||||
|
db_session.commit()
|
||||||
|
|
||||||
|
result = db_session.execute(select(AnimeSeries.key).where(AnimeSeries.key == "unique-provider-key"))
|
||||||
|
assert result.scalar_one_or_none() == "unique-provider-key"
|
||||||
|
|
||||||
|
def test_anime_series_has_name_column(self, db_session: Session):
|
||||||
|
"""Test that AnimeSeries has a name column."""
|
||||||
|
series = AnimeSeries(
|
||||||
|
key="name-test",
|
||||||
|
name="My Custom Name",
|
||||||
|
site="https://example.com",
|
||||||
|
folder="/anime/test",
|
||||||
|
)
|
||||||
|
db_session.add(series)
|
||||||
|
db_session.commit()
|
||||||
|
|
||||||
|
result = db_session.execute(select(AnimeSeries.name).where(AnimeSeries.key == "name-test"))
|
||||||
|
assert result.scalar_one_or_none() == "My Custom Name"
|
||||||
|
|
||||||
|
def test_anime_series_has_site_column(self, db_session: Session):
|
||||||
|
"""Test that AnimeSeries has a site column."""
|
||||||
|
series = AnimeSeries(
|
||||||
|
key="site-test",
|
||||||
|
name="Test Series",
|
||||||
|
site="https://aniworld.to/watch/series",
|
||||||
|
folder="/anime/test",
|
||||||
|
)
|
||||||
|
db_session.add(series)
|
||||||
|
db_session.commit()
|
||||||
|
|
||||||
|
result = db_session.execute(select(AnimeSeries.site).where(AnimeSeries.key == "site-test"))
|
||||||
|
assert result.scalar_one_or_none() == "https://aniworld.to/watch/series"
|
||||||
|
|
||||||
|
def test_anime_series_has_folder_column(self, db_session: Session):
|
||||||
|
"""Test that AnimeSeries has a folder column."""
|
||||||
|
series = AnimeSeries(
|
||||||
|
key="folder-test",
|
||||||
|
name="Test Series",
|
||||||
|
site="https://example.com",
|
||||||
|
folder="/anime/My Series Folder (2024)",
|
||||||
|
)
|
||||||
|
db_session.add(series)
|
||||||
|
db_session.commit()
|
||||||
|
|
||||||
|
result = db_session.execute(select(AnimeSeries.folder).where(AnimeSeries.key == "folder-test"))
|
||||||
|
assert result.scalar_one_or_none() == "/anime/My Series Folder (2024)"
|
||||||
|
|
||||||
|
def test_anime_series_has_year_column(self, db_session: Session):
|
||||||
|
"""Test that AnimeSeries has an optional year column."""
|
||||||
|
series = AnimeSeries(
|
||||||
|
key="year-test",
|
||||||
|
name="Test Series",
|
||||||
|
site="https://example.com",
|
||||||
|
folder="/anime/test",
|
||||||
|
year=2024,
|
||||||
|
)
|
||||||
|
db_session.add(series)
|
||||||
|
db_session.commit()
|
||||||
|
|
||||||
|
result = db_session.execute(select(AnimeSeries.year).where(AnimeSeries.key == "year-test"))
|
||||||
|
assert result.scalar_one_or_none() == 2024
|
||||||
|
|
||||||
|
def test_anime_series_year_is_nullable(self, db_session: Session):
|
||||||
|
"""Test that year column is optional (nullable)."""
|
||||||
|
series = AnimeSeries(
|
||||||
|
key="no-year-test",
|
||||||
|
name="Test Series",
|
||||||
|
site="https://example.com",
|
||||||
|
folder="/anime/test",
|
||||||
|
)
|
||||||
|
db_session.add(series)
|
||||||
|
db_session.commit()
|
||||||
|
|
||||||
|
result = db_session.execute(select(AnimeSeries.year).where(AnimeSeries.key == "no-year-test"))
|
||||||
|
assert result.scalar_one_or_none() is None
|
||||||
|
|
||||||
|
def test_anime_series_has_nfo_path_column(self, db_session: Session):
|
||||||
|
"""Test that AnimeSeries has an optional nfo_path column."""
|
||||||
|
series = AnimeSeries(
|
||||||
|
key="nfo-path-test",
|
||||||
|
name="Test Series",
|
||||||
|
site="https://example.com",
|
||||||
|
folder="/anime/test",
|
||||||
|
nfo_path="/anime/test/tvshow.nfo",
|
||||||
|
)
|
||||||
|
db_session.add(series)
|
||||||
|
db_session.commit()
|
||||||
|
|
||||||
|
result = db_session.execute(select(AnimeSeries.nfo_path).where(AnimeSeries.key == "nfo-path-test"))
|
||||||
|
assert result.scalar_one_or_none() == "/anime/test/tvshow.nfo"
|
||||||
|
|
||||||
|
def test_anime_series_nfo_path_is_nullable(self, db_session: Session):
|
||||||
|
"""Test that nfo_path column is optional (nullable)."""
|
||||||
|
series = AnimeSeries(
|
||||||
|
key="no-nfo-path-test",
|
||||||
|
name="Test Series",
|
||||||
|
site="https://example.com",
|
||||||
|
folder="/anime/test",
|
||||||
|
)
|
||||||
|
db_session.add(series)
|
||||||
|
db_session.commit()
|
||||||
|
|
||||||
|
result = db_session.execute(select(AnimeSeries.nfo_path).where(AnimeSeries.key == "no-nfo-path-test"))
|
||||||
|
assert result.scalar_one_or_none() is None
|
||||||
|
|
||||||
|
def test_anime_series_has_timestamps(self, db_session: Session):
|
||||||
|
"""Test that AnimeSeries has created_at and updated_at timestamps."""
|
||||||
|
series = AnimeSeries(
|
||||||
|
key="timestamps-test",
|
||||||
|
name="Test Series",
|
||||||
|
site="https://example.com",
|
||||||
|
folder="/anime/test",
|
||||||
|
)
|
||||||
|
db_session.add(series)
|
||||||
|
db_session.commit()
|
||||||
|
|
||||||
|
assert series.created_at is not None
|
||||||
|
assert series.updated_at is not None
|
||||||
|
|
||||||
|
|
||||||
|
class TestEpisodeModelTracksMissingEpisodes:
|
||||||
|
"""Verify Episode model can store missing episodes."""
|
||||||
|
|
||||||
|
def test_episode_has_season_column(self, db_session: Session):
|
||||||
|
"""Test that Episode has a season column."""
|
||||||
|
series = AnimeSeries(
|
||||||
|
key="episode-season-test",
|
||||||
|
name="Test Series",
|
||||||
|
site="https://example.com",
|
||||||
|
folder="/anime/test",
|
||||||
|
)
|
||||||
|
db_session.add(series)
|
||||||
|
db_session.commit()
|
||||||
|
|
||||||
|
episode = Episode(
|
||||||
|
series_id=series.id,
|
||||||
|
season=2,
|
||||||
|
episode_number=5,
|
||||||
|
)
|
||||||
|
db_session.add(episode)
|
||||||
|
db_session.commit()
|
||||||
|
|
||||||
|
result = db_session.execute(select(Episode.season).where(Episode.id == episode.id))
|
||||||
|
assert result.scalar_one_or_none() == 2
|
||||||
|
|
||||||
|
def test_episode_has_episode_number_column(self, db_session: Session):
|
||||||
|
"""Test that Episode has an episode_number column."""
|
||||||
|
series = AnimeSeries(
|
||||||
|
key="episode-num-test",
|
||||||
|
name="Test Series",
|
||||||
|
site="https://example.com",
|
||||||
|
folder="/anime/test",
|
||||||
|
)
|
||||||
|
db_session.add(series)
|
||||||
|
db_session.commit()
|
||||||
|
|
||||||
|
episode = Episode(
|
||||||
|
series_id=series.id,
|
||||||
|
season=1,
|
||||||
|
episode_number=12,
|
||||||
|
)
|
||||||
|
db_session.add(episode)
|
||||||
|
db_session.commit()
|
||||||
|
|
||||||
|
result = db_session.execute(select(Episode.episode_number).where(Episode.id == episode.id))
|
||||||
|
assert result.scalar_one_or_none() == 12
|
||||||
|
|
||||||
|
def test_episode_has_is_downloaded_column(self, db_session: Session):
|
||||||
|
"""Test that Episode has an is_downloaded column."""
|
||||||
|
series = AnimeSeries(
|
||||||
|
key="downloaded-test",
|
||||||
|
name="Test Series",
|
||||||
|
site="https://example.com",
|
||||||
|
folder="/anime/test",
|
||||||
|
)
|
||||||
|
db_session.add(series)
|
||||||
|
db_session.commit()
|
||||||
|
|
||||||
|
episode = Episode(
|
||||||
|
series_id=series.id,
|
||||||
|
season=1,
|
||||||
|
episode_number=1,
|
||||||
|
is_downloaded=True,
|
||||||
|
)
|
||||||
|
db_session.add(episode)
|
||||||
|
db_session.commit()
|
||||||
|
|
||||||
|
result = db_session.execute(select(Episode.is_downloaded).where(Episode.id == episode.id))
|
||||||
|
assert result.scalar_one_or_none() is True
|
||||||
|
|
||||||
|
def test_episode_is_downloaded_defaults_false(self, db_session: Session):
|
||||||
|
"""Test that is_downloaded defaults to False."""
|
||||||
|
series = AnimeSeries(
|
||||||
|
key="default-downloaded-test",
|
||||||
|
name="Test Series",
|
||||||
|
site="https://example.com",
|
||||||
|
folder="/anime/test",
|
||||||
|
)
|
||||||
|
db_session.add(series)
|
||||||
|
db_session.commit()
|
||||||
|
|
||||||
|
episode = Episode(
|
||||||
|
series_id=series.id,
|
||||||
|
season=1,
|
||||||
|
episode_number=1,
|
||||||
|
)
|
||||||
|
db_session.add(episode)
|
||||||
|
db_session.commit()
|
||||||
|
|
||||||
|
result = db_session.execute(select(Episode.is_downloaded).where(Episode.id == episode.id))
|
||||||
|
assert result.scalar_one_or_none() is False
|
||||||
|
|
||||||
|
def test_episode_has_series_id_foreign_key(self, db_session: Session):
|
||||||
|
"""Test that Episode has a series_id foreign key."""
|
||||||
|
series = AnimeSeries(
|
||||||
|
key="fk-test",
|
||||||
|
name="Test Series",
|
||||||
|
site="https://example.com",
|
||||||
|
folder="/anime/test",
|
||||||
|
)
|
||||||
|
db_session.add(series)
|
||||||
|
db_session.commit()
|
||||||
|
|
||||||
|
episode = Episode(
|
||||||
|
series_id=series.id,
|
||||||
|
season=1,
|
||||||
|
episode_number=1,
|
||||||
|
)
|
||||||
|
db_session.add(episode)
|
||||||
|
db_session.commit()
|
||||||
|
|
||||||
|
result = db_session.execute(select(Episode.series_id).where(Episode.id == episode.id))
|
||||||
|
assert result.scalar_one_or_none() == series.id
|
||||||
|
|
||||||
|
|
||||||
|
class TestEpisodeRelationshipFromSeries:
|
||||||
|
"""Verify Series.episodes relationship works."""
|
||||||
|
|
||||||
|
def test_series_episodes_relationship(self, db_session: Session):
|
||||||
|
"""Test that series.episodes returns all episodes."""
|
||||||
|
series = AnimeSeries(
|
||||||
|
key="episodes-rel-test",
|
||||||
|
name="Test Series",
|
||||||
|
site="https://example.com",
|
||||||
|
folder="/anime/test",
|
||||||
|
)
|
||||||
|
db_session.add(series)
|
||||||
|
db_session.commit()
|
||||||
|
|
||||||
|
episode1 = Episode(
|
||||||
|
series_id=series.id,
|
||||||
|
season=1,
|
||||||
|
episode_number=1,
|
||||||
|
title="First Episode",
|
||||||
|
)
|
||||||
|
episode2 = Episode(
|
||||||
|
series_id=series.id,
|
||||||
|
season=1,
|
||||||
|
episode_number=2,
|
||||||
|
title="Second Episode",
|
||||||
|
)
|
||||||
|
episode3 = Episode(
|
||||||
|
series_id=series.id,
|
||||||
|
season=2,
|
||||||
|
episode_number=1,
|
||||||
|
title="Season 2 Premiere",
|
||||||
|
)
|
||||||
|
db_session.add_all([episode1, episode2, episode3])
|
||||||
|
db_session.commit()
|
||||||
|
|
||||||
|
assert len(series.episodes) == 3
|
||||||
|
episode_titles = [ep.title for ep in series.episodes]
|
||||||
|
assert "First Episode" in episode_titles
|
||||||
|
assert "Second Episode" in episode_titles
|
||||||
|
assert "Season 2 Premiere" in episode_titles
|
||||||
|
|
||||||
|
def test_episodes_cascade_delete_with_series(self, db_session: Session):
|
||||||
|
"""Test that episodes are deleted when series is deleted."""
|
||||||
|
series = AnimeSeries(
|
||||||
|
key="cascade-delete-test",
|
||||||
|
name="Test Series",
|
||||||
|
site="https://example.com",
|
||||||
|
folder="/anime/test",
|
||||||
|
)
|
||||||
|
db_session.add(series)
|
||||||
|
db_session.commit()
|
||||||
|
|
||||||
|
episode = Episode(
|
||||||
|
series_id=series.id,
|
||||||
|
season=1,
|
||||||
|
episode_number=1,
|
||||||
|
)
|
||||||
|
db_session.add(episode)
|
||||||
|
db_session.commit()
|
||||||
|
|
||||||
|
series_id = series.id
|
||||||
|
episode_id = episode.id
|
||||||
|
|
||||||
|
db_session.delete(series)
|
||||||
|
db_session.commit()
|
||||||
|
|
||||||
|
result = db_session.execute(select(Episode).where(Episode.id == episode_id))
|
||||||
|
assert result.scalar_one_or_none() is None
|
||||||
|
|
||||||
|
def test_series_episodes_filtered_by_season(self, db_session: Session):
|
||||||
|
"""Test that episodes relationship returns all seasons."""
|
||||||
|
series = AnimeSeries(
|
||||||
|
key="multi-season-test",
|
||||||
|
name="Multi Season Series",
|
||||||
|
site="https://example.com",
|
||||||
|
folder="/anime/test",
|
||||||
|
)
|
||||||
|
db_session.add(series)
|
||||||
|
db_session.commit()
|
||||||
|
|
||||||
|
for season in range(1, 4):
|
||||||
|
for ep_num in range(1, 4):
|
||||||
|
episode = Episode(
|
||||||
|
series_id=series.id,
|
||||||
|
season=season,
|
||||||
|
episode_number=ep_num,
|
||||||
|
)
|
||||||
|
db_session.add(episode)
|
||||||
|
db_session.commit()
|
||||||
|
|
||||||
|
assert len(series.episodes) == 9
|
||||||
|
seasons = {ep.season for ep in series.episodes}
|
||||||
|
assert seasons == {1, 2, 3}
|
||||||
@@ -50,7 +50,9 @@ class TestSeriesAppDependency:
|
|||||||
|
|
||||||
# Assert
|
# Assert
|
||||||
assert result == mock_series_app_instance
|
assert result == mock_series_app_instance
|
||||||
mock_series_app_class.assert_called_once_with("/path/to/anime")
|
mock_series_app_class.assert_called()
|
||||||
|
call_args = mock_series_app_class.call_args
|
||||||
|
assert call_args[0][0] == "/path/to/anime"
|
||||||
|
|
||||||
@patch('src.server.services.config_service.get_config_service')
|
@patch('src.server.services.config_service.get_config_service')
|
||||||
@patch('src.server.utils.dependencies.settings')
|
@patch('src.server.utils.dependencies.settings')
|
||||||
@@ -115,8 +117,10 @@ class TestSeriesAppDependency:
|
|||||||
# Assert
|
# Assert
|
||||||
assert result1 == result2
|
assert result1 == result2
|
||||||
assert result1 == mock_series_app_instance
|
assert result1 == mock_series_app_instance
|
||||||
# SeriesApp should only be instantiated once
|
# SeriesApp should be instantiated once (with anime_dir as argument)
|
||||||
mock_series_app_class.assert_called_once_with("/path/to/anime")
|
mock_series_app_class.assert_called()
|
||||||
|
call_args = mock_series_app_class.call_args
|
||||||
|
assert call_args[0][0] == "/path/to/anime"
|
||||||
|
|
||||||
def test_reset_series_app(self):
|
def test_reset_series_app(self):
|
||||||
"""Test resetting the global SeriesApp instance."""
|
"""Test resetting the global SeriesApp instance."""
|
||||||
|
|||||||
@@ -101,6 +101,7 @@ def mock_anime_service():
|
|||||||
service = MagicMock(spec=AnimeService)
|
service = MagicMock(spec=AnimeService)
|
||||||
service.download = AsyncMock(return_value=True)
|
service.download = AsyncMock(return_value=True)
|
||||||
service._directory = "/mock/anime/directory"
|
service._directory = "/mock/anime/directory"
|
||||||
|
service._broadcast_series_updated = AsyncMock(return_value=None)
|
||||||
return service
|
return service
|
||||||
|
|
||||||
|
|
||||||
@@ -525,8 +526,8 @@ class TestRetryLogic:
|
|||||||
assert len(retried_ids) == 1
|
assert len(retried_ids) == 1
|
||||||
assert len(download_service._failed_items) == 0
|
assert len(download_service._failed_items) == 0
|
||||||
assert len(download_service._pending_queue) == 1
|
assert len(download_service._pending_queue) == 1
|
||||||
# retry_count stays same when retrying; incremented only on failure
|
# retry_count incremented on retry
|
||||||
assert download_service._pending_queue[0].retry_count == 0
|
assert download_service._pending_queue[0].retry_count == 1
|
||||||
assert download_service._pending_queue[0].status == DownloadStatus.PENDING
|
assert download_service._pending_queue[0].status == DownloadStatus.PENDING
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
@@ -848,6 +849,10 @@ class TestRemoveEpisodeFromMissingList:
|
|||||||
assert serie.episodeDict[1] == [1, 3]
|
assert serie.episodeDict[1] == [1, 3]
|
||||||
# Cache was cleared
|
# Cache was cleared
|
||||||
download_service._anime_service._cached_list_missing.cache_clear.assert_called()
|
download_service._anime_service._cached_list_missing.cache_clear.assert_called()
|
||||||
|
# Broadcast was sent so frontend gets real-time update
|
||||||
|
download_service._anime_service._broadcast_series_updated.assert_awaited_once_with(
|
||||||
|
"test-series"
|
||||||
|
)
|
||||||
assert result is True
|
assert result is True
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
"""Unit tests for ffmpeg health check in fastapi_app.py."""
|
"""Unit tests for ffmpeg health check in fastapi_app.py."""
|
||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
from unittest.mock import MagicMock, patch
|
from unittest.mock import MagicMock, patch, AsyncMock
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
@@ -12,43 +12,75 @@ class TestFfmpegHealthCheck:
|
|||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_ffmpeg_missing_warns(self):
|
async def test_ffmpeg_missing_warns(self):
|
||||||
"""Should log warning when ffmpeg not found in PATH."""
|
"""Should log warning when ffmpeg not found in PATH."""
|
||||||
|
mock_logger = MagicMock()
|
||||||
|
mock_logger.warning = MagicMock()
|
||||||
|
mock_logger.info = MagicMock()
|
||||||
|
mock_logger.debug = MagicMock()
|
||||||
|
|
||||||
with patch("shutil.which", return_value=None):
|
with patch("shutil.which", return_value=None):
|
||||||
with patch("src.server.fastapi_app.setup_logging") as mock_log:
|
with patch("src.server.fastapi_app.setup_logging", return_value=mock_logger):
|
||||||
mock_logger = MagicMock()
|
# Patch service getters at their actual definition modules
|
||||||
mock_log.return_value = mock_logger
|
with patch("src.server.services.config_service.get_config_service"):
|
||||||
|
with patch("src.server.services.progress_service.get_progress_service"):
|
||||||
|
with patch("src.server.services.websocket_service.get_websocket_service"):
|
||||||
|
with patch("src.server.utils.dependencies.get_anime_service"):
|
||||||
|
with patch("src.server.utils.dependencies.get_download_service"):
|
||||||
|
with patch("src.server.utils.dependencies.get_background_loader_service"):
|
||||||
|
with patch("src.server.services.scheduler_service.get_scheduler_service") as mock_get_sched:
|
||||||
|
mock_sched = MagicMock()
|
||||||
|
mock_sched.start = AsyncMock(return_value=None)
|
||||||
|
mock_get_sched.return_value = mock_sched
|
||||||
|
with patch("src.server.database.connection.init_db", new_callable=AsyncMock):
|
||||||
|
with patch("src.server.services.initialization_service.perform_initial_setup", new_callable=AsyncMock):
|
||||||
|
with patch("src.server.services.initialization_service.perform_nfo_scan_if_needed", new_callable=AsyncMock):
|
||||||
|
with patch("src.server.services.initialization_service.perform_media_scan_if_needed", new_callable=AsyncMock):
|
||||||
|
from src.server.fastapi_app import lifespan
|
||||||
|
app = MagicMock()
|
||||||
|
|
||||||
from src.server.fastapi_app import lifespan
|
async with lifespan(app):
|
||||||
app = MagicMock()
|
pass
|
||||||
|
|
||||||
with pytest.raises(StopIteration):
|
# Should have logged a warning about ffmpeg
|
||||||
async with lifespan(app):
|
warning_calls = [
|
||||||
pass
|
c for c in mock_logger.warning.call_args_list
|
||||||
|
if "ffmpeg" in str(c)
|
||||||
# Should have logged a warning about ffmpeg
|
]
|
||||||
warning_calls = [
|
assert len(warning_calls) >= 1
|
||||||
c for c in mock_logger.warning.call_args_list
|
|
||||||
if "ffmpeg" in str(c)
|
|
||||||
]
|
|
||||||
assert len(warning_calls) >= 1
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_ffmpeg_present_no_warning(self):
|
async def test_ffmpeg_present_no_warning(self):
|
||||||
"""Should not log warning when ffmpeg is found."""
|
"""Should not log warning when ffmpeg is found."""
|
||||||
|
mock_logger = MagicMock()
|
||||||
|
mock_logger.warning = MagicMock()
|
||||||
|
mock_logger.info = MagicMock()
|
||||||
|
mock_logger.debug = MagicMock()
|
||||||
|
|
||||||
with patch("shutil.which", return_value="/usr/bin/ffmpeg"):
|
with patch("shutil.which", return_value="/usr/bin/ffmpeg"):
|
||||||
with patch("src.server.fastapi_app.setup_logging") as mock_log:
|
with patch("src.server.fastapi_app.setup_logging", return_value=mock_logger):
|
||||||
mock_logger = MagicMock()
|
# Patch service getters at their actual definition modules
|
||||||
mock_log.return_value = mock_logger
|
with patch("src.server.services.config_service.get_config_service"):
|
||||||
|
with patch("src.server.services.progress_service.get_progress_service"):
|
||||||
|
with patch("src.server.services.websocket_service.get_websocket_service"):
|
||||||
|
with patch("src.server.utils.dependencies.get_anime_service"):
|
||||||
|
with patch("src.server.utils.dependencies.get_download_service"):
|
||||||
|
with patch("src.server.utils.dependencies.get_background_loader_service"):
|
||||||
|
with patch("src.server.services.scheduler_service.get_scheduler_service") as mock_get_sched:
|
||||||
|
mock_sched = MagicMock()
|
||||||
|
mock_sched.start = AsyncMock(return_value=None)
|
||||||
|
mock_get_sched.return_value = mock_sched
|
||||||
|
with patch("src.server.database.connection.init_db", new_callable=AsyncMock):
|
||||||
|
with patch("src.server.services.initialization_service.perform_initial_setup", new_callable=AsyncMock):
|
||||||
|
with patch("src.server.services.initialization_service.perform_nfo_scan_if_needed", new_callable=AsyncMock):
|
||||||
|
with patch("src.server.services.initialization_service.perform_media_scan_if_needed", new_callable=AsyncMock):
|
||||||
|
from src.server.fastapi_app import lifespan
|
||||||
|
app = MagicMock()
|
||||||
|
|
||||||
from src.server.fastapi_app import lifespan
|
async with lifespan(app):
|
||||||
app = MagicMock()
|
pass
|
||||||
|
|
||||||
with pytest.raises(StopIteration):
|
# Should NOT have logged a warning about ffmpeg
|
||||||
async with lifespan(app):
|
warning_calls = [
|
||||||
pass
|
c for c in mock_logger.warning.call_args_list
|
||||||
|
if "ffmpeg" in str(c)
|
||||||
# Should NOT have logged a warning about ffmpeg
|
]
|
||||||
warning_calls = [
|
assert len(warning_calls) == 0
|
||||||
c for c in mock_logger.warning.call_args_list
|
|
||||||
if "ffmpeg" in str(c)
|
|
||||||
]
|
|
||||||
assert len(warning_calls) == 0
|
|
||||||
@@ -160,7 +160,7 @@ class TestSyncAnimeFolders:
|
|||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_sync_anime_folders_without_progress(self):
|
async def test_sync_anime_folders_without_progress(self):
|
||||||
"""Test syncing anime folders without progress service."""
|
"""Test syncing anime folders without progress service."""
|
||||||
with patch('src.server.services.initialization_service.sync_series_from_data_files',
|
with patch('src.server.services.initialization_service.sync_legacy_series_to_db',
|
||||||
new_callable=AsyncMock, return_value=42) as mock_sync:
|
new_callable=AsyncMock, return_value=42) as mock_sync:
|
||||||
result = await _sync_anime_folders()
|
result = await _sync_anime_folders()
|
||||||
|
|
||||||
@@ -172,7 +172,7 @@ class TestSyncAnimeFolders:
|
|||||||
"""Test syncing anime folders with progress updates."""
|
"""Test syncing anime folders with progress updates."""
|
||||||
mock_progress = AsyncMock()
|
mock_progress = AsyncMock()
|
||||||
|
|
||||||
with patch('src.server.services.initialization_service.sync_series_from_data_files',
|
with patch('src.server.services.initialization_service.sync_legacy_series_to_db',
|
||||||
new_callable=AsyncMock, return_value=10) as mock_sync:
|
new_callable=AsyncMock, return_value=10) as mock_sync:
|
||||||
result = await _sync_anime_folders(progress_service=mock_progress)
|
result = await _sync_anime_folders(progress_service=mock_progress)
|
||||||
|
|
||||||
|
|||||||
@@ -1809,3 +1809,107 @@ class TestNegativeCache:
|
|||||||
assert "expired_key" not in tmdb_client._negative_cache
|
assert "expired_key" not in tmdb_client._negative_cache
|
||||||
assert "valid_key" in tmdb_client._negative_cache
|
assert "valid_key" in tmdb_client._negative_cache
|
||||||
|
|
||||||
|
|
||||||
|
class TestNFOIDOverride:
|
||||||
|
"""Tests for manual TMDB ID override via NFO."""
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_create_tvshow_nfo_uses_existing_tmdb_id(self, nfo_service, tmp_path, mock_tmdb_data, mock_content_ratings_de):
|
||||||
|
"""Test that existing TMDB ID in NFO skips search."""
|
||||||
|
# Create series folder with existing NFO containing TMDB ID
|
||||||
|
series_folder = tmp_path / "Attack on Titan"
|
||||||
|
series_folder.mkdir()
|
||||||
|
nfo_path = series_folder / "tvshow.nfo"
|
||||||
|
nfo_path.write_text("""<?xml version="1.0" encoding="UTF-8"?>
|
||||||
|
<tvshow>
|
||||||
|
<title>Attack on Titan</title>
|
||||||
|
<tmdbid>1429</tmdbid>
|
||||||
|
</tvshow>
|
||||||
|
""", encoding="utf-8")
|
||||||
|
|
||||||
|
with patch.object(nfo_service.tmdb_client, 'get_tv_show_details', new_callable=AsyncMock) as mock_details, \
|
||||||
|
patch.object(nfo_service.tmdb_client, 'get_tv_show_content_ratings', new_callable=AsyncMock) as mock_ratings, \
|
||||||
|
patch.object(nfo_service, '_download_media_files', new_callable=AsyncMock), \
|
||||||
|
patch.object(nfo_service.tmdb_client, '_ensure_session', new_callable=AsyncMock):
|
||||||
|
|
||||||
|
mock_details.return_value = mock_tmdb_data
|
||||||
|
mock_ratings.return_value = mock_content_ratings_de
|
||||||
|
|
||||||
|
nfo_path_result = await nfo_service.create_tvshow_nfo(
|
||||||
|
"Attack on Titan",
|
||||||
|
"Attack on Titan",
|
||||||
|
download_poster=False, download_logo=False, download_fanart=False
|
||||||
|
)
|
||||||
|
|
||||||
|
# Verify NFO was created
|
||||||
|
assert nfo_path_result.exists()
|
||||||
|
|
||||||
|
# Verify get_tv_show_details was called directly with the ID (no search)
|
||||||
|
mock_details.assert_called_once_with(1429, append_to_response="credits,external_ids,images")
|
||||||
|
|
||||||
|
# Verify search was NOT called
|
||||||
|
# (we can check by verifying no search_tv_show mock was set up)
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_create_tvshow_nfo_searches_when_no_tmdb_id(self, nfo_service, tmp_path, mock_tmdb_data, mock_content_ratings_de):
|
||||||
|
"""Test that search is used when NFO has no TMDB ID."""
|
||||||
|
# Create series folder without existing NFO
|
||||||
|
series_folder = tmp_path / "Test Anime"
|
||||||
|
series_folder.mkdir()
|
||||||
|
|
||||||
|
with patch.object(nfo_service.tmdb_client, 'search_tv_show', new_callable=AsyncMock) as mock_search, \
|
||||||
|
patch.object(nfo_service.tmdb_client, 'get_tv_show_details', new_callable=AsyncMock) as mock_details, \
|
||||||
|
patch.object(nfo_service.tmdb_client, 'get_tv_show_content_ratings', new_callable=AsyncMock) as mock_ratings, \
|
||||||
|
patch.object(nfo_service, '_download_media_files', new_callable=AsyncMock), \
|
||||||
|
patch.object(nfo_service.tmdb_client, '_ensure_session', new_callable=AsyncMock):
|
||||||
|
|
||||||
|
mock_search.return_value = {
|
||||||
|
"results": [{
|
||||||
|
"id": 1429,
|
||||||
|
"name": "Test Anime",
|
||||||
|
"first_air_date": "2024-01-01",
|
||||||
|
"overview": "Test overview"
|
||||||
|
}]
|
||||||
|
}
|
||||||
|
mock_details.return_value = mock_tmdb_data
|
||||||
|
mock_ratings.return_value = mock_content_ratings_de
|
||||||
|
|
||||||
|
nfo_path = await nfo_service.create_tvshow_nfo(
|
||||||
|
"Test Anime",
|
||||||
|
"Test Anime",
|
||||||
|
download_poster=False, download_logo=False, download_fanart=False
|
||||||
|
)
|
||||||
|
|
||||||
|
# Verify search was called
|
||||||
|
mock_search.assert_called()
|
||||||
|
|
||||||
|
|
||||||
|
class TestSearchMultiStrategy:
|
||||||
|
"""Tests for search/multi fallback strategy."""
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_search_multi_strategy_used_as_fallback(self, nfo_service, mock_tmdb_data):
|
||||||
|
"""Test that search/multi is tried after regular search fails."""
|
||||||
|
mock_search = AsyncMock()
|
||||||
|
mock_multi = AsyncMock()
|
||||||
|
|
||||||
|
# First: regular search fails
|
||||||
|
# Second: multi search returns TV result
|
||||||
|
mock_search.return_value = {"results": []}
|
||||||
|
mock_multi.return_value = {
|
||||||
|
"results": [
|
||||||
|
{"media_type": "movie", "id": 123},
|
||||||
|
{"media_type": "tv", "id": 456, "name": "Found Show", "first_air_date": "2024-01-01"}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
|
||||||
|
with patch.object(nfo_service.tmdb_client, 'search_tv_show', mock_search), \
|
||||||
|
patch.object(nfo_service.tmdb_client, 'search_multi', mock_multi):
|
||||||
|
|
||||||
|
result, source = await nfo_service._search_with_fallback(
|
||||||
|
"Unknown Show", 2024, None
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result["id"] == 456
|
||||||
|
assert source == "multi_search"
|
||||||
|
|
||||||
|
|||||||
@@ -27,7 +27,9 @@ def _make_episode(season: int = 1, episode: int = 1) -> EpisodeIdentifier:
|
|||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
def mock_anime_service():
|
def mock_anime_service():
|
||||||
return MagicMock(spec=["download_episode"])
|
service = MagicMock(spec=["download_episode"])
|
||||||
|
service._directory = "/tmp/test_anime"
|
||||||
|
return service
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
|
|||||||
@@ -555,7 +555,41 @@ class TestStartupRecovery:
|
|||||||
"src.server.services.scheduler_service.logger"
|
"src.server.services.scheduler_service.logger"
|
||||||
) as mock_logger:
|
) as mock_logger:
|
||||||
await scheduler_service.start()
|
await scheduler_service.start()
|
||||||
# Check that next_run was logged
|
|
||||||
info_calls = [str(c) for c in mock_logger.info.call_args_list]
|
info_calls = [str(c) for c in mock_logger.info.call_args_list]
|
||||||
assert any("next_run" in c for c in info_calls)
|
assert any("next_run" in str(c) or "Scheduler" in str(c) for c in info_calls)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# 12.8 ensure_started() – idempotent startup
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
class TestEnsureStarted:
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_ensure_started_when_not_running(
|
||||||
|
self, scheduler_service, mock_config_service
|
||||||
|
):
|
||||||
|
"""ensure_started() calls start() when scheduler is not running."""
|
||||||
|
# Mock start method
|
||||||
|
scheduler_service.start = AsyncMock()
|
||||||
|
|
||||||
|
# Call ensure_started
|
||||||
|
await scheduler_service.ensure_started()
|
||||||
|
|
||||||
|
# Verify start() was called exactly once
|
||||||
|
scheduler_service.start.assert_called_once()
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_ensure_started_when_already_running(self, scheduler_service):
|
||||||
|
"""ensure_started() returns immediately when already running (idempotent)."""
|
||||||
|
# Set up as already running
|
||||||
|
scheduler_service._is_running = True
|
||||||
|
|
||||||
|
# Mock start method
|
||||||
|
scheduler_service.start = AsyncMock()
|
||||||
|
|
||||||
|
# Call ensure_started
|
||||||
|
await scheduler_service.ensure_started()
|
||||||
|
|
||||||
|
# Verify start() was NOT called
|
||||||
|
scheduler_service.start.assert_not_called()
|
||||||
|
|
||||||
|
|||||||
291
tests/unit/test_serie_list_db_loading.py
Normal file
291
tests/unit/test_serie_list_db_loading.py
Normal file
@@ -0,0 +1,291 @@
|
|||||||
|
"""Tests for SerieList database loading functionality."""
|
||||||
|
|
||||||
|
from unittest.mock import AsyncMock, MagicMock, patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from src.core.entities.series import Serie
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def mock_session_factory():
|
||||||
|
"""Create a mock async session factory."""
|
||||||
|
mock_session = AsyncMock()
|
||||||
|
mock_session_factory = MagicMock(return_value=mock_session)
|
||||||
|
return mock_session_factory, mock_session
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def sample_anime_series():
|
||||||
|
"""Create a sample AnimeSeries DB model for testing."""
|
||||||
|
mock = MagicMock()
|
||||||
|
mock.key = "attack-on-titan"
|
||||||
|
mock.name = "Attack on Titan"
|
||||||
|
mock.site = "aniworld.to"
|
||||||
|
mock.folder = "Attack on Titan (2013)"
|
||||||
|
mock.year = 2013
|
||||||
|
mock.episodes = [
|
||||||
|
MagicMock(season=1, episode_number=1),
|
||||||
|
MagicMock(season=1, episode_number=2),
|
||||||
|
MagicMock(season=1, episode_number=3),
|
||||||
|
MagicMock(season=2, episode_number=1),
|
||||||
|
MagicMock(season=2, episode_number=2),
|
||||||
|
]
|
||||||
|
return mock
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def sample_serie():
|
||||||
|
"""Create a sample Serie for testing."""
|
||||||
|
return Serie(
|
||||||
|
key="attack-on-titan",
|
||||||
|
name="Attack on Titan",
|
||||||
|
site="aniworld.to",
|
||||||
|
folder="Attack on Titan (2013)",
|
||||||
|
episodeDict={1: [1, 2, 3], 2: [1, 2]},
|
||||||
|
year=2013
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class TestLoadAllFromDb:
|
||||||
|
"""Test load_all_from_db method."""
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_load_all_from_db(self, mock_session_factory, sample_anime_series):
|
||||||
|
"""Verify SerieList loads all series from DB."""
|
||||||
|
mock_factory, mock_session = mock_session_factory
|
||||||
|
|
||||||
|
with patch(
|
||||||
|
"src.server.database.connection.get_async_session_factory",
|
||||||
|
return_value=mock_factory
|
||||||
|
):
|
||||||
|
with patch(
|
||||||
|
"src.server.database.service.AnimeSeriesService.get_all",
|
||||||
|
return_value=[sample_anime_series]
|
||||||
|
):
|
||||||
|
from src.core.entities.SerieList import SerieList
|
||||||
|
|
||||||
|
serie_list = SerieList("/tmp", skip_load=True)
|
||||||
|
count = await serie_list.load_all_from_db()
|
||||||
|
|
||||||
|
assert count == 1
|
||||||
|
assert "attack-on-titan" in serie_list.keyDict
|
||||||
|
serie = serie_list.keyDict["attack-on-titan"]
|
||||||
|
assert serie.name == "Attack on Titan"
|
||||||
|
assert serie.key == "attack-on-titan"
|
||||||
|
assert serie.year == 2013
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_load_all_from_db_multiple_series(
|
||||||
|
self, mock_session_factory, sample_anime_series
|
||||||
|
):
|
||||||
|
"""Verify SerieList loads multiple series from DB."""
|
||||||
|
mock_factory, mock_session = mock_session_factory
|
||||||
|
|
||||||
|
mock_series2 = MagicMock()
|
||||||
|
mock_series2.key = "one-piece"
|
||||||
|
mock_series2.name = "One Piece"
|
||||||
|
mock_series2.site = "aniworld.to"
|
||||||
|
mock_series2.folder = "One Piece"
|
||||||
|
mock_series2.year = 1999
|
||||||
|
mock_series2.episodes = []
|
||||||
|
|
||||||
|
with patch(
|
||||||
|
"src.server.database.connection.get_async_session_factory",
|
||||||
|
return_value=mock_factory
|
||||||
|
):
|
||||||
|
with patch(
|
||||||
|
"src.server.database.service.AnimeSeriesService.get_all",
|
||||||
|
return_value=[sample_anime_series, mock_series2]
|
||||||
|
):
|
||||||
|
from src.core.entities.SerieList import SerieList
|
||||||
|
|
||||||
|
serie_list = SerieList("/tmp", skip_load=True)
|
||||||
|
count = await serie_list.load_all_from_db()
|
||||||
|
|
||||||
|
assert count == 2
|
||||||
|
assert "attack-on-titan" in serie_list.keyDict
|
||||||
|
assert "one-piece" in serie_list.keyDict
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_load_all_from_db_rebuilds_episode_dict(
|
||||||
|
self, mock_session_factory, sample_anime_series
|
||||||
|
):
|
||||||
|
"""Verify episode_dict is correctly built from Episode records."""
|
||||||
|
mock_factory, mock_session = mock_session_factory
|
||||||
|
|
||||||
|
with patch(
|
||||||
|
"src.server.database.connection.get_async_session_factory",
|
||||||
|
return_value=mock_factory
|
||||||
|
):
|
||||||
|
with patch(
|
||||||
|
"src.server.database.service.AnimeSeriesService.get_all",
|
||||||
|
return_value=[sample_anime_series]
|
||||||
|
):
|
||||||
|
from src.core.entities.SerieList import SerieList
|
||||||
|
|
||||||
|
serie_list = SerieList("/tmp", skip_load=True)
|
||||||
|
await serie_list.load_all_from_db()
|
||||||
|
|
||||||
|
serie = serie_list.keyDict["attack-on-titan"]
|
||||||
|
assert 1 in serie.episodeDict
|
||||||
|
assert 2 in serie.episodeDict
|
||||||
|
assert sorted(serie.episodeDict[1]) == [1, 2, 3]
|
||||||
|
assert sorted(serie.episodeDict[2]) == [1, 2]
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_load_all_from_db_no_series(self, mock_session_factory):
|
||||||
|
"""Verify SerieList handles empty DB."""
|
||||||
|
mock_factory, mock_session = mock_session_factory
|
||||||
|
|
||||||
|
with patch(
|
||||||
|
"src.server.database.connection.get_async_session_factory",
|
||||||
|
return_value=mock_factory
|
||||||
|
):
|
||||||
|
with patch(
|
||||||
|
"src.server.database.service.AnimeSeriesService.get_all",
|
||||||
|
return_value=[]
|
||||||
|
):
|
||||||
|
from src.core.entities.SerieList import SerieList
|
||||||
|
|
||||||
|
serie_list = SerieList("/tmp", skip_load=True)
|
||||||
|
count = await serie_list.load_all_from_db()
|
||||||
|
|
||||||
|
assert count == 0
|
||||||
|
assert len(serie_list.keyDict) == 0
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_load_all_from_db_db_not_initialized(self, mock_session_factory):
|
||||||
|
"""Verify SerieList handles uninitialized DB gracefully."""
|
||||||
|
mock_factory, mock_session = mock_session_factory
|
||||||
|
|
||||||
|
with patch(
|
||||||
|
"src.server.database.connection.get_async_session_factory",
|
||||||
|
return_value=mock_factory
|
||||||
|
):
|
||||||
|
with patch(
|
||||||
|
"src.server.database.service.AnimeSeriesService.get_all",
|
||||||
|
side_effect=RuntimeError("Database not initialized")
|
||||||
|
):
|
||||||
|
from src.core.entities.SerieList import SerieList
|
||||||
|
|
||||||
|
serie_list = SerieList("/tmp", skip_load=True)
|
||||||
|
count = await serie_list.load_all_from_db()
|
||||||
|
|
||||||
|
assert count == 0
|
||||||
|
assert len(serie_list.keyDict) == 0
|
||||||
|
|
||||||
|
|
||||||
|
class TestLoadSingleSeriesFromDb:
|
||||||
|
"""Test _load_single_series_from_db method."""
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_load_single_series_from_db(
|
||||||
|
self, mock_session_factory, sample_anime_series
|
||||||
|
):
|
||||||
|
"""Verify SerieList loads a single series from DB by folder."""
|
||||||
|
mock_factory, mock_session = mock_session_factory
|
||||||
|
|
||||||
|
with patch(
|
||||||
|
"src.server.database.connection.get_async_session_factory",
|
||||||
|
return_value=mock_factory
|
||||||
|
):
|
||||||
|
with patch(
|
||||||
|
"src.server.database.service.AnimeSeriesService.get_by_folder",
|
||||||
|
return_value=sample_anime_series
|
||||||
|
):
|
||||||
|
from src.core.entities.SerieList import SerieList
|
||||||
|
|
||||||
|
serie_list = SerieList("/tmp", skip_load=True)
|
||||||
|
serie = await serie_list._load_single_series_from_db("Attack on Titan (2013)")
|
||||||
|
|
||||||
|
assert serie is not None
|
||||||
|
assert serie.key == "attack-on-titan"
|
||||||
|
assert "attack-on-titan" in serie_list.keyDict
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_load_single_series_from_db_not_found(
|
||||||
|
self, mock_session_factory
|
||||||
|
):
|
||||||
|
"""Verify SerieList handles series not found in DB."""
|
||||||
|
mock_factory, mock_session = mock_session_factory
|
||||||
|
|
||||||
|
with patch(
|
||||||
|
"src.server.database.connection.get_async_session_factory",
|
||||||
|
return_value=mock_factory
|
||||||
|
):
|
||||||
|
with patch(
|
||||||
|
"src.server.database.service.AnimeSeriesService.get_by_folder",
|
||||||
|
return_value=None
|
||||||
|
):
|
||||||
|
from src.core.entities.SerieList import SerieList
|
||||||
|
|
||||||
|
serie_list = SerieList("/tmp", skip_load=True)
|
||||||
|
serie = await serie_list._load_single_series_from_db("Unknown Series")
|
||||||
|
|
||||||
|
assert serie is None
|
||||||
|
assert len(serie_list.keyDict) == 0
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_load_single_series_from_db_db_not_initialized(
|
||||||
|
self, mock_session_factory
|
||||||
|
):
|
||||||
|
"""Verify SerieList handles uninitialized DB gracefully."""
|
||||||
|
mock_factory, mock_session = mock_session_factory
|
||||||
|
|
||||||
|
with patch(
|
||||||
|
"src.server.database.connection.get_async_session_factory",
|
||||||
|
return_value=mock_factory
|
||||||
|
):
|
||||||
|
with patch(
|
||||||
|
"src.server.database.service.AnimeSeriesService.get_by_folder",
|
||||||
|
side_effect=RuntimeError("Database not initialized")
|
||||||
|
):
|
||||||
|
from src.core.entities.SerieList import SerieList
|
||||||
|
|
||||||
|
serie_list = SerieList("/tmp", skip_load=True)
|
||||||
|
serie = await serie_list._load_single_series_from_db("Some Folder")
|
||||||
|
|
||||||
|
assert serie is None
|
||||||
|
|
||||||
|
|
||||||
|
class TestInvalidateCache:
|
||||||
|
"""Test invalidate_cache method."""
|
||||||
|
|
||||||
|
def test_invalidate_cache_clears_keydict(self, sample_serie):
|
||||||
|
"""Verify invalidate_cache clears the in-memory cache."""
|
||||||
|
from src.core.entities.SerieList import SerieList
|
||||||
|
|
||||||
|
serie_list = SerieList("/tmp", skip_load=True)
|
||||||
|
serie_list.keyDict["attack-on-titan"] = sample_serie
|
||||||
|
assert len(serie_list.keyDict) == 1
|
||||||
|
|
||||||
|
serie_list.invalidate_cache()
|
||||||
|
|
||||||
|
assert len(serie_list.keyDict) == 0
|
||||||
|
|
||||||
|
def test_invalidate_cache_allows_reload(self, mock_session_factory, sample_anime_series):
|
||||||
|
"""Verify cache can be reloaded after invalidation."""
|
||||||
|
mock_factory, mock_session = mock_session_factory
|
||||||
|
|
||||||
|
with patch(
|
||||||
|
"src.server.database.connection.get_async_session_factory",
|
||||||
|
return_value=mock_factory
|
||||||
|
):
|
||||||
|
with patch(
|
||||||
|
"src.server.database.service.AnimeSeriesService.get_all",
|
||||||
|
return_value=[sample_anime_series]
|
||||||
|
):
|
||||||
|
from src.core.entities.SerieList import SerieList
|
||||||
|
|
||||||
|
serie_list = SerieList("/tmp", skip_load=True)
|
||||||
|
serie_list.keyDict["some-key"] = MagicMock()
|
||||||
|
|
||||||
|
serie_list.invalidate_cache()
|
||||||
|
assert len(serie_list.keyDict) == 0
|
||||||
|
|
||||||
|
# Reload
|
||||||
|
import asyncio
|
||||||
|
asyncio.get_event_loop().run_until_complete(serie_list.load_all_from_db())
|
||||||
|
|
||||||
|
assert len(serie_list.keyDict) == 1
|
||||||
@@ -75,12 +75,12 @@ class TestSerieScannerInitialization:
|
|||||||
class TestSerieScannerScan:
|
class TestSerieScannerScan:
|
||||||
"""Test file-based scan operations."""
|
"""Test file-based scan operations."""
|
||||||
|
|
||||||
def test_file_based_scan_works(
|
def test_scan_persists_to_db(
|
||||||
self, temp_directory, mock_loader, sample_serie
|
self, temp_directory, mock_loader, sample_serie
|
||||||
):
|
):
|
||||||
"""Test file-based scan works properly."""
|
"""Test scan persists series to database."""
|
||||||
scanner = SerieScanner(temp_directory, mock_loader)
|
scanner = SerieScanner(temp_directory, mock_loader)
|
||||||
|
|
||||||
with patch.object(scanner, 'get_total_to_scan', return_value=1):
|
with patch.object(scanner, 'get_total_to_scan', return_value=1):
|
||||||
with patch.object(
|
with patch.object(
|
||||||
scanner,
|
scanner,
|
||||||
@@ -100,12 +100,15 @@ class TestSerieScannerScan:
|
|||||||
return_value=({1: [2, 3]}, "aniworld.to")
|
return_value=({1: [2, 3]}, "aniworld.to")
|
||||||
):
|
):
|
||||||
with patch.object(
|
with patch.object(
|
||||||
sample_serie, 'save_to_file'
|
scanner, '_persist_serie_to_db'
|
||||||
) as mock_save:
|
) as mock_persist:
|
||||||
scanner.scan()
|
scanner.scan()
|
||||||
|
|
||||||
# Verify file was saved
|
# Verify DB persistence was called
|
||||||
mock_save.assert_called_once()
|
mock_persist.assert_called_once()
|
||||||
|
# Check the serie passed matches
|
||||||
|
call_args = mock_persist.call_args
|
||||||
|
assert call_args[0][0].key == "attack-on-titan"
|
||||||
|
|
||||||
def test_keydict_populated_after_scan(
|
def test_keydict_populated_after_scan(
|
||||||
self, temp_directory, mock_loader, sample_serie
|
self, temp_directory, mock_loader, sample_serie
|
||||||
|
|||||||
124
tests/unit/test_serie_scanner_db_lookup.py
Normal file
124
tests/unit/test_serie_scanner_db_lookup.py
Normal file
@@ -0,0 +1,124 @@
|
|||||||
|
"""Tests for SerieScanner DB lookup functionality."""
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import tempfile
|
||||||
|
from unittest.mock import MagicMock, patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from src.core.entities.series import Serie
|
||||||
|
from src.core.SerieScanner import SerieScanner
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def mock_loader():
|
||||||
|
"""Create a mock Loader instance."""
|
||||||
|
loader = MagicMock()
|
||||||
|
loader.get_season_episode_count = MagicMock(return_value={1: 12})
|
||||||
|
loader.is_language = MagicMock(return_value=True)
|
||||||
|
loader.get_year = MagicMock(return_value=2026)
|
||||||
|
return loader
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def temp_directory():
|
||||||
|
"""Create a temporary directory with subdirectories for testing."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmpdir:
|
||||||
|
anime_folder = os.path.join(tmpdir, "Rooster Fighter (2026)")
|
||||||
|
os.makedirs(anime_folder, exist_ok=True)
|
||||||
|
mp4_path = os.path.join(anime_folder, "S01E001.mp4")
|
||||||
|
with open(mp4_path, "w") as f:
|
||||||
|
f.write("dummy mp4")
|
||||||
|
yield tmpdir
|
||||||
|
|
||||||
|
|
||||||
|
class TestGetSerieFromFolderDbLookup:
|
||||||
|
"""Test __read_data_from_file DB lookup behavior."""
|
||||||
|
|
||||||
|
def test_db_hit_returns_serie_from_db(self, temp_directory, mock_loader):
|
||||||
|
"""DB lookup resolves folder -> Serie returned."""
|
||||||
|
from src.server.database.models import AnimeSeries
|
||||||
|
from src.server.database import service as anime_series_service
|
||||||
|
|
||||||
|
mock_session = MagicMock()
|
||||||
|
mock_anime_series = MagicMock()
|
||||||
|
mock_anime_series.key = "rooster-fighter"
|
||||||
|
mock_anime_series.name = "Rooster Fighter"
|
||||||
|
mock_anime_series.site = "aniworld.to"
|
||||||
|
mock_anime_series.folder = "Rooster Fighter (2026)"
|
||||||
|
mock_anime_series.year = 2026
|
||||||
|
mock_anime_series.episodes = []
|
||||||
|
mock_session.execute.return_value.scalar_one_or_none.return_value = mock_anime_series
|
||||||
|
|
||||||
|
with patch("src.core.SerieScanner.get_sync_session", return_value=mock_session):
|
||||||
|
scanner = SerieScanner(temp_directory, mock_loader)
|
||||||
|
result = scanner._SerieScanner__read_data_from_file("Rooster Fighter (2026)")
|
||||||
|
|
||||||
|
assert result is not None
|
||||||
|
assert result.key == "rooster-fighter"
|
||||||
|
assert result.name == "Rooster Fighter"
|
||||||
|
assert result.year == 2026
|
||||||
|
|
||||||
|
def test_db_miss_falls_back_to_provider_callback(self, temp_directory, mock_loader):
|
||||||
|
"""DB miss -> _db_lookup callback called."""
|
||||||
|
lookup = MagicMock(return_value=Serie(
|
||||||
|
key="rooster-fighter",
|
||||||
|
name="Rooster Fighter",
|
||||||
|
site="aniworld.to",
|
||||||
|
folder="Rooster Fighter (2026)",
|
||||||
|
episodeDict={},
|
||||||
|
))
|
||||||
|
scanner = SerieScanner(temp_directory, mock_loader, db_lookup=lookup)
|
||||||
|
|
||||||
|
result = scanner._SerieScanner__read_data_from_file("Rooster Fighter (2026)")
|
||||||
|
|
||||||
|
assert result is not None
|
||||||
|
assert result.key == "rooster-fighter"
|
||||||
|
lookup.assert_called_once_with("Rooster Fighter (2026)")
|
||||||
|
|
||||||
|
def test_legacy_key_file_as_last_resort(self, temp_directory, mock_loader):
|
||||||
|
"""No DB, no callback -> legacy 'key' file used with deprecation warning."""
|
||||||
|
folder = os.path.join(temp_directory, "Legacy Series")
|
||||||
|
os.makedirs(folder, exist_ok=True)
|
||||||
|
with open(os.path.join(folder, "key"), "w") as f:
|
||||||
|
f.write("legacy-key")
|
||||||
|
|
||||||
|
scanner = SerieScanner(temp_directory, mock_loader)
|
||||||
|
|
||||||
|
with patch.object(logging.getLogger("src.core.SerieScanner"), "warning") as mock_warning:
|
||||||
|
result = scanner._SerieScanner__read_data_from_file("Legacy Series")
|
||||||
|
|
||||||
|
assert result is not None
|
||||||
|
assert result.key == "legacy-key"
|
||||||
|
mock_warning.assert_called()
|
||||||
|
warning_calls = [str(c) for c in mock_warning.call_args_list]
|
||||||
|
assert any("deprecated" in c or "v3.0.0" in c for c in warning_calls)
|
||||||
|
|
||||||
|
def test_db_lookup_exception_caught_and_logged(self, temp_directory, mock_loader):
|
||||||
|
"""DB exception -> fallback to provider callback."""
|
||||||
|
def bad_lookup(folder):
|
||||||
|
raise RuntimeError("DB connection failed")
|
||||||
|
|
||||||
|
scanner = SerieScanner(temp_directory, mock_loader, db_lookup=bad_lookup)
|
||||||
|
|
||||||
|
with patch.object(logging.getLogger("src.core.SerieScanner"), "warning") as mock_warning:
|
||||||
|
result = scanner._SerieScanner__read_data_from_file("Rooster Fighter (2026)")
|
||||||
|
mock_warning.assert_called()
|
||||||
|
assert any("DB lookup failed" in str(c) for c in mock_warning.call_args_list)
|
||||||
|
|
||||||
|
|
||||||
|
class TestGetSerieFromFolderEdgeCases:
|
||||||
|
"""Edge case tests for __read_data_from_file."""
|
||||||
|
|
||||||
|
def test_empty_folder_name_returns_none(self, temp_directory, mock_loader):
|
||||||
|
"""Empty folder name -> returns None (no DB lookup attempted)."""
|
||||||
|
scanner = SerieScanner(temp_directory, mock_loader)
|
||||||
|
result = scanner._SerieScanner__read_data_from_file("")
|
||||||
|
assert result is None
|
||||||
|
|
||||||
|
def test_nonexistent_folder_no_exception(self, temp_directory, mock_loader):
|
||||||
|
"""Folder doesn't exist -> returns None without raising."""
|
||||||
|
scanner = SerieScanner(temp_directory, mock_loader)
|
||||||
|
result = scanner._SerieScanner__read_data_from_file("Nonexistent Folder")
|
||||||
|
assert result is None
|
||||||
225
tests/unit/test_serie_scanner_db_writes.py
Normal file
225
tests/unit/test_serie_scanner_db_writes.py
Normal file
@@ -0,0 +1,225 @@
|
|||||||
|
"""Tests for SerieScanner DB persistence functionality."""
|
||||||
|
|
||||||
|
from unittest.mock import AsyncMock, MagicMock, patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from src.core.entities.series import Serie
|
||||||
|
from src.core.SerieScanner import SerieScanner
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def mock_session_factory():
|
||||||
|
"""Create a mock async session factory."""
|
||||||
|
mock_session = AsyncMock()
|
||||||
|
mock_session_factory = MagicMock(return_value=mock_session)
|
||||||
|
return mock_session_factory, mock_session
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def sample_serie():
|
||||||
|
"""Create a sample Serie for testing."""
|
||||||
|
return Serie(
|
||||||
|
key="attack-on-titan",
|
||||||
|
name="Attack on Titan",
|
||||||
|
site="aniworld.to",
|
||||||
|
folder="Attack on Titan (2013)",
|
||||||
|
episodeDict={1: [1, 2, 3], 2: [1, 2]},
|
||||||
|
year=2013
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class TestPersistSerieToDb:
|
||||||
|
"""Test _persist_serie_to_db method."""
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_creates_new_series_when_not_exists(
|
||||||
|
self, mock_session_factory, sample_serie
|
||||||
|
):
|
||||||
|
"""Verify new series is created in DB."""
|
||||||
|
mock_factory, mock_session = mock_session_factory
|
||||||
|
|
||||||
|
with patch(
|
||||||
|
"src.server.database.connection.get_async_session_factory",
|
||||||
|
return_value=mock_factory
|
||||||
|
):
|
||||||
|
with patch(
|
||||||
|
"src.server.database.service.AnimeSeriesService.get_by_key",
|
||||||
|
return_value=None
|
||||||
|
):
|
||||||
|
mock_anime_series = MagicMock()
|
||||||
|
mock_anime_series.id = 1
|
||||||
|
with patch(
|
||||||
|
"src.server.database.service.AnimeSeriesService.create",
|
||||||
|
return_value=mock_anime_series
|
||||||
|
):
|
||||||
|
scanner = SerieScanner("/tmp", MagicMock())
|
||||||
|
await scanner._persist_serie_to_db(sample_serie)
|
||||||
|
|
||||||
|
from src.server.database.service import AnimeSeriesService
|
||||||
|
AnimeSeriesService.create.assert_called_once()
|
||||||
|
call_kwargs = AnimeSeriesService.create.call_args[1]
|
||||||
|
assert call_kwargs["key"] == "attack-on-titan"
|
||||||
|
assert call_kwargs["name"] == "Attack on Titan"
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_updates_existing_series(self, mock_session_factory, sample_serie):
|
||||||
|
"""Verify existing series is updated."""
|
||||||
|
mock_factory, mock_session = mock_session_factory
|
||||||
|
|
||||||
|
mock_existing = MagicMock()
|
||||||
|
mock_existing.id = 42
|
||||||
|
mock_existing.key = "attack-on-titan"
|
||||||
|
|
||||||
|
scanner = SerieScanner("/tmp", MagicMock())
|
||||||
|
|
||||||
|
with patch(
|
||||||
|
"src.server.database.connection.get_async_session_factory",
|
||||||
|
return_value=mock_factory
|
||||||
|
):
|
||||||
|
with patch(
|
||||||
|
"src.server.database.service.AnimeSeriesService.get_by_key",
|
||||||
|
return_value=mock_existing
|
||||||
|
):
|
||||||
|
with patch(
|
||||||
|
"src.server.database.service.AnimeSeriesService.update",
|
||||||
|
new_callable=AsyncMock
|
||||||
|
) as mock_update:
|
||||||
|
with patch.object(
|
||||||
|
scanner,
|
||||||
|
"_sync_episodes_to_db",
|
||||||
|
new_callable=AsyncMock
|
||||||
|
):
|
||||||
|
await scanner._persist_serie_to_db(sample_serie)
|
||||||
|
|
||||||
|
mock_update.assert_called_once()
|
||||||
|
call_args = mock_update.call_args[0]
|
||||||
|
assert call_args[1] == 42 # series_id
|
||||||
|
|
||||||
|
|
||||||
|
class TestSyncEpisodesToDb:
|
||||||
|
"""Test _sync_episodes_to_db method."""
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_preserves_downloaded_episodes(self):
|
||||||
|
"""Verify downloaded episodes are not removed even when no longer missing."""
|
||||||
|
mock_session = AsyncMock()
|
||||||
|
|
||||||
|
# S01E1 was downloaded (file exists), S01E2 was missing but file now exists
|
||||||
|
# Both are no longer in episode_dict
|
||||||
|
existing_eps = [
|
||||||
|
MagicMock(id=1, season=1, episode_number=1, is_downloaded=True),
|
||||||
|
MagicMock(id=2, season=1, episode_number=2, is_downloaded=True),
|
||||||
|
]
|
||||||
|
|
||||||
|
with patch(
|
||||||
|
"src.server.database.service.EpisodeService.get_by_series",
|
||||||
|
return_value=existing_eps
|
||||||
|
):
|
||||||
|
with patch(
|
||||||
|
"src.server.database.service.EpisodeService.delete_by_series",
|
||||||
|
new_callable=AsyncMock
|
||||||
|
) as mock_delete:
|
||||||
|
scanner = SerieScanner("/tmp", MagicMock())
|
||||||
|
# Neither S01E1 nor S01E2 are missing now
|
||||||
|
await scanner._sync_episodes_to_db(
|
||||||
|
mock_session, 1, {} # No episodes missing
|
||||||
|
)
|
||||||
|
|
||||||
|
# Neither should be deleted since both are downloaded
|
||||||
|
mock_delete.assert_not_called()
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_removes_missing_episodes_when_no_longer_missing(self):
|
||||||
|
"""Verify episodes removed from DB if file now exists."""
|
||||||
|
mock_session = AsyncMock()
|
||||||
|
|
||||||
|
existing_eps = [
|
||||||
|
MagicMock(id=1, season=1, episode_number=1, is_downloaded=False),
|
||||||
|
MagicMock(id=2, season=1, episode_number=2, is_downloaded=False),
|
||||||
|
]
|
||||||
|
|
||||||
|
with patch(
|
||||||
|
"src.server.database.service.EpisodeService.get_by_series",
|
||||||
|
return_value=existing_eps
|
||||||
|
):
|
||||||
|
with patch(
|
||||||
|
"src.server.database.service.EpisodeService.delete_by_series",
|
||||||
|
new_callable=AsyncMock
|
||||||
|
) as mock_delete:
|
||||||
|
with patch(
|
||||||
|
"src.server.database.service.EpisodeService.create",
|
||||||
|
new_callable=AsyncMock
|
||||||
|
):
|
||||||
|
scanner = SerieScanner("/tmp", MagicMock())
|
||||||
|
await scanner._sync_episodes_to_db(
|
||||||
|
mock_session, 1, {1: [1]} # Only S01E01 now missing
|
||||||
|
)
|
||||||
|
|
||||||
|
# S01E02 should be deleted since no longer missing
|
||||||
|
mock_delete.assert_called_once()
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_adds_new_missing_episodes(self):
|
||||||
|
"""Verify new missing episodes are added."""
|
||||||
|
mock_session = AsyncMock()
|
||||||
|
|
||||||
|
existing_eps = [
|
||||||
|
MagicMock(id=1, season=1, episode_number=1, is_downloaded=False),
|
||||||
|
]
|
||||||
|
|
||||||
|
with patch(
|
||||||
|
"src.server.database.service.EpisodeService.get_by_series",
|
||||||
|
return_value=existing_eps
|
||||||
|
):
|
||||||
|
with patch(
|
||||||
|
"src.server.database.service.EpisodeService.create",
|
||||||
|
new_callable=AsyncMock
|
||||||
|
) as mock_create:
|
||||||
|
scanner = SerieScanner("/tmp", MagicMock())
|
||||||
|
await scanner._sync_episodes_to_db(
|
||||||
|
mock_session, 1, {1: [1, 2, 3]} # S01E01, S01E02, S01E03
|
||||||
|
)
|
||||||
|
|
||||||
|
# S01E02 and S01E03 should be created
|
||||||
|
assert mock_create.call_count == 2
|
||||||
|
|
||||||
|
|
||||||
|
class TestPersistSerieToDbErrorHandling:
|
||||||
|
"""Test error handling in _persist_serie_to_db."""
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_logs_error_when_db_unavailable(self, sample_serie):
|
||||||
|
"""Verify DB unavailability is logged but doesn't crash."""
|
||||||
|
with patch(
|
||||||
|
"src.server.database.connection.get_async_session_factory",
|
||||||
|
side_effect=RuntimeError("DB not initialized")
|
||||||
|
):
|
||||||
|
scanner = SerieScanner("/tmp", MagicMock())
|
||||||
|
# Should not raise
|
||||||
|
await scanner._persist_serie_to_db(sample_serie)
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_rollback_on_failure(self, mock_session_factory, sample_serie):
|
||||||
|
"""Verify rollback on DB failure."""
|
||||||
|
mock_factory, mock_session = mock_session_factory
|
||||||
|
|
||||||
|
mock_existing = MagicMock()
|
||||||
|
mock_existing.id = 1
|
||||||
|
|
||||||
|
with patch(
|
||||||
|
"src.server.database.connection.get_async_session_factory",
|
||||||
|
return_value=mock_session
|
||||||
|
):
|
||||||
|
with patch(
|
||||||
|
"src.server.database.service.AnimeSeriesService.get_by_key",
|
||||||
|
return_value=mock_existing
|
||||||
|
):
|
||||||
|
with patch(
|
||||||
|
"src.server.database.service.AnimeSeriesService.update",
|
||||||
|
side_effect=Exception("DB error")
|
||||||
|
):
|
||||||
|
scanner = SerieScanner("/tmp", MagicMock())
|
||||||
|
# Should not raise but should rollback
|
||||||
|
await scanner._persist_serie_to_db(sample_serie)
|
||||||
|
mock_session.rollback.assert_called_once()
|
||||||
@@ -444,6 +444,77 @@ class TestTMDBClientSessionLeak:
|
|||||||
"Unexpected warning about unclosed session"
|
"Unexpected warning about unclosed session"
|
||||||
|
|
||||||
|
|
||||||
|
class TestTMDBClientLifecycleIntegration:
|
||||||
|
"""Integration tests for TMDBClient lifecycle management."""
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_context_manager_no_resource_warning(self, caplog):
|
||||||
|
"""Test async with TMDBClient produces no ResourceWarning."""
|
||||||
|
import logging
|
||||||
|
import warnings
|
||||||
|
caplog.set_level(logging.WARNING)
|
||||||
|
|
||||||
|
# Use context manager properly - should not leak
|
||||||
|
async with TMDBClient(api_key="test_key") as client:
|
||||||
|
await client._ensure_session()
|
||||||
|
assert client.session is not None
|
||||||
|
|
||||||
|
# Session should be closed after context exit
|
||||||
|
assert client.session is None or client.session.closed
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_exception_safety_during_api_call(self, caplog):
|
||||||
|
"""Test session is closed even when exception raised during API call."""
|
||||||
|
import logging
|
||||||
|
caplog.set_level(logging.WARNING)
|
||||||
|
|
||||||
|
close_called = False
|
||||||
|
|
||||||
|
class TrackingSession:
|
||||||
|
def __init__(self):
|
||||||
|
self.closed = False
|
||||||
|
|
||||||
|
async def close(self):
|
||||||
|
nonlocal close_called
|
||||||
|
close_called = True
|
||||||
|
self.closed = True
|
||||||
|
|
||||||
|
async def get(self, url, **kwargs):
|
||||||
|
raise TMDBAPIError("Simulated API failure")
|
||||||
|
|
||||||
|
client = TMDBClient(api_key="test_key")
|
||||||
|
client.session = TrackingSession()
|
||||||
|
|
||||||
|
# Exception during context should still close session
|
||||||
|
with pytest.raises(TMDBAPIError):
|
||||||
|
async with client:
|
||||||
|
raise TMDBAPIError("Simulated API failure")
|
||||||
|
|
||||||
|
assert close_called, "Session was not closed after API exception"
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_reuse_session_across_multiple_requests(self, caplog):
|
||||||
|
"""Test session is reused across multiple requests without leaks."""
|
||||||
|
import logging
|
||||||
|
caplog.set_level(logging.WARNING)
|
||||||
|
|
||||||
|
client = TMDBClient(api_key="test_key")
|
||||||
|
|
||||||
|
async with client as c:
|
||||||
|
# First request
|
||||||
|
await c._ensure_session()
|
||||||
|
session1 = c.session
|
||||||
|
|
||||||
|
# Second request should reuse same session
|
||||||
|
await c._ensure_session()
|
||||||
|
session2 = c.session
|
||||||
|
|
||||||
|
assert session1 is session2, "Session should be reused"
|
||||||
|
|
||||||
|
# After context exit, session should be closed
|
||||||
|
assert client.session is None or client.session.closed
|
||||||
|
|
||||||
|
|
||||||
class TestTMDBClientConnectorClosed:
|
class TestTMDBClientConnectorClosed:
|
||||||
"""Test handling of 'Connector is closed' errors."""
|
"""Test handling of 'Connector is closed' errors."""
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user