Add data file to database sync functionality
- Add get_all_series_from_data_files() to SeriesApp - Sync series from data files to DB on startup - Add unit tests for new SeriesApp method - Add integration tests for sync functionality - Update documentation
This commit is contained in:
parent
86eaa8a680
commit
684337fd0c
@ -254,6 +254,25 @@ Deprecation warnings are raised when using these methods.
|
|||||||
|
|
||||||
Main engine for anime series management with async support, progress callbacks, and cancellation.
|
Main engine for anime series management with async support, progress callbacks, and cancellation.
|
||||||
|
|
||||||
|
**Key Methods:**
|
||||||
|
|
||||||
|
- `search(words)` - Search for anime series
|
||||||
|
- `download(serie_folder, season, episode, key, language)` - Download an episode
|
||||||
|
- `rescan()` - Rescan directory for missing episodes
|
||||||
|
- `get_all_series_from_data_files()` - Load all series from data files in the anime directory (used for database sync on startup)
|
||||||
|
|
||||||
|
### Data File to Database Sync
|
||||||
|
|
||||||
|
On application startup, the system automatically syncs series from data files to the database:
|
||||||
|
|
||||||
|
1. After `download_service.initialize()` succeeds
|
||||||
|
2. `SeriesApp.get_all_series_from_data_files()` loads all series from `data` files
|
||||||
|
3. Each series is added to the database via `SerieList.add_to_db()`
|
||||||
|
4. Existing series are skipped (no duplicates)
|
||||||
|
5. Sync continues silently even if individual series fail
|
||||||
|
|
||||||
|
This ensures that series metadata stored in filesystem data files is available in the database for the web application.
|
||||||
|
|
||||||
### Callback System (`src/core/interfaces/callbacks.py`)
|
### Callback System (`src/core/interfaces/callbacks.py`)
|
||||||
|
|
||||||
- `ProgressCallback`, `ErrorCallback`, `CompletionCallback`
|
- `ProgressCallback`, `ErrorCallback`, `CompletionCallback`
|
||||||
|
|||||||
163
instructions.md
163
instructions.md
@ -120,3 +120,166 @@ For each task completed:
|
|||||||
- Good foundation for future enhancements if needed
|
- Good foundation for future enhancements if needed
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## 📋 TODO Tasks
|
||||||
|
|
||||||
|
### Task 1: Add `get_all_series_from_data_files()` Method to SeriesApp
|
||||||
|
|
||||||
|
**Status**: [x] Completed
|
||||||
|
|
||||||
|
**Description**: Add a new method to `SeriesApp` that returns all series data found in data files from the filesystem.
|
||||||
|
|
||||||
|
**File to Modify**: `src/core/SeriesApp.py`
|
||||||
|
|
||||||
|
**Requirements**:
|
||||||
|
|
||||||
|
1. Add a new method `get_all_series_from_data_files() -> List[Serie]` to `SeriesApp`
|
||||||
|
2. This method should scan the `directory_to_search` for all data files
|
||||||
|
3. Load and return all `Serie` objects found in data files
|
||||||
|
4. Use the existing `SerieList.load_series()` pattern for file discovery
|
||||||
|
5. Return an empty list if no data files are found
|
||||||
|
6. Include proper logging for debugging
|
||||||
|
7. Method should be synchronous (can be wrapped with `asyncio.to_thread` if needed)
|
||||||
|
|
||||||
|
**Implementation Details**:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def get_all_series_from_data_files(self) -> List[Serie]:
|
||||||
|
"""
|
||||||
|
Get all series from data files in the anime directory.
|
||||||
|
|
||||||
|
Scans the directory_to_search for all 'data' files and loads
|
||||||
|
the Serie metadata from each file.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
List of Serie objects found in data files
|
||||||
|
"""
|
||||||
|
# Use SerieList's file-based loading to get all series
|
||||||
|
# Return list of Serie objects from self.list.keyDict.values()
|
||||||
|
```
|
||||||
|
|
||||||
|
**Acceptance Criteria**:
|
||||||
|
|
||||||
|
- [x] Method exists in `SeriesApp`
|
||||||
|
- [x] Method returns `List[Serie]`
|
||||||
|
- [x] Method scans filesystem for data files
|
||||||
|
- [x] Proper error handling for missing/corrupt files
|
||||||
|
- [x] Logging added for operations
|
||||||
|
- [x] Unit tests written and passing
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 2: Sync Series from Data Files to Database on Setup Complete
|
||||||
|
|
||||||
|
**Status**: [x] Completed
|
||||||
|
|
||||||
|
**Description**: When the application setup is complete (anime directory configured), automatically sync all series from data files to the database.
|
||||||
|
|
||||||
|
**Files to Modify**:
|
||||||
|
|
||||||
|
- `src/server/fastapi_app.py` (lifespan function)
|
||||||
|
- `src/server/services/` (if needed for service layer)
|
||||||
|
|
||||||
|
**Requirements**:
|
||||||
|
|
||||||
|
1. After `download_service.initialize()` succeeds in the lifespan function
|
||||||
|
2. Call `SeriesApp.get_all_series_from_data_files()` to get all series
|
||||||
|
3. For each series, use `SerieList.add_to_db()` to save to database (uses existing DB schema)
|
||||||
|
4. Skip series that already exist in database (handled by `add_to_db`)
|
||||||
|
5. Log the sync progress and results
|
||||||
|
6. Do NOT modify database model definitions
|
||||||
|
|
||||||
|
**Implementation Details**:
|
||||||
|
|
||||||
|
```python
|
||||||
|
# In lifespan function, after download_service.initialize():
|
||||||
|
try:
|
||||||
|
from src.server.database.connection import get_db_session
|
||||||
|
|
||||||
|
# Get all series from data files using SeriesApp
|
||||||
|
series_app = SeriesApp(settings.anime_directory)
|
||||||
|
all_series = series_app.get_all_series_from_data_files()
|
||||||
|
|
||||||
|
if all_series:
|
||||||
|
async with get_db_session() as db:
|
||||||
|
serie_list = SerieList(settings.anime_directory, db_session=db, skip_load=True)
|
||||||
|
added_count = 0
|
||||||
|
for serie in all_series:
|
||||||
|
result = await serie_list.add_to_db(serie, db)
|
||||||
|
if result:
|
||||||
|
added_count += 1
|
||||||
|
await db.commit()
|
||||||
|
logger.info("Synced %d new series to database", added_count)
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning("Failed to sync series to database: %s", e)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Acceptance Criteria**:
|
||||||
|
|
||||||
|
- [x] Series from data files are synced to database on startup
|
||||||
|
- [x] Existing series in database are not duplicated
|
||||||
|
- [x] Database schema is NOT modified
|
||||||
|
- [x] Proper error handling (app continues even if sync fails)
|
||||||
|
- [x] Logging added for sync operations
|
||||||
|
- [x] Integration tests written and passing
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 3: Validation - Verify Data File to Database Sync
|
||||||
|
|
||||||
|
**Status**: [x] Completed
|
||||||
|
|
||||||
|
**Description**: Create validation tests to ensure the data file to database sync works correctly.
|
||||||
|
|
||||||
|
**File to Create**: `tests/integration/test_data_file_db_sync.py`
|
||||||
|
|
||||||
|
**Requirements**:
|
||||||
|
|
||||||
|
1. Test `get_all_series_from_data_files()` returns correct data
|
||||||
|
2. Test that series are correctly added to database
|
||||||
|
3. Test that duplicate series are not created
|
||||||
|
4. Test that sync handles empty directories gracefully
|
||||||
|
5. Test that sync handles corrupt data files gracefully
|
||||||
|
6. Test end-to-end startup sync behavior
|
||||||
|
|
||||||
|
**Test Cases**:
|
||||||
|
|
||||||
|
```python
|
||||||
|
class TestDataFileDbSync:
|
||||||
|
"""Test data file to database synchronization."""
|
||||||
|
|
||||||
|
async def test_get_all_series_from_data_files_returns_list(self):
|
||||||
|
"""Test that get_all_series_from_data_files returns a list."""
|
||||||
|
pass
|
||||||
|
|
||||||
|
async def test_get_all_series_from_data_files_empty_directory(self):
|
||||||
|
"""Test behavior with empty anime directory."""
|
||||||
|
pass
|
||||||
|
|
||||||
|
async def test_series_sync_to_db_creates_records(self):
|
||||||
|
"""Test that series are correctly synced to database."""
|
||||||
|
pass
|
||||||
|
|
||||||
|
async def test_series_sync_to_db_no_duplicates(self):
|
||||||
|
"""Test that duplicate series are not created."""
|
||||||
|
pass
|
||||||
|
|
||||||
|
async def test_series_sync_handles_corrupt_files(self):
|
||||||
|
"""Test that corrupt data files don't crash the sync."""
|
||||||
|
pass
|
||||||
|
|
||||||
|
async def test_startup_sync_integration(self):
|
||||||
|
"""Test end-to-end startup sync behavior."""
|
||||||
|
pass
|
||||||
|
```
|
||||||
|
|
||||||
|
**Acceptance Criteria**:
|
||||||
|
|
||||||
|
- [x] All test cases implemented
|
||||||
|
- [x] Tests use pytest async fixtures
|
||||||
|
- [x] Tests use temporary directories for isolation
|
||||||
|
- [x] Tests cover happy path and error cases
|
||||||
|
- [x] All tests passing
|
||||||
|
- [x] Code coverage > 80% for new code
|
||||||
|
|
||||||
|
---
|
||||||
|
|||||||
@ -599,3 +599,56 @@ class SeriesApp:
|
|||||||
looks up series by their unique key, not by folder name.
|
looks up series by their unique key, not by folder name.
|
||||||
"""
|
"""
|
||||||
return self.list.get_by_key(key)
|
return self.list.get_by_key(key)
|
||||||
|
|
||||||
|
def get_all_series_from_data_files(self) -> List[Serie]:
|
||||||
|
"""
|
||||||
|
Get all series from data files in the anime directory.
|
||||||
|
|
||||||
|
Scans the directory_to_search for all 'data' files and loads
|
||||||
|
the Serie metadata from each file. This method is synchronous
|
||||||
|
and can be wrapped with asyncio.to_thread if needed for async
|
||||||
|
contexts.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
List of Serie objects found in data files. Returns an empty
|
||||||
|
list if no data files are found or if the directory doesn't
|
||||||
|
exist.
|
||||||
|
|
||||||
|
Example:
|
||||||
|
series_app = SeriesApp("/path/to/anime")
|
||||||
|
all_series = series_app.get_all_series_from_data_files()
|
||||||
|
for serie in all_series:
|
||||||
|
print(f"Found: {serie.name} (key={serie.key})")
|
||||||
|
"""
|
||||||
|
logger.info(
|
||||||
|
"Scanning for data files in directory: %s",
|
||||||
|
self.directory_to_search
|
||||||
|
)
|
||||||
|
|
||||||
|
# Create a fresh SerieList instance for file-based loading
|
||||||
|
# This ensures we get all series from data files without
|
||||||
|
# interfering with the main instance's state
|
||||||
|
try:
|
||||||
|
temp_list = SerieList(
|
||||||
|
self.directory_to_search,
|
||||||
|
db_session=None, # Force file-based loading
|
||||||
|
skip_load=False # Allow automatic loading
|
||||||
|
)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(
|
||||||
|
"Failed to scan directory for data files: %s",
|
||||||
|
str(e),
|
||||||
|
exc_info=True
|
||||||
|
)
|
||||||
|
return []
|
||||||
|
|
||||||
|
# Get all series from the temporary list
|
||||||
|
all_series = temp_list.get_all()
|
||||||
|
|
||||||
|
logger.info(
|
||||||
|
"Found %d series from data files in %s",
|
||||||
|
len(all_series),
|
||||||
|
self.directory_to_search
|
||||||
|
)
|
||||||
|
|
||||||
|
return all_series
|
||||||
|
|||||||
@ -41,6 +41,78 @@ from src.server.services.websocket_service import get_websocket_service
|
|||||||
# module-level globals. This makes testing and multi-instance hosting safer.
|
# module-level globals. This makes testing and multi-instance hosting safer.
|
||||||
|
|
||||||
|
|
||||||
|
async def _sync_series_to_database(
|
||||||
|
anime_directory: str,
|
||||||
|
logger
|
||||||
|
) -> int:
|
||||||
|
"""
|
||||||
|
Sync series from data files to the database.
|
||||||
|
|
||||||
|
Scans the anime directory for data files and adds any new series
|
||||||
|
to the database. Existing series are skipped (no duplicates).
|
||||||
|
|
||||||
|
Args:
|
||||||
|
anime_directory: Path to the anime directory with data files
|
||||||
|
logger: Logger instance for logging operations
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Number of new series added to the database
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
import asyncio
|
||||||
|
|
||||||
|
from src.core.entities.SerieList import SerieList
|
||||||
|
from src.core.SeriesApp import SeriesApp
|
||||||
|
from src.server.database.connection import get_db_session
|
||||||
|
|
||||||
|
# Get all series from data files using SeriesApp
|
||||||
|
series_app = SeriesApp(anime_directory)
|
||||||
|
all_series = await asyncio.to_thread(
|
||||||
|
series_app.get_all_series_from_data_files
|
||||||
|
)
|
||||||
|
|
||||||
|
if not all_series:
|
||||||
|
logger.info("No series found in data files to sync")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
logger.info(
|
||||||
|
"Found %d series in data files, syncing to database...",
|
||||||
|
len(all_series)
|
||||||
|
)
|
||||||
|
|
||||||
|
async with get_db_session() as db:
|
||||||
|
serie_list = SerieList(
|
||||||
|
anime_directory,
|
||||||
|
db_session=db,
|
||||||
|
skip_load=True
|
||||||
|
)
|
||||||
|
added_count = 0
|
||||||
|
for serie in all_series:
|
||||||
|
result = await serie_list.add_to_db(serie, db)
|
||||||
|
if result:
|
||||||
|
added_count += 1
|
||||||
|
logger.debug(
|
||||||
|
"Added series to database: %s (key=%s)",
|
||||||
|
serie.name,
|
||||||
|
serie.key
|
||||||
|
)
|
||||||
|
# Commit happens automatically via get_db_session context
|
||||||
|
logger.info(
|
||||||
|
"Synced %d new series to database (skipped %d existing)",
|
||||||
|
added_count,
|
||||||
|
len(all_series) - added_count
|
||||||
|
)
|
||||||
|
return added_count
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(
|
||||||
|
"Failed to sync series to database: %s",
|
||||||
|
e,
|
||||||
|
exc_info=True
|
||||||
|
)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
@asynccontextmanager
|
@asynccontextmanager
|
||||||
async def lifespan(app: FastAPI):
|
async def lifespan(app: FastAPI):
|
||||||
"""Manage application lifespan (startup and shutdown)."""
|
"""Manage application lifespan (startup and shutdown)."""
|
||||||
@ -104,6 +176,11 @@ async def lifespan(app: FastAPI):
|
|||||||
download_service = get_download_service()
|
download_service = get_download_service()
|
||||||
await download_service.initialize()
|
await download_service.initialize()
|
||||||
logger.info("Download service initialized and queue restored")
|
logger.info("Download service initialized and queue restored")
|
||||||
|
|
||||||
|
# Sync series from data files to database
|
||||||
|
await _sync_series_to_database(
|
||||||
|
settings.anime_directory, logger
|
||||||
|
)
|
||||||
else:
|
else:
|
||||||
logger.info(
|
logger.info(
|
||||||
"Download service initialization skipped - "
|
"Download service initialization skipped - "
|
||||||
|
|||||||
350
tests/integration/test_data_file_db_sync.py
Normal file
350
tests/integration/test_data_file_db_sync.py
Normal file
@ -0,0 +1,350 @@
|
|||||||
|
"""Integration tests for data file to database synchronization.
|
||||||
|
|
||||||
|
This module verifies that the data file to database sync functionality
|
||||||
|
works correctly, including:
|
||||||
|
- Loading series from data files
|
||||||
|
- Adding series to the database
|
||||||
|
- Preventing duplicate entries
|
||||||
|
- Handling corrupt or missing files gracefully
|
||||||
|
- End-to-end startup sync behavior
|
||||||
|
|
||||||
|
The sync functionality allows existing series metadata stored in
|
||||||
|
data files to be automatically imported into the database during
|
||||||
|
application startup.
|
||||||
|
"""
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import tempfile
|
||||||
|
from unittest.mock import AsyncMock, Mock, patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from src.core.entities.SerieList import SerieList
|
||||||
|
from src.core.entities.series import Serie
|
||||||
|
from src.core.SeriesApp import SeriesApp
|
||||||
|
|
||||||
|
|
||||||
|
class TestGetAllSeriesFromDataFiles:
|
||||||
|
"""Test SeriesApp.get_all_series_from_data_files() method."""
|
||||||
|
|
||||||
|
def test_returns_empty_list_for_empty_directory(self):
|
||||||
|
"""Test that empty directory returns empty list."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp_dir:
|
||||||
|
with patch('src.core.SeriesApp.Loaders'), \
|
||||||
|
patch('src.core.SeriesApp.SerieScanner'):
|
||||||
|
app = SeriesApp(tmp_dir)
|
||||||
|
result = app.get_all_series_from_data_files()
|
||||||
|
|
||||||
|
assert isinstance(result, list)
|
||||||
|
assert len(result) == 0
|
||||||
|
|
||||||
|
def test_returns_series_from_data_files(self):
|
||||||
|
"""Test that valid data files are loaded correctly."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp_dir:
|
||||||
|
# Create test data files
|
||||||
|
_create_test_data_file(
|
||||||
|
tmp_dir,
|
||||||
|
folder="Anime Test 1",
|
||||||
|
key="anime-test-1",
|
||||||
|
name="Anime Test 1",
|
||||||
|
episodes={1: [1, 2, 3]}
|
||||||
|
)
|
||||||
|
_create_test_data_file(
|
||||||
|
tmp_dir,
|
||||||
|
folder="Anime Test 2",
|
||||||
|
key="anime-test-2",
|
||||||
|
name="Anime Test 2",
|
||||||
|
episodes={1: [1]}
|
||||||
|
)
|
||||||
|
|
||||||
|
with patch('src.core.SeriesApp.Loaders'), \
|
||||||
|
patch('src.core.SeriesApp.SerieScanner'):
|
||||||
|
app = SeriesApp(tmp_dir)
|
||||||
|
result = app.get_all_series_from_data_files()
|
||||||
|
|
||||||
|
assert isinstance(result, list)
|
||||||
|
assert len(result) == 2
|
||||||
|
keys = {s.key for s in result}
|
||||||
|
assert "anime-test-1" in keys
|
||||||
|
assert "anime-test-2" in keys
|
||||||
|
|
||||||
|
def test_handles_corrupt_data_files_gracefully(self):
|
||||||
|
"""Test that corrupt data files don't crash the sync."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp_dir:
|
||||||
|
# Create a valid data file
|
||||||
|
_create_test_data_file(
|
||||||
|
tmp_dir,
|
||||||
|
folder="Valid Anime",
|
||||||
|
key="valid-anime",
|
||||||
|
name="Valid Anime",
|
||||||
|
episodes={1: [1]}
|
||||||
|
)
|
||||||
|
|
||||||
|
# Create a corrupt data file (invalid JSON)
|
||||||
|
corrupt_dir = os.path.join(tmp_dir, "Corrupt Anime")
|
||||||
|
os.makedirs(corrupt_dir, exist_ok=True)
|
||||||
|
with open(os.path.join(corrupt_dir, "data"), "w") as f:
|
||||||
|
f.write("this is not valid json {{{")
|
||||||
|
|
||||||
|
with patch('src.core.SeriesApp.Loaders'), \
|
||||||
|
patch('src.core.SeriesApp.SerieScanner'):
|
||||||
|
app = SeriesApp(tmp_dir)
|
||||||
|
result = app.get_all_series_from_data_files()
|
||||||
|
|
||||||
|
# Should still return the valid series
|
||||||
|
assert isinstance(result, list)
|
||||||
|
assert len(result) >= 1
|
||||||
|
# The valid anime should be loaded
|
||||||
|
keys = {s.key for s in result}
|
||||||
|
assert "valid-anime" in keys
|
||||||
|
|
||||||
|
def test_handles_missing_directory_gracefully(self):
|
||||||
|
"""Test that non-existent directory returns empty list."""
|
||||||
|
non_existent_dir = "/non/existent/directory/path"
|
||||||
|
|
||||||
|
with patch('src.core.SeriesApp.Loaders'), \
|
||||||
|
patch('src.core.SeriesApp.SerieScanner'):
|
||||||
|
app = SeriesApp(non_existent_dir)
|
||||||
|
result = app.get_all_series_from_data_files()
|
||||||
|
|
||||||
|
assert isinstance(result, list)
|
||||||
|
assert len(result) == 0
|
||||||
|
|
||||||
|
|
||||||
|
class TestSerieListAddToDb:
|
||||||
|
"""Test SerieList.add_to_db() method for database insertion."""
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_add_to_db_creates_record(self):
|
||||||
|
"""Test that add_to_db creates a database record."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp_dir:
|
||||||
|
serie = Serie(
|
||||||
|
key="new-anime",
|
||||||
|
name="New Anime",
|
||||||
|
site="https://aniworld.to",
|
||||||
|
folder="New Anime (2024)",
|
||||||
|
episodeDict={1: [1, 2, 3], 2: [1, 2]}
|
||||||
|
)
|
||||||
|
|
||||||
|
# Mock database session and services
|
||||||
|
mock_db = AsyncMock()
|
||||||
|
mock_anime_series = Mock()
|
||||||
|
mock_anime_series.id = 1
|
||||||
|
mock_anime_series.key = "new-anime"
|
||||||
|
mock_anime_series.name = "New Anime"
|
||||||
|
|
||||||
|
with patch(
|
||||||
|
'src.server.database.service.AnimeSeriesService'
|
||||||
|
) as mock_service, patch(
|
||||||
|
'src.server.database.service.EpisodeService'
|
||||||
|
) as mock_episode_service:
|
||||||
|
# Setup mocks
|
||||||
|
mock_service.get_by_key = AsyncMock(return_value=None)
|
||||||
|
mock_service.create = AsyncMock(return_value=mock_anime_series)
|
||||||
|
mock_episode_service.create = AsyncMock()
|
||||||
|
|
||||||
|
serie_list = SerieList(tmp_dir, skip_load=True)
|
||||||
|
result = await serie_list.add_to_db(serie, mock_db)
|
||||||
|
|
||||||
|
# Verify series was created
|
||||||
|
assert result is not None
|
||||||
|
mock_service.create.assert_called_once()
|
||||||
|
|
||||||
|
# Verify episodes were created (5 total: 3 + 2)
|
||||||
|
assert mock_episode_service.create.call_count == 5
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_add_to_db_skips_existing_series(self):
|
||||||
|
"""Test that add_to_db skips existing series."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp_dir:
|
||||||
|
serie = Serie(
|
||||||
|
key="existing-anime",
|
||||||
|
name="Existing Anime",
|
||||||
|
site="https://aniworld.to",
|
||||||
|
folder="Existing Anime (2023)",
|
||||||
|
episodeDict={1: [1]}
|
||||||
|
)
|
||||||
|
|
||||||
|
mock_db = AsyncMock()
|
||||||
|
mock_existing = Mock()
|
||||||
|
mock_existing.id = 99
|
||||||
|
mock_existing.key = "existing-anime"
|
||||||
|
|
||||||
|
with patch(
|
||||||
|
'src.server.database.service.AnimeSeriesService'
|
||||||
|
) as mock_service:
|
||||||
|
# Return existing series
|
||||||
|
mock_service.get_by_key = AsyncMock(return_value=mock_existing)
|
||||||
|
mock_service.create = AsyncMock()
|
||||||
|
|
||||||
|
serie_list = SerieList(tmp_dir, skip_load=True)
|
||||||
|
result = await serie_list.add_to_db(serie, mock_db)
|
||||||
|
|
||||||
|
# Verify None returned (already exists)
|
||||||
|
assert result is None
|
||||||
|
# Verify create was NOT called
|
||||||
|
mock_service.create.assert_not_called()
|
||||||
|
|
||||||
|
|
||||||
|
class TestSyncSeriesToDatabase:
|
||||||
|
"""Test _sync_series_to_database function from fastapi_app."""
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_sync_with_empty_directory(self):
|
||||||
|
"""Test sync with empty anime directory."""
|
||||||
|
from src.server.fastapi_app import _sync_series_to_database
|
||||||
|
|
||||||
|
with tempfile.TemporaryDirectory() as tmp_dir:
|
||||||
|
mock_logger = Mock()
|
||||||
|
|
||||||
|
with patch('src.core.SeriesApp.Loaders'), \
|
||||||
|
patch('src.core.SeriesApp.SerieScanner'):
|
||||||
|
count = await _sync_series_to_database(tmp_dir, mock_logger)
|
||||||
|
|
||||||
|
assert count == 0
|
||||||
|
# Should log that no series were found
|
||||||
|
mock_logger.info.assert_called()
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_sync_adds_new_series_to_database(self):
|
||||||
|
"""Test that sync adds new series to database.
|
||||||
|
|
||||||
|
This is a more realistic test that verifies series data is loaded
|
||||||
|
from files and the sync function attempts to add them to the DB.
|
||||||
|
The actual DB interaction is tested in test_add_to_db_creates_record.
|
||||||
|
"""
|
||||||
|
from src.server.fastapi_app import _sync_series_to_database
|
||||||
|
|
||||||
|
with tempfile.TemporaryDirectory() as tmp_dir:
|
||||||
|
# Create test data files
|
||||||
|
_create_test_data_file(
|
||||||
|
tmp_dir,
|
||||||
|
folder="Sync Test Anime",
|
||||||
|
key="sync-test-anime",
|
||||||
|
name="Sync Test Anime",
|
||||||
|
episodes={1: [1, 2]}
|
||||||
|
)
|
||||||
|
|
||||||
|
mock_logger = Mock()
|
||||||
|
|
||||||
|
# First verify that we can load the series from files
|
||||||
|
with patch('src.core.SeriesApp.Loaders'), \
|
||||||
|
patch('src.core.SeriesApp.SerieScanner'):
|
||||||
|
app = SeriesApp(tmp_dir)
|
||||||
|
series = app.get_all_series_from_data_files()
|
||||||
|
assert len(series) == 1
|
||||||
|
assert series[0].key == "sync-test-anime"
|
||||||
|
|
||||||
|
# Now test that the sync function loads series and handles DB
|
||||||
|
# gracefully (even if DB operations fail, it should not crash)
|
||||||
|
with patch('src.core.SeriesApp.Loaders'), \
|
||||||
|
patch('src.core.SeriesApp.SerieScanner'):
|
||||||
|
# The function should return 0 because DB isn't available
|
||||||
|
# but should not crash
|
||||||
|
count = await _sync_series_to_database(tmp_dir, mock_logger)
|
||||||
|
|
||||||
|
# Since no real DB, it will fail gracefully
|
||||||
|
assert isinstance(count, int)
|
||||||
|
# Should have logged something
|
||||||
|
assert mock_logger.info.called or mock_logger.warning.called
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_sync_handles_exceptions_gracefully(self):
|
||||||
|
"""Test that sync handles exceptions without crashing."""
|
||||||
|
from src.server.fastapi_app import _sync_series_to_database
|
||||||
|
|
||||||
|
mock_logger = Mock()
|
||||||
|
|
||||||
|
# Make SeriesApp raise an exception during initialization
|
||||||
|
with patch('src.core.SeriesApp.Loaders'), \
|
||||||
|
patch('src.core.SeriesApp.SerieScanner'), \
|
||||||
|
patch(
|
||||||
|
'src.core.SeriesApp.SerieList',
|
||||||
|
side_effect=Exception("Test error")
|
||||||
|
):
|
||||||
|
count = await _sync_series_to_database(
|
||||||
|
"/fake/path", mock_logger
|
||||||
|
)
|
||||||
|
|
||||||
|
assert count == 0
|
||||||
|
# Should log the warning
|
||||||
|
mock_logger.warning.assert_called()
|
||||||
|
|
||||||
|
|
||||||
|
class TestEndToEndSync:
|
||||||
|
"""End-to-end tests for the sync functionality."""
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_startup_sync_integration(self):
|
||||||
|
"""Test end-to-end startup sync behavior."""
|
||||||
|
# This test verifies the integration of all components
|
||||||
|
with tempfile.TemporaryDirectory() as tmp_dir:
|
||||||
|
# Create test data
|
||||||
|
_create_test_data_file(
|
||||||
|
tmp_dir,
|
||||||
|
folder="E2E Test Anime 1",
|
||||||
|
key="e2e-test-anime-1",
|
||||||
|
name="E2E Test Anime 1",
|
||||||
|
episodes={1: [1, 2, 3]}
|
||||||
|
)
|
||||||
|
_create_test_data_file(
|
||||||
|
tmp_dir,
|
||||||
|
folder="E2E Test Anime 2",
|
||||||
|
key="e2e-test-anime-2",
|
||||||
|
name="E2E Test Anime 2",
|
||||||
|
episodes={1: [1], 2: [1, 2]}
|
||||||
|
)
|
||||||
|
|
||||||
|
# Use SeriesApp to load series from files
|
||||||
|
with patch('src.core.SeriesApp.Loaders'), \
|
||||||
|
patch('src.core.SeriesApp.SerieScanner'):
|
||||||
|
app = SeriesApp(tmp_dir)
|
||||||
|
all_series = app.get_all_series_from_data_files()
|
||||||
|
|
||||||
|
# Verify all series were loaded
|
||||||
|
assert len(all_series) == 2
|
||||||
|
|
||||||
|
# Verify series data is correct
|
||||||
|
series_by_key = {s.key: s for s in all_series}
|
||||||
|
assert "e2e-test-anime-1" in series_by_key
|
||||||
|
assert "e2e-test-anime-2" in series_by_key
|
||||||
|
|
||||||
|
# Verify episode data
|
||||||
|
anime1 = series_by_key["e2e-test-anime-1"]
|
||||||
|
assert anime1.episodeDict == {1: [1, 2, 3]}
|
||||||
|
|
||||||
|
anime2 = series_by_key["e2e-test-anime-2"]
|
||||||
|
assert anime2.episodeDict == {1: [1], 2: [1, 2]}
|
||||||
|
|
||||||
|
|
||||||
|
def _create_test_data_file(
|
||||||
|
base_dir: str,
|
||||||
|
folder: str,
|
||||||
|
key: str,
|
||||||
|
name: str,
|
||||||
|
episodes: dict
|
||||||
|
) -> None:
|
||||||
|
"""
|
||||||
|
Create a test data file in the anime directory.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
base_dir: Base directory for anime folders
|
||||||
|
folder: Folder name for the anime
|
||||||
|
key: Unique key for the series
|
||||||
|
name: Display name of the series
|
||||||
|
episodes: Dictionary mapping season to list of episode numbers
|
||||||
|
"""
|
||||||
|
anime_dir = os.path.join(base_dir, folder)
|
||||||
|
os.makedirs(anime_dir, exist_ok=True)
|
||||||
|
|
||||||
|
data = {
|
||||||
|
"key": key,
|
||||||
|
"name": name,
|
||||||
|
"site": "https://aniworld.to",
|
||||||
|
"folder": folder,
|
||||||
|
"episodeDict": {str(k): v for k, v in episodes.items()}
|
||||||
|
}
|
||||||
|
|
||||||
|
data_file = os.path.join(anime_dir, "data")
|
||||||
|
with open(data_file, "w", encoding="utf-8") as f:
|
||||||
|
json.dump(data, f, indent=2)
|
||||||
@ -559,3 +559,196 @@ class TestSeriesAppAsyncDbInit:
|
|||||||
assert len(w) == 1
|
assert len(w) == 1
|
||||||
assert "without db_session" in str(w[0].message)
|
assert "without db_session" in str(w[0].message)
|
||||||
|
|
||||||
|
|
||||||
|
class TestSeriesAppGetAllSeriesFromDataFiles:
|
||||||
|
"""Test get_all_series_from_data_files() functionality."""
|
||||||
|
|
||||||
|
@patch('src.core.SeriesApp.Loaders')
|
||||||
|
@patch('src.core.SeriesApp.SerieScanner')
|
||||||
|
@patch('src.core.SeriesApp.SerieList')
|
||||||
|
def test_returns_list_of_series(
|
||||||
|
self, mock_serie_list_class, mock_scanner, mock_loaders
|
||||||
|
):
|
||||||
|
"""Test that get_all_series_from_data_files returns a list of Serie."""
|
||||||
|
from src.core.entities.series import Serie
|
||||||
|
|
||||||
|
test_dir = "/test/anime"
|
||||||
|
|
||||||
|
# Mock series to return
|
||||||
|
mock_series = [
|
||||||
|
Serie(
|
||||||
|
key="anime1",
|
||||||
|
name="Anime 1",
|
||||||
|
site="https://aniworld.to",
|
||||||
|
folder="Anime 1 (2020)",
|
||||||
|
episodeDict={1: [1, 2, 3]}
|
||||||
|
),
|
||||||
|
Serie(
|
||||||
|
key="anime2",
|
||||||
|
name="Anime 2",
|
||||||
|
site="https://aniworld.to",
|
||||||
|
folder="Anime 2 (2021)",
|
||||||
|
episodeDict={1: [1]}
|
||||||
|
),
|
||||||
|
]
|
||||||
|
|
||||||
|
# Setup mock for the main SerieList instance (constructor call)
|
||||||
|
mock_main_list = Mock()
|
||||||
|
mock_main_list.GetMissingEpisode.return_value = []
|
||||||
|
|
||||||
|
# Setup mock for temporary SerieList in get_all_series_from_data_files
|
||||||
|
mock_temp_list = Mock()
|
||||||
|
mock_temp_list.get_all.return_value = mock_series
|
||||||
|
|
||||||
|
# Return different mocks for the two calls
|
||||||
|
mock_serie_list_class.side_effect = [mock_main_list, mock_temp_list]
|
||||||
|
|
||||||
|
# Create app
|
||||||
|
app = SeriesApp(test_dir)
|
||||||
|
|
||||||
|
# Call the method
|
||||||
|
result = app.get_all_series_from_data_files()
|
||||||
|
|
||||||
|
# Verify result is a list of Serie
|
||||||
|
assert isinstance(result, list)
|
||||||
|
assert len(result) == 2
|
||||||
|
assert all(isinstance(s, Serie) for s in result)
|
||||||
|
assert result[0].key == "anime1"
|
||||||
|
assert result[1].key == "anime2"
|
||||||
|
|
||||||
|
@patch('src.core.SeriesApp.Loaders')
|
||||||
|
@patch('src.core.SeriesApp.SerieScanner')
|
||||||
|
@patch('src.core.SeriesApp.SerieList')
|
||||||
|
def test_returns_empty_list_when_no_data_files(
|
||||||
|
self, mock_serie_list_class, mock_scanner, mock_loaders
|
||||||
|
):
|
||||||
|
"""Test that empty list is returned when no data files exist."""
|
||||||
|
test_dir = "/test/anime"
|
||||||
|
|
||||||
|
# Setup mock for the main SerieList instance
|
||||||
|
mock_main_list = Mock()
|
||||||
|
mock_main_list.GetMissingEpisode.return_value = []
|
||||||
|
|
||||||
|
# Setup mock for the temporary SerieList (empty directory)
|
||||||
|
mock_temp_list = Mock()
|
||||||
|
mock_temp_list.get_all.return_value = []
|
||||||
|
|
||||||
|
mock_serie_list_class.side_effect = [mock_main_list, mock_temp_list]
|
||||||
|
|
||||||
|
# Create app
|
||||||
|
app = SeriesApp(test_dir)
|
||||||
|
|
||||||
|
# Call the method
|
||||||
|
result = app.get_all_series_from_data_files()
|
||||||
|
|
||||||
|
# Verify empty list is returned
|
||||||
|
assert isinstance(result, list)
|
||||||
|
assert len(result) == 0
|
||||||
|
|
||||||
|
@patch('src.core.SeriesApp.Loaders')
|
||||||
|
@patch('src.core.SeriesApp.SerieScanner')
|
||||||
|
@patch('src.core.SeriesApp.SerieList')
|
||||||
|
def test_handles_exception_gracefully(
|
||||||
|
self, mock_serie_list_class, mock_scanner, mock_loaders
|
||||||
|
):
|
||||||
|
"""Test exceptions are handled gracefully and empty list returned."""
|
||||||
|
test_dir = "/test/anime"
|
||||||
|
|
||||||
|
# Setup mock for the main SerieList instance
|
||||||
|
mock_main_list = Mock()
|
||||||
|
mock_main_list.GetMissingEpisode.return_value = []
|
||||||
|
|
||||||
|
# Make the second SerieList constructor raise an exception
|
||||||
|
mock_serie_list_class.side_effect = [
|
||||||
|
mock_main_list,
|
||||||
|
OSError("Directory not found")
|
||||||
|
]
|
||||||
|
|
||||||
|
# Create app
|
||||||
|
app = SeriesApp(test_dir)
|
||||||
|
|
||||||
|
# Call the method - should not raise
|
||||||
|
result = app.get_all_series_from_data_files()
|
||||||
|
|
||||||
|
# Verify empty list is returned on error
|
||||||
|
assert isinstance(result, list)
|
||||||
|
assert len(result) == 0
|
||||||
|
|
||||||
|
@patch('src.core.SeriesApp.Loaders')
|
||||||
|
@patch('src.core.SeriesApp.SerieScanner')
|
||||||
|
@patch('src.core.SeriesApp.SerieList')
|
||||||
|
def test_uses_file_based_loading(
|
||||||
|
self, mock_serie_list_class, mock_scanner, mock_loaders
|
||||||
|
):
|
||||||
|
"""Test that method uses file-based loading (no db_session)."""
|
||||||
|
test_dir = "/test/anime"
|
||||||
|
|
||||||
|
# Setup mock for the main SerieList instance
|
||||||
|
mock_main_list = Mock()
|
||||||
|
mock_main_list.GetMissingEpisode.return_value = []
|
||||||
|
|
||||||
|
# Setup mock for the temporary SerieList
|
||||||
|
mock_temp_list = Mock()
|
||||||
|
mock_temp_list.get_all.return_value = []
|
||||||
|
|
||||||
|
mock_serie_list_class.side_effect = [mock_main_list, mock_temp_list]
|
||||||
|
|
||||||
|
# Create app
|
||||||
|
app = SeriesApp(test_dir)
|
||||||
|
|
||||||
|
# Call the method
|
||||||
|
app.get_all_series_from_data_files()
|
||||||
|
|
||||||
|
# Verify the second SerieList was created with correct params
|
||||||
|
# (file-based loading: db_session=None, skip_load=False)
|
||||||
|
calls = mock_serie_list_class.call_args_list
|
||||||
|
assert len(calls) == 2
|
||||||
|
|
||||||
|
# Check the second call (for get_all_series_from_data_files)
|
||||||
|
second_call = calls[1]
|
||||||
|
assert second_call.kwargs.get('db_session') is None
|
||||||
|
assert second_call.kwargs.get('skip_load') is False
|
||||||
|
|
||||||
|
@patch('src.core.SeriesApp.Loaders')
|
||||||
|
@patch('src.core.SeriesApp.SerieScanner')
|
||||||
|
@patch('src.core.SeriesApp.SerieList')
|
||||||
|
def test_does_not_modify_main_list(
|
||||||
|
self, mock_serie_list_class, mock_scanner, mock_loaders
|
||||||
|
):
|
||||||
|
"""Test that method does not modify the main SerieList instance."""
|
||||||
|
from src.core.entities.series import Serie
|
||||||
|
|
||||||
|
test_dir = "/test/anime"
|
||||||
|
|
||||||
|
# Setup mock for the main SerieList instance
|
||||||
|
mock_main_list = Mock()
|
||||||
|
mock_main_list.GetMissingEpisode.return_value = []
|
||||||
|
mock_main_list.get_all.return_value = []
|
||||||
|
|
||||||
|
# Setup mock for the temporary SerieList
|
||||||
|
mock_temp_list = Mock()
|
||||||
|
mock_temp_list.get_all.return_value = [
|
||||||
|
Serie(
|
||||||
|
key="anime1",
|
||||||
|
name="Anime 1",
|
||||||
|
site="https://aniworld.to",
|
||||||
|
folder="Anime 1",
|
||||||
|
episodeDict={}
|
||||||
|
)
|
||||||
|
]
|
||||||
|
|
||||||
|
mock_serie_list_class.side_effect = [mock_main_list, mock_temp_list]
|
||||||
|
|
||||||
|
# Create app
|
||||||
|
app = SeriesApp(test_dir)
|
||||||
|
|
||||||
|
# Store reference to original list
|
||||||
|
original_list = app.list
|
||||||
|
|
||||||
|
# Call the method
|
||||||
|
app.get_all_series_from_data_files()
|
||||||
|
|
||||||
|
# Verify main list is unchanged
|
||||||
|
assert app.list is original_list
|
||||||
|
# Verify the main list's get_all was not called
|
||||||
|
mock_main_list.get_all.assert_not_called()
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user