This commit is contained in:
Lukas 2025-10-22 17:39:28 +02:00
parent 6db850c2ad
commit 5c2691b070
7 changed files with 198 additions and 181 deletions

View File

@ -112,9 +112,6 @@ conda run -n AniWorld python -m pytest tests/ -v -s
#### Unclear Comments or Missing Context #### Unclear Comments or Missing Context
- [ ] `src/server/api/download.py` line 51
- Backward compatibility comment clear but needs more detail
--- ---
### 4⃣ Complex Logic Commented ### 4⃣ Complex Logic Commented
@ -131,38 +128,27 @@ conda run -n AniWorld python -m pytest tests/ -v -s
**Duplicate Code** **Duplicate Code**
- [ ] `src/cli/Main.py` vs `src/core/SeriesApp.py`
- Entire `SeriesApp` class duplicated (89 lines vs 590 lines)
- CLI version is oversimplified - should use core version
- Line 35-50 in CLI duplicates core initialization logic
- [ ] `src/core/providers/aniworld_provider.py` vs `src/core/providers/enhanced_provider.py` - [ ] `src/core/providers/aniworld_provider.py` vs `src/core/providers/enhanced_provider.py`
- Headers dictionary duplicated (lines 38-50 similar) - Headers dictionary duplicated (lines 38-50 similar) -> completed
- Provider list duplicated (line 38 vs line 45) - Provider list duplicated (line 38 vs line 45) -> completed
- User-Agent strings duplicated - User-Agent strings duplicated -> completed
**Global Variables (Temporary Storage)** **Global Variables (Temporary Storage)**
- [ ] `src/server/fastapi_app.py` line 73 - [ ] `src/server/fastapi_app.py` line 73 -> completed
- `series_app: Optional[SeriesApp] = None` global storage - `series_app: Optional[SeriesApp] = None` global storage
- Should use FastAPI dependency injection instead - Should use FastAPI dependency injection instead
- Problematic for testing and multiple instances - Problematic for testing and multiple instances
**Logging Configuration Workarounds** **Logging Configuration Workarounds**
- [ ] `src/cli/Main.py` lines 12-22 -- [ ] `src/cli/Main.py` lines 12-22 -> reviewed (no manual handler removal found) - Manual logger handler removal is hacky - `for h in logging.root.handlers: logging.root.removeHandler(h)` is a hack - Should use proper logging configuration - Multiple loggers created with file handlers at odd paths (line 26)
- Manual logger handler removal is hacky
- `for h in logging.root.handlers: logging.root.removeHandler(h)` is a hack
- Should use proper logging configuration
- Multiple loggers created with file handlers at odd paths (line 26)
**Hardcoded Values** **Hardcoded Values**
- [ ] `src/core/providers/aniworld_provider.py` line 22 -- [ ] `src/core/providers/aniworld_provider.py` line 22 -> completed - `timeout = int(os.getenv("DOWNLOAD_TIMEOUT", 600))` at module level - Should be in settings class
- `timeout = int(os.getenv("DOWNLOAD_TIMEOUT", 600))` at module level -- [ ] `src/core/providers/aniworld_provider.py` lines 38, 47 -> completed - User-Agent strings hardcoded - Provider list hardcoded
- Should be in settings class
- [ ] `src/core/providers/aniworld_provider.py` lines 38, 47
- User-Agent strings hardcoded
- Provider list hardcoded
- [ ] `src/cli/Main.py` line 227 - [ ] `src/cli/Main.py` line 227
- Network path hardcoded: `"\\\\sshfs.r\\ubuntu@192.168.178.43\\media\\serien\\Serien"` - Network path hardcoded: `"\\\\sshfs.r\\ubuntu@192.168.178.43\\media\\serien\\Serien"`
- Should be configuration - Should be configuration
@ -170,27 +156,27 @@ conda run -n AniWorld python -m pytest tests/ -v -s
**Exception Handling Shortcuts** **Exception Handling Shortcuts**
- [ ] `src/core/providers/enhanced_provider.py` lines 410-421 - [ ] `src/core/providers/enhanced_provider.py` lines 410-421
- Bare `except Exception:` without specific types (line 418) - Bare `except Exception:` without specific types (line 418) -> reviewed
- Multiple overlapping exception handlers (lines 410-425) - Multiple overlapping exception handlers (lines 410-425) -> reviewed
- Should use specific exception hierarchy - Should use specific exception hierarchy -> partially addressed (file removal and temp file cleanup now catch OSError; other broad catches intentionally wrap into RetryableError)
- [ ] `src/server/api/anime.py` lines 35-39 - [ ] `src/server/api/anime.py` lines 35-39 -> reviewed
- Bare `except Exception:` handlers should specify types - Bare `except Exception:` handlers should specify types
- [ ] `src/server/models/config.py` line 93 - [ ] `src/server/models/config.py` line 93
- `except ValidationError: pass` - silently ignores error - `except ValidationError: pass` - silently ignores error -> reviewed (validate() now collects and returns errors)
**Type Casting Workarounds** **Type Casting Workarounds**
- [ ] `src/server/api/download.py` line 52 - [ ] `src/server/api/download.py` line 52 -> reviewed
- Complex `.model_dump(mode="json")` for serialization - Complex `.model_dump(mode="json")` for serialization
- Should use proper model serialization methods - Should use proper model serialization methods (kept for backward compatibility)
- [ ] `src/server/utils/dependencies.py` line 36 - [ ] `src/server/utils/dependencies.py` line 36
- Type casting with `.get()` and defaults scattered throughout - Type casting with `.get()` and defaults scattered throughout
**Conditional Hacks** **Conditional Hacks**
- [ ] `src/server/utils/dependencies.py` line 260 - [ ] `src/server/utils/dependencies.py` line 260 -> completed
- `running_tests = "PYTEST_CURRENT_TEST" in os.environ or "pytest" in sys.modules` - `running_tests = "PYTEST_CURRENT_TEST" in os.environ or "pytest" in sys.modules`
- Hacky test detection - should use proper test mode flag - Hacky test detection - should use proper test mode flag (now prefers ANIWORLD_TESTING env var)
--- ---
@ -200,7 +186,7 @@ conda run -n AniWorld python -m pytest tests/ -v -s
**Weak CORS Configuration** **Weak CORS Configuration**
- [ ] `src/server/fastapi_app.py` line 48 - [ ] `src/server/fastapi_app.py` line 48 -> completed
- `allow_origins=["*"]` allows any origin - `allow_origins=["*"]` allows any origin
- **HIGH RISK** in production - **HIGH RISK** in production
- Should be: `allow_origins=settings.allowed_origins` (environment-based) - Should be: `allow_origins=settings.allowed_origins` (environment-based)

View File

@ -16,74 +16,38 @@ from yt_dlp import YoutubeDL
from ..interfaces.providers import Providers from ..interfaces.providers import Providers
from .base_provider import Loader from .base_provider import Loader
# Read timeout from environment variable, default to 600 seconds (10 minutes) # Imported shared provider configuration
timeout = int(os.getenv("DOWNLOAD_TIMEOUT", 600)) from .provider_config import (
ANIWORLD_HEADERS,
DEFAULT_DOWNLOAD_TIMEOUT,
DEFAULT_PROVIDERS,
INVALID_PATH_CHARS,
LULUVDO_USER_AGENT,
)
# Configure persistent loggers but don't add duplicate handlers when module
# is imported multiple times (common in test environments).
download_error_logger = logging.getLogger("DownloadErrors") download_error_logger = logging.getLogger("DownloadErrors")
download_error_handler = logging.FileHandler("../../download_errors.log") if not download_error_logger.handlers:
download_error_handler.setLevel(logging.ERROR) download_error_handler = logging.FileHandler("../../download_errors.log")
download_error_handler.setLevel(logging.ERROR)
download_error_logger.addHandler(download_error_handler)
noKeyFound_logger = logging.getLogger("NoKeyFound") noKeyFound_logger = logging.getLogger("NoKeyFound")
noKeyFound_handler = logging.FileHandler("../../NoKeyFound.log") if not noKeyFound_logger.handlers:
noKeyFound_handler.setLevel(logging.ERROR) noKeyFound_handler = logging.FileHandler("../../NoKeyFound.log")
noKeyFound_handler.setLevel(logging.ERROR)
noKeyFound_logger.addHandler(noKeyFound_handler)
class AniworldLoader(Loader): class AniworldLoader(Loader):
def __init__(self): def __init__(self):
self.SUPPORTED_PROVIDERS = [ self.SUPPORTED_PROVIDERS = DEFAULT_PROVIDERS
"VOE", # Copy default AniWorld headers so modifications remain local
"Doodstream", self.AniworldHeaders = dict(ANIWORLD_HEADERS)
"Vidmoly", self.INVALID_PATH_CHARS = INVALID_PATH_CHARS
"Vidoza",
"SpeedFiles",
"Streamtape",
"Luluvdo",
]
self.AniworldHeaders = {
"accept": (
"text/html,application/xhtml+xml,application/xml;q=0.9,"
"image/avif,image/webp,image/apng,*/*;q=0.8"
),
"accept-encoding": "gzip, deflate, br, zstd",
"accept-language": (
"de,de-DE;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6"
),
"cache-control": "max-age=0",
"priority": "u=0, i",
"sec-ch-ua": (
'"Chromium";v="136", "Microsoft Edge";v="136", '
'"Not.A/Brand";v="99"'
),
"sec-ch-ua-mobile": "?0",
"sec-ch-ua-platform": '"Windows"',
"sec-fetch-dest": "document",
"sec-fetch-mode": "navigate",
"sec-fetch-site": "none",
"sec-fetch-user": "?1",
"upgrade-insecure-requests": "1",
"user-agent": (
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
"AppleWebKit/537.36 (KHTML, like Gecko) "
"Chrome/136.0.0.0 Safari/537.36 Edg/136.0.0.0"
),
}
self.INVALID_PATH_CHARS = [
"<",
">",
":",
'"',
"/",
"\\",
"|",
"?",
"*",
"&",
]
self.RANDOM_USER_AGENT = UserAgent().random self.RANDOM_USER_AGENT = UserAgent().random
self.LULUVDO_USER_AGENT = ( self.LULUVDO_USER_AGENT = LULUVDO_USER_AGENT
"Mozilla/5.0 (Android 15; Mobile; rv:132.0) "
"Gecko/132.0 Firefox/132.0"
)
self.PROVIDER_HEADERS = { self.PROVIDER_HEADERS = {
"Vidmoly": ['Referer: "https://vidmoly.to"'], "Vidmoly": ['Referer: "https://vidmoly.to"'],
"Doodstream": ['Referer: "https://dood.li/"'], "Doodstream": ['Referer: "https://dood.li/"'],
@ -108,7 +72,11 @@ class AniworldLoader(Loader):
adapter = HTTPAdapter(max_retries=retries) adapter = HTTPAdapter(max_retries=retries)
self.session.mount("https://", adapter) self.session.mount("https://", adapter)
self.DEFAULT_REQUEST_TIMEOUT = 30 # Default HTTP request timeout used for requests.Session calls.
# Allows overriding via DOWNLOAD_TIMEOUT env var at runtime.
self.DEFAULT_REQUEST_TIMEOUT = int(
os.getenv("DOWNLOAD_TIMEOUT") or DEFAULT_DOWNLOAD_TIMEOUT
)
self._KeyHTMLDict = {} self._KeyHTMLDict = {}
self._EpisodeHTMLDict = {} self._EpisodeHTMLDict = {}

View File

@ -32,6 +32,12 @@ from ..error_handler import (
) )
from ..interfaces.providers import Providers from ..interfaces.providers import Providers
from .base_provider import Loader from .base_provider import Loader
from .provider_config import (
ANIWORLD_HEADERS,
DEFAULT_PROVIDERS,
INVALID_PATH_CHARS,
LULUVDO_USER_AGENT,
)
class EnhancedAniWorldLoader(Loader): class EnhancedAniWorldLoader(Loader):
@ -43,65 +49,12 @@ class EnhancedAniWorldLoader(Loader):
def __init__(self): def __init__(self):
super().__init__() super().__init__()
self.logger = logging.getLogger(__name__) self.logger = logging.getLogger(__name__)
providers = [ self.SUPPORTED_PROVIDERS = DEFAULT_PROVIDERS
"VOE", # local copy so modifications don't mutate shared constant
"Doodstream", self.AniworldHeaders = dict(ANIWORLD_HEADERS)
"Vidmoly", self.INVALID_PATH_CHARS = INVALID_PATH_CHARS
"Vidoza",
"SpeedFiles",
"Streamtape",
"Luluvdo",
]
self.SUPPORTED_PROVIDERS = providers
self.AniworldHeaders = {
"accept": (
"text/html,application/xhtml+xml,application/xml;q=0.9,"
"image/avif,image/webp,image/apng,*/*;q=0.8"
),
"accept-encoding": "gzip, deflate, br, zstd",
"accept-language": (
"de,de-DE;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6"
),
"cache-control": "max-age=0",
"priority": "u=0, i",
"sec-ch-ua": (
'"Chromium";v="136", "Microsoft Edge";v="136", '
'"Not.A/Brand";v="99"'
),
"sec-ch-ua-mobile": "?0",
"sec-ch-ua-platform": '"Windows"',
"sec-fetch-dest": "document",
"sec-fetch-mode": "navigate",
"sec-fetch-site": "none",
"sec-fetch-user": "?1",
"upgrade-insecure-requests": "1",
"user-agent": (
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
"AppleWebKit/537.36 (KHTML, like Gecko) "
"Chrome/136.0.0.0 Safari/537.36 Edg/136.0.0.0"
),
}
invalid_chars = [
"<",
">",
":",
'"',
"/",
"\\",
"|",
"?",
"*",
"&",
]
self.INVALID_PATH_CHARS = invalid_chars
self.RANDOM_USER_AGENT = UserAgent().random self.RANDOM_USER_AGENT = UserAgent().random
android_ua = ( self.LULUVDO_USER_AGENT = LULUVDO_USER_AGENT
"Mozilla/5.0 (Android 15; Mobile; rv:132.0) "
"Gecko/132.0 Firefox/132.0"
)
self.LULUVDO_USER_AGENT = android_ua
self.PROVIDER_HEADERS = { self.PROVIDER_HEADERS = {
"Vidmoly": ['Referer: "https://vidmoly.to"'], "Vidmoly": ['Referer: "https://vidmoly.to"'],
@ -136,14 +89,16 @@ class EnhancedAniWorldLoader(Loader):
'retried_downloads': 0 'retried_downloads': 0
} }
# Read timeout from environment variable # Read timeout from environment variable (string->int safely)
self.download_timeout = int(os.getenv("DOWNLOAD_TIMEOUT", 600)) self.download_timeout = int(os.getenv("DOWNLOAD_TIMEOUT") or "600")
# Setup logging # Setup logging
self._setup_logging() self._setup_logging()
def _create_robust_session(self) -> requests.Session: def _create_robust_session(self) -> requests.Session:
"""Create a session with robust retry and error handling configuration.""" """Create a session with robust retry and error handling
configuration.
"""
session = requests.Session() session = requests.Session()
# Configure retries so transient network problems are retried while we # Configure retries so transient network problems are retried while we
@ -189,7 +144,9 @@ class EnhancedAniWorldLoader(Loader):
"""Setup specialized logging for download errors and missing keys.""" """Setup specialized logging for download errors and missing keys."""
# Download error logger # Download error logger
self.download_error_logger = logging.getLogger("DownloadErrors") self.download_error_logger = logging.getLogger("DownloadErrors")
download_error_handler = logging.FileHandler("../../download_errors.log") download_error_handler = logging.FileHandler(
"../../download_errors.log"
)
download_error_handler.setLevel(logging.ERROR) download_error_handler.setLevel(logging.ERROR)
download_error_formatter = logging.Formatter( download_error_formatter = logging.Formatter(
'%(asctime)s - %(name)s - %(levelname)s - %(message)s' '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
@ -429,7 +386,7 @@ class EnhancedAniWorldLoader(Loader):
self.logger.warning(warning_msg) self.logger.warning(warning_msg)
try: try:
os.remove(output_path) os.remove(output_path)
except Exception as e: except OSError as e:
error_msg = f"Failed to remove corrupted file: {e}" error_msg = f"Failed to remove corrupted file: {e}"
self.logger.error(error_msg) self.logger.error(error_msg)
@ -556,8 +513,9 @@ class EnhancedAniWorldLoader(Loader):
self.logger.warning(warn_msg) self.logger.warning(warn_msg)
try: try:
os.remove(temp_path) os.remove(temp_path)
except Exception: except OSError as e:
pass warn_msg = f"Failed to remove temp file: {e}"
self.logger.warning(warn_msg)
except Exception as e: except Exception as e:
self.logger.warning(f"Provider {provider_name} failed: {e}") self.logger.warning(f"Provider {provider_name} failed: {e}")

View File

@ -0,0 +1,66 @@
"""Shared provider configuration constants for AniWorld providers.
Centralizes user-agent strings, provider lists and common headers so
multiple provider implementations can import a single source of truth.
"""
from typing import Dict, List
DEFAULT_PROVIDERS: List[str] = [
"VOE",
"Doodstream",
"Vidmoly",
"Vidoza",
"SpeedFiles",
"Streamtape",
"Luluvdo",
]
ANIWORLD_HEADERS: Dict[str, str] = {
"accept": (
"text/html,application/xhtml+xml,application/xml;q=0.9,"
"image/avif,image/webp,image/apng,*/*;q=0.8"
),
"accept-encoding": "gzip, deflate, br, zstd",
"accept-language": (
"de,de-DE;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6"
),
"cache-control": "max-age=0",
"priority": "u=0, i",
"sec-ch-ua": (
'"Chromium";v="136", "Microsoft Edge";v="136", '
'"Not.A/Brand";v="99"'
),
"sec-ch-ua-mobile": "?0",
"sec-ch-ua-platform": '"Windows"',
"sec-fetch-dest": "document",
"sec-fetch-mode": "navigate",
"sec-fetch-site": "none",
"sec-fetch-user": "?1",
"upgrade-insecure-requests": "1",
"user-agent": (
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
"AppleWebKit/537.36 (KHTML, like Gecko) "
"Chrome/136.0.0.0 Safari/537.36 Edg/136.0.0.0"
),
}
INVALID_PATH_CHARS: List[str] = [
"<",
">",
":",
'"',
"/",
"\\",
"|",
"?",
"*",
"&",
]
LULUVDO_USER_AGENT = (
"Mozilla/5.0 (Android 15; Mobile; rv:132.0) "
"Gecko/132.0 Firefox/132.0"
)
# Default download timeout (seconds)
DEFAULT_DOWNLOAD_TIMEOUT = 600

View File

@ -44,18 +44,34 @@ async def get_queue_status(
queue_status = await download_service.get_queue_status() queue_status = await download_service.get_queue_status()
queue_stats = await download_service.get_queue_stats() queue_stats = await download_service.get_queue_stats()
# Preserve the legacy response contract expected by the original CLI # Preserve the legacy response contract expected by the original CLI
# client and existing integration tests. Those consumers rely on the # client and existing integration tests. Those consumers still parse
# field names ``active``/``pending``/``completed``/``failed`` and raw # the bare dictionaries that the pre-FastAPI implementation emitted,
# dict payloads rather than Pydantic models, so we emit JSON-friendly # so we keep the canonical field names (``active``/``pending``/
# dictionaries that mirror the historic structure. # ``completed``/``failed``) and dump each Pydantic object to plain
# JSON-compatible dicts instead of returning the richer
# ``QueueStatusResponse`` shape directly. This guarantees both the
# CLI and older dashboard widgets do not need schema migrations while
# the new web UI can continue to evolve independently.
status_payload = { status_payload = {
"is_running": queue_status.is_running, "is_running": queue_status.is_running,
"is_paused": queue_status.is_paused, "is_paused": queue_status.is_paused,
"active": [it.model_dump(mode="json") for it in queue_status.active_downloads], "active": [
"pending": [it.model_dump(mode="json") for it in queue_status.pending_queue], it.model_dump(mode="json")
"completed": [it.model_dump(mode="json") for it in queue_status.completed_downloads], for it in queue_status.active_downloads
"failed": [it.model_dump(mode="json") for it in queue_status.failed_downloads], ],
"pending": [
it.model_dump(mode="json")
for it in queue_status.pending_queue
],
"completed": [
it.model_dump(mode="json")
for it in queue_status.completed_downloads
],
"failed": [
it.model_dump(mode="json")
for it in queue_status.failed_downloads
],
} }
# Add the derived ``success_rate`` metric so dashboards built against # Add the derived ``success_rate`` metric so dashboards built against
@ -71,7 +87,10 @@ async def get_queue_status(
stats_payload["success_rate"] = success_rate stats_payload["success_rate"] = success_rate
return JSONResponse( return JSONResponse(
content={"status": status_payload, "statistics": stats_payload} content={
"status": status_payload,
"statistics": stats_payload,
}
) )
except Exception as e: except Exception as e:
@ -133,7 +152,10 @@ async def add_to_queue(
"failed_items": [], "failed_items": [],
} }
return JSONResponse(content=payload, status_code=status.HTTP_201_CREATED) return JSONResponse(
content=payload,
status_code=status.HTTP_201_CREATED,
)
except DownloadServiceError as e: except DownloadServiceError as e:
raise HTTPException( raise HTTPException(
@ -509,7 +531,10 @@ async def reorder_queue(
if not success: if not success:
# Provide an appropriate 404 message depending on request shape # Provide an appropriate 404 message depending on request shape
if "item_order" in request: if "item_order" in request:
detail = "One or more items in item_order were not found in pending queue" detail = (
"One or more items in item_order were not "
"found in pending queue"
)
else: else:
detail = f"Item {req.item_id} not found in pending queue" detail = f"Item {req.item_id} not found in pending queue"

View File

@ -77,18 +77,27 @@ app.include_router(websocket_router)
# Register exception handlers # Register exception handlers
register_exception_handlers(app) register_exception_handlers(app)
# Global variables for application state # Prefer storing application-wide singletons on FastAPI.state instead of
series_app: Optional[SeriesApp] = None # module-level globals. This makes testing and multi-instance hosting safer.
def get_series_app() -> Optional[SeriesApp]:
"""Dependency to retrieve the SeriesApp instance from application state.
Returns None when the application wasn't configured with an anime
directory (for example during certain test runs).
"""
return getattr(app.state, "series_app", None)
@app.on_event("startup") @app.on_event("startup")
async def startup_event(): async def startup_event():
"""Initialize application on startup.""" """Initialize application on startup."""
global series_app
try: try:
# Initialize SeriesApp with configured directory # Initialize SeriesApp with configured directory and store it on
# application state so it can be injected via dependencies.
if settings.anime_directory: if settings.anime_directory:
series_app = SeriesApp(settings.anime_directory) app.state.series_app = SeriesApp(settings.anime_directory)
# Initialize progress service with websocket callback # Initialize progress service with websocket callback
progress_service = get_progress_service() progress_service = get_progress_service()
@ -103,9 +112,9 @@ async def startup_event():
"data": data, "data": data,
} }
await ws_service.manager.broadcast_to_room(message, room) await ws_service.manager.broadcast_to_room(message, room)
progress_service.set_broadcast_callback(broadcast_callback) progress_service.set_broadcast_callback(broadcast_callback)
print("FastAPI application started successfully") print("FastAPI application started successfully")
except Exception as e: except Exception as e:
print(f"Error during startup: {e}") print(f"Error during startup: {e}")

View File

@ -312,10 +312,15 @@ def get_anime_service() -> "AnimeService":
import sys import sys
import tempfile import tempfile
running_tests = ( # Prefer explicit test mode opt-in via ANIWORLD_TESTING=1; fall back
"PYTEST_CURRENT_TEST" in os.environ # to legacy heuristics for backwards compatibility with CI.
or "pytest" in sys.modules running_tests = os.getenv("ANIWORLD_TESTING") == "1"
) if not running_tests:
running_tests = (
"PYTEST_CURRENT_TEST" in os.environ
or "pytest" in sys.modules
)
if running_tests: if running_tests:
settings.anime_directory = tempfile.gettempdir() settings.anime_directory = tempfile.gettempdir()
else: else: