Commit remaining tracked changes

docs/InstructionsLogging.md

@@ -85,6 +85,10 @@ except Exception:
 
 > 💡 When in doubt, favor **fewer, richer logs** over many noisy logs.
 
-# importent
+---
 
-note which file is done.
+## 5. Logging Audit Task List
+
+For a guided checklist of files and logging improvements, see **`docs/tasks.md`**. This is where we track which files have been reviewed and which logging items still need attention.
+
+> ✅ After applying the guidelines above, update `docs/tasks.md` to indicate which tasks are complete.

docs/instructions.md

@@ -117,4 +117,3 @@ For each task completed:
 
 ---
 
-## TODO List:

src/server/fastapi_app.py

@@ -6,6 +6,7 @@ configuration, middleware setup, static file serving, and Jinja2 template
 integration.
 """
 import asyncio
+import logging
 from contextlib import asynccontextmanager
 from pathlib import Path
 
@@ -51,7 +52,7 @@ async def _check_incomplete_series_on_startup(background_loader) -> None:
     Args:
         background_loader: BackgroundLoaderService instance
     """
-    logger = setup_logging(log_level="INFO")
+    logger = logging.getLogger("aniworld")
     
     try:
         from src.server.database.connection import get_db_session
@@ -96,11 +97,11 @@ async def _check_incomplete_series_on_startup(background_loader) -> None:
                 else:
                     logger.info("All series data is complete. No background loading needed.")
                 
-            except Exception as e:
-                logger.exception("Error checking incomplete series: %s", e)
+            except Exception:
+                logger.exception("Error checking incomplete series")
             
-    except Exception as e:
-        logger.exception("Failed to check incomplete series on startup: %s", e)
+    except Exception:
+        logger.exception("Failed to check incomplete series on startup")
 
 
 @asynccontextmanager

src/server/services/background_loader_service.py

@@ -22,6 +22,7 @@ from typing import Any, Dict, List, Optional
 
 import structlog
 
+from src.core.services.nfo_factory import get_nfo_factory
 from src.server.services.websocket_service import WebSocketService
 
 logger = structlog.get_logger(__name__)
@@ -550,9 +551,22 @@ class BackgroundLoaderService:
             await self._broadcast_status(task, "Generating NFO file...")
             logger.info("Creating new NFO for %s", task.key)
             
-            # Use existing NFOService to create NFO with all images
-            # This reuses all existing TMDB API logic and image downloading
-            nfo_path = await self.series_app.nfo_service.create_tvshow_nfo(
+            # Create a fresh NFOService for this task to avoid shared TMDB session closure
+            try:
+                factory = get_nfo_factory()
+                nfo_service = factory.create()
+            except ValueError:
+                logger.warning(
+                    "NFOService unavailable for %s, skipping NFO/images",
+                    task.key
+                )
+                task.progress["nfo"] = False
+                task.progress["logo"] = False
+                task.progress["images"] = False
+                return False
+
+            try:
+                nfo_path = await nfo_service.create_tvshow_nfo(
                     serie_name=task.name,
                     serie_folder=task.folder,
                     year=task.year,
@@ -560,6 +574,8 @@ class BackgroundLoaderService:
                     download_logo=True,
                     download_fanart=True
                 )
+            finally:
+                await nfo_service.close()
 
             # Update task progress
             task.progress["nfo"] = True

tasks.md

@@ -1,361 +0,0 @@
-# Logging Audit & Improvement Tasks
-
-This tasks list is written for an AI agent to systematically rework and add logging across the AniWorld codebase. Each task includes a specific file and the intended logging improvement.
-
-> 📌 **Goal:** Improve log clarity, reduce noise, ensure consistent structured context, and add missing logs for important events and failures.
-
-### How to execute these tasks
-
-1. **For each file** in the list, open it and locate existing `logger.*` calls.
-2. **Ask:**
-    - Is the message actionable and human-readable?
-    - Does it include key context (ids, names, paths, operation results)?
-    - Is the level correct (DEBUG/INFO/WARNING/ERROR/CRITICAL)?
-    - Are exceptions logged using `logger.exception(...)` or `exc_info=True`?
-    - Are we avoiding log spam (e.g., loops, repeated messages)?
-3. **Add or adjust logs** by:
-    - Moving noisy INFO logs to DEBUG unless they represent important state changes.
-    - Adding INFO-level logs at boundary points (entry/exit of operations, request endpoints).
-    - Including structured `extra={...}` fields where helpful.
-    - Logging minimal useful object representations (e.g., `series.id`, `file_path`).
-4. **Validate** that changes do not expose secrets or PII in logs.
-5. **Follow the project logging style** in `docs/InstructionsLogging.md` when drafting or refactoring log calls.
-
----
-
-## 1) Core Logging Infrastructure
-
-- **`src/infrastructure/logging/logger.py`**
-    - Ensure the project has a single, consistent logging entry point for all modules.
-    - Validate that logger names are consistent and allow per-module filtering.
-    - Confirm structured logging support (extra/context fields) is correctly implemented.
-
-- **`src/server/config/logging_config.py`**
-    - Verify log format templates are clear, consistent, and include timestamp + level + module.
-    - Make sure log level configuration is reliable and respects env/config.
-    - Confirm file rotation, log file creation, and log handlers are configured properly.
-
-- **`src/server/utils/logging.py`**
-    - Confirm helper utilities for creating loggers and configuring log levels are correct.
-    - Ensure default logger levels (download, security, performance) are clear and non-spammy.
-
-- **`src/server/utils/log_manager.py`**
-    - Review any dynamic log level changes and confirm they are safe and properly logged.
-
----
-
-## 2) Startup & Runtime Lifecycle Logging
-
-- **`src/server/fastapi_app.py`**
-    - Refactor startup/shutdown logs to ensure each major step (db init, service initialization, background tasks) is logged at an appropriate level.
-    - Add missing context (e.g., which service or component is starting/stopping) in the messages.
-    - Ensure exceptions are logged via `logger.exception(...)` where appropriate.
-
-- **`src/server/services/initialization_service.py`**
-    - Add logs for key initialization steps and failures.
-
-- **`src/server/services/cache_service.py`**
-    - Ensure cache hits/misses (if relevant) are logged at DEBUG and errors are logged at WARNING/ERROR.
-
-- **`src/server/services/notification_service.py`**
-    - Add logs for notification events and failures.
-
----
-
-## 3) API / Controller Logging
-
-- **`src/server/api/anime.py`**
-- **`src/server/api/nfo.py`**
-- **`src/server/api/config.py`**
-- **`src/server/api/health.py`**
-- **`src/server/api/scheduler.py`**
-- **`src/server/api/logging.py`**
-- **`src/server/api/websocket.py`**
-
-For each API module:
-
-- Ensure each endpoint logs at INFO when invoked (at least the route and key identifiers).
-- Add DEBUG logs for important steps inside handlers (data validation, service calls, responses) without spamming.
-- Ensure errors/exceptions are logged with stack traces.
-
----
-
-## 4) Middleware and Error Handling
-
-- **`src/server/middleware/error_handler.py`**
-    - Make sure handled exceptions are logged appropriately.
-    - Avoid logging the same error multiple times at the same level.
-
-- **`src/server/middleware/security.py`**
-    - Ensure security-related events (auth failures, suspicious activity) are logged at WARNING/ERROR with context.
-
----
-
-## 5) Database & Transaction Logging
-
-- **`src/server/database/connection.py`**
-    - Log key connection lifecycle events and failures.
-
-- **`src/server/database/init.py`**
-    - Ensure DB initialization steps are logged.
-
-- **`src/server/database/service.py`**
-    - Review logs for CRUD operations on series and episodes.
-    - Confirm that success logs include ids and key values.
-    - Ensure exceptions are logged with full context.
-
-- **`src/server/database/system_settings_service.py`**
-    - Confirm logs for scan completion/reset are accurate and not noisy.
-
-- **`src/server/database/transaction.py`**
-    - Ensure transaction start/commit/rollback logs are DEBUG, and warnings/errors for failures are logged.
-    - Confirm savepoint logs include savepoint names.
-
----
-
-## 6) Background Services & Worker Logging
-
-- **`src/server/services/background_loader_service.py`**
-    - Add/info logs for job start/completion, and any batch summaries.
-
-- **`src/server/services/scheduler_service.py`**
-    - Ensure job scheduling, execution, and failure are logged.
-
-- **`src/server/services/download_service.py`**
-    - Add informative logs for downloads (start, success, failure, retry).
-
-- **`src/server/services/queue_repository.py`**
-    - Ensure queue operations are logged at suitable levels (DEBUG for enqueue/dequeue, WARNING for errors).
-
-- **`src/server/services/progress_service.py`**
-    - Add logs for progress updates and any errors.
-
-- **`src/server/services/scan_service.py`**
-    - Add logs for scan start/completion, skipped items, and failures.
-
-- **`src/server/services/anime_service.py`**
-    - Ensure high-level operations (create/update/delete) are logged with identifiers.
-
-- **`src/server/services/websocket_service.py`**
-    - Add logs for connection/disconnection events and key message paths.
-
----
-
-## 7) Server Utilities & Helpers
-
-- **`src/server/utils/dependencies.py`**
-    - Ensure dependency injections and resource retrievals are logged at DEBUG when there is failure.
-
-- **`src/server/utils/error_tracking.py`**
-    - Verify errors are logged with context and stack trace.
-
-- **`src/server/utils/metrics.py`**
-    - Ensure metrics collection failures are logged appropriately.
-
-- **`src/server/utils/system.py`**
-    - Add logs for system-level operations (memory, disk checks) if missing.
-
-- **`src/server/utils/template_helpers.py`**
-    - Add logs for template rendering errors.
-
----
-
-## 8) Core Application Logging
-
-- **`src/core/SeriesApp.py`**
-- **`src/core/SerieScanner.py`**
-
-For core application logic:
-
-- Ensure key events (scan start/end, errors, decisions) are logged at correct level, and logs are not too noisy when scanning large directories.
-
----
-
-## 9) Providers / Integration Logging
-
-- **`src/core/providers/aniworld_provider.py`**
-- **`src/core/providers/enhanced_provider.py`**
-- **`src/core/providers/monitored_provider.py`**
-- **`src/core/providers/failover.py`**
-- **`src/core/providers/config_manager.py`**
-- **`src/core/providers/health_monitor.py`**
-
-For provider modules:
-
-- Ensure failures / retries / fallbacks are logged at WARNING or ERROR with context.
-- Log important state changes at INFO (e.g., provider enabled/disabled).
-
----
-
-## 10) Core Services / Utility Logging
-
-- **`src/core/services/nfo_service.py`**
-- **`src/core/services/nfo_factory.py`**
-- **`src/core/services/nfo_repair_service.py`**
-- **`src/core/services/series_manager_service.py`**
-- **`src/core/services/tmdb_client.py`**
-
-Ensure all operations that can fail or produce important output are logged with enough context (series id/name, operation result, duration).
-
-- **`src/core/utils/nfo_generator.py`**
-- **`src/core/utils/nfo_mapper.py`**
-- **`src/core/utils/image_downloader.py`**
-
-Ensure any external calls (HTTP, file I/O) and failures are logged with reasons.
-
----
-
-## 11) CLI & Scripts
-
-- **`src/cli/nfo_cli.py`**
-    - Ensure startup logs are concise; reduce repetitive output; group statistics.
-
-- **`Docker/test_vpn.py`**
-    - Ensure logging is clear and not too verbose (especially in loops).
-
----
-
-## 12) Tests (Verify Logging Behavior)
-
-- **`tests/unit/test_infrastructure_logger.py`**
-- **`tests/unit/test_logging.py`**
-- **`tests/unit/test_nfo_update_parsing.py`**
-- **`tests/unit/test_parallel_anime_add.py`**
-- **`tests/api/test_concurrent_anime_add.py`**
-- **`tests/performance/test_nfo_batch_performance.py`**
-- **`tests/performance/test_websocket_load.py`**
-- **`tests/security/test_encryption_security.py`**
-
-Tasks:
-
-- Ensure log capture/formatting tests are up to date.
-- Add tests to validate that key operations log at expected levels (INFO/DEBUG/ERROR) and include required context.
-
----
-
-## 13) Follow-up Tasks (Post-Refactor)
-
-1. Run the full test suite to confirm logging changes do not break assertions.
-2. Verify logs during normal runs are not overly noisy; tune log levels accordingly.
-3. Ensure deployment/production log behavior matches expectations (rotate logs, preserve errors, and hide debug output).
-
----
-
-## 14) Full Python File Inventory (excluding tests)
-
-Below is the complete list of all `.py` files in the repo (excluding anything under `tests/`).
-
-### 14a) Mutable logging tasks (edit these files)
-
-These are files where logging should be added/updated as part of the audit.
-
-- **`Docker/test_vpn.py`** – Review logging clarity, avoid verbose loops, and ensure errors are logged with context.
-- **`run_server.py`** – Ensure server startup logs (and any error paths) have clear context and use proper log levels.
-- **`src/cli/nfo_cli.py`** – Confirm CLI output is logged appropriately with summary reporting.
-- **`src/config/settings.py`** – Ensure config parsing failures are logged and invalid values are handled or reported.
-- **`src/core/entities/nfo_models.py`** – Add logs for model validation errors or unexpected data.
-- **`src/core/entities/SerieList.py`** – Ensure list operations and conversions log useful identifiers.
-- **`src/core/entities/series.py`** – Log key series state changes and validation failures.
-- **`src/core/error_handler.py`** – Ensure caught errors are logged and propagated correctly.
-- **`src/core/exceptions/Exceptions.py`** – Validate exception definitions support useful logging messages.
-- **`src/core/interfaces/callbacks.py`** – Log callback errors or unexpected callback payloads.
-- **`src/core/interfaces/providers.py`** – Ensure provider interface errors are logged clearly.
-- **`src/core/providers/aniworld_provider.py`** – Add logs for API calls, retries, and failure reasons.
-- **`src/core/providers/base_provider.py`** – Ensure base provider logs setup and teardown.
-- **`src/core/providers/config_manager.py`** – Log config load/refresh and validation errors.
-- **`src/core/providers/enhanced_provider.py`** – Review logging for error reporting and rate limiting.
-- **`src/core/providers/failover.py`** – Ensure failover decisions and switchovers are logged.
-- **`src/core/providers/health_monitor.py`** – Add logs for health-check results and degraded state.
-- **`src/core/providers/monitored_provider.py`** – Ensure monitored provider metrics and errors are logged.
-- **`src/core/providers/provider_config.py`** – Log config validation and defaulting behavior.
-- **`src/core/providers/provider_factory.py`** – Ensure factory creation errors are logged with context.
-- **`src/core/providers/streaming/Provider.py`** – Add logs for stream start/stop and errors.
-- **`src/core/providers/streaming/voe.py`** – Log VOE fetch attempts and failures.
-- **`src/core/SeriesApp.py`** – Confirm app lifecycle events and errors are logged.
-- **`src/core/SerieScanner.py`** – Ensure scan start/end, skipped items, and errors are logged.
-- **`src/core/services/nfo_factory.py`** – Log NFO generation steps and failures.
-- **`src/core/services/nfo_repair_service.py`** – Ensure repair actions and failures are logged.
-- **`src/core/services/nfo_service.py`** – Add logs for NFO operations and exceptions.
-- **`src/core/services/series_manager_service.py`** – Log series management actions and key identifiers.
-- **`src/core/services/tmdb_client.py`** – Ensure external API calls and failures are logged.
-- **`src/core/utils/image_downloader.py`** – Add logs for download attempts, successes, and errors.
-- **`src/core/utils/nfo_generator.py`** – Ensure generation steps and failures are logged.
-- **`src/core/utils/nfo_mapper.py`** – Add logs for mapping failures and questionable data.
-- **`src/infrastructure/logging/logger.py`** – Validate centralized logger behavior and structured logging support.
-- **`src/infrastructure/logging/uvicorn_config.py`** – Ensure Uvicorn logging config is correct and includes context.
-- **`src/infrastructure/security/config_encryption.py`** – Log encryption/decryption failures and warnings.
-- **`src/infrastructure/security/database_integrity.py`** – Add logs for integrity check results and failures.
-- **`src/infrastructure/security/file_integrity.py`** – Ensure file integrity checks log results and errors.
-- **`src/server/api/anime.py`** – Confirm endpoints log requests, key params, and errors.
-- **`src/server/api/auth.py`** – Ensure auth failures and token operations are logged.
-- **`src/server/api/config.py`** – Add logs for config read/write and validation.
-- **`src/server/api/download.py`** – Log download request lifecycle and errors.
-- **`src/server/api/health.py`** – Ensure health checks log status and failure reasons.
-- **`src/server/api/logging.py`** – Ensure log endpoint behavior is clear and properly logged.
-- **`src/server/api/nfo.py`** – Add logs for NFO endpoint actions and errors.
-- **`src/server/api/scheduler.py`** – Log scheduling actions, triggers, and failures.
-- **`src/server/api/websocket.py`** – Ensure websocket connect/disconnect and message errors are logged.
-- **`src/server/controllers/error_controller.py`** – Add logs for handled errors.
-- **`src/server/controllers/page_controller.py`** – Add logs for page rendering errors.
-- **`src/server/database/base.py`** – Ensure base DB operations log unexpected issues.
-- **`src/server/database/connection.py`** – Add connection lifecycle and error logs.
-- **`src/server/database/init.py`** – Ensure DB init steps are logged.
-- **`src/server/database/models.py`** – Validate model validation errors are logged.
-- **`src/server/database/service.py`** – Confirm CRUD operations log with context.
-- **`src/server/database/system_settings_service.py`** – Ensure system settings actions log correctly.
-- **`src/server/database/transaction.py`** – Verify transaction start/commit/rollback logging.
-- **`src/server/fastapi_app.py`** – Review startup/shutdown and error logging.
-- **`src/server/middleware/auth.py`** – Ensure auth middleware logs auth events and failures.
-- **`src/server/middleware/error_handler.py`** – Confirm error handling logs exceptions clearly.
-- **`src/server/middleware/security.py`** – Ensure security events are logged with context.
-- **`src/server/middleware/setup_redirect.py`** – Add logs for redirect behavior and errors.
-- **`src/server/models/anime.py`** – Ensure model validation errors are logged.
-- **`src/server/models/auth.py`** – Confirm auth model logging for invalid data.
-- **`src/server/models/config.py`** – Add logs for config model validation.
-- **`src/server/models/download.py`** – Ensure download model failures are logged.
-- **`src/server/models/nfo.py`** – Ensure NFO model issues are logged.
-- **`src/server/models/websocket.py`** – Confirm websocket model errors are logged.
-- **`src/server/services/anime_service.py`** – Ensure operations log relevant ids and results.
-- **`src/server/services/auth_service.py`** – Add logs for auth operations and failures.
-- **`src/server/services/background_loader_service.py`** – Ensure background jobs log start/stop and errors.
-- **`src/server/services/cache_service.py`** – Confirm caching logs are sane and useful.
-- **`src/server/services/config_service.py`** – Ensure config updates and errors are logged.
-- **`src/server/services/download_service.py`** – Add download lifecycle logs.
-- **`src/server/services/initialization_service.py`** – Confirm initialization steps are logged.
-- **`src/server/services/notification_service.py`** – Ensure notification dispatch/failure logs exist.
-- **`src/server/services/progress_service.py`** – Add progress update logs.
-- **`src/server/services/queue_repository.py`** – Ensure queue operations are logged.
-- **`src/server/services/scan_service.py`** – Confirm scan lifecycle and failure logging.
-- **`src/server/services/scheduler_service.py`** – Ensure scheduler actions and failures are logged.
-- **`src/server/services/websocket_service.py`** – Confirm websocket lifecycle and message errors are logged.
-- **`src/server/utils/dependencies.py`** – Ensure dependency errors are logged.
-- **`src/server/utils/error_tracking.py`** – Verify error tracking logs contain context.
-- **`src/server/utils/filesystem.py`** – Add filesystem operation logs and errors.
-- **`src/server/utils/logging.py`** – Ensure helper logging utilities behave as expected.
-- **`src/server/utils/log_manager.py`** – Review dynamic log-level changes and logs.
-- **`src/server/utils/media.py`** – Add logs for media operations and failures.
-- **`src/server/utils/metrics.py`** – Ensure metric collection/logging failures are captured.
-- **`src/server/utils/system.py`** – Add system-level operation logs and error handling.
-- **`src/server/utils/template_helpers.py`** – Ensure template rendering errors are logged.
-- **`src/server/utils/templates.py`** – Add logs for template loading and render errors.
-- **`src/server/utils/validators.py`** – Ensure validation failures are logged appropriately.
-- **`src/server/web/middleware/__init__.py`** – Verify middleware initialization logging.
-
-### 14b) Reference / low-touch files (review only)
-
-- **`src/core/__init__.py`** – Verify any module-level logging is appropriate.
-- **`src/core/providers/__init__.py`** – Verify module-level logging is appropriate.
-- **`src/infrastructure/logging/__init__.py`** – Confirm module exports and initialization logs are appropriate.
-- **`src/infrastructure/security/__init__.py`** – Verify module-level logging is appropriate.
-- **`src/server/api/__init__.py`** – Verify any module setup logging.
-- **`src/server/config/__init__.py`** – Verify config initialization logging.
-- **`src/server/controllers/__init__.py`** – Verify module import logging is non-spammy.
-- **`src/server/database/__init__.py`** – Confirm any initialization logs.
-- **`src/server/exceptions/__init__.py`** – Confirm module-level logging is appropriate.
-- **`src/server/models/__init__.py`** – Verify any module init logs.
-- **`src/server/utils/__init__.py`** – Confirm initialization logging is appropriate.
-- **`src/server/web/middleware/__init__.py`** – Verify middleware initialization logging.
-
----
-
-> ✅ **Done:** This `tasks.md` file enumerates each relevant source file and a clear instruction for how to improve or add logging in that file.