Aniworld/tasks.md

# Logging Audit & Improvement Tasks

This tasks list is written for an AI agent to systematically rework and add logging across the AniWorld codebase. Each task includes a specific file and the intended logging improvement.

> 📌 **Goal:** Improve log clarity, reduce noise, ensure consistent structured context, and add missing logs for important events and failures.

### How to execute these tasks

1. **For each file** in the list, open it and locate existing `logger.*` calls.
2. **Ask:**
    - Is the message actionable and human-readable?
    - Does it include key context (ids, names, paths, operation results)?
    - Is the level correct (DEBUG/INFO/WARNING/ERROR/CRITICAL)?
    - Are exceptions logged using `logger.exception(...)` or `exc_info=True`?
    - Are we avoiding log spam (e.g., loops, repeated messages)?
3. **Add or adjust logs** by:
    - Moving noisy INFO logs to DEBUG unless they represent important state changes.
    - Adding INFO-level logs at boundary points (entry/exit of operations, request endpoints).
    - Including structured `extra={...}` fields where helpful.
    - Logging minimal useful object representations (e.g., `series.id`, `file_path`).
4. **Validate** that changes do not expose secrets or PII in logs.
5. **Follow the project logging style** in `docs/InstructionsLogging.md` when drafting or refactoring log calls.

---

## 1) Core Logging Infrastructure

- **`src/infrastructure/logging/logger.py`**
    - Ensure the project has a single, consistent logging entry point for all modules.
    - Validate that logger names are consistent and allow per-module filtering.
    - Confirm structured logging support (extra/context fields) is correctly implemented.

- **`src/server/config/logging_config.py`**
    - Verify log format templates are clear, consistent, and include timestamp + level + module.
    - Make sure log level configuration is reliable and respects env/config.
    - Confirm file rotation, log file creation, and log handlers are configured properly.

- **`src/server/utils/logging.py`**
    - Confirm helper utilities for creating loggers and configuring log levels are correct.
    - Ensure default logger levels (download, security, performance) are clear and non-spammy.

- **`src/server/utils/log_manager.py`**
    - Review any dynamic log level changes and confirm they are safe and properly logged.

---

## 2) Startup & Runtime Lifecycle Logging

- **`src/server/fastapi_app.py`**
    - Refactor startup/shutdown logs to ensure each major step (db init, service initialization, background tasks) is logged at an appropriate level.
    - Add missing context (e.g., which service or component is starting/stopping) in the messages.
    - Ensure exceptions are logged via `logger.exception(...)` where appropriate.

- **`src/server/services/initialization_service.py`**
    - Add logs for key initialization steps and failures.

- **`src/server/services/cache_service.py`**
    - Ensure cache hits/misses (if relevant) are logged at DEBUG and errors are logged at WARNING/ERROR.

- **`src/server/services/notification_service.py`**
    - Add logs for notification events and failures.

---

## 3) API / Controller Logging

- **`src/server/api/anime.py`**
- **`src/server/api/nfo.py`**
- **`src/server/api/config.py`**
- **`src/server/api/health.py`**
- **`src/server/api/scheduler.py`**
- **`src/server/api/logging.py`**
- **`src/server/api/websocket.py`**

For each API module:

- Ensure each endpoint logs at INFO when invoked (at least the route and key identifiers).
- Add DEBUG logs for important steps inside handlers (data validation, service calls, responses) without spamming.
- Ensure errors/exceptions are logged with stack traces.

---

## 4) Middleware and Error Handling

- **`src/server/middleware/error_handler.py`**
    - Make sure handled exceptions are logged appropriately.
    - Avoid logging the same error multiple times at the same level.

- **`src/server/middleware/security.py`**
    - Ensure security-related events (auth failures, suspicious activity) are logged at WARNING/ERROR with context.

---

## 5) Database & Transaction Logging

- **`src/server/database/connection.py`**
    - Log key connection lifecycle events and failures.

- **`src/server/database/init.py`**
    - Ensure DB initialization steps are logged.

- **`src/server/database/service.py`**
    - Review logs for CRUD operations on series and episodes.
    - Confirm that success logs include ids and key values.
    - Ensure exceptions are logged with full context.

- **`src/server/database/system_settings_service.py`**
    - Confirm logs for scan completion/reset are accurate and not noisy.

- **`src/server/database/transaction.py`**
    - Ensure transaction start/commit/rollback logs are DEBUG, and warnings/errors for failures are logged.
    - Confirm savepoint logs include savepoint names.

---

## 6) Background Services & Worker Logging

- **`src/server/services/background_loader_service.py`**
    - Add/info logs for job start/completion, and any batch summaries.

- **`src/server/services/scheduler_service.py`**
    - Ensure job scheduling, execution, and failure are logged.

- **`src/server/services/download_service.py`**
    - Add informative logs for downloads (start, success, failure, retry).

- **`src/server/services/queue_repository.py`**
    - Ensure queue operations are logged at suitable levels (DEBUG for enqueue/dequeue, WARNING for errors).

- **`src/server/services/progress_service.py`**
    - Add logs for progress updates and any errors.

- **`src/server/services/scan_service.py`**
    - Add logs for scan start/completion, skipped items, and failures.

- **`src/server/services/anime_service.py`**
    - Ensure high-level operations (create/update/delete) are logged with identifiers.

- **`src/server/services/websocket_service.py`**
    - Add logs for connection/disconnection events and key message paths.

---

## 7) Server Utilities & Helpers

- **`src/server/utils/dependencies.py`**
    - Ensure dependency injections and resource retrievals are logged at DEBUG when there is failure.

- **`src/server/utils/error_tracking.py`**
    - Verify errors are logged with context and stack trace.

- **`src/server/utils/metrics.py`**
    - Ensure metrics collection failures are logged appropriately.

- **`src/server/utils/system.py`**
    - Add logs for system-level operations (memory, disk checks) if missing.

- **`src/server/utils/template_helpers.py`**
    - Add logs for template rendering errors.

---

## 8) Core Application Logging

- **`src/core/SeriesApp.py`**
- **`src/core/SerieScanner.py`**

For core application logic:

- Ensure key events (scan start/end, errors, decisions) are logged at correct level, and logs are not too noisy when scanning large directories.

---

## 9) Providers / Integration Logging

- **`src/core/providers/aniworld_provider.py`**
- **`src/core/providers/enhanced_provider.py`**
- **`src/core/providers/monitored_provider.py`**
- **`src/core/providers/failover.py`**
- **`src/core/providers/config_manager.py`**
- **`src/core/providers/health_monitor.py`**

For provider modules:

- Ensure failures / retries / fallbacks are logged at WARNING or ERROR with context.
- Log important state changes at INFO (e.g., provider enabled/disabled).

---

## 10) Core Services / Utility Logging

- **`src/core/services/nfo_service.py`**
- **`src/core/services/nfo_factory.py`**
- **`src/core/services/nfo_repair_service.py`**
- **`src/core/services/series_manager_service.py`**
- **`src/core/services/tmdb_client.py`**

Ensure all operations that can fail or produce important output are logged with enough context (series id/name, operation result, duration).

- **`src/core/utils/nfo_generator.py`**
- **`src/core/utils/nfo_mapper.py`**
- **`src/core/utils/image_downloader.py`**

Ensure any external calls (HTTP, file I/O) and failures are logged with reasons.

---

## 11) CLI & Scripts

- **`src/cli/nfo_cli.py`**
    - Ensure startup logs are concise; reduce repetitive output; group statistics.

- **`Docker/test_vpn.py`**
    - Ensure logging is clear and not too verbose (especially in loops).

---

## 12) Tests (Verify Logging Behavior)

- **`tests/unit/test_infrastructure_logger.py`**
- **`tests/unit/test_logging.py`**
- **`tests/unit/test_nfo_update_parsing.py`**
- **`tests/unit/test_parallel_anime_add.py`**
- **`tests/api/test_concurrent_anime_add.py`**
- **`tests/performance/test_nfo_batch_performance.py`**
- **`tests/performance/test_websocket_load.py`**
- **`tests/security/test_encryption_security.py`**

Tasks:

- Ensure log capture/formatting tests are up to date.
- Add tests to validate that key operations log at expected levels (INFO/DEBUG/ERROR) and include required context.

---

## 13) Follow-up Tasks (Post-Refactor)

1. Run the full test suite to confirm logging changes do not break assertions.
2. Verify logs during normal runs are not overly noisy; tune log levels accordingly.
3. Ensure deployment/production log behavior matches expectations (rotate logs, preserve errors, and hide debug output).

---

## 14) Full Python File Inventory (excluding tests)

Below is the complete list of all `.py` files in the repo (excluding anything under `tests/`).

### 14a) Mutable logging tasks (edit these files)

These are files where logging should be added/updated as part of the audit.

- **`Docker/test_vpn.py`** – Review logging clarity, avoid verbose loops, and ensure errors are logged with context.
- **`run_server.py`** – Ensure server startup logs (and any error paths) have clear context and use proper log levels.
- **`src/cli/nfo_cli.py`** – Confirm CLI output is logged appropriately with summary reporting.
- **`src/config/settings.py`** – Ensure config parsing failures are logged and invalid values are handled or reported.
- **`src/core/entities/nfo_models.py`** – Add logs for model validation errors or unexpected data.
- **`src/core/entities/SerieList.py`** – Ensure list operations and conversions log useful identifiers.
- **`src/core/entities/series.py`** – Log key series state changes and validation failures.
- **`src/core/error_handler.py`** – Ensure caught errors are logged and propagated correctly.
- **`src/core/exceptions/Exceptions.py`** – Validate exception definitions support useful logging messages.
- **`src/core/interfaces/callbacks.py`** – Log callback errors or unexpected callback payloads.
- **`src/core/interfaces/providers.py`** – Ensure provider interface errors are logged clearly.
- **`src/core/providers/aniworld_provider.py`** – Add logs for API calls, retries, and failure reasons.
- **`src/core/providers/base_provider.py`** – Ensure base provider logs setup and teardown.
- **`src/core/providers/config_manager.py`** – Log config load/refresh and validation errors.
- **`src/core/providers/enhanced_provider.py`** – Review logging for error reporting and rate limiting.
- **`src/core/providers/failover.py`** – Ensure failover decisions and switchovers are logged.
- **`src/core/providers/health_monitor.py`** – Add logs for health-check results and degraded state.
- **`src/core/providers/monitored_provider.py`** – Ensure monitored provider metrics and errors are logged.
- **`src/core/providers/provider_config.py`** – Log config validation and defaulting behavior.
- **`src/core/providers/provider_factory.py`** – Ensure factory creation errors are logged with context.
- **`src/core/providers/streaming/Provider.py`** – Add logs for stream start/stop and errors.
- **`src/core/providers/streaming/voe.py`** – Log VOE fetch attempts and failures.
- **`src/core/SeriesApp.py`** – Confirm app lifecycle events and errors are logged.
- **`src/core/SerieScanner.py`** – Ensure scan start/end, skipped items, and errors are logged.
- **`src/core/services/nfo_factory.py`** – Log NFO generation steps and failures.
- **`src/core/services/nfo_repair_service.py`** – Ensure repair actions and failures are logged.
- **`src/core/services/nfo_service.py`** – Add logs for NFO operations and exceptions.
- **`src/core/services/series_manager_service.py`** – Log series management actions and key identifiers.
- **`src/core/services/tmdb_client.py`** – Ensure external API calls and failures are logged.
- **`src/core/utils/image_downloader.py`** – Add logs for download attempts, successes, and errors.
- **`src/core/utils/nfo_generator.py`** – Ensure generation steps and failures are logged.
- **`src/core/utils/nfo_mapper.py`** – Add logs for mapping failures and questionable data.
- **`src/infrastructure/logging/logger.py`** – Validate centralized logger behavior and structured logging support.
- **`src/infrastructure/logging/uvicorn_config.py`** – Ensure Uvicorn logging config is correct and includes context.
- **`src/infrastructure/security/config_encryption.py`** – Log encryption/decryption failures and warnings.
- **`src/infrastructure/security/database_integrity.py`** – Add logs for integrity check results and failures.
- **`src/infrastructure/security/file_integrity.py`** – Ensure file integrity checks log results and errors.
- **`src/server/api/anime.py`** – Confirm endpoints log requests, key params, and errors.
- **`src/server/api/auth.py`** – Ensure auth failures and token operations are logged.
- **`src/server/api/config.py`** – Add logs for config read/write and validation.
- **`src/server/api/download.py`** – Log download request lifecycle and errors.
- **`src/server/api/health.py`** – Ensure health checks log status and failure reasons.
- **`src/server/api/logging.py`** – Ensure log endpoint behavior is clear and properly logged.
- **`src/server/api/nfo.py`** – Add logs for NFO endpoint actions and errors.
- **`src/server/api/scheduler.py`** – Log scheduling actions, triggers, and failures.
- **`src/server/api/websocket.py`** – Ensure websocket connect/disconnect and message errors are logged.
- **`src/server/controllers/error_controller.py`** – Add logs for handled errors.
- **`src/server/controllers/page_controller.py`** – Add logs for page rendering errors.
- **`src/server/database/base.py`** – Ensure base DB operations log unexpected issues.
- **`src/server/database/connection.py`** – Add connection lifecycle and error logs.
- **`src/server/database/init.py`** – Ensure DB init steps are logged.
- **`src/server/database/models.py`** – Validate model validation errors are logged.
- **`src/server/database/service.py`** – Confirm CRUD operations log with context.
- **`src/server/database/system_settings_service.py`** – Ensure system settings actions log correctly.
- **`src/server/database/transaction.py`** – Verify transaction start/commit/rollback logging.
- **`src/server/fastapi_app.py`** – Review startup/shutdown and error logging.
- **`src/server/middleware/auth.py`** – Ensure auth middleware logs auth events and failures.
- **`src/server/middleware/error_handler.py`** – Confirm error handling logs exceptions clearly.
- **`src/server/middleware/security.py`** – Ensure security events are logged with context.
- **`src/server/middleware/setup_redirect.py`** – Add logs for redirect behavior and errors.
- **`src/server/models/anime.py`** – Ensure model validation errors are logged.
- **`src/server/models/auth.py`** – Confirm auth model logging for invalid data.
- **`src/server/models/config.py`** – Add logs for config model validation.
- **`src/server/models/download.py`** – Ensure download model failures are logged.
- **`src/server/models/nfo.py`** – Ensure NFO model issues are logged.
- **`src/server/models/websocket.py`** – Confirm websocket model errors are logged.
- **`src/server/services/anime_service.py`** – Ensure operations log relevant ids and results.
- **`src/server/services/auth_service.py`** – Add logs for auth operations and failures.
- **`src/server/services/background_loader_service.py`** – Ensure background jobs log start/stop and errors.
- **`src/server/services/cache_service.py`** – Confirm caching logs are sane and useful.
- **`src/server/services/config_service.py`** – Ensure config updates and errors are logged.
- **`src/server/services/download_service.py`** – Add download lifecycle logs.
- **`src/server/services/initialization_service.py`** – Confirm initialization steps are logged.
- **`src/server/services/notification_service.py`** – Ensure notification dispatch/failure logs exist.
- **`src/server/services/progress_service.py`** – Add progress update logs.
- **`src/server/services/queue_repository.py`** – Ensure queue operations are logged.
- **`src/server/services/scan_service.py`** – Confirm scan lifecycle and failure logging.
- **`src/server/services/scheduler_service.py`** – Ensure scheduler actions and failures are logged.
- **`src/server/services/websocket_service.py`** – Confirm websocket lifecycle and message errors are logged.
- **`src/server/utils/dependencies.py`** – Ensure dependency errors are logged.
- **`src/server/utils/error_tracking.py`** – Verify error tracking logs contain context.
- **`src/server/utils/filesystem.py`** – Add filesystem operation logs and errors.
- **`src/server/utils/logging.py`** – Ensure helper logging utilities behave as expected.
- **`src/server/utils/log_manager.py`** – Review dynamic log-level changes and logs.
- **`src/server/utils/media.py`** – Add logs for media operations and failures.
- **`src/server/utils/metrics.py`** – Ensure metric collection/logging failures are captured.
- **`src/server/utils/system.py`** – Add system-level operation logs and error handling.
- **`src/server/utils/template_helpers.py`** – Ensure template rendering errors are logged.
- **`src/server/utils/templates.py`** – Add logs for template loading and render errors.
- **`src/server/utils/validators.py`** – Ensure validation failures are logged appropriately.
- **`src/server/web/middleware/__init__.py`** – Verify middleware initialization logging.

### 14b) Reference / low-touch files (review only)

- **`src/core/__init__.py`** – Verify any module-level logging is appropriate.
- **`src/core/providers/__init__.py`** – Verify module-level logging is appropriate.
- **`src/infrastructure/logging/__init__.py`** – Confirm module exports and initialization logs are appropriate.
- **`src/infrastructure/security/__init__.py`** – Verify module-level logging is appropriate.
- **`src/server/api/__init__.py`** – Verify any module setup logging.
- **`src/server/config/__init__.py`** – Verify config initialization logging.
- **`src/server/controllers/__init__.py`** – Verify module import logging is non-spammy.
- **`src/server/database/__init__.py`** – Confirm any initialization logs.
- **`src/server/exceptions/__init__.py`** – Confirm module-level logging is appropriate.
- **`src/server/models/__init__.py`** – Verify any module init logs.
- **`src/server/utils/__init__.py`** – Confirm initialization logging is appropriate.
- **`src/server/web/middleware/__init__.py`** – Verify middleware initialization logging.

---

> ✅ **Done:** This `tasks.md` file enumerates each relevant source file and a clear instruction for how to improve or add logging in that file.