diff --git a/docs/instructions.md b/docs/instructions.md index a11b4f5..ef6f3db 100644 --- a/docs/instructions.md +++ b/docs/instructions.md @@ -119,392 +119,91 @@ For each task completed: ## TODO List: -### ✅ **Task 1: Provider System Integration Tests** (Priority: CRITICAL) — COMPLETED (211 tests passing) +### High Priority - Test Failures (136 total) -**Objective**: Create unit and integration tests for core provider orchestration system (6 files) that handles provider selection, failover, and health monitoring. +#### 1. TMDB API Resilience Tests (26 failures) +**Location**: `tests/integration/test_tmdb_resilience.py`, `tests/unit/test_tmdb_rate_limiting.py` +**Issue**: `TypeError: 'coroutine' object does not support the asynchronous context manager protocol` +**Root cause**: Mock session.get() returns coroutine instead of async context manager +**Impact**: All TMDB API resilience and timeout tests failing +- [ ] Fix mock setup in TMDB resilience tests +- [ ] Fix mock setup in TMDB rate limiting tests +- [ ] Ensure AsyncMock context managers are properly configured -**Target Files to Test**: +#### 2. Config Backup/Restore Tests (18 failures) +**Location**: `tests/integration/test_config_backup_restore.py` +**Issue**: Authentication failures (401 Unauthorized) +**Root cause**: authenticated_client fixture not properly authenticating +**Affected tests**: +- [ ] test_create_backup_with_default_name +- [ ] test_multiple_backups_can_be_created +- [ ] test_list_backups_returns_array +- [ ] test_list_backups_contains_metadata +- [ ] test_list_backups_shows_recently_created +- [ ] test_restore_nonexistent_backup_fails +- [ ] test_restore_backup_with_valid_backup +- [ ] test_restore_creates_backup_before_restoring +- [ ] test_restored_config_matches_backup +- [ ] test_delete_existing_backup +- [ ] test_delete_removes_backup_from_list +- [ ] test_delete_removes_backup_file +- [ ] test_delete_nonexistent_backup_fails +- [ ] test_full_backup_restore_workflow +- [ ] test_restore_with_invalid_backup_name +- [ ] test_concurrent_backup_operations +- [ ] test_backup_with_very_long_custom_name +- [ ] test_backup_preserves_all_configuration_sections -- `src/core/providers/base_provider.py` - Abstract base class and interface -- `src/core/providers/aniworld_provider.py` - Main provider (664 lines, core functionality) -- `src/core/providers/provider_factory.py` - Provider instantiation logic -- `src/core/providers/enhanced_provider.py` - Enhanced features and caching -- `src/core/providers/monitored_provider.py` - Monitoring wrapper with metrics -- `src/core/providers/config_manager.py` - Provider configuration management +#### 3. Background Loader Service Tests (10 failures) +**Location**: `tests/integration/test_async_series_loading.py`, `tests/unit/test_background_loader_session.py`, `tests/integration/test_anime_add_nfo_isolation.py` +**Issues**: Service initialization, task processing, NFO loading +- [ ] test_loader_start_stop - Fix worker_task vs worker_tasks attribute +- [ ] test_add_series_loading_task - Tasks not being added to active_tasks +- [ ] test_multiple_tasks_concurrent - Active tasks not being tracked +- [ ] test_no_duplicate_tasks - No tasks registered +- [ ] test_adding_tasks_is_fast - Active tasks empty +- [ ] test_load_series_data_loads_missing_episodes - _load_episodes not called +- [ ] test_add_anime_loads_nfo_only_for_new_anime - NFO service not called +- [ ] test_add_anime_has_nfo_check_is_isolated - has_nfo check not called +- [ ] test_multiple_anime_added_each_loads_independently - NFO service call count wrong +- [ ] test_nfo_service_receives_correct_parameters - Call args is None -**Create Test Files**: -**Unit Tests**: +#### 4. Performance Tests (4 failures) +**Location**: `tests/performance/test_large_library.py`, `tests/performance/test_api_load.py` +**Issues**: Missing attributes, database not initialized, service not initialized +- [ ] test_scanner_progress_reporting_1000_series - AttributeError: '_SerieClass' missing +- [ ] test_database_query_performance_1000_series - Database not initialized +- [ ] test_concurrent_scan_prevention - get_anime_service() missing required argument +- [ ] test_health_endpoint_load - RPS too low (37.27 < 50 expected) -- `tests/unit/test_base_provider.py` - Abstract methods, interface contracts, inheritance -- `tests/unit/test_aniworld_provider.py` - Anime catalog scraping, episode listing, streaming link extraction (mock HTML responses) -- `tests/unit/test_provider_factory.py` - Factory instantiation, dependency injection, provider registration -- `tests/unit/test_enhanced_provider.py` - Caching behavior, optimization features, decorator patterns -- `tests/unit/test_monitored_provider.py` - Metrics collection, health checks, monitoring integration -- `tests/unit/test_provider_config_manager.py` - Configuration loading, validation, defaults +#### 5. NFO Tracking Tests (4 failures) +**Location**: `tests/unit/test_anime_service.py` +**Issue**: `TypeError: object MagicMock can't be used in 'await' expression` +**Root cause**: Database mocks not properly configured for async +- [ ] test_update_nfo_status_success +- [ ] test_update_nfo_status_not_found +- [ ] test_get_series_without_nfo +- [ ] test_get_nfo_statistics -**Integration Tests**: +#### 6. Concurrent Anime Add Tests (2 failures) +**Location**: `tests/api/test_concurrent_anime_add.py` +**Issue**: `RuntimeError: BackgroundLoaderService not initialized` +**Root cause**: Service not initialized in test setup +- [ ] test_concurrent_anime_add_requests +- [ ] test_same_anime_concurrent_add -- `tests/integration/test_provider_failover_scenarios.py` - End-to-end provider switching when streaming fails -- `tests/integration/test_provider_selection.py` - Provider selection based on availability, health status, priority +#### 7. Other Test Failures (3 failures) +- [ ] test_get_database_session_handles_http_exception - Database not initialized +- [ ] test_anime_endpoint_returns_series_after_loading - Empty response (expects 2, got 0) -**Test Coverage Requirements**: - -- Provider instantiation via factory pattern (all provider types) -- Failover from failed provider to healthy backup (3+ provider scenario) -- Health monitoring and circuit breaker patterns -- Configuration loading from config.json and validation -- Aniworld catalog scraping with mocked HTML responses -- Episode listing and metadata extraction -- Multi-provider scenarios with different health states -- Provider priority and selection algorithm - -**Expected Outcome**: ~80 tests total, 90%+ coverage for provider system - -**Implementation Notes**: - -- Mock HTML responses for aniworld_provider tests using BeautifulSoup fixtures -- Test factory pattern returns correct provider instances -- Integration tests should test full failover workflow: healthy provider → fails → switches to backup → succeeds -- Use existing `test_provider_health.py` and `test_provider_failover.py` as reference -- Mock external dependencies (HTTP, file system, database) -- Test concurrent provider usage scenarios +### Summary +- **Total failures**: 136 out of 2503 tests +- **Pass rate**: 94.6% +- **Main issues**: + 1. AsyncMock configuration for TMDB tests + 2. Authentication in backup/restore tests + 3. Background loader service lifecycle + 4. Database mock configuration for async operations + 5. Service initialization in tests --- - -### ✅ **Task 2: Security Infrastructure Tests** (Priority: CRITICAL) — COMPLETED (75 tests passing) - -**Objective**: Create comprehensive tests for security modules handling encryption and database integrity (2 critical files). - -**Target Files to Test**: - -- `src/infrastructure/security/config_encryption.py` - Configuration encryption/decryption -- `src/infrastructure/security/database_integrity.py` - Database integrity checks and validation - -**Create Test Files**: - -- `tests/unit/test_config_encryption.py`: - - Encryption/decryption of sensitive configuration values - - Key rotation and management lifecycle - - AES-256 encryption validation - - Decrypt failures with wrong key - - Empty/null value handling - - Multiple encryption rounds - - Performance of encryption operations -- `tests/unit/test_database_integrity.py`: - - Database checksum calculation and validation - - Corruption detection mechanisms - - Integrity verification on application startup - - Backup restoration on corruption detection - - Schema validation against expected structure - - Transaction integrity checks -- `tests/security/test_encryption_security.py`: - - Key strength validation (minimum bits) - - Timing attack prevention - - Secure key storage validation - - Environment variable security - - Encrypted data format validation - - Key compromise scenarios - -**Test Coverage Requirements**: - -- Encryption algorithm correctness (encrypt → decrypt → original value) -- Key management lifecycle (generation, rotation, revocation) -- Database integrity check mechanisms -- Corruption detection and recovery workflows -- Security edge cases (key compromise, brute force attempts) -- Performance testing for encryption operations (should not slow down app significantly) - -**Expected Outcome**: ~40 tests total, 95%+ coverage for security modules - -**Implementation Notes**: - -- Read security module files first to understand cryptography library used -- Test both successful and failed encryption/decryption scenarios -- Mock file system for encrypted key storage tests -- Use in-memory databases for integrity testing -- Simulate database corruption scenarios -- Follow security testing best practices from `tests/security/` directory -- Ensure tests don't expose sensitive data in logs or output - ---- - -### ✅ **Task 3: Error Handling Tests** (Priority: HIGH) — COMPLETED (74 tests passing) - -**Objective**: Create comprehensive tests for error handling and recovery mechanisms (2 files) to ensure robust error management across the application. - -**Target Files to Test**: - -- `src/core/error_handler.py` - Core error handling and retry logic -- `src/server/middleware/error_handler.py` - API error handling middleware - -**Create Test Files**: - -- `tests/unit/test_core_error_handler.py`: - - Retry logic with exponential backoff - - Maximum retry limits enforcement - - Error classification (transient vs permanent errors) - - Error recovery strategies - - Circuit breaker integration - - Timeout handling - - Resource cleanup on errors -- `tests/unit/test_middleware_error_handler.py`: - - HTTP error response formatting (JSON structure) - - Stack trace sanitization in production mode - - Error logging integration with structlog - - Custom exception handling (AnimeNotFound, ProviderError, etc.) - - 400/404/500 error responses - - Error context preservation - - CORS headers on error responses -- `tests/integration/test_error_recovery_workflows.py`: - - End-to-end error recovery: download fails → retry → success - - Provider failover on errors (primary fails → backup succeeds) - - Database transaction rollback on errors - - User notification on errors via WebSocket - - Cascading error handling (error in one service affects others) - - Error recovery after temporary outages - -**Test Coverage Requirements**: - -- Transient vs permanent error distinction -- Retry exhaustion scenarios (max retries reached) -- Error reporting to users (proper messages, no stack traces) -- Error logging with proper context -- Recovery workflows for common errors -- Error handling doesn't leak resources (connections, file handles) - -**Expected Outcome**: ~50 tests total, 90%+ coverage for error handling - -**Implementation Notes**: - -- Test retry logic with controlled failure scenarios -- Mock external services to simulate errors -- Verify exponential backoff timing -- Test error message clarity and usefulness -- Integration tests should verify end-to-end recovery -- Use `pytest.raises` for exception testing -- Mock time.sleep for faster retry tests - ---- - -### ✅ **Task 4: Services & Utilities Tests** (Priority: MEDIUM) — COMPLETED (64 tests passing) - -**Objective**: Create tests for undertested service and utility modules to increase coverage of business logic and helper functions (5 files). - -**Target Files to Test**: - -- `src/core/services/series_manager_service.py` - Series orchestration logic -- `src/core/services/nfo_factory.py` - NFO service factory pattern -- `src/server/utils/media.py` - Media file validation utilities -- `src/server/utils/templates.py` - Template rendering utilities -- `src/server/controllers/error_controller.py` - Error page controller - -**Create Test Files**: - -- `tests/unit/test_series_manager_service.py`: - - Series orchestration and lifecycle management - - Episode management (add, remove, update) - - Season handling and organization - - Series state management - - Interaction with SeriesApp -- `tests/unit/test_nfo_factory.py`: - - Factory pattern instantiation of NFO services - - Dependency injection setup - - Service lifecycle (singleton vs transient) - - Configuration passing to services -- `tests/unit/test_media_utils.py`: - - Media file validation (video formats) - - Codec detection (H.264, H.265, etc.) - - Metadata extraction (duration, resolution) - - File size checks and validation - - Corrupt file detection -- `tests/unit/test_templates_utils.py`: - - Template rendering with Jinja2 - - Context injection and variable passing - - Error page rendering - - Template caching behavior - - Custom filters and functions -- `tests/unit/test_error_controller.py`: - - 404 page rendering with context - - 500 error page with safe error info - - Error context passing to templates - - Static file errors - - API error responses - -**Test Coverage Requirements**: - -- Service initialization patterns and dependency setup -- Factory method correctness and proper instance types -- Media file operations with various formats -- Template rendering edge cases (missing variables, errors) -- Error controller response formatting - -**Expected Outcome**: ~60 tests total, 85%+ coverage for each module - -**Implementation Notes**: - -- Mock file system for media utility tests -- Use temporary files for media validation tests -- Mock Jinja2 environment for template tests -- Test both success and error paths -- Verify proper resource cleanup -- Use existing service test patterns as reference - ---- - -### ✅ **Task 5: Infrastructure Logging Tests** (Priority: MEDIUM) — COMPLETED (49 tests passing) - -**Objective**: Create tests for logging infrastructure to ensure proper log configuration, formatting, and rotation (2 files). - -**Target Files to Test**: - -- `src/infrastructure/logging/logger.py` - Main logger configuration -- `src/infrastructure/logging/uvicorn_config.py` - Uvicorn logging configuration - -**Create Test Files**: - -- `tests/unit/test_infrastructure_logger.py`: - - Logger initialization and setup - - Log level configuration (DEBUG, INFO, WARNING, ERROR) - - Log formatting (JSON, text formats) - - File rotation behavior - - Multiple handler setup (console, file, syslog) - - Structured logging with context - - Logger hierarchy and propagation -- `tests/unit/test_uvicorn_logging_config.py`: - - Uvicorn access log configuration - - Error log configuration - - Log format customization for HTTP requests - - Integration with main application logger - - Log level filtering for Uvicorn logs - - Performance logging (request timing) - -**Test Coverage Requirements**: - -- Logger configuration loading from settings -- Log output format validation (JSON structure, fields) -- Log level filtering works correctly -- File rotation behavior (size-based, time-based) -- Integration with structlog for structured logging -- Performance impact is minimal - -**Expected Outcome**: ~30 tests total, 80%+ coverage for logging infrastructure - -**Implementation Notes**: - -- Use temporary log files for testing -- Capture log output using logging.handlers.MemoryHandler -- Test log rotation without waiting for actual rotation triggers -- Verify log format matches expected structure -- Mock file system for file handler tests -- Test various log levels and ensure filtering works -- Verify no sensitive data in logs - ---- - -### ✅ **Task 6: CLI Tool Tests** (Priority: LOW) — COMPLETED (25 tests passing) - -**Objective**: Create tests for NFO command-line interface tool used for DevOps and maintenance workflows (1 file). - -**Target File to Test**: - -- `src/cli/nfo_cli.py` - NFO management CLI commands - -**Create Test Files**: - -- `tests/unit/test_nfo_cli.py`: - - Command parsing (argparse or click) - - Argument validation (required args, types) - - Batch operations (multiple NFO files) - - Error reporting and user-friendly messages - - Output formatting (table, JSON, text) - - Help text generation - - Exit codes (0 for success, non-zero for errors) -- `tests/integration/test_cli_workflows.py`: - - NFO creation via CLI end-to-end - - Batch NFO update workflow - - CLI + database integration - - CLI + API integration (if CLI calls API) - - Error handling in CLI workflows - - File system operations (read/write NFO files) - -**Test Coverage Requirements**: - -- CLI argument parsing for all commands -- Batch processing multiple files -- Error messages are clear and actionable -- Output formatting matches specification -- Integration with core services (NFO service) -- File operations work correctly - -**Expected Outcome**: ~35 tests total, 80%+ coverage for CLI module - -**Implementation Notes**: - -- Read `src/cli/nfo_cli.py` first to understand commands -- Use `subprocess` or `click.testing.CliRunner` for integration tests -- Mock file system operations -- Test with various command-line arguments -- Verify exit codes are correct -- Test help text generation -- Use temporary directories for file operations -- Follow patterns from existing CLI tests if any exist - ---- - -### ✅ **Task 7: Edge Case & Regression Tests** (Priority: MEDIUM) — COMPLETED (69 tests passing) - -**Objective**: Add edge case coverage and regression tests across existing modules to catch rare bugs and prevent reintroduction of fixed bugs (4 new test files). - -**Create Test Files**: - -- `tests/unit/test_provider_edge_cases.py`: - - Malformed HTML responses from providers - - Missing episode data in provider responses - - Invalid streaming URLs (malformed, expired) - - Unicode characters in anime titles - - Special characters in filenames - - Empty responses from providers - - Partial data from providers - - Provider timeout scenarios -- `tests/integration/test_concurrent_operations.py`: - - Concurrent downloads from same provider - - Parallel NFO generation for multiple series - - Race conditions in queue management - - Database lock contention under load - - WebSocket broadcasts during concurrent operations - - Cache consistency with concurrent writes -- `tests/api/test_rate_limiting_edge_cases.py`: - - Rate limiting with multiple IP addresses - - Rate limit reset behavior - - Burst traffic handling - - Rate limit per-user vs per-IP - - Rate limit with authenticated vs anonymous users - - Rate limit bypass attempts -- `tests/integration/test_database_edge_cases.py`: - - Database lock contention scenarios - - Large transaction rollback (100+ operations) - - Connection pool exhaustion - - Slow query handling - - Database file growth and vacuum - - Concurrent write conflicts - - Foreign key constraint violations - -**Test Coverage Requirements**: - -- Edge cases that aren't covered by existing tests -- Known bugs that were fixed (regression tests) -- Concurrent operation safety -- Resource exhaustion scenarios -- Boundary conditions (empty data, very large data) - -**Expected Outcome**: ~50 tests total, targeting known edge cases and regression scenarios - -**Implementation Notes**: - -- Review git history for bug fixes to create regression tests -- Test boundary conditions (0, 1, max values) -- Simulate resource exhaustion (disk full, memory limit) -- Test concurrent operations with threading/asyncio -- Use property-based testing with hypothesis if appropriate -- Mock external services to simulate edge cases -- Test error recovery from edge cases