Add failed tests to TODO list (136 failures)
This commit is contained in:
@@ -119,392 +119,91 @@ For each task completed:
|
||||
|
||||
## TODO List:
|
||||
|
||||
### ✅ **Task 1: Provider System Integration Tests** (Priority: CRITICAL) — COMPLETED (211 tests passing)
|
||||
### High Priority - Test Failures (136 total)
|
||||
|
||||
**Objective**: Create unit and integration tests for core provider orchestration system (6 files) that handles provider selection, failover, and health monitoring.
|
||||
#### 1. TMDB API Resilience Tests (26 failures)
|
||||
**Location**: `tests/integration/test_tmdb_resilience.py`, `tests/unit/test_tmdb_rate_limiting.py`
|
||||
**Issue**: `TypeError: 'coroutine' object does not support the asynchronous context manager protocol`
|
||||
**Root cause**: Mock session.get() returns coroutine instead of async context manager
|
||||
**Impact**: All TMDB API resilience and timeout tests failing
|
||||
- [ ] Fix mock setup in TMDB resilience tests
|
||||
- [ ] Fix mock setup in TMDB rate limiting tests
|
||||
- [ ] Ensure AsyncMock context managers are properly configured
|
||||
|
||||
**Target Files to Test**:
|
||||
#### 2. Config Backup/Restore Tests (18 failures)
|
||||
**Location**: `tests/integration/test_config_backup_restore.py`
|
||||
**Issue**: Authentication failures (401 Unauthorized)
|
||||
**Root cause**: authenticated_client fixture not properly authenticating
|
||||
**Affected tests**:
|
||||
- [ ] test_create_backup_with_default_name
|
||||
- [ ] test_multiple_backups_can_be_created
|
||||
- [ ] test_list_backups_returns_array
|
||||
- [ ] test_list_backups_contains_metadata
|
||||
- [ ] test_list_backups_shows_recently_created
|
||||
- [ ] test_restore_nonexistent_backup_fails
|
||||
- [ ] test_restore_backup_with_valid_backup
|
||||
- [ ] test_restore_creates_backup_before_restoring
|
||||
- [ ] test_restored_config_matches_backup
|
||||
- [ ] test_delete_existing_backup
|
||||
- [ ] test_delete_removes_backup_from_list
|
||||
- [ ] test_delete_removes_backup_file
|
||||
- [ ] test_delete_nonexistent_backup_fails
|
||||
- [ ] test_full_backup_restore_workflow
|
||||
- [ ] test_restore_with_invalid_backup_name
|
||||
- [ ] test_concurrent_backup_operations
|
||||
- [ ] test_backup_with_very_long_custom_name
|
||||
- [ ] test_backup_preserves_all_configuration_sections
|
||||
|
||||
- `src/core/providers/base_provider.py` - Abstract base class and interface
|
||||
- `src/core/providers/aniworld_provider.py` - Main provider (664 lines, core functionality)
|
||||
- `src/core/providers/provider_factory.py` - Provider instantiation logic
|
||||
- `src/core/providers/enhanced_provider.py` - Enhanced features and caching
|
||||
- `src/core/providers/monitored_provider.py` - Monitoring wrapper with metrics
|
||||
- `src/core/providers/config_manager.py` - Provider configuration management
|
||||
#### 3. Background Loader Service Tests (10 failures)
|
||||
**Location**: `tests/integration/test_async_series_loading.py`, `tests/unit/test_background_loader_session.py`, `tests/integration/test_anime_add_nfo_isolation.py`
|
||||
**Issues**: Service initialization, task processing, NFO loading
|
||||
- [ ] test_loader_start_stop - Fix worker_task vs worker_tasks attribute
|
||||
- [ ] test_add_series_loading_task - Tasks not being added to active_tasks
|
||||
- [ ] test_multiple_tasks_concurrent - Active tasks not being tracked
|
||||
- [ ] test_no_duplicate_tasks - No tasks registered
|
||||
- [ ] test_adding_tasks_is_fast - Active tasks empty
|
||||
- [ ] test_load_series_data_loads_missing_episodes - _load_episodes not called
|
||||
- [ ] test_add_anime_loads_nfo_only_for_new_anime - NFO service not called
|
||||
- [ ] test_add_anime_has_nfo_check_is_isolated - has_nfo check not called
|
||||
- [ ] test_multiple_anime_added_each_loads_independently - NFO service call count wrong
|
||||
- [ ] test_nfo_service_receives_correct_parameters - Call args is None
|
||||
|
||||
**Create Test Files**:
|
||||
**Unit Tests**:
|
||||
#### 4. Performance Tests (4 failures)
|
||||
**Location**: `tests/performance/test_large_library.py`, `tests/performance/test_api_load.py`
|
||||
**Issues**: Missing attributes, database not initialized, service not initialized
|
||||
- [ ] test_scanner_progress_reporting_1000_series - AttributeError: '_SerieClass' missing
|
||||
- [ ] test_database_query_performance_1000_series - Database not initialized
|
||||
- [ ] test_concurrent_scan_prevention - get_anime_service() missing required argument
|
||||
- [ ] test_health_endpoint_load - RPS too low (37.27 < 50 expected)
|
||||
|
||||
- `tests/unit/test_base_provider.py` - Abstract methods, interface contracts, inheritance
|
||||
- `tests/unit/test_aniworld_provider.py` - Anime catalog scraping, episode listing, streaming link extraction (mock HTML responses)
|
||||
- `tests/unit/test_provider_factory.py` - Factory instantiation, dependency injection, provider registration
|
||||
- `tests/unit/test_enhanced_provider.py` - Caching behavior, optimization features, decorator patterns
|
||||
- `tests/unit/test_monitored_provider.py` - Metrics collection, health checks, monitoring integration
|
||||
- `tests/unit/test_provider_config_manager.py` - Configuration loading, validation, defaults
|
||||
#### 5. NFO Tracking Tests (4 failures)
|
||||
**Location**: `tests/unit/test_anime_service.py`
|
||||
**Issue**: `TypeError: object MagicMock can't be used in 'await' expression`
|
||||
**Root cause**: Database mocks not properly configured for async
|
||||
- [ ] test_update_nfo_status_success
|
||||
- [ ] test_update_nfo_status_not_found
|
||||
- [ ] test_get_series_without_nfo
|
||||
- [ ] test_get_nfo_statistics
|
||||
|
||||
**Integration Tests**:
|
||||
#### 6. Concurrent Anime Add Tests (2 failures)
|
||||
**Location**: `tests/api/test_concurrent_anime_add.py`
|
||||
**Issue**: `RuntimeError: BackgroundLoaderService not initialized`
|
||||
**Root cause**: Service not initialized in test setup
|
||||
- [ ] test_concurrent_anime_add_requests
|
||||
- [ ] test_same_anime_concurrent_add
|
||||
|
||||
- `tests/integration/test_provider_failover_scenarios.py` - End-to-end provider switching when streaming fails
|
||||
- `tests/integration/test_provider_selection.py` - Provider selection based on availability, health status, priority
|
||||
#### 7. Other Test Failures (3 failures)
|
||||
- [ ] test_get_database_session_handles_http_exception - Database not initialized
|
||||
- [ ] test_anime_endpoint_returns_series_after_loading - Empty response (expects 2, got 0)
|
||||
|
||||
**Test Coverage Requirements**:
|
||||
|
||||
- Provider instantiation via factory pattern (all provider types)
|
||||
- Failover from failed provider to healthy backup (3+ provider scenario)
|
||||
- Health monitoring and circuit breaker patterns
|
||||
- Configuration loading from config.json and validation
|
||||
- Aniworld catalog scraping with mocked HTML responses
|
||||
- Episode listing and metadata extraction
|
||||
- Multi-provider scenarios with different health states
|
||||
- Provider priority and selection algorithm
|
||||
|
||||
**Expected Outcome**: ~80 tests total, 90%+ coverage for provider system
|
||||
|
||||
**Implementation Notes**:
|
||||
|
||||
- Mock HTML responses for aniworld_provider tests using BeautifulSoup fixtures
|
||||
- Test factory pattern returns correct provider instances
|
||||
- Integration tests should test full failover workflow: healthy provider → fails → switches to backup → succeeds
|
||||
- Use existing `test_provider_health.py` and `test_provider_failover.py` as reference
|
||||
- Mock external dependencies (HTTP, file system, database)
|
||||
- Test concurrent provider usage scenarios
|
||||
### Summary
|
||||
- **Total failures**: 136 out of 2503 tests
|
||||
- **Pass rate**: 94.6%
|
||||
- **Main issues**:
|
||||
1. AsyncMock configuration for TMDB tests
|
||||
2. Authentication in backup/restore tests
|
||||
3. Background loader service lifecycle
|
||||
4. Database mock configuration for async operations
|
||||
5. Service initialization in tests
|
||||
|
||||
---
|
||||
|
||||
### ✅ **Task 2: Security Infrastructure Tests** (Priority: CRITICAL) — COMPLETED (75 tests passing)
|
||||
|
||||
**Objective**: Create comprehensive tests for security modules handling encryption and database integrity (2 critical files).
|
||||
|
||||
**Target Files to Test**:
|
||||
|
||||
- `src/infrastructure/security/config_encryption.py` - Configuration encryption/decryption
|
||||
- `src/infrastructure/security/database_integrity.py` - Database integrity checks and validation
|
||||
|
||||
**Create Test Files**:
|
||||
|
||||
- `tests/unit/test_config_encryption.py`:
|
||||
- Encryption/decryption of sensitive configuration values
|
||||
- Key rotation and management lifecycle
|
||||
- AES-256 encryption validation
|
||||
- Decrypt failures with wrong key
|
||||
- Empty/null value handling
|
||||
- Multiple encryption rounds
|
||||
- Performance of encryption operations
|
||||
- `tests/unit/test_database_integrity.py`:
|
||||
- Database checksum calculation and validation
|
||||
- Corruption detection mechanisms
|
||||
- Integrity verification on application startup
|
||||
- Backup restoration on corruption detection
|
||||
- Schema validation against expected structure
|
||||
- Transaction integrity checks
|
||||
- `tests/security/test_encryption_security.py`:
|
||||
- Key strength validation (minimum bits)
|
||||
- Timing attack prevention
|
||||
- Secure key storage validation
|
||||
- Environment variable security
|
||||
- Encrypted data format validation
|
||||
- Key compromise scenarios
|
||||
|
||||
**Test Coverage Requirements**:
|
||||
|
||||
- Encryption algorithm correctness (encrypt → decrypt → original value)
|
||||
- Key management lifecycle (generation, rotation, revocation)
|
||||
- Database integrity check mechanisms
|
||||
- Corruption detection and recovery workflows
|
||||
- Security edge cases (key compromise, brute force attempts)
|
||||
- Performance testing for encryption operations (should not slow down app significantly)
|
||||
|
||||
**Expected Outcome**: ~40 tests total, 95%+ coverage for security modules
|
||||
|
||||
**Implementation Notes**:
|
||||
|
||||
- Read security module files first to understand cryptography library used
|
||||
- Test both successful and failed encryption/decryption scenarios
|
||||
- Mock file system for encrypted key storage tests
|
||||
- Use in-memory databases for integrity testing
|
||||
- Simulate database corruption scenarios
|
||||
- Follow security testing best practices from `tests/security/` directory
|
||||
- Ensure tests don't expose sensitive data in logs or output
|
||||
|
||||
---
|
||||
|
||||
### ✅ **Task 3: Error Handling Tests** (Priority: HIGH) — COMPLETED (74 tests passing)
|
||||
|
||||
**Objective**: Create comprehensive tests for error handling and recovery mechanisms (2 files) to ensure robust error management across the application.
|
||||
|
||||
**Target Files to Test**:
|
||||
|
||||
- `src/core/error_handler.py` - Core error handling and retry logic
|
||||
- `src/server/middleware/error_handler.py` - API error handling middleware
|
||||
|
||||
**Create Test Files**:
|
||||
|
||||
- `tests/unit/test_core_error_handler.py`:
|
||||
- Retry logic with exponential backoff
|
||||
- Maximum retry limits enforcement
|
||||
- Error classification (transient vs permanent errors)
|
||||
- Error recovery strategies
|
||||
- Circuit breaker integration
|
||||
- Timeout handling
|
||||
- Resource cleanup on errors
|
||||
- `tests/unit/test_middleware_error_handler.py`:
|
||||
- HTTP error response formatting (JSON structure)
|
||||
- Stack trace sanitization in production mode
|
||||
- Error logging integration with structlog
|
||||
- Custom exception handling (AnimeNotFound, ProviderError, etc.)
|
||||
- 400/404/500 error responses
|
||||
- Error context preservation
|
||||
- CORS headers on error responses
|
||||
- `tests/integration/test_error_recovery_workflows.py`:
|
||||
- End-to-end error recovery: download fails → retry → success
|
||||
- Provider failover on errors (primary fails → backup succeeds)
|
||||
- Database transaction rollback on errors
|
||||
- User notification on errors via WebSocket
|
||||
- Cascading error handling (error in one service affects others)
|
||||
- Error recovery after temporary outages
|
||||
|
||||
**Test Coverage Requirements**:
|
||||
|
||||
- Transient vs permanent error distinction
|
||||
- Retry exhaustion scenarios (max retries reached)
|
||||
- Error reporting to users (proper messages, no stack traces)
|
||||
- Error logging with proper context
|
||||
- Recovery workflows for common errors
|
||||
- Error handling doesn't leak resources (connections, file handles)
|
||||
|
||||
**Expected Outcome**: ~50 tests total, 90%+ coverage for error handling
|
||||
|
||||
**Implementation Notes**:
|
||||
|
||||
- Test retry logic with controlled failure scenarios
|
||||
- Mock external services to simulate errors
|
||||
- Verify exponential backoff timing
|
||||
- Test error message clarity and usefulness
|
||||
- Integration tests should verify end-to-end recovery
|
||||
- Use `pytest.raises` for exception testing
|
||||
- Mock time.sleep for faster retry tests
|
||||
|
||||
---
|
||||
|
||||
### ✅ **Task 4: Services & Utilities Tests** (Priority: MEDIUM) — COMPLETED (64 tests passing)
|
||||
|
||||
**Objective**: Create tests for undertested service and utility modules to increase coverage of business logic and helper functions (5 files).
|
||||
|
||||
**Target Files to Test**:
|
||||
|
||||
- `src/core/services/series_manager_service.py` - Series orchestration logic
|
||||
- `src/core/services/nfo_factory.py` - NFO service factory pattern
|
||||
- `src/server/utils/media.py` - Media file validation utilities
|
||||
- `src/server/utils/templates.py` - Template rendering utilities
|
||||
- `src/server/controllers/error_controller.py` - Error page controller
|
||||
|
||||
**Create Test Files**:
|
||||
|
||||
- `tests/unit/test_series_manager_service.py`:
|
||||
- Series orchestration and lifecycle management
|
||||
- Episode management (add, remove, update)
|
||||
- Season handling and organization
|
||||
- Series state management
|
||||
- Interaction with SeriesApp
|
||||
- `tests/unit/test_nfo_factory.py`:
|
||||
- Factory pattern instantiation of NFO services
|
||||
- Dependency injection setup
|
||||
- Service lifecycle (singleton vs transient)
|
||||
- Configuration passing to services
|
||||
- `tests/unit/test_media_utils.py`:
|
||||
- Media file validation (video formats)
|
||||
- Codec detection (H.264, H.265, etc.)
|
||||
- Metadata extraction (duration, resolution)
|
||||
- File size checks and validation
|
||||
- Corrupt file detection
|
||||
- `tests/unit/test_templates_utils.py`:
|
||||
- Template rendering with Jinja2
|
||||
- Context injection and variable passing
|
||||
- Error page rendering
|
||||
- Template caching behavior
|
||||
- Custom filters and functions
|
||||
- `tests/unit/test_error_controller.py`:
|
||||
- 404 page rendering with context
|
||||
- 500 error page with safe error info
|
||||
- Error context passing to templates
|
||||
- Static file errors
|
||||
- API error responses
|
||||
|
||||
**Test Coverage Requirements**:
|
||||
|
||||
- Service initialization patterns and dependency setup
|
||||
- Factory method correctness and proper instance types
|
||||
- Media file operations with various formats
|
||||
- Template rendering edge cases (missing variables, errors)
|
||||
- Error controller response formatting
|
||||
|
||||
**Expected Outcome**: ~60 tests total, 85%+ coverage for each module
|
||||
|
||||
**Implementation Notes**:
|
||||
|
||||
- Mock file system for media utility tests
|
||||
- Use temporary files for media validation tests
|
||||
- Mock Jinja2 environment for template tests
|
||||
- Test both success and error paths
|
||||
- Verify proper resource cleanup
|
||||
- Use existing service test patterns as reference
|
||||
|
||||
---
|
||||
|
||||
### ✅ **Task 5: Infrastructure Logging Tests** (Priority: MEDIUM) — COMPLETED (49 tests passing)
|
||||
|
||||
**Objective**: Create tests for logging infrastructure to ensure proper log configuration, formatting, and rotation (2 files).
|
||||
|
||||
**Target Files to Test**:
|
||||
|
||||
- `src/infrastructure/logging/logger.py` - Main logger configuration
|
||||
- `src/infrastructure/logging/uvicorn_config.py` - Uvicorn logging configuration
|
||||
|
||||
**Create Test Files**:
|
||||
|
||||
- `tests/unit/test_infrastructure_logger.py`:
|
||||
- Logger initialization and setup
|
||||
- Log level configuration (DEBUG, INFO, WARNING, ERROR)
|
||||
- Log formatting (JSON, text formats)
|
||||
- File rotation behavior
|
||||
- Multiple handler setup (console, file, syslog)
|
||||
- Structured logging with context
|
||||
- Logger hierarchy and propagation
|
||||
- `tests/unit/test_uvicorn_logging_config.py`:
|
||||
- Uvicorn access log configuration
|
||||
- Error log configuration
|
||||
- Log format customization for HTTP requests
|
||||
- Integration with main application logger
|
||||
- Log level filtering for Uvicorn logs
|
||||
- Performance logging (request timing)
|
||||
|
||||
**Test Coverage Requirements**:
|
||||
|
||||
- Logger configuration loading from settings
|
||||
- Log output format validation (JSON structure, fields)
|
||||
- Log level filtering works correctly
|
||||
- File rotation behavior (size-based, time-based)
|
||||
- Integration with structlog for structured logging
|
||||
- Performance impact is minimal
|
||||
|
||||
**Expected Outcome**: ~30 tests total, 80%+ coverage for logging infrastructure
|
||||
|
||||
**Implementation Notes**:
|
||||
|
||||
- Use temporary log files for testing
|
||||
- Capture log output using logging.handlers.MemoryHandler
|
||||
- Test log rotation without waiting for actual rotation triggers
|
||||
- Verify log format matches expected structure
|
||||
- Mock file system for file handler tests
|
||||
- Test various log levels and ensure filtering works
|
||||
- Verify no sensitive data in logs
|
||||
|
||||
---
|
||||
|
||||
### ✅ **Task 6: CLI Tool Tests** (Priority: LOW) — COMPLETED (25 tests passing)
|
||||
|
||||
**Objective**: Create tests for NFO command-line interface tool used for DevOps and maintenance workflows (1 file).
|
||||
|
||||
**Target File to Test**:
|
||||
|
||||
- `src/cli/nfo_cli.py` - NFO management CLI commands
|
||||
|
||||
**Create Test Files**:
|
||||
|
||||
- `tests/unit/test_nfo_cli.py`:
|
||||
- Command parsing (argparse or click)
|
||||
- Argument validation (required args, types)
|
||||
- Batch operations (multiple NFO files)
|
||||
- Error reporting and user-friendly messages
|
||||
- Output formatting (table, JSON, text)
|
||||
- Help text generation
|
||||
- Exit codes (0 for success, non-zero for errors)
|
||||
- `tests/integration/test_cli_workflows.py`:
|
||||
- NFO creation via CLI end-to-end
|
||||
- Batch NFO update workflow
|
||||
- CLI + database integration
|
||||
- CLI + API integration (if CLI calls API)
|
||||
- Error handling in CLI workflows
|
||||
- File system operations (read/write NFO files)
|
||||
|
||||
**Test Coverage Requirements**:
|
||||
|
||||
- CLI argument parsing for all commands
|
||||
- Batch processing multiple files
|
||||
- Error messages are clear and actionable
|
||||
- Output formatting matches specification
|
||||
- Integration with core services (NFO service)
|
||||
- File operations work correctly
|
||||
|
||||
**Expected Outcome**: ~35 tests total, 80%+ coverage for CLI module
|
||||
|
||||
**Implementation Notes**:
|
||||
|
||||
- Read `src/cli/nfo_cli.py` first to understand commands
|
||||
- Use `subprocess` or `click.testing.CliRunner` for integration tests
|
||||
- Mock file system operations
|
||||
- Test with various command-line arguments
|
||||
- Verify exit codes are correct
|
||||
- Test help text generation
|
||||
- Use temporary directories for file operations
|
||||
- Follow patterns from existing CLI tests if any exist
|
||||
|
||||
---
|
||||
|
||||
### ✅ **Task 7: Edge Case & Regression Tests** (Priority: MEDIUM) — COMPLETED (69 tests passing)
|
||||
|
||||
**Objective**: Add edge case coverage and regression tests across existing modules to catch rare bugs and prevent reintroduction of fixed bugs (4 new test files).
|
||||
|
||||
**Create Test Files**:
|
||||
|
||||
- `tests/unit/test_provider_edge_cases.py`:
|
||||
- Malformed HTML responses from providers
|
||||
- Missing episode data in provider responses
|
||||
- Invalid streaming URLs (malformed, expired)
|
||||
- Unicode characters in anime titles
|
||||
- Special characters in filenames
|
||||
- Empty responses from providers
|
||||
- Partial data from providers
|
||||
- Provider timeout scenarios
|
||||
- `tests/integration/test_concurrent_operations.py`:
|
||||
- Concurrent downloads from same provider
|
||||
- Parallel NFO generation for multiple series
|
||||
- Race conditions in queue management
|
||||
- Database lock contention under load
|
||||
- WebSocket broadcasts during concurrent operations
|
||||
- Cache consistency with concurrent writes
|
||||
- `tests/api/test_rate_limiting_edge_cases.py`:
|
||||
- Rate limiting with multiple IP addresses
|
||||
- Rate limit reset behavior
|
||||
- Burst traffic handling
|
||||
- Rate limit per-user vs per-IP
|
||||
- Rate limit with authenticated vs anonymous users
|
||||
- Rate limit bypass attempts
|
||||
- `tests/integration/test_database_edge_cases.py`:
|
||||
- Database lock contention scenarios
|
||||
- Large transaction rollback (100+ operations)
|
||||
- Connection pool exhaustion
|
||||
- Slow query handling
|
||||
- Database file growth and vacuum
|
||||
- Concurrent write conflicts
|
||||
- Foreign key constraint violations
|
||||
|
||||
**Test Coverage Requirements**:
|
||||
|
||||
- Edge cases that aren't covered by existing tests
|
||||
- Known bugs that were fixed (regression tests)
|
||||
- Concurrent operation safety
|
||||
- Resource exhaustion scenarios
|
||||
- Boundary conditions (empty data, very large data)
|
||||
|
||||
**Expected Outcome**: ~50 tests total, targeting known edge cases and regression scenarios
|
||||
|
||||
**Implementation Notes**:
|
||||
|
||||
- Review git history for bug fixes to create regression tests
|
||||
- Test boundary conditions (0, 1, max values)
|
||||
- Simulate resource exhaustion (disk full, memory limit)
|
||||
- Test concurrent operations with threading/asyncio
|
||||
- Use property-based testing with hypothesis if appropriate
|
||||
- Mock external services to simulate edge cases
|
||||
- Test error recovery from edge cases
|
||||
|
||||
Reference in New Issue
Block a user