lukas.pupkalipinski/Aniworld
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
846176f1149007592288dbb81217b38b0db7014b
Aniworld/docs/instructions.md

25 KiB
Raw Blame History

Aniworld Web Application Development Instructions

This document provides detailed tasks for AI agents to implement a modern web application for the Aniworld anime download manager. All tasks should follow the coding guidelines specified in the project's copilot instructions.

Project Overview

The goal is to create a FastAPI-based web application that provides a modern interface for the existing Aniworld anime download functionality. The core anime logic should remain in SeriesApp.py while the web layer provides REST API endpoints and a responsive UI.

Architecture Principles

  • Single Responsibility: Each file/class has one clear purpose
  • Dependency Injection: Use FastAPI's dependency system
  • Clean Separation: Web layer calls core logic, never the reverse
  • File Size Limit: Maximum 500 lines per file
  • Type Hints: Use comprehensive type annotations
  • Error Handling: Proper exception handling and logging

Additional Implementation Guidelines

Code Style and Standards

  • Type Hints: Use comprehensive type annotations throughout all modules
  • Docstrings: Follow PEP 257 for function and class documentation
  • Error Handling: Implement custom exception classes with meaningful messages
  • Logging: Use structured logging with appropriate log levels
  • Security: Validate all inputs and sanitize outputs
  • Performance: Use async/await patterns for I/O operations

📞 Escalation

If you encounter:

  • Architecture issues requiring design decisions
  • Tests that conflict with documented requirements
  • Breaking changes needed
  • Unclear requirements or expectations

Document the issue and escalate rather than guessing.

<EFBFBD> Credentials

Admin Login:

  • Username: admin
  • Password: Hallo123!

<EFBFBD>📚 Helpful Commands

# Run all tests
conda run -n AniWorld python -m pytest tests/ -v --tb=short

# Run specific test file
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py -v

# Run specific test class
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py::TestWebSocketService -v

# Run specific test
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py::TestWebSocketService::test_broadcast_download_progress -v

# Run with extra verbosity
conda run -n AniWorld python -m pytest tests/ -vv

# Run with full traceback
conda run -n AniWorld python -m pytest tests/ -v --tb=long

# Run and stop at first failure
conda run -n AniWorld python -m pytest tests/ -v -x

# Run tests matching pattern
conda run -n AniWorld python -m pytest tests/ -v -k "auth"

# Show all print statements
conda run -n AniWorld python -m pytest tests/ -v -s

#Run app
conda run -n AniWorld python -m uvicorn src.server.fastapi_app:app --host 127.0.0.1 --port 8000 --reload

Implementation Notes

  1. Incremental Development: Implement features incrementally, testing each component thoroughly before moving to the next
  2. Code Review: Review all generated code for adherence to project standards
  3. Documentation: Document all public APIs and complex logic
  4. Testing: Maintain test coverage above 80% for all new code
  5. Performance: Profile and optimize critical paths, especially download and streaming operations
  6. Security: Regular security audits and dependency updates
  7. Monitoring: Implement comprehensive monitoring and alerting
  8. Maintenance: Plan for regular maintenance and updates

Task Completion Checklist

For each task completed:

  • Implementation follows coding standards
  • Unit tests written and passing
  • Integration tests passing
  • Documentation updated
  • Error handling implemented
  • Logging added
  • Security considerations addressed
  • Performance validated
  • Code reviewed
  • Task marked as complete in instructions.md
  • Infrastructure.md updated and other docs
  • Changes committed to git; keep your messages in git short and clear
  • Take the next task

TODO List:

Phase 1: Critical Security & Infrastructure Tests (P0)

Task 1: Implement Security Middleware Tests ✅

Priority: P0 | Effort: Medium | Coverage Target: 90%+ | Status: COMPLETE

Objective: Test all security middleware components to ensure security headers and rate limiting work correctly.

Files to Test:

What Was Tested:

  1. Security headers correctly added (HSTS, X-Frame-Options, CSP, Referrer-Policy, X-Content-Type-Options) ✅
  2. CSP policy directives properly formatted ✅
  3. XSS protection escaping works correctly ✅
  4. Rate limiting tracks requests per IP and enforces limits ✅
  5. Rate limit cleanup removes old history to prevent memory leaks ✅
  6. Middleware order doesn't cause conflicts ✅
  7. Error responses include security headers ✅
  8. Request sanitization blocks SQL injection and XSS attacks ✅
  9. Content type and request size validation ✅
  10. Origin-based rate limiting for CORS requests ✅

Results:

  • Test File: tests/unit/test_security_middleware.py
  • Tests Created: 48 comprehensive tests
  • Coverage Achieved: 95% total (security.py: 97%, auth.py: 92%)
  • Target: 90%+ ✅ EXCEEDED
  • All Tests Passing: ✅

Bug Fixes:

  • Fixed MutableHeaders.pop() AttributeError in security.py (lines 100-101) - changed to use del with try/except

Notes:

  • Documented current limitation where '/' in PUBLIC_PATHS causes all paths to match as public
  • Rate limiting functionality thoroughly tested including cleanup and per-IP tracking
  • All security header configurations tested with various options
  • CSP tested in both enforcement and report-only modes

Task 2: Implement Notification Service Tests ✅

Priority: P0 | Effort: Large | Coverage Target: 85%+ | Status: COMPLETE

Objective: Comprehensively test email sending, webhook delivery, and in-app notifications.

Files to Test:

What Was Tested:

  1. Email sending via SMTP with credentials validation ✅
  2. Email template rendering (plain text and HTML) ✅
  3. Webhook payload creation and delivery ✅
  4. HTTP retries with exponential backoff ✅
  5. In-app notification storage and retrieval ✅
  6. Notification history pagination and filtering ✅
  7. Multi-channel dispatch (email + webhook + in-app) ✅
  8. Error handling and logging for failed notifications ✅
  9. Notification preferences (quiet hours, priority filtering) ✅
  10. Notification deduplication and limits ✅

Results:

  • Test File: tests/unit/test_notification_service.py
  • Tests Created: 50 comprehensive tests (47 passed, 3 skipped)
  • Coverage Achieved: 90%
  • Target: 85%+ ✅ EXCEEDED
  • All Required Tests Passing: ✅

Test Coverage by Component:

  • EmailNotificationService: Initialization, SMTP sending, error handling
  • WebhookNotificationService: HTTP requests, retries, exponential backoff, timeout handling
  • InAppNotificationService: Add, retrieve, mark as read, clear notifications, max limits
  • NotificationService: Preferences, quiet hours, priority filtering, multi-channel dispatch
  • Helper functions: Notification type-specific helpers (download complete, failed, queue complete, system error)

Notes:

  • 3 tests skipped if aiosmtplib not installed (optional dependency)
  • Comprehensive testing of retry logic with exponential backoff (2^attempt)
  • Quiet hours tested including midnight-spanning periods
  • Critical notifications bypass quiet hours as expected
  • All notification channels tested independently and together

Task 3: Implement Database Transaction Tests ✅

Priority: P0 | Effort: Large | Coverage Target: 90%+ | Status: COMPLETE

Objective: Ensure database transactions handle rollback, nesting, and error recovery correctly.

Files to Test:

What Was Tested:

  1. Basic transaction commit and rollback (sync and async) ✅
  2. Nested transactions using savepoints ✅
  3. Async transaction context manager ✅
  4. Savepoint creation and rollback ✅
  5. Error during transaction rolls back all changes ✅
  6. @transactional decorator for sync and async functions ✅
  7. Transaction propagation modes (REQUIRED, REQUIRES_NEW, NESTED) ✅
  8. atomic() and atomic_sync() context managers ✅
  9. Explicit commit/rollback within transactions ✅
  10. Transaction logging and error handling ✅

Results:

  • Test File: tests/unit/test_transaction.py
  • Tests Created: 66 comprehensive tests
  • Coverage Achieved: 90% (213/226 statements, 48/64 branches)
  • Target: 90%+ ✅ MET EXACTLY
  • All Tests Passing: ✅

Test Coverage by Component:

  • TransactionPropagation: Enum values and members
  • TransactionContext: Enter/exit, commit/rollback, savepoints, multiple nesting
  • SavepointContext: Rollback, idempotency, commit behavior
  • AsyncTransactionContext: All async equivalents of sync tests
  • AsyncSavepointContext: Async savepoint operations
  • atomic(): REQUIRED, NESTED propagation, commit/rollback
  • atomic_sync(): Sync context manager operations
  • @transactional: Decorator on async/sync functions, propagation, error handling
  • _extract_session(): Session extraction from kwargs/args
  • Utility functions: is_in_transaction(), get_transaction_depth()
  • Complex scenarios: Nested transactions, partial rollback, multiple operations

Notes:

  • Comprehensive testing of both synchronous and asynchronous transaction contexts
  • Transaction propagation modes thoroughly tested with different scenarios
  • Savepoint functionality validated including automatic naming and explicit rollback
  • Decorator tested with various parameter configurations
  • All error paths tested to ensure proper rollback behavior
  • Fixed file name discrepancy: actual file is transaction.py (not transactions.py)

Test File: tests/unit/test_database_transactions.py

Phase 2: Core Service & Initialization Tests (P1)

Task 4: Implement Initialization Service Tests ✅

Priority: P1 | Effort: Large | Coverage Target: 85%+ | Status: COMPLETE

Objective: Test complete application startup orchestration and configuration loading.

Files to Test:

What Was Tested:

  1. Generic scan status checking and marking functions ✅
  2. Initial scan status checking and completion marking ✅
  3. Anime folder syncing with series database ✅
  4. Series loading into memory cache ✅
  5. Anime directory validation ✅
  6. Complete initial setup orchestration ✅
  7. NFO scan status, configuration, and execution ✅
  8. Media scan status and execution ✅
  9. Error handling and recovery (OSError, RuntimeError, ValueError) ✅
  10. Full initialization sequences with progress tracking ✅

Results:

  • Test File: tests/unit/test_initialization_service.py
  • Tests Created: 46 comprehensive tests
  • Coverage Achieved: 96.65% (135/137 statements, 38/42 branches)
  • Target: 85%+ ✅ SIGNIFICANTLY EXCEEDED
  • All Tests Passing: ✅

Test Coverage by Component:

  • _check_scan_status(): Generic status checking with error handling
  • _mark_scan_completed(): Generic completion marking with error handling
  • Initial scan: Status checking, marking, and validation
  • _sync_anime_folders(): With/without progress service
  • _load_series_into_memory(): With/without progress service
  • _validate_anime_directory(): Configuration validation
  • perform_initial_setup(): Full orchestration, error handling, idempotency
  • NFO scan: Configuration checks, execution, error handling
  • perform_nfo_scan_if_needed(): Complete NFO scan flow with progress
  • Media scan: Status, execution, completion marking
  • perform_media_scan_if_needed(): Complete media scan flow
  • Integration tests: Full sequences, partial recovery, idempotency

Notes:

  • All initialization phases tested (initial setup, NFO scan, media scan)
  • Progress service integration tested thoroughly
  • Error handling validated for all scan types
  • Idempotency verified - repeated calls don't re-execute completed scans
  • Partial initialization recovery tested
  • Configuration validation prevents execution when directory not set
  • NFO scan configuration checks (API key, feature flags)
  • All patches correctly target imported functions

Task 5: Implement Series NFO Management Tests ✅

Priority: P1 | Effort: Large | Coverage Target: 80%+ | Status: COMPLETE

Objective: Test NFO metadata creation, updates, and media file downloads.

Files to Test:

What Was Tested:

  1. NFO file creation from TMDB data ✅
  2. NFO file updates with fresh metadata ✅
  3. Media file downloads (poster, logo, fanart) ✅
  4. Concurrent NFO processing for multiple series ✅
  5. Error recovery if TMDB API fails ✅
  6. Year extraction from series names ✅
  7. TMDB-to-NFO model conversion ✅
  8. FSK rating extraction from German content ratings ✅
  9. NFO ID parsing (TMDB, TVDB, IMDb) ✅
  10. Edge cases (empty data, malformed XML, missing fields) ✅

Results:

  • Test File: tests/unit/test_nfo_service.py
  • Tests Created: 73 comprehensive tests
  • Coverage Achieved: 90.65% (202/222 statements, 79/88 branches)
  • Target: 80%+ ✅ SIGNIFICANTLY EXCEEDED
  • All Tests Passing: ✅

Test Coverage by Component:

  • FSK rating extraction with German content ratings mapping
  • Year extraction from series names with various formats
  • TMDB-to-NFO model conversion with all fields
  • NFO creation from TMDB search and details
  • NFO updates with fresh data and optional media refresh
  • Media file downloads (poster, logo, fanart) with size configuration
  • NFO ID parsing (uniqueid elements and fallback elements)
  • Error handling for API failures, missing data, invalid XML
  • Configuration options (image sizes, auto-create)
  • Concurrent operations and cleanup

Notes:

  • Comprehensive testing of TMDB integration with mocked API client
  • All media download paths tested (poster, logo, fanart)
  • FSK rating extraction handles multiple German rating formats
  • Year extraction from series names works with parentheses format
  • NFO model conversion preserves all metadata from TMDB
  • Concurrent operations tested to ensure no conflicts
  • Edge cases covered for robustness

Task 6: Implement Page Controller Tests ✅

Priority: P1 | Effort: Medium | Coverage Target: 85%+ | Status: COMPLETE

Objective: Test page rendering, routing, and error handling.

Files to Test:

What Was Tested:

  1. Root endpoint (/) rendering index.html ✅
  2. Setup endpoint (/setup) rendering setup.html ✅
  3. Login endpoint (/login) rendering login.html ✅
  4. Queue endpoint (/queue) rendering queue.html ✅
  5. Loading endpoint (/loading) rendering loading.html ✅
  6. Template context generation with base context ✅
  7. Series context preparation and sorting ✅
  8. Template validation and availability checking ✅
  9. Series lookup by key ✅
  10. Filter series by missing episodes ✅

Results:

  • Test File: tests/unit/test_page_controller.py
  • Tests Created: 37 comprehensive tests
  • Page Controller Coverage: 100% (19/19 statements)
  • Template Helpers Coverage: 98.28% (42/42 statements, 15/16 branches)
  • Target: 85%+ ✅ SIGNIFICANTLY EXCEEDED
  • All Tests Passing: ✅

Test Coverage by Component:

  • All 5 page endpoints tested with mocked render_template
  • Base context generation with request and title
  • Title generation from template names
  • Series context preparation with sorting options
  • Series lookup and filtering by missing episodes
  • Template existence validation
  • Available templates listing
  • Edge cases (empty data, missing fields, case sensitivity)

Notes:

  • 100% coverage of page_controller.py endpoints
  • 98.28% coverage of template_helpers.py utilities
  • All template helper functions tested comprehensively
  • Request object properly mocked for all endpoint tests
  • Series data preparation validates required 'key' field
  • Filtering logic correctly identifies series with missing episodes

Phase 3: Background Tasks & Cache Tests (P2)

Task 7: Implement Background Task Tests ✅

Priority: P2 | Effort: Medium | Coverage Target: 80%+ | Status: COMPLETE

Objective: Test background loading tasks and error recovery.

Files to Test:

What Was Tested:

  1. Task queuing and worker orchestration ✅
  2. Series loading task initialization and status tracking ✅
  3. LoadingStatus enumeration values ✅
  4. Service startup with configurable workers ✅
  5. Service shutdown and graceful cleanup ✅
  6. Adding tasks to the loading queue ✅
  7. Duplicate task prevention ✅
  8. Status broadcasting via WebSocket ✅
  9. Finding series directories ✅
  10. Scanning episodes from series directories ✅
  11. NFO creation (new and existing files) ✅
  12. Checking missing data (episodes, NFO, logos, images) ✅
  13. Missing episodes scanning and sync ✅
  14. Error handling and recovery ✅
  15. Concurrent task processing ✅
  16. Task progress tracking lifecycle ✅

Results:

  • Test File: tests/unit/test_background_loader_service.py
  • Tests Created: 46 comprehensive tests
  • Coverage Achieved: 82% (247/300 statements, 52/80 branches)
  • Target: 80%+ ✅ EXCEEDED BY 2%
  • All Tests Passing: ✅

Test Coverage by Component:

  • SeriesLoadingTask data class initialization
  • LoadingStatus enumeration and status values
  • Service initialization with proper configuration
  • Start/stop lifecycle with worker management
  • Queue operations (add, duplicate prevention, processing)
  • Missing data detection (episodes, NFO, logos, images)
  • WebSocket status broadcasting with all payload types
  • Directory operations (finding, scanning episodes, error handling)
  • NFO loading (new creation, existing files, without NFO service)
  • Episode scanning with anime service sync
  • Error handling for API failures, missing data, invalid operations
  • Concurrent task processing and worker limit enforcement
  • Task progress tracking and status lifecycle

Notes:

  • Service supports configurable number of concurrent workers (default: 5)
  • Workers run indefinitely until shutdown, processing tasks from queue
  • Task queuing prevents duplicates for the same series key
  • WebSocket broadcasts include metadata and timestamp for frontend sync
  • Error handling ensures failures in one task don't affect others
  • All async operations properly tested with pytest-asyncio
  • Task progress individually tracks episodes, NFO, logos, images

Task 8: Implement Cache Service Tests

Priority: P2 | Effort: Medium | Coverage Target: 80%+

Objective: Test caching layers and cache invalidation.

Files to Test:

What to Test:

  1. Cache set and get operations
  2. Cache TTL expiration
  3. Cache invalidation strategies
  4. Cache statistics and monitoring
  5. Distributed cache consistency (Redis)
  6. In-memory cache under memory pressure
  7. Concurrent cache access
  8. Cache warmup on startup
  9. Cache key namespacing
  10. Cache bypass for sensitive data

Success Criteria:

  • Cache hit/miss tracking works
  • TTL respected correctly
  • Distributed cache consistent
  • Test coverage ≥80%

Test File: tests/unit/test_cache_service.py

Phase 4: Error Tracking & Utilities (P3)

Task 9: Implement Error Tracking Tests

Priority: P3 | Effort: Medium | Coverage Target: 85%+

Objective: Test error tracking and observability features.

Files to Test:

What to Test:

  1. Error tracking and history storage
  2. Error statistics calculation
  3. Error deduplication
  4. Request context management
  5. Error correlation IDs
  6. Error severity levels
  7. Error history pagination
  8. Error cleanup/retention
  9. Thread safety in error tracking
  10. Performance under high error rates

Success Criteria:

  • Errors tracked accurately with timestamps
  • Statistics calculated correctly
  • Request context preserved across async calls
  • Test coverage ≥85%

Test File: tests/unit/test_error_tracking.py

Task 10: Implement Settings Validation Tests

Priority: P3 | Effort: Small | Coverage Target: 80%+

Objective: Test configuration settings validation and defaults.

Files to Test:

What to Test:

  1. Environment variable parsing
  2. Settings defaults applied correctly
  3. Invalid settings raise validation errors
  4. Settings serialization and deserialization
  5. Secrets not exposed in logs
  6. Path validation for configured directories
  7. Range validation for numeric settings
  8. Format validation for URLs and IPs
  9. Required settings can't be empty
  10. Settings migration from old versions

Success Criteria:

  • All settings validated with proper error messages
  • Invalid configurations caught early
  • Test coverage ≥80%

Test File: tests/unit/test_settings_validation.py

Phase 5: Integration Tests (P1)

Task 11: Implement End-to-End Workflow Tests

Priority: P1 | Effort: Extra Large | Coverage Target: 75%+

Objective: Test complete workflows from start to finish.

What to Test:

  1. Setup Flow: Initialize app → Configure settings → Create master password → Ready
  2. Library Scan Flow: Scan filesystem → Find missing episodes → Update database → Display in UI
  3. NFO Creation Flow: Select series → Fetch TMDB data → Create NFO files → Download media
  4. Download Flow: Add episode to queue → Start download → Monitor progress → Complete
  5. Error Recovery Flow: Download fails → Retry → Success or permanently failed
  6. Multi-Series Flow: Multiple series in library → Concurrent NFO processing → Concurrent downloads

Success Criteria:

  • Full workflows complete without errors
  • Database state consistent throughout
  • UI reflects actual system state
  • Error recovery works for all failure points
  • Test coverage ≥75%

Test File: tests/integration/test_end_to_end_workflows.py

Coverage Summary

Phase Priority Tasks Target Coverage Status Results
Phase 1 P0 3 tasks 85-90% ✅ COMPLETE 164 tests, 91.88% avg coverage
Phase 2 P1 3 tasks 80-85% ✅ COMPLETE 156 tests, 96.31% avg coverage
Phase 3 P2 2 tasks 80% ⏳ IN PROGRESS 46/2 tests (46 complete, 82%)
Phase 4 P3 2 tasks 80-85% Not Started 0/2 complete
Phase 5 P1 1 task 75% Not Started 0/1 complete

Phases 1-3 Summary (COMPLETE/IN PROGRESS)

  • Phase 1-2 Total Tests: 320 tests
  • Phase 1-2 Total Coverage: 93.76% average
  • Phase 3 Task 7 Tests: 46 tests
  • Phase 3 Task 7 Coverage: 82%
  • All Tests Passing: ✅ 100%
  • Tasks: 7/11 complete with git commits

Testing Guidelines for AI Agents

When implementing these tests:

  1. Use existing fixtures from tests/conftest.py - db_session, app, mock_config
  2. Mock external services - TMDB API, SMTP, Redis, webhooks
  3. Test both happy paths and edge cases - success, errors, timeouts, retries
  4. Verify database state - Use db_session to check persisted data
  5. Test async code - Use pytest.mark.asyncio and proper async test patterns
  6. Measure coverage - Run pytest --cov to verify targets met
  7. Document test intent - Use clear test names and docstrings
  8. Follow project conventions - 80+ line limit per test method, clear arrange-act-assert pattern

Execution Order

  1. Start with Phase 1 (P0) - These are critical for production stability
  2. Then Phase 2 (P1) - Core features depend on these
  3. Then Phase 5 (P1) - End-to-end validation
  4. Then Phase 3 (P2) - Performance and optimization
  5. Finally Phase 4 (P3) - Observability and monitoring

Run tests continuously: pytest tests/ -v --cov --cov-report=html after each task completion.

Reference in New Issue View Git Blame Copy Permalink