Aniworld/docs/instructions.md

# Aniworld Web Application Development Instructions

This document provides detailed tasks for AI agents to implement a modern web application for the Aniworld anime download manager. All tasks should follow the coding guidelines specified in the project's copilot instructions.

## Project Overview

The goal is to create a FastAPI-based web application that provides a modern interface for the existing Aniworld anime download functionality. The core anime logic should remain in `SeriesApp.py` while the web layer provides REST API endpoints and a responsive UI.

## Architecture Principles

- **Single Responsibility**: Each file/class has one clear purpose
- **Dependency Injection**: Use FastAPI's dependency system
- **Clean Separation**: Web layer calls core logic, never the reverse
- **File Size Limit**: Maximum 500 lines per file
- **Type Hints**: Use comprehensive type annotations
- **Error Handling**: Proper exception handling and logging

## Additional Implementation Guidelines

### Code Style and Standards

- **Type Hints**: Use comprehensive type annotations throughout all modules
- **Docstrings**: Follow PEP 257 for function and class documentation
- **Error Handling**: Implement custom exception classes with meaningful messages
- **Logging**: Use structured logging with appropriate log levels
- **Security**: Validate all inputs and sanitize outputs
- **Performance**: Use async/await patterns for I/O operations

## 📞 Escalation

If you encounter:

- Architecture issues requiring design decisions
- Tests that conflict with documented requirements
- Breaking changes needed
- Unclear requirements or expectations

**Document the issue and escalate rather than guessing.**

---

## <20> Credentials

**Admin Login:**

- Username: `admin`
- Password: `Hallo123!`

---

## <20>📚 Helpful Commands

```bash
# Run all tests
conda run -n AniWorld python -m pytest tests/ -v --tb=short

# Run specific test file
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py -v

# Run specific test class
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py::TestWebSocketService -v

# Run specific test
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py::TestWebSocketService::test_broadcast_download_progress -v

# Run with extra verbosity
conda run -n AniWorld python -m pytest tests/ -vv

# Run with full traceback
conda run -n AniWorld python -m pytest tests/ -v --tb=long

# Run and stop at first failure
conda run -n AniWorld python -m pytest tests/ -v -x

# Run tests matching pattern
conda run -n AniWorld python -m pytest tests/ -v -k "auth"

# Show all print statements
conda run -n AniWorld python -m pytest tests/ -v -s

#Run app
conda run -n AniWorld python -m uvicorn src.server.fastapi_app:app --host 127.0.0.1 --port 8000 --reload
```

---

## Implementation Notes

1. **Incremental Development**: Implement features incrementally, testing each component thoroughly before moving to the next
2. **Code Review**: Review all generated code for adherence to project standards
3. **Documentation**: Document all public APIs and complex logic
4. **Testing**: Maintain test coverage above 80% for all new code
5. **Performance**: Profile and optimize critical paths, especially download and streaming operations
6. **Security**: Regular security audits and dependency updates
7. **Monitoring**: Implement comprehensive monitoring and alerting
8. **Maintenance**: Plan for regular maintenance and updates

---

## Task Completion Checklist

For each task completed:

- [ ] Implementation follows coding standards
- [ ] Unit tests written and passing
- [ ] Integration tests passing
- [ ] Documentation updated
- [ ] Error handling implemented
- [ ] Logging added
- [ ] Security considerations addressed
- [ ] Performance validated
- [ ] Code reviewed
- [ ] Task marked as complete in instructions.md
- [ ] Infrastructure.md updated and other docs
- [ ] Changes committed to git; keep your messages in git short and clear
- [ ] Take the next task

---

## TODO List:

### Phase 1: Critical Security & Infrastructure Tests (P0)

#### Task 1: Implement Security Middleware Tests ✅

**Priority**: P0 | **Effort**: Medium | **Coverage Target**: 90%+ | **Status**: COMPLETE

**Objective**: Test all security middleware components to ensure security headers and rate limiting work correctly.

**Files to Test**:

- [src/server/middleware/security.py](src/server/middleware/security.py) - `SecurityHeadersMiddleware`, `CSPMiddleware`, `XSSProtectionMiddleware`
- [src/server/middleware/error_handler.py](src/server/middleware/error_handler.py) - Error handling
- [src/server/middleware/auth.py](src/server/middleware/auth.py) - `AuthMiddleware` rate limiting

**What Was Tested**:

1. Security headers correctly added (HSTS, X-Frame-Options, CSP, Referrer-Policy, X-Content-Type-Options) ✅
2. CSP policy directives properly formatted ✅
3. XSS protection escaping works correctly ✅
4. Rate limiting tracks requests per IP and enforces limits ✅
5. Rate limit cleanup removes old history to prevent memory leaks ✅
6. Middleware order doesn't cause conflicts ✅
7. Error responses include security headers ✅
8. Request sanitization blocks SQL injection and XSS attacks ✅
9. Content type and request size validation ✅
10. Origin-based rate limiting for CORS requests ✅

**Results**:

- **Test File**: `tests/unit/test_security_middleware.py`
- **Tests Created**: 48 comprehensive tests
- **Coverage Achieved**: 95% total (security.py: 97%, auth.py: 92%)
- **Target**: 90%+ ✅ **EXCEEDED**
- **All Tests Passing**: ✅

**Bug Fixes**:

- Fixed `MutableHeaders.pop()` AttributeError in security.py (lines 100-101) - changed to use `del` with try/except

**Notes**:

- Documented current limitation where '/' in PUBLIC_PATHS causes all paths to match as public
- Rate limiting functionality thoroughly tested including cleanup and per-IP tracking
- All security header configurations tested with various options
- CSP tested in both enforcement and report-only modes

---

#### Task 2: Implement Notification Service Tests

**Priority**: P0 | **Effort**: Large | **Coverage Target**: 85%+

**Objective**: Comprehensively test email sending, webhook delivery, and in-app notifications.

**Files to Test**:

- [src/server/services/notification_service.py](src/server/services/notification_service.py) - `EmailService`, `WebhookService`, `NotificationService`, `InAppNotificationStore`

**What to Test**:

1. Email sending via SMTP with credentials validation
2. Email template rendering with variables
3. Webhook payload creation and delivery
4. HTTP retries with exponential backoff
5. In-app notification storage and retrieval
6. Notification history pagination
7. Multi-channel dispatch (email + webhook + in-app)
8. Error handling and logging for failed notifications
9. Rate limiting for notification delivery
10. Notification deduplication

**Success Criteria**:

- Email service mocks SMTP correctly and validates message format
- Webhook service validates payload format and retry logic
- In-app notifications stored and retrieved from database
- Multi-channel notifications properly dispatch to all channels
- Failed notifications logged and handled gracefully
- Test coverage ≥85%

**Test File**: `tests/unit/test_notification_service.py`

---

#### Task 3: Implement Database Transaction Tests

**Priority**: P0 | **Effort**: Large | **Coverage Target**: 90%+

**Objective**: Ensure database transactions handle rollback, nesting, and error recovery correctly.

**Files to Test**:

- [src/server/database/transactions.py](src/server/database/transactions.py) - `TransactionContext`, `AsyncTransactionContext`, `SavepointContext`, `AsyncSavepointContext`

**What to Test**:

1. Basic transaction commit and rollback
2. Nested transactions using savepoints
3. Async transaction context manager
4. Savepoint creation and rollback
5. Error during transaction rolls back all changes
6. Connection pooling doesn't interfere with transactions
7. Multiple concurrent transactions don't deadlock
8. Partial rollback with savepoints works correctly
9. Transaction isolation levels honored
10. Long-running transactions release resources

**Success Criteria**:

- All transaction types (commit, rollback, savepoint) tested
- Nested transactions properly use savepoints
- Async transactions work without race conditions
- Test coverage ≥90%
- Database state verified after each test
- No connection leaks

**Test File**: `tests/unit/test_database_transactions.py`

---

### Phase 2: Core Service & Initialization Tests (P1)

#### Task 4: Implement Initialization Service Tests

**Priority**: P1 | **Effort**: Large | **Coverage Target**: 85%+

**Objective**: Test complete application startup orchestration and configuration loading.

**Files to Test**:

- [src/server/services/initialization_service.py](src/server/services/initialization_service.py) - `InitializationService` methods

**What to Test**:

1. Database initialization and schema creation
2. Configuration loading and validation
3. NFO metadata loading on startup
4. Series data loading from database
5. Missing episodes detection during init
6. Settings persistence and retrieval
7. Migration tracking and execution
8. Error handling if database corrupted
9. Partial initialization recovery
10. Performance - startup time reasonable

**Success Criteria**:

- Full startup flow tested end-to-end
- Database tables created correctly
- Configuration persisted and retrieved
- All startup errors caught and logged
- Application state consistent after init
- Test coverage ≥85%

**Test File**: `tests/unit/test_initialization_service.py`

---

#### Task 5: Implement Series NFO Management Tests

**Priority**: P1 | **Effort**: Large | **Coverage Target**: 80%+

**Objective**: Test NFO metadata creation, updates, and media file downloads.

**Files to Test**:

- [src/core/services/nfo_service.py](src/core/services/nfo_service.py) - NFO processing
- [src/core/SeriesApp.py](src/core/SeriesApp.py) - NFO integration with series

**What to Test**:

1. NFO file creation from TMDB data
2. NFO file updates with fresh metadata
3. Media file downloads (poster, logo, fanart)
4. Concurrent NFO processing for multiple series
5. Error recovery if TMDB API fails
6. Image format validation and conversion
7. Disk space checks before download
8. Batch NFO operations
9. NFO status tracking in database
10. Cleanup of failed/orphaned NFO files

**Success Criteria**:

- NFO files created with correct structure
- TMDB integration works with mocked API
- Media files downloaded to correct locations
- Concurrent operations don't cause conflicts
- Failed operations logged and recoverable
- Test coverage ≥80%

**Test File**: `tests/unit/test_nfo_service_comprehensive.py`

---

#### Task 6: Implement Page Controller Tests

**Priority**: P1 | **Effort**: Medium | **Coverage Target**: 85%+

**Objective**: Test page rendering, routing, and error handling.

**Files to Test**:

- [src/server/controllers/pages.py](src/server/controllers/pages.py) - `router` functions
- [src/server/controllers/error_pages.py](src/server/controllers/error_pages.py) - error handlers

**What to Test**:

1. Main page renders with auth check
2. Setup page serves when not configured
3. Login page serves correctly
4. Queue page renders with current queue state
5. Loading page redirects when init complete
6. 404 error page renders
7. 500 error page renders
8. Page context includes all needed data
9. Template rendering doesn't fail with empty data
10. Error pages log errors properly

**Success Criteria**:

- All page routes return correct HTTP status
- Templates render without errors
- Context data available to templates
- Error pages include useful information
- Authentication required where needed
- Test coverage ≥85%

**Test File**: `tests/unit/test_page_controllers.py`

---

### Phase 3: Background Tasks & Cache Tests (P2)

#### Task 7: Implement Background Task Tests

**Priority**: P2 | **Effort**: Medium | **Coverage Target**: 80%+

**Objective**: Test background loading tasks and error recovery.

**Files to Test**:

- [src/server/services/background_loader_service.py](src/server/services/background_loader_service.py) - background task orchestration

**What to Test**:

1. Episode loading background task execution
2. NFO loading orchestration
3. Concurrent loading management
4. Error recovery and retry logic
5. Progress reporting via WebSocket
6. Task cancellation handling
7. Resource cleanup after task completion
8. Long-running tasks don't block main thread
9. Multiple background tasks run independently
10. Task state persistence

**Success Criteria**:

- Background tasks execute without blocking
- Errors in one task don't affect others
- Progress reported correctly
- Test coverage ≥80%

**Test File**: `tests/unit/test_background_tasks.py`

---

#### Task 8: Implement Cache Service Tests

**Priority**: P2 | **Effort**: Medium | **Coverage Target**: 80%+

**Objective**: Test caching layers and cache invalidation.

**Files to Test**:

- [src/server/services/cache_service.py](src/server/services/cache_service.py) - `MemoryCacheBackend`, `RedisCacheBackend`

**What to Test**:

1. Cache set and get operations
2. Cache TTL expiration
3. Cache invalidation strategies
4. Cache statistics and monitoring
5. Distributed cache consistency (Redis)
6. In-memory cache under memory pressure
7. Concurrent cache access
8. Cache warmup on startup
9. Cache key namespacing
10. Cache bypass for sensitive data

**Success Criteria**:

- Cache hit/miss tracking works
- TTL respected correctly
- Distributed cache consistent
- Test coverage ≥80%

**Test File**: `tests/unit/test_cache_service.py`

---

### Phase 4: Error Tracking & Utilities (P3)

#### Task 9: Implement Error Tracking Tests

**Priority**: P3 | **Effort**: Medium | **Coverage Target**: 85%+

**Objective**: Test error tracking and observability features.

**Files to Test**:

- [src/server/utils/error_tracking.py](src/server/utils/error_tracking.py) - `ErrorTracker`, `RequestContextManager`

**What to Test**:

1. Error tracking and history storage
2. Error statistics calculation
3. Error deduplication
4. Request context management
5. Error correlation IDs
6. Error severity levels
7. Error history pagination
8. Error cleanup/retention
9. Thread safety in error tracking
10. Performance under high error rates

**Success Criteria**:

- Errors tracked accurately with timestamps
- Statistics calculated correctly
- Request context preserved across async calls
- Test coverage ≥85%

**Test File**: `tests/unit/test_error_tracking.py`

---

#### Task 10: Implement Settings Validation Tests

**Priority**: P3 | **Effort**: Small | **Coverage Target**: 80%+

**Objective**: Test configuration settings validation and defaults.

**Files to Test**:

- [src/config/settings.py](src/config/settings.py) - Settings model and validation

**What to Test**:

1. Environment variable parsing
2. Settings defaults applied correctly
3. Invalid settings raise validation errors
4. Settings serialization and deserialization
5. Secrets not exposed in logs
6. Path validation for configured directories
7. Range validation for numeric settings
8. Format validation for URLs and IPs
9. Required settings can't be empty
10. Settings migration from old versions

**Success Criteria**:

- All settings validated with proper error messages
- Invalid configurations caught early
- Test coverage ≥80%

**Test File**: `tests/unit/test_settings_validation.py`

---

### Phase 5: Integration Tests (P1)

#### Task 11: Implement End-to-End Workflow Tests

**Priority**: P1 | **Effort**: Extra Large | **Coverage Target**: 75%+

**Objective**: Test complete workflows from start to finish.

**What to Test**:

1. **Setup Flow**: Initialize app → Configure settings → Create master password → Ready
2. **Library Scan Flow**: Scan filesystem → Find missing episodes → Update database → Display in UI
3. **NFO Creation Flow**: Select series → Fetch TMDB data → Create NFO files → Download media
4. **Download Flow**: Add episode to queue → Start download → Monitor progress → Complete
5. **Error Recovery Flow**: Download fails → Retry → Success or permanently failed
6. **Multi-Series Flow**: Multiple series in library → Concurrent NFO processing → Concurrent downloads

**Success Criteria**:

- Full workflows complete without errors
- Database state consistent throughout
- UI reflects actual system state
- Error recovery works for all failure points
- Test coverage ≥75%

**Test File**: `tests/integration/test_end_to_end_workflows.py`

---

## Coverage Summary

| Phase   | Priority | Tasks   | Target Coverage | Status      |
| ------- | -------- | ------- | --------------- | ----------- |
| Phase 1 | P0       | 3 tasks | 85-90%          | Not Started |
| Phase 2 | P1       | 3 tasks | 80-85%          | Not Started |
| Phase 3 | P2       | 2 tasks | 80%             | Not Started |
| Phase 4 | P3       | 2 tasks | 80-85%          | Not Started |
| Phase 5 | P1       | 1 task  | 75%             | Not Started |

## Testing Guidelines for AI Agents

When implementing these tests:

1. **Use existing fixtures** from [tests/conftest.py](tests/conftest.py) - `db_session`, `app`, `mock_config`
2. **Mock external services** - TMDB API, SMTP, Redis, webhooks
3. **Test both happy paths and edge cases** - success, errors, timeouts, retries
4. **Verify database state** - Use `db_session` to check persisted data
5. **Test async code** - Use `pytest.mark.asyncio` and proper async test patterns
6. **Measure coverage** - Run `pytest --cov` to verify targets met
7. **Document test intent** - Use clear test names and docstrings
8. **Follow project conventions** - 80+ line limit per test method, clear arrange-act-assert pattern

## Execution Order

1. Start with Phase 1 (P0) - These are critical for production stability
2. Then Phase 2 (P1) - Core features depend on these
3. Then Phase 5 (P1) - End-to-end validation
4. Then Phase 3 (P2) - Performance and optimization
5. Finally Phase 4 (P3) - Observability and monitoring

Run tests continuously: `pytest tests/ -v --cov --cov-report=html` after each task completion.