Aniworld/instructions.md

901 lines
25 KiB
Markdown

# Aniworld Web Application Development Instructions
This document provides detailed tasks for AI agents to implement a modern web application for the Aniworld anime download manager. All tasks should follow the coding guidelines specified in the project's copilot instructions.
## Project Overview
The goal is to create a FastAPI-based web application that provides a modern interface for the existing Aniworld anime download functionality. The core anime logic should remain in `SeriesApp.py` while the web layer provides REST API endpoints and a responsive UI.
## Architecture Principles
- **Single Responsibility**: Each file/class has one clear purpose
- **Dependency Injection**: Use FastAPI's dependency system
- **Clean Separation**: Web layer calls core logic, never the reverse
- **File Size Limit**: Maximum 500 lines per file
- **Type Hints**: Use comprehensive type annotations
- **Error Handling**: Proper exception handling and logging
## Additional Implementation Guidelines
### Code Style and Standards
- **Type Hints**: Use comprehensive type annotations throughout all modules
- **Docstrings**: Follow PEP 257 for function and class documentation
- **Error Handling**: Implement custom exception classes with meaningful messages
- **Logging**: Use structured logging with appropriate log levels
- **Security**: Validate all inputs and sanitize outputs
- **Performance**: Use async/await patterns for I/O operations
## 📞 Escalation
If you encounter:
- Architecture issues requiring design decisions
- Tests that conflict with documented requirements
- Breaking changes needed
- Unclear requirements or expectations
**Document the issue and escalate rather than guessing.**
---
## 📚 Helpful Commands
```bash
# Run all tests
conda run -n AniWorld python -m pytest tests/ -v --tb=short
# Run specific test file
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py -v
# Run specific test class
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py::TestWebSocketService -v
# Run specific test
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py::TestWebSocketService::test_broadcast_download_progress -v
# Run with extra verbosity
conda run -n AniWorld python -m pytest tests/ -vv
# Run with full traceback
conda run -n AniWorld python -m pytest tests/ -v --tb=long
# Run and stop at first failure
conda run -n AniWorld python -m pytest tests/ -v -x
# Run tests matching pattern
conda run -n AniWorld python -m pytest tests/ -v -k "auth"
# Show all print statements
conda run -n AniWorld python -m pytest tests/ -v -s
#Run app
conda run -n AniWorld python -m uvicorn src.server.fastapi_app:app --host 127.0.0.1 --port 8000 --reload
```
---
## Final Implementation Notes
1. **Incremental Development**: Implement features incrementally, testing each component thoroughly before moving to the next
2. **Code Review**: Review all generated code for adherence to project standards
3. **Documentation**: Document all public APIs and complex logic
4. **Testing**: Maintain test coverage above 80% for all new code
5. **Performance**: Profile and optimize critical paths, especially download and streaming operations
6. **Security**: Regular security audits and dependency updates
7. **Monitoring**: Implement comprehensive monitoring and alerting
8. **Maintenance**: Plan for regular maintenance and updates
## Task Completion Checklist
For each task completed:
- [ ] Implementation follows coding standards
- [ ] Unit tests written and passing
- [ ] Integration tests passing
- [ ] Documentation updated
- [ ] Error handling implemented
- [ ] Logging added
- [ ] Security considerations addressed
- [ ] Performance validated
- [ ] Code reviewed
- [ ] Task marked as complete in instructions.md
- [ ] Infrastructure.md updated
- [ ] Changes committed to git
---
### Prerequisites
1. Server is running: `conda run -n AniWorld python -m uvicorn src.server.fastapi_app:app --host 127.0.0.1 --port 8000 --reload`
2. Password: `Hallo123!`
3. Login via browser at `http://127.0.0.1:8000/login`
**Deployment Steps:**
1. Commit all changes to git repository
2. Create deployment tag (e.g., `v1.0.0-queue-simplified`)
3. Deploy to production environment
4. Monitor logs for any unexpected behavior
5. Verify production queue functionality
### Notes
- This is a simplification that removes complexity while maintaining core functionality
- Improves user experience with explicit manual control
- Easier to understand, test, and maintain
- Good foundation for future enhancements if needed
- No database schema changes required
- WebSocket infrastructure remains unchanged
---
# 🎯 CRITICAL: Series Identifier Standardization
## Overview
**Problem:** The codebase currently uses multiple identifiers inconsistently:
- `key` (provider identifier, e.g., "attack-on-titan")
- `folder` (filesystem folder name, e.g., "Attack on Titan (2013)")
- `serie_id` (sometimes key, sometimes folder)
- `serie_folder` (filesystem path used as identifier)
**Solution:** Standardize on `key` as the single source of truth for all series identification.
**Benefits:**
- Single, consistent identifier throughout the codebase
- `key` is unique, provider-assigned, URL-safe
- `folder` becomes metadata only (not used for lookups)
- Clearer separation of concerns
- Easier to maintain and debug
**Scope:** This affects core logic, services, API endpoints, frontend, WebSocket events, and tests.
---
## Task Series: Identifier Standardization
### Phase 1: Core Models and Data Layer
**All Phase 1 tasks completed and committed to git**
---
### Phase 2: Core Application Layer
**Task 2.1 completed and committed to git**
---
### Phase 3: Service Layer
**All Phase 3 tasks completed and committed to git (November 2025)**
- Task 3.1: Update DownloadService ✅
- Task 3.2: Update AnimeService ✅
- Task 3.3: Update ProgressService ✅
- Task 3.4: Update ScanService ✅ (November 27, 2025)
---
### Phase 4: API Layer
#### Task 4.1: Update Anime API Endpoints to Use Key ✅ (November 27, 2025)
#### Task 4.2: Update Download API Endpoints to Use Key ✅ (November 27, 2025)
---
#### Task 4.3: Update Queue API Endpoints to Use Key
**File:** [`src/server/api/queue.py`](src/server/api/queue.py)
**Objective:** Ensure queue API endpoints use `key` for series identification.
**Steps:**
1. Open [`src/server/api/queue.py`](src/server/api/queue.py)
2. Review all queue-related endpoints
3. Update request/response models to use `key` as identifier
4. Ensure queue status includes `key` in item data
5. Update queue manipulation endpoints to accept `key`
6. Update docstrings and OpenAPI documentation
**Success Criteria:**
- [ ] All queue endpoints use `key`
- [ ] Queue status responses include `key`
- [ ] Request models use `key` for identification
- [ ] All queue API tests pass
**Test Command:**
```bash
conda run -n AniWorld python -m pytest tests/api/ -k "queue" -v
```
---
#### Task 4.4: Update WebSocket API Endpoints to Use Key
**File:** `src/server/api/websocket.py`
**Objective:** Ensure WebSocket API endpoint handlers use `key` for series identification.
**Steps:**
1. Open `src/server/api/websocket.py`
2. Review all WebSocket message handlers
3. Ensure messages use `key` for series identification
4. Update message schemas to include `key` field
5. Keep `folder` for display purposes
6. Update docstrings
**Success Criteria:**
- [ ] All WebSocket handlers use `key`
- [ ] Message schemas include `key`
- [ ] All WebSocket API tests pass
**Test Command:**
```bash
conda run -n AniWorld python -m pytest tests/api/ -k "websocket" -v
```
---
#### Task 4.5: Update Pydantic Models to Use Key
**Files:**
- `src/server/models/anime.py`
- `src/server/models/download.py`
**Objective:** Ensure all Pydantic models use `key` as the series identifier.
**Steps:**
1. Open `src/server/models/anime.py`
2. Review all models (e.g., `AnimeDetail`, `AnimeSummary`, `SearchResult`)
3. Ensure `key` is the primary identifier field
4. Add `folder` as optional metadata field
5. Update field validators to validate `key` format
6. Repeat for `src/server/models/download.py`
7. Update `DownloadRequest` and related models
8. Update all docstrings and field descriptions
**Success Criteria:**
- [ ] All anime models use `key` as identifier
- [ ] All download models use `key` as identifier
- [ ] Field validators ensure `key` format is correct
- [ ] `folder` is optional metadata
- [ ] All model tests pass
**Test Command:**
```bash
conda run -n AniWorld python -m pytest tests/unit/ -k "models" -v
```
---
#### Task 4.6: Update Validators to Use Key
**File:** `src/server/utils/validators.py`
**Objective:** Ensure validation functions validate `key` instead of `folder`.
**Steps:**
1. Open `src/server/utils/validators.py`
2. Review all validation functions
3. Add `validate_series_key()` function if not exists
4. Update any validators that check series identifiers
5. Ensure validators accept `key` format (URL-safe, lowercase with hyphens)
6. Add tests for key validation
**Success Criteria:**
- [ ] Validators validate `key` format
- [ ] No validators use `folder` for identification
- [ ] Validation functions well-documented
- [ ] All validator tests pass
**Test Command:**
```bash
conda run -n AniWorld python -m pytest tests/unit/ -k "validator" -v
```
---
#### Task 4.7: Update Template Helpers to Use Key
**File:** `src/server/utils/template_helpers.py`
**Objective:** Ensure template helpers pass `key` to templates for series identification.
**Steps:**
1. Open `src/server/utils/template_helpers.py`
2. Review all helper functions
3. Ensure functions that handle series data use `key`
4. Update any filtering or sorting to use `key`
5. Ensure `folder` is available for display purposes
6. Update docstrings
**Success Criteria:**
- [ ] All helpers use `key` for identification
- [ ] `folder` available for display
- [ ] No breaking changes to template interface
- [ ] All template helper tests pass
**Test Command:**
```bash
conda run -n AniWorld python -m pytest tests/unit/ -k "template" -v
```
---
### Phase 5: Frontend
#### Task 5.1: Update Frontend JavaScript to Use Key
**File:** [`src/server/web/static/js/app.js`](src/server/web/static/js/app.js)
**Objective:** Update frontend to use `key` as the primary series identifier instead of `folder`.
**Steps:**
1. Open [`src/server/web/static/js/app.js`](src/server/web/static/js/app.js)
2. Update `seriesData` storage to index by `key`
3. Update `selectedSeries` Set to use `key` instead of `folder`
4. Update `createSerieCard()`:
- Use `data-key` attribute instead of `data-folder`
- Display `folder` as metadata only
5. Update `toggleSerieSelection()` to use `key`
6. Update `downloadSelected()`:
- Send `key` as `serie_id`
- Include `folder` for filesystem operations
7. Update all event handlers and lookups to use `key`
8. Keep `folder` visible in UI for user reference
**Success Criteria:**
- [ ] Frontend uses `key` for all series operations
- [ ] `folder` displayed in UI but not used for identification
- [ ] Selection tracking uses `key`
- [ ] All frontend interactions work correctly
**Manual Test:**
1. Start server
2. Login to web interface
3. Verify series list displays correctly
4. Test selecting series (should use key internally)
5. Test downloading episodes
6. Verify search functionality
---
#### Task 5.2: Update WebSocket Events to Use Key
**File:** [`src/server/services/websocket_service.py`](src/server/services/websocket_service.py)
**Objective:** Ensure WebSocket events use `key` for series identification.
**Steps:**
1. Open [`src/server/services/websocket_service.py`](src/server/services/websocket_service.py)
2. Update `broadcast_download_progress()`:
- Include `key` in event data
- Keep `folder` for display purposes
3. Update `broadcast_scan_progress()` similarly
4. Update all event broadcasts to include `key`
5. Update event handler subscriptions to use `key`
**Success Criteria:**
- [ ] All WebSocket events include `key`
- [ ] Events also include `folder` for display
- [ ] All WebSocket tests pass
**Test Command:**
```bash
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py -v
```
---
#### Task 5.3: Update Additional Frontend JavaScript Files
**Files:**
- `src/server/web/static/js/websocket.js`
- `src/server/web/static/js/queue.js`
- `src/server/web/static/js/download.js`
- `src/server/web/static/js/utils.js`
**Objective:** Ensure all frontend JavaScript modules use `key` for series identification.
**Steps:**
1. **Update `websocket.js`:**
- Open `src/server/web/static/js/websocket.js`
- Review all message handlers
- Ensure event payloads use `key` for identification
- Update event listeners to use `key`
- Keep `folder` for display
2. **Update `queue.js`:**
- Open `src/server/web/static/js/queue.js`
- Update queue item identification to use `key`
- Update queue manipulation functions
- Ensure queue display shows both `key` and `folder`
3. **Update `download.js`:**
- Open `src/server/web/static/js/download.js`
- Update download request building to use `key`
- Update progress tracking to use `key`
- Keep `folder` for file path operations
4. **Update `utils.js`:**
- Open `src/server/web/static/js/utils.js`
- Review utility functions that handle series data
- Ensure utilities use `key` for identification
- Update any series-related helper functions
**Success Criteria:**
- [ ] All JavaScript modules use `key` for identification
- [ ] WebSocket handlers use `key`
- [ ] Queue operations use `key`
- [ ] Download operations use `key`
- [ ] Utilities handle `key` correctly
- [ ] `folder` displayed in UI where appropriate
- [ ] All frontend functionality works correctly
**Manual Test:**
1. Test WebSocket connectivity and events
2. Test queue management
3. Test download functionality
4. Verify all series operations use `key`
---
#### Task 5.4: Update HTML Templates to Use Key
**Files:** All templates in `src/server/web/templates/`
**Objective:** Ensure all HTML templates use `key` for series identification in data attributes and forms.
**Steps:**
1. Review all template files:
- `index.html`
- `anime_detail.html`
- `search.html`
- Any other templates using series data
2. For each template:
- Update data attributes to use `data-key` instead of `data-folder`
- Keep `data-folder` for display purposes if needed
- Update form inputs to submit `key` as identifier
- Update JavaScript references to use `key`
- Display `folder` for user-friendly names
3. Update template variables:
- Ensure templates receive `key` from backend
- Verify `folder` is available for display
- Update any template logic that filters/sorts by series
**Success Criteria:**
- [ ] All templates use `data-key` for identification
- [ ] Forms submit `key` as identifier
- [ ] `folder` displayed for user reference
- [ ] No templates use `folder` for identification
- [ ] All template rendering works correctly
**Manual Test:**
1. Verify all pages render correctly
2. Test form submissions
3. Verify JavaScript interactions
4. Check data attributes in browser dev tools
---
### Phase 6: Database Layer
#### Task 6.1: Verify Database Models Use Key Correctly
**File:** [`src/server/database/models.py`](src/server/database/models.py)
**Objective:** Verify and document that database models correctly use `key` as unique identifier.
**Steps:**
1. Open [`src/server/database/models.py`](src/server/database/models.py)
2. Verify `AnimeSeries.key` is unique and indexed
3. Verify `AnimeSeries.folder` is not used for lookups
4. Update docstrings to clarify identifier usage
5. Ensure all relationships use `id` (primary key) not `key` or `folder`
**Success Criteria:**
- [ ] `key` is unique and indexed
- [ ] `folder` is metadata only
- [ ] Docstrings are clear
- [ ] All database model tests pass
**Test Command:**
```bash
conda run -n AniWorld python -m pytest tests/unit/test_database_models.py -v
```
---
#### Task 6.2: Update Database Services to Use Key
**File:** [`src/server/database/service.py`](src/server/database/service.py)
**Objective:** Ensure all database service methods use `key` for series lookup.
**Steps:**
1. Open [`src/server/database/service.py`](src/server/database/service.py)
2. Verify `AnimeSeriesService.get_by_key()` is used for lookups
3. Verify no methods use `folder` for identification
4. Update any methods that incorrectly use `folder` for lookups
5. Add migration helper if needed to update existing data
**Success Criteria:**
- [ ] All service methods use `key` for lookups
- [ ] `folder` never used as identifier
- [ ] All database service tests pass
**Test Command:**
```bash
conda run -n AniWorld python -m pytest tests/unit/test_database_service.py -v
```
---
### Phase 7: Testing and Validation
#### Task 7.1: Update All Test Fixtures to Use Key
**Files:** All test files in [`tests/`](tests/)
**Objective:** Ensure all test fixtures and mocks use `key` consistently.
**Steps:**
1. Search for all test files using `folder` as identifier
2. Update `FakeSerie` class in [`tests/api/test_anime_endpoints.py`](tests/api/test_anime_endpoints.py):
- Ensure `key` is the primary identifier
3. Update all test fixtures to use `key`
4. Update mock data to use realistic `key` values
5. Ensure tests verify both `key` and `folder` are present but only `key` is used for operations
**Success Criteria:**
- [ ] All test fixtures use `key` as identifier
- [ ] Tests verify `key` is used for operations
- [ ] Tests verify `folder` is present as metadata
- [ ] All tests pass
**Test Command:**
```bash
conda run -n AniWorld python -m pytest tests/ -v
```
---
#### Task 7.2: Add Integration Tests for Identifier Consistency
**File:** Create new file `tests/integration/test_identifier_consistency.py`
**Objective:** Create integration tests to verify `key` is used consistently across all layers.
**Steps:**
1. Create [`tests/integration/test_identifier_consistency.py`](tests/integration/test_identifier_consistency.py)
2. Write test to verify:
- API endpoint returns `key` as identifier
- Download service uses `key`
- Database lookups use `key`
- WebSocket events include `key`
3. Write test to verify `folder` is never used for lookups
4. Write test for end-to-end flow using `key`
**Success Criteria:**
- [ ] Integration test file created
- [ ] Tests verify `key` usage across all layers
- [ ] Tests verify `folder` not used for identification
- [ ] All integration tests pass
**Test Command:**
```bash
conda run -n AniWorld python -m pytest tests/integration/test_identifier_consistency.py -v
```
---
### Phase 8: Documentation and Cleanup
#### Task 8.1: Update Infrastructure Documentation
**File:** [`infrastructure.md`](infrastructure.md)
**Objective:** Document the identifier standardization in infrastructure documentation.
**Steps:**
1. Open [`infrastructure.md`](infrastructure.md)
2. Add section explaining identifier usage:
- `key`: Unique series identifier (provider-assigned)
- `folder`: Filesystem folder name (metadata only)
- `id`: Database primary key (internal use)
3. Update all API documentation to show `key` as identifier
4. Update data model documentation
5. Add migration notes if needed
**Success Criteria:**
- [ ] Documentation clearly explains identifier usage
- [ ] All API examples use `key`
- [ ] Data model section updated
- [ ] Migration notes added if applicable
---
#### Task 8.2: Update README and Developer Documentation
**Files:** [`README.md`](README.md), [`docs/`](docs/)
**Objective:** Update all developer-facing documentation.
**Steps:**
1. Update main README to explain identifier usage
2. Update any developer guides to use `key`
3. Add note about backward compatibility with `folder`
4. Update code examples to use `key`
**Success Criteria:**
- [ ] README updated
- [ ] Developer guides updated
- [ ] Code examples use `key`
- [ ] Backward compatibility documented
---
#### Task 8.3: Add Deprecation Warnings for Folder-Based Lookups
**Files:** Various service and API files
**Objective:** Add deprecation warnings where `folder` is still accepted for backward compatibility.
**Steps:**
1. Identify all methods that accept `folder` for lookups
2. Add deprecation warnings using Python's `warnings` module
3. Update docstrings to indicate deprecation
4. Plan removal timeline (e.g., next major version)
**Success Criteria:**
- [ ] Deprecation warnings added
- [ ] Docstrings indicate deprecation
- [ ] Removal timeline documented
- [ ] Tests updated to suppress warnings
---
### Phase 9: Final Validation
#### Task 9.1: Run Full Test Suite
**Objective:** Verify all changes work together correctly.
**Steps:**
1. Run complete test suite:
```bash
conda run -n AniWorld python -m pytest tests/ -v --tb=short
```
2. Fix any failing tests
3. Verify test coverage is maintained
4. Run integration tests
5. Run manual UI tests
**Success Criteria:**
- [ ] All unit tests pass
- [ ] All integration tests pass
- [ ] All API tests pass
- [ ] Test coverage >= 80%
- [ ] Manual UI testing successful
---
#### Task 9.2: Manual End-to-End Testing
**Objective:** Manually verify all features work with the new identifier system.
**Steps:**
1. Start server: `conda run -n AniWorld python -m uvicorn src.server.fastapi_app:app --host 127.0.0.1 --port 8000 --reload`
2. Login to web interface
3. Test search functionality (verify results show `key`)
4. Test adding new series (verify uses `key`)
5. Test downloading episodes (verify uses `key`)
6. Test WebSocket events (verify events include `key`)
7. Verify database contains correct `key` values
8. Test rescan functionality
**Success Criteria:**
- [ ] Search works correctly
- [ ] Adding series works
- [ ] Downloads work correctly
- [ ] WebSocket events work
- [ ] Database entries correct
- [ ] Rescan functionality works
---
#### Task 9.3: Performance and Load Testing
**Objective:** Ensure identifier changes don't impact performance.
**Steps:**
1. Run performance tests on key operations:
- Series lookup by `key`
- Database queries using `key`
- API response times
2. Compare with baseline if available
3. Identify any performance regressions
4. Optimize if needed
**Success Criteria:**
- [ ] No significant performance regression
- [ ] Lookup by `key` is fast
- [ ] Database queries optimized
- [ ] API response times acceptable
---
### Phase 10: Deployment
#### Task 10.1: Create Migration Script
**Objective:** Create script to migrate existing data if needed.
**Steps:**
1. Create [`scripts/migrate_identifiers.py`](scripts/migrate_identifiers.py)
2. Script should:
- Check all series have valid `key` values
- Update any references that incorrectly use `folder`
- Validate database integrity
- Create backup before migration
3. Add rollback capability
4. Test migration on test data
**Success Criteria:**
- [ ] Migration script created
- [ ] Script validates data
- [ ] Backup functionality works
- [ ] Rollback capability tested
- [ ] Migration tested on test data
---
#### Task 10.2: Update Deployment Documentation
**File:** Update deployment section in [`instructions.md`](instructions.md)
**Objective:** Document deployment steps for identifier changes.
**Steps:**
1. Add pre-deployment checklist
2. Document migration steps
3. Add rollback procedure
4. Document verification steps
5. Add troubleshooting guide
**Success Criteria:**
- [ ] Deployment steps documented
- [ ] Migration procedure clear
- [ ] Rollback procedure documented
- [ ] Verification steps listed
- [ ] Troubleshooting guide added
---
#### Task 10.3: Deploy to Production
**Objective:** Deploy changes to production environment.
**Steps:**
1. Create deployment tag: `v2.0.0-identifier-standardization`
2. Backup production database
3. Run migration script
4. Deploy new code
5. Monitor logs for errors
6. Verify production functionality
7. Monitor for 24 hours
**Success Criteria:**
- [ ] Deployment tag created
- [ ] Database backed up
- [ ] Migration successful
- [ ] Code deployed
- [ ] No errors in logs
- [ ] All features working
- [ ] 24-hour monitoring completed
---
## Task Tracking
### Completion Status
- [x] Phase 1: Core Models and Data Layer ✅
- [x] Phase 2: Core Application Layer ✅
- [x] Phase 3: Service Layer ✅
- [ ] Phase 4: API Layer
- [x] Task 4.1: Update Anime API Endpoints ✅ **Completed November 27, 2025**
- [x] Task 4.2: Update Download API Endpoints ✅ **Completed November 27, 2025**
- [ ] Task 4.3: Update Queue API Endpoints
- [ ] Task 4.4: Update WebSocket API Endpoints
- [ ] Task 4.5: Update Pydantic Models
- [ ] Task 4.6: Update Validators
- [ ] Task 4.7: Update Template Helpers
- [ ] Phase 5: Frontend
- [ ] Task 5.1: Update Frontend JavaScript
- [ ] Task 5.2: Update WebSocket Events
- [ ] Task 5.3: Update Additional Frontend JavaScript Files
- [ ] Task 5.4: Update HTML Templates
- [ ] Phase 6: Database Layer
- [ ] Task 6.1: Verify Database Models
- [ ] Task 6.2: Update Database Services
- [ ] Phase 7: Testing and Validation
- [ ] Task 7.1: Update Test Fixtures
- [ ] Task 7.2: Add Integration Tests
- [ ] Phase 8: Documentation and Cleanup
- [ ] Task 8.1: Update Infrastructure Documentation
- [ ] Task 8.2: Update README
- [ ] Task 8.3: Add Deprecation Warnings
- [ ] Phase 9: Final Validation