Aniworld/instructions.md

# Aniworld Web Application Development Instructions

This document provides detailed tasks for AI agents to implement a modern web application for the Aniworld anime download manager. All tasks should follow the coding guidelines specified in the project's copilot instructions.

## Project Overview

The goal is to create a FastAPI-based web application that provides a modern interface for the existing Aniworld anime download functionality. The core anime logic should remain in `SeriesApp.py` while the web layer provides REST API endpoints and a responsive UI.

## Architecture Principles

-   **Single Responsibility**: Each file/class has one clear purpose
-   **Dependency Injection**: Use FastAPI's dependency system
-   **Clean Separation**: Web layer calls core logic, never the reverse
-   **File Size Limit**: Maximum 500 lines per file
-   **Type Hints**: Use comprehensive type annotations
-   **Error Handling**: Proper exception handling and logging

## Additional Implementation Guidelines

### Code Style and Standards

-   **Type Hints**: Use comprehensive type annotations throughout all modules
-   **Docstrings**: Follow PEP 257 for function and class documentation
-   **Error Handling**: Implement custom exception classes with meaningful messages
-   **Logging**: Use structured logging with appropriate log levels
-   **Security**: Validate all inputs and sanitize outputs
-   **Performance**: Use async/await patterns for I/O operations

## 📞 Escalation

If you encounter:

-   Architecture issues requiring design decisions
-   Tests that conflict with documented requirements
-   Breaking changes needed
-   Unclear requirements or expectations

**Document the issue and escalate rather than guessing.**

---

## 📚 Helpful Commands

```bash
# Run all tests
conda run -n AniWorld python -m pytest tests/ -v --tb=short

# Run specific test file
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py -v

# Run specific test class
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py::TestWebSocketService -v

# Run specific test
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py::TestWebSocketService::test_broadcast_download_progress -v

# Run with extra verbosity
conda run -n AniWorld python -m pytest tests/ -vv

# Run with full traceback
conda run -n AniWorld python -m pytest tests/ -v --tb=long

# Run and stop at first failure
conda run -n AniWorld python -m pytest tests/ -v -x

# Run tests matching pattern
conda run -n AniWorld python -m pytest tests/ -v -k "auth"

# Show all print statements
conda run -n AniWorld python -m pytest tests/ -v -s

#Run app
conda run -n AniWorld python -m uvicorn src.server.fastapi_app:app --host 127.0.0.1 --port 8000 --reload
```

---

## Final Implementation Notes

1. **Incremental Development**: Implement features incrementally, testing each component thoroughly before moving to the next
2. **Code Review**: Review all generated code for adherence to project standards
3. **Documentation**: Document all public APIs and complex logic
4. **Testing**: Maintain test coverage above 80% for all new code
5. **Performance**: Profile and optimize critical paths, especially download and streaming operations
6. **Security**: Regular security audits and dependency updates
7. **Monitoring**: Implement comprehensive monitoring and alerting
8. **Maintenance**: Plan for regular maintenance and updates

## Task Completion Checklist

For each task completed:

-   [ ] Implementation follows coding standards
-   [ ] Unit tests written and passing
-   [ ] Integration tests passing
-   [ ] Documentation updated
-   [ ] Error handling implemented
-   [ ] Logging added
-   [ ] Security considerations addressed
-   [ ] Performance validated
-   [ ] Code reviewed
-   [ ] Task marked as complete in instructions.md
-   [ ] Infrastructure.md updated
-   [ ] Changes committed to git
-   [ ] Take the next task

---

### Prerequisites

1. Server is running: `conda run -n AniWorld python -m uvicorn src.server.fastapi_app:app --host 127.0.0.1 --port 8000 --reload`
2. Password: `Hallo123!`
3. Login via browser at `http://127.0.0.1:8000/login`

**Deployment Steps:**

1. Commit all changes to git repository
2. Create deployment tag (e.g., `v1.0.0-queue-simplified`)
3. Deploy to production environment
4. Monitor logs for any unexpected behavior
5. Verify production queue functionality

### Notes

-   This is a simplification that removes complexity while maintaining core functionality
-   Improves user experience with explicit manual control
-   Easier to understand, test, and maintain
-   Good foundation for future enhancements if needed
-   No database schema changes required
-   WebSocket infrastructure remains unchanged

---

# 🎯 CRITICAL: Series Identifier Standardization

## Overview

**Problem:** The codebase currently uses multiple identifiers inconsistently:

-   `key` (provider identifier, e.g., "attack-on-titan")
-   `folder` (filesystem folder name, e.g., "Attack on Titan (2013)")
-   `serie_id` (sometimes key, sometimes folder)
-   `serie_folder` (filesystem path used as identifier)

**Solution:** Standardize on `key` as the single source of truth for all series identification.

**Benefits:**

-   Single, consistent identifier throughout the codebase
-   `key` is unique, provider-assigned, URL-safe
-   `folder` becomes metadata only (not used for lookups)
-   Clearer separation of concerns
-   Easier to maintain and debug

**Scope:** This affects core logic, services, API endpoints, frontend, WebSocket events, and tests.

---

## Task Series: Identifier Standardization

### Phase 4: API Layer

#### Task 4.1: Update Anime API Endpoints to Use Key ✅ (November 27, 2025)

#### Task 4.2: Update Download API Endpoints to Use Key ✅ (November 27, 2025)

#### Task 4.3: Update Queue API Endpoints to Use Key ✅ (November 27, 2025)

#### Task 4.4: Update WebSocket API Endpoints to Use Key ✅ (November 27, 2025)

#### Task 4.5: Update Pydantic Models to Use Key ✅ (November 27, 2025)

---

#### Task 4.6: Update Validators to Use Key

**File:** `src/server/utils/validators.py`

**Objective:** Ensure validation functions validate `key` instead of `folder`.

**Steps:**

1. Open `src/server/utils/validators.py`
2. Review all validation functions
3. Add `validate_series_key()` function if not exists
4. Update any validators that check series identifiers
5. Ensure validators accept `key` format (URL-safe, lowercase with hyphens)
6. Add tests for key validation

**Success Criteria:**

-   [ ] Validators validate `key` format
-   [ ] No validators use `folder` for identification
-   [ ] Validation functions well-documented
-   [ ] All validator tests pass

**Test Command:**

```bash
conda run -n AniWorld python -m pytest tests/unit/ -k "validator" -v
```

---

#### Task 4.7: Update Template Helpers to Use Key

**File:** `src/server/utils/template_helpers.py`

**Objective:** Ensure template helpers pass `key` to templates for series identification.

**Steps:**

1. Open `src/server/utils/template_helpers.py`
2. Review all helper functions
3. Ensure functions that handle series data use `key`
4. Update any filtering or sorting to use `key`
5. Ensure `folder` is available for display purposes
6. Update docstrings

**Success Criteria:**

-   [ ] All helpers use `key` for identification
-   [ ] `folder` available for display
-   [ ] No breaking changes to template interface
-   [ ] All template helper tests pass

**Test Command:**

```bash
conda run -n AniWorld python -m pytest tests/unit/ -k "template" -v
```

---

### Phase 5: Frontend

#### Task 5.1: Update Frontend JavaScript to Use Key

**File:** [`src/server/web/static/js/app.js`](src/server/web/static/js/app.js)

**Objective:** Update frontend to use `key` as the primary series identifier instead of `folder`.

**Steps:**

1. Open [`src/server/web/static/js/app.js`](src/server/web/static/js/app.js)
2. Update `seriesData` storage to index by `key`
3. Update `selectedSeries` Set to use `key` instead of `folder`
4. Update `createSerieCard()`:
    - Use `data-key` attribute instead of `data-folder`
    - Display `folder` as metadata only
5. Update `toggleSerieSelection()` to use `key`
6. Update `downloadSelected()`:
    - Send `key` as `serie_id`
    - Include `folder` for filesystem operations
7. Update all event handlers and lookups to use `key`
8. Keep `folder` visible in UI for user reference

**Success Criteria:**

-   [ ] Frontend uses `key` for all series operations
-   [ ] `folder` displayed in UI but not used for identification
-   [ ] Selection tracking uses `key`
-   [ ] All frontend interactions work correctly

**Manual Test:**

1. Start server
2. Login to web interface
3. Verify series list displays correctly
4. Test selecting series (should use key internally)
5. Test downloading episodes
6. Verify search functionality

---

#### Task 5.2: Update WebSocket Events to Use Key

**File:** [`src/server/services/websocket_service.py`](src/server/services/websocket_service.py)

**Objective:** Ensure WebSocket events use `key` for series identification.

**Steps:**

1. Open [`src/server/services/websocket_service.py`](src/server/services/websocket_service.py)
2. Update `broadcast_download_progress()`:
    - Include `key` in event data
    - Keep `folder` for display purposes
3. Update `broadcast_scan_progress()` similarly
4. Update all event broadcasts to include `key`
5. Update event handler subscriptions to use `key`

**Success Criteria:**

-   [ ] All WebSocket events include `key`
-   [ ] Events also include `folder` for display
-   [ ] All WebSocket tests pass

**Test Command:**

```bash
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py -v
```

---

#### Task 5.3: Update Additional Frontend JavaScript Files

**Files:**

-   `src/server/web/static/js/websocket.js`
-   `src/server/web/static/js/queue.js`
-   `src/server/web/static/js/download.js`
-   `src/server/web/static/js/utils.js`

**Objective:** Ensure all frontend JavaScript modules use `key` for series identification.

**Steps:**

1. **Update `websocket.js`:**

    - Open `src/server/web/static/js/websocket.js`
    - Review all message handlers
    - Ensure event payloads use `key` for identification
    - Update event listeners to use `key`
    - Keep `folder` for display

2. **Update `queue.js`:**

    - Open `src/server/web/static/js/queue.js`
    - Update queue item identification to use `key`
    - Update queue manipulation functions
    - Ensure queue display shows both `key` and `folder`

3. **Update `download.js`:**

    - Open `src/server/web/static/js/download.js`
    - Update download request building to use `key`
    - Update progress tracking to use `key`
    - Keep `folder` for file path operations

4. **Update `utils.js`:**
    - Open `src/server/web/static/js/utils.js`
    - Review utility functions that handle series data
    - Ensure utilities use `key` for identification
    - Update any series-related helper functions

**Success Criteria:**

-   [ ] All JavaScript modules use `key` for identification
-   [ ] WebSocket handlers use `key`
-   [ ] Queue operations use `key`
-   [ ] Download operations use `key`
-   [ ] Utilities handle `key` correctly
-   [ ] `folder` displayed in UI where appropriate
-   [ ] All frontend functionality works correctly

**Manual Test:**

1. Test WebSocket connectivity and events
2. Test queue management
3. Test download functionality
4. Verify all series operations use `key`

---

#### Task 5.4: Update HTML Templates to Use Key

**Files:** All templates in `src/server/web/templates/`

**Objective:** Ensure all HTML templates use `key` for series identification in data attributes and forms.

**Steps:**

1. Review all template files:

    - `index.html`
    - `anime_detail.html`
    - `search.html`
    - Any other templates using series data

2. For each template:

    - Update data attributes to use `data-key` instead of `data-folder`
    - Keep `data-folder` for display purposes if needed
    - Update form inputs to submit `key` as identifier
    - Update JavaScript references to use `key`
    - Display `folder` for user-friendly names

3. Update template variables:
    - Ensure templates receive `key` from backend
    - Verify `folder` is available for display
    - Update any template logic that filters/sorts by series

**Success Criteria:**

-   [ ] All templates use `data-key` for identification
-   [ ] Forms submit `key` as identifier
-   [ ] `folder` displayed for user reference
-   [ ] No templates use `folder` for identification
-   [ ] All template rendering works correctly

**Manual Test:**

1. Verify all pages render correctly
2. Test form submissions
3. Verify JavaScript interactions
4. Check data attributes in browser dev tools

---

### Phase 6: Database Layer

#### Task 6.1: Verify Database Models Use Key Correctly

**File:** [`src/server/database/models.py`](src/server/database/models.py)

**Objective:** Verify and document that database models correctly use `key` as unique identifier.

**Steps:**

1. Open [`src/server/database/models.py`](src/server/database/models.py)
2. Verify `AnimeSeries.key` is unique and indexed
3. Verify `AnimeSeries.folder` is not used for lookups
4. Update docstrings to clarify identifier usage
5. Ensure all relationships use `id` (primary key) not `key` or `folder`

**Success Criteria:**

-   [ ] `key` is unique and indexed
-   [ ] `folder` is metadata only
-   [ ] Docstrings are clear
-   [ ] All database model tests pass

**Test Command:**

```bash
conda run -n AniWorld python -m pytest tests/unit/test_database_models.py -v
```

---

#### Task 6.2: Update Database Services to Use Key

**File:** [`src/server/database/service.py`](src/server/database/service.py)

**Objective:** Ensure all database service methods use `key` for series lookup.

**Steps:**

1. Open [`src/server/database/service.py`](src/server/database/service.py)
2. Verify `AnimeSeriesService.get_by_key()` is used for lookups
3. Verify no methods use `folder` for identification
4. Update any methods that incorrectly use `folder` for lookups
5. Add migration helper if needed to update existing data

**Success Criteria:**

-   [ ] All service methods use `key` for lookups
-   [ ] `folder` never used as identifier
-   [ ] All database service tests pass

**Test Command:**

```bash
conda run -n AniWorld python -m pytest tests/unit/test_database_service.py -v
```

---

### Phase 7: Testing and Validation

#### Task 7.1: Update All Test Fixtures to Use Key

**Files:** All test files in [`tests/`](tests/)

**Objective:** Ensure all test fixtures and mocks use `key` consistently.

**Steps:**

1. Search for all test files using `folder` as identifier
2. Update `FakeSerie` class in [`tests/api/test_anime_endpoints.py`](tests/api/test_anime_endpoints.py):
    - Ensure `key` is the primary identifier
3. Update all test fixtures to use `key`
4. Update mock data to use realistic `key` values
5. Ensure tests verify both `key` and `folder` are present but only `key` is used for operations

**Success Criteria:**

-   [ ] All test fixtures use `key` as identifier
-   [ ] Tests verify `key` is used for operations
-   [ ] Tests verify `folder` is present as metadata
-   [ ] All tests pass

**Test Command:**

```bash
conda run -n AniWorld python -m pytest tests/ -v
```

---

#### Task 7.2: Add Integration Tests for Identifier Consistency

**File:** Create new file `tests/integration/test_identifier_consistency.py`

**Objective:** Create integration tests to verify `key` is used consistently across all layers.

**Steps:**

1. Create [`tests/integration/test_identifier_consistency.py`](tests/integration/test_identifier_consistency.py)
2. Write test to verify:
    - API endpoint returns `key` as identifier
    - Download service uses `key`
    - Database lookups use `key`
    - WebSocket events include `key`
3. Write test to verify `folder` is never used for lookups
4. Write test for end-to-end flow using `key`

**Success Criteria:**

-   [ ] Integration test file created
-   [ ] Tests verify `key` usage across all layers
-   [ ] Tests verify `folder` not used for identification
-   [ ] All integration tests pass

**Test Command:**

```bash
conda run -n AniWorld python -m pytest tests/integration/test_identifier_consistency.py -v
```

---

### Phase 8: Documentation and Cleanup

#### Task 8.1: Update Infrastructure Documentation

**File:** [`infrastructure.md`](infrastructure.md)

**Objective:** Document the identifier standardization in infrastructure documentation.

**Steps:**

1. Open [`infrastructure.md`](infrastructure.md)
2. Add section explaining identifier usage:
    - `key`: Unique series identifier (provider-assigned)
    - `folder`: Filesystem folder name (metadata only)
    - `id`: Database primary key (internal use)
3. Update all API documentation to show `key` as identifier
4. Update data model documentation
5. Add migration notes if needed

**Success Criteria:**

-   [ ] Documentation clearly explains identifier usage
-   [ ] All API examples use `key`
-   [ ] Data model section updated
-   [ ] Migration notes added if applicable

---

#### Task 8.2: Update README and Developer Documentation

**Files:** [`README.md`](README.md), [`docs/`](docs/)

**Objective:** Update all developer-facing documentation.

**Steps:**

1. Update main README to explain identifier usage
2. Update any developer guides to use `key`
3. Add note about backward compatibility with `folder`
4. Update code examples to use `key`

**Success Criteria:**

-   [ ] README updated
-   [ ] Developer guides updated
-   [ ] Code examples use `key`
-   [ ] Backward compatibility documented

---

#### Task 8.3: Add Deprecation Warnings for Folder-Based Lookups

**Files:** Various service and API files

**Objective:** Add deprecation warnings where `folder` is still accepted for backward compatibility.

**Steps:**

1. Identify all methods that accept `folder` for lookups
2. Add deprecation warnings using Python's `warnings` module
3. Update docstrings to indicate deprecation
4. Plan removal timeline (e.g., next major version)

**Success Criteria:**

-   [ ] Deprecation warnings added
-   [ ] Docstrings indicate deprecation
-   [ ] Removal timeline documented
-   [ ] Tests updated to suppress warnings

---

### Phase 9: Final Validation

#### Task 9.1: Run Full Test Suite

**Objective:** Verify all changes work together correctly.

**Steps:**

1. Run complete test suite:
    ```bash
    conda run -n AniWorld python -m pytest tests/ -v --tb=short
    ```
2. Fix any failing tests
3. Verify test coverage is maintained
4. Run integration tests
5. Run manual UI tests

**Success Criteria:**

-   [ ] All unit tests pass
-   [ ] All integration tests pass
-   [ ] All API tests pass
-   [ ] Test coverage >= 80%
-   [ ] Manual UI testing successful

---

#### Task 9.2: Manual End-to-End Testing

**Objective:** Manually verify all features work with the new identifier system.

**Steps:**

1. Start server: `conda run -n AniWorld python -m uvicorn src.server.fastapi_app:app --host 127.0.0.1 --port 8000 --reload`
2. Login to web interface
3. Test search functionality (verify results show `key`)
4. Test adding new series (verify uses `key`)
5. Test downloading episodes (verify uses `key`)
6. Test WebSocket events (verify events include `key`)
7. Verify database contains correct `key` values
8. Test rescan functionality

**Success Criteria:**

-   [ ] Search works correctly
-   [ ] Adding series works
-   [ ] Downloads work correctly
-   [ ] WebSocket events work
-   [ ] Database entries correct
-   [ ] Rescan functionality works

---

#### Task 9.3: Performance and Load Testing

**Objective:** Ensure identifier changes don't impact performance.

**Steps:**

1. Run performance tests on key operations:
    - Series lookup by `key`
    - Database queries using `key`
    - API response times
2. Compare with baseline if available
3. Identify any performance regressions
4. Optimize if needed

**Success Criteria:**

-   [ ] No significant performance regression
-   [ ] Lookup by `key` is fast
-   [ ] Database queries optimized
-   [ ] API response times acceptable

---

### Phase 10: Deployment

#### Task 10.1: Create Migration Script

**Objective:** Create script to migrate existing data if needed.

**Steps:**

1. Create [`scripts/migrate_identifiers.py`](scripts/migrate_identifiers.py)
2. Script should:
    - Check all series have valid `key` values
    - Update any references that incorrectly use `folder`
    - Validate database integrity
    - Create backup before migration
3. Add rollback capability
4. Test migration on test data

**Success Criteria:**

-   [ ] Migration script created
-   [ ] Script validates data
-   [ ] Backup functionality works
-   [ ] Rollback capability tested
-   [ ] Migration tested on test data

---

#### Task 10.2: Update Deployment Documentation

**File:** Update deployment section in [`instructions.md`](instructions.md)

**Objective:** Document deployment steps for identifier changes.

**Steps:**

1. Add pre-deployment checklist
2. Document migration steps
3. Add rollback procedure
4. Document verification steps
5. Add troubleshooting guide

**Success Criteria:**

-   [ ] Deployment steps documented
-   [ ] Migration procedure clear
-   [ ] Rollback procedure documented
-   [ ] Verification steps listed
-   [ ] Troubleshooting guide added

---

#### Task 10.3: Deploy to Production

**Objective:** Deploy changes to production environment.

**Steps:**

1. Create deployment tag: `v2.0.0-identifier-standardization`
2. Backup production database
3. Run migration script
4. Deploy new code
5. Monitor logs for errors
6. Verify production functionality
7. Monitor for 24 hours

**Success Criteria:**

-   [ ] Deployment tag created
-   [ ] Database backed up
-   [ ] Migration successful
-   [ ] Code deployed
-   [ ] No errors in logs
-   [ ] All features working
-   [ ] 24-hour monitoring completed

---

## Task Tracking

### Completion Status

-   [x] Phase 1: Core Models and Data Layer ✅
-   [x] Phase 2: Core Application Layer ✅
-   [x] Phase 3: Service Layer ✅
-   [ ] Phase 4: API Layer
    -   [x] Task 4.1: Update Anime API Endpoints ✅ **Completed November 27, 2025**
    -   [x] Task 4.2: Update Download API Endpoints ✅ **Completed November 27, 2025**
    -   [x] Task 4.3: Update Queue API Endpoints ✅ **Completed November 27, 2025**
    -   [x] Task 4.4: Update WebSocket API Endpoints ✅ **Completed November 27, 2025**
    -   [x] Task 4.5: Update Pydantic Models ✅ **Completed November 27, 2025**
    -   [ ] Task 4.6: Update Validators
    -   [ ] Task 4.7: Update Template Helpers
-   [ ] Phase 5: Frontend
    -   [ ] Task 5.1: Update Frontend JavaScript
    -   [ ] Task 5.2: Update WebSocket Events
    -   [ ] Task 5.3: Update Additional Frontend JavaScript Files
    -   [ ] Task 5.4: Update HTML Templates
-   [ ] Phase 6: Database Layer
    -   [ ] Task 6.1: Verify Database Models
    -   [ ] Task 6.2: Update Database Services
-   [ ] Phase 7: Testing and Validation
    -   [ ] Task 7.1: Update Test Fixtures
    -   [ ] Task 7.2: Add Integration Tests
-   [ ] Phase 8: Documentation and Cleanup
    -   [ ] Task 8.1: Update Infrastructure Documentation
    -   [ ] Task 8.2: Update README
    -   [ ] Task 8.3: Add Deprecation Warnings
-   [ ] Phase 9: Final Validation