Aniworld/fix_test_instruction.md

# Test Fixing Instructions for AniWorld Project

## 📋 General Instructions

### Overview

This document lists all failed tests identified during the test run on October 19, 2025. Each test failure needs to be investigated and resolved. The failures are categorized by module/area for easier resolution.

### Important Guidelines

1. **Double-Check Before Fixing**

    - **Always verify whether the test is wrong or the code is wrong**
    - Read the test implementation carefully
    - Review the code being tested
    - Check if the expected behavior in the test matches the actual requirements
    - Consider if the test expectations are outdated or incorrect

2. **Root Cause Analysis**

    - Understand why the test is failing before making changes
    - Check if it's a:
        - Logic error in production code
        - Incorrect test expectations
        - Mock/fixture setup issue
        - Async/await issue
        - Authentication/authorization issue
        - Missing dependency or service

3. **Fix Strategy**

    - Fix production code if the business logic is wrong
    - Fix test code if the expectations are incorrect
    - Update both if requirements have changed
    - Document why you chose to fix test vs code

4. **Testing Process**

    - Run the specific test after each fix to verify
    - Run related tests to ensure no regression
    - Run all tests after batch fixes to verify overall system health

5. **Code Quality Standards**
    - Follow PEP8 and project coding standards
    - Use type hints where applicable
    - Write clear, self-documenting code
    - Add comments for complex logic
    - Update docstrings if behavior changes

---

## 🔴 Critical Issues to Address First

### 3. Authentication/Authorization Failures

**Priority:** HIGH

**Files Affected:**

-   `tests/api/test_config_endpoints.py`
-   `tests/api/test_download_endpoints.py`
-   `tests/integration/test_auth_flow.py`

**Tests Failing:**

#### Config Endpoints (7 failures)

1. `test_validate_invalid_config`
2. `test_update_config_unauthorized`
3. `test_list_backups` (403 instead of 200)
4. `test_create_backup` (403 instead of expected)
5. `test_restore_backup` (403 instead of expected)
6. `test_delete_backup` (403 instead of expected)
7. `test_config_persistence` (403 instead of expected)

#### Download Endpoints (2 failures)

8. `test_get_queue_status_unauthorized`
9. `test_queue_endpoints_require_auth`

#### Auth Flow Integration (43 failures)

Multiple test classes affected - all authentication flow tests

**Symptoms:**

-   403 Forbidden responses when 200 expected
-   Authentication/authorization not working as expected
-   Tokens not being validated correctly

**Root Cause Options:**

1. Middleware not properly checking authentication
2. Test fixtures not setting up auth correctly
3. Token generation/validation broken
4. Session management issues

**Investigation Required:**

-   Review `src/server/middleware/auth.py`
-   Check `src/server/services/auth_service.py`
-   Verify test fixtures for authentication setup
-   Check if routes are properly protected

---

### 4. Frontend Integration Test Errors

**Priority:** HIGH

**Files Affected:**

-   `tests/frontend/test_existing_ui_integration.py`

**Tests Failing:**

1. `TestFrontendAuthentication::test_auth_status_endpoint_not_configured`
2. `TestFrontendAuthentication::test_auth_status_configured_not_authenticated`
3. `TestFrontendAuthentication::test_login_returns_jwt_token`
4. `TestFrontendAuthentication::test_unauthorized_request_returns_401`
5. `TestFrontendJavaScriptIntegration::test_frontend_handles_401_gracefully`

**Additional Test Errors (with assertion errors):**

-   37 more tests in this file with ERROR status

**Investigation Required:**

-   Check test setup and fixtures
-   Verify client initialization
-   Review async handling

---

### 5. WebSocket Integration Test Failures

**Priority:** MEDIUM

**Files Affected:**

-   `tests/integration/test_websocket.py`

**Tests Failing (48 total):**

#### Connection Management (7 failures)

1. `TestWebSocketConnection::test_websocket_endpoint_exists` (ERROR)
2. `TestWebSocketConnection::test_connection_manager_tracks_connections`
3. `TestWebSocketConnection::test_disconnect_removes_connection`
4. `TestWebSocketConnection::test_room_assignment_on_connection`
5. `TestWebSocketConnection::test_multiple_rooms_support`

#### Message Broadcasting (4 failures)

6. `TestMessageBroadcasting::test_broadcast_to_all_connections`
7. `TestMessageBroadcasting::test_broadcast_to_specific_room`
8. `TestMessageBroadcasting::test_broadcast_with_json_message`
9. `TestMessageBroadcasting::test_broadcast_handles_disconnected_clients`

#### Progress Integration (3 failures)

10. `TestProgressIntegration::test_download_progress_broadcasts_to_websocket`
11. `TestProgressIntegration::test_download_complete_notification`
12. `TestProgressIntegration::test_download_failed_notification`

#### Queue Status Broadcasting (3 failures)

13. `TestQueueStatusBroadcasting::test_queue_status_update_broadcast`
14. `TestQueueStatusBroadcasting::test_queue_item_added_notification`
15. `TestQueueStatusBroadcasting::test_queue_item_removed_notification`

#### System Messaging (2 failures)

16. `TestSystemMessaging::test_system_notification_broadcast`
17. `TestSystemMessaging::test_error_message_broadcast`

#### Concurrent Connections (2 failures)

18. `TestConcurrentConnections::test_multiple_clients_in_same_room`
19. `TestConcurrentConnections::test_concurrent_broadcasts_to_different_rooms`

#### Connection Error Handling (3 failures)

20. `TestConnectionErrorHandling::test_handle_send_failure`
21. `TestConnectionErrorHandling::test_handle_multiple_send_failures`
22. `TestConnectionErrorHandling::test_cleanup_after_disconnect`

#### Message Formatting (2 failures)

23. `TestMessageFormatting::test_message_structure_validation`
24. `TestMessageFormatting::test_different_message_types`

#### Room Management (3 failures)

25. `TestRoomManagement::test_room_creation_on_first_connection`
26. `TestRoomManagement::test_room_cleanup_when_empty`
27. `TestRoomManagement::test_client_can_be_in_one_room`

#### Complete Workflow (2 failures)

28. `TestCompleteWebSocketWorkflow::test_full_download_notification_workflow`
29. `TestCompleteWebSocketWorkflow::test_multi_room_workflow`

**Investigation Required:**

-   WebSocket endpoint routing
-   Connection manager implementation
-   Room management logic
-   Broadcast mechanism

---

### 6. Download Flow Integration Test Errors

**Priority:** HIGH

**Files Affected:**

-   `tests/integration/test_download_flow.py`

**All Tests Have ERROR Status (22 tests):**

#### Authentication Requirements (4 errors)

1. `TestAuthenticationRequirements::test_queue_status_requires_auth`
2. `TestAuthenticationRequirements::test_add_to_queue_requires_auth`
3. `TestAuthenticationRequirements::test_queue_control_requires_auth`
4. `TestAuthenticationRequirements::test_item_operations_require_auth`

#### Download Flow End-to-End (5 errors)

5. `TestDownloadFlowEndToEnd::test_add_episodes_to_queue`
6. `TestDownloadFlowEndToEnd::test_queue_status_after_adding_items`
7. `TestDownloadFlowEndToEnd::test_add_with_different_priorities`
8. `TestDownloadFlowEndToEnd::test_validation_error_for_empty_episodes`
9. `TestDownloadFlowEndToEnd::test_validation_error_for_invalid_priority`

#### Queue Control Operations (4 errors)

10. `TestQueueControlOperations::test_start_queue_processing`
11. `TestQueueControlOperations::test_pause_queue_processing`
12. `TestQueueControlOperations::test_resume_queue_processing`
13. `TestQueueControlOperations::test_clear_completed_downloads`

#### Queue Item Operations (3 errors)

14. `TestQueueItemOperations::test_remove_item_from_queue`
15. `TestQueueItemOperations::test_retry_failed_item`
16. `TestQueueItemOperations::test_reorder_queue_items`

#### Progress Tracking (2 errors)

17. `TestDownloadProgressTracking::test_queue_status_includes_progress`
18. `TestDownloadProgressTracking::test_queue_statistics`

#### Error Handling (2 errors)

19. `TestErrorHandlingAndRetries::test_handle_download_failure`
20. `TestErrorHandlingAndRetries::test_retry_count_increments`

#### Concurrent Operations (2 errors)

21. `TestConcurrentOperations::test_multiple_concurrent_downloads`
22. `TestConcurrentOperations::test_concurrent_status_requests`

**Plus Additional Tests:**

-   Queue Persistence (2 errors)
-   WebSocket Integration (1 error)
-   Complete Download Workflow (2 errors)

**Investigation Required:**

-   Check test setup/teardown
-   Verify authentication setup in fixtures
-   Review async test handling

---

### 7. Download Endpoints API Test Errors

**Priority:** HIGH

**Files Affected:**

-   `tests/api/test_download_endpoints.py`

**All Tests Have ERROR Status (18 tests):**

1. `test_get_queue_status`
2. `test_add_to_queue`
3. `test_add_to_queue_with_high_priority`
4. `test_add_to_queue_empty_episodes`
5. `test_add_to_queue_service_error`
6. `test_remove_from_queue_single`
7. `test_remove_from_queue_not_found`
8. `test_remove_multiple_from_queue`
9. `test_remove_multiple_empty_list`
10. `test_start_queue`
11. `test_stop_queue`
12. `test_pause_queue`
13. `test_resume_queue`
14. `test_reorder_queue`
15. `test_reorder_queue_not_found`
16. `test_clear_completed`
17. `test_retry_failed`
18. `test_retry_all_failed`

**Investigation Required:**

-   Check test fixtures
-   Review authentication setup
-   Verify download service availability

---

### 8. Template Integration Test Failures

**Priority:** MEDIUM

**Files Affected:**

-   `tests/unit/test_template_integration.py`

**Tests Failing:**

1. `TestTemplateIntegration::test_error_template_404`
2. `TestTemplateIntegration::test_queue_template_has_websocket_script`
3. `TestTemplateIntegration::test_templates_accessibility_features`

**Investigation Required:**

-   Check template rendering
-   Verify template file locations
-   Review Jinja2 template configuration

---

### 9. Frontend Integration Smoke Tests

**Priority:** MEDIUM

**Files Affected:**

-   `tests/integration/test_frontend_integration_smoke.py`

**Tests Failing:**

1. `TestFrontendIntegration::test_login_returns_jwt_token`
2. `TestFrontendIntegration::test_authenticated_endpoints_require_bearer_token`
3. `TestFrontendIntegration::test_queue_endpoints_accessible_with_token`

**Investigation Required:**

-   Check authentication flow
-   Verify token generation
-   Review endpoint protection

---

## ⚠️ Non-Critical Warnings to Address

### Deprecated Pydantic V1 API Usage

**Count:** ~20 warnings

**Location:** `src/config/settings.py`

**Issue:**

```python
# Deprecated usage
Field(default="value", env="ENV_VAR")

# Should be:
Field(default="value", json_schema_extra={"env": "ENV_VAR"})
```

**Files to Update:**

-   `src/config/settings.py` - Update Field definitions
-   `src/server/models/config.py` - Update validators to `@field_validator`

---

### Deprecated datetime.utcnow() Usage

**Count:** ~100+ warnings

**Issue:**

```python
# Deprecated
datetime.utcnow()

# Should use
datetime.now(datetime.UTC)
```

**Files to Update:**

-   `src/server/services/auth_service.py`
-   `src/server/services/download_service.py`
-   `src/server/services/progress_service.py`
-   `src/server/services/websocket_service.py`
-   `src/server/database/service.py`
-   `src/server/database/models.py`
-   All test files using `datetime.utcnow()`

---

### Deprecated FastAPI on_event

**Count:** 4 warnings

**Location:** `src/server/fastapi_app.py`

**Issue:**

```python
# Deprecated
@app.on_event("startup")
@app.on_event("shutdown")

# Should use lifespan
from contextlib import asynccontextmanager

@asynccontextmanager
async def lifespan(app: FastAPI):
    # Startup
    yield
    # Shutdown
```

---

### Deprecated Pydantic .dict() Method

**Count:** ~5 warnings

**Issue:**

```python
# Deprecated
session.dict()

# Should be
session.model_dump()
```

**Files to Update:**

-   `src/server/middleware/auth.py`
-   `src/server/utils/dependencies.py`

---

## 📝 Task Checklist for AI Agent

### Phase 1: Critical Async Issues

-   [ ] Fix all async/await issues in `test_frontend_auth_integration.py` (10 tests)
-   [ ] Verify test methods are properly marked as async
-   [ ] Run and verify: `pytest tests/integration/test_frontend_auth_integration.py -v`

### Phase 2: WebSocket Broadcast Issues

-   [ ] Investigate WebSocket service broadcast implementation
-   [ ] Fix mock configuration in `test_websocket_service.py` (7 tests)
-   [ ] Fix connection lifecycle management
-   [ ] Run and verify: `pytest tests/unit/test_websocket_service.py -v`

### Phase 3: Authentication System

-   [ ] Debug auth middleware and service
-   [ ] Fix auth flow integration tests (43 tests)
-   [ ] Fix config endpoint auth issues (7 tests)
-   [ ] Fix download endpoint auth issues (2 tests)
-   [ ] Run and verify: `pytest tests/integration/test_auth_flow.py -v`
-   [ ] Run and verify: `pytest tests/api/test_config_endpoints.py -v`

### Phase 4: Frontend Integration

-   [ ] Fix frontend auth integration tests (5 failures + 37 errors)
-   [ ] Fix frontend integration smoke tests (3 failures)
-   [ ] Run and verify: `pytest tests/frontend/ -v`

### Phase 5: WebSocket Integration

-   [ ] Fix websocket integration tests (48 failures)
-   [ ] Test connection management
-   [ ] Test broadcasting mechanism
-   [ ] Run and verify: `pytest tests/integration/test_websocket.py -v`

### Phase 6: Download Flow

-   [ ] Fix download flow integration tests (22+ errors)
-   [ ] Fix download endpoint API tests (18 errors)
-   [ ] Run and verify: `pytest tests/integration/test_download_flow.py -v`
-   [ ] Run and verify: `pytest tests/api/test_download_endpoints.py -v`

### Phase 7: Template Integration

-   [ ] Fix template integration tests (3 failures)
-   [ ] Run and verify: `pytest tests/unit/test_template_integration.py -v`

### Phase 8: Deprecation Warnings

-   [ ] Update Pydantic V2 Field definitions
-   [ ] Replace `datetime.utcnow()` with `datetime.now(datetime.UTC)`
-   [ ] Update FastAPI to use lifespan instead of on_event
-   [ ] Replace `.dict()` with `.model_dump()`
-   [ ] Run and verify: `pytest tests/ -v --tb=short`

### Phase 9: Final Verification

-   [ ] Run all tests: `pytest tests/ -v`
-   [ ] Verify all tests pass
-   [ ] Verify warnings are reduced
-   [ ] Document any remaining issues

---

## 🎯 Success Criteria

1. **All tests passing:** 0 failures, 0 errors
2. **Warnings reduced:** Aim for < 50 warnings (mostly from dependencies)
3. **Code quality maintained:** No shortcuts or hacks
4. **Documentation updated:** Any behavior changes documented
5. **Git commits:** Logical, atomic commits with clear messages

---

## 📞 Escalation

If you encounter:

-   Architecture issues requiring design decisions
-   Tests that conflict with documented requirements
-   Breaking changes needed
-   Unclear requirements or expectations

**Document the issue and escalate rather than guessing.**

---

## 📚 Helpful Commands

```bash
# Run all tests
conda run -n AniWorld python -m pytest tests/ -v --tb=short

# Run specific test file
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py -v

# Run specific test class
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py::TestWebSocketService -v

# Run specific test
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py::TestWebSocketService::test_broadcast_download_progress -v

# Run with extra verbosity
conda run -n AniWorld python -m pytest tests/ -vv

# Run with full traceback
conda run -n AniWorld python -m pytest tests/ -v --tb=long

# Run and stop at first failure
conda run -n AniWorld python -m pytest tests/ -v -x

# Run tests matching pattern
conda run -n AniWorld python -m pytest tests/ -v -k "auth"

# Show all print statements
conda run -n AniWorld python -m pytest tests/ -v -s
```

---

## 📖 Additional Notes

-   This document was generated on: October 19, 2025
-   Test run took: 6.63 seconds
-   Python environment: AniWorld (conda)
-   Framework: pytest with FastAPI TestClient

**Remember:** The goal is not just to make tests pass, but to ensure the system works correctly and reliably!