Aniworld/fix_test_instruction.md

# Test Fixing Instructions for AniWorld Project

## 🎉 Current Progress (Updated: October 21, 2025)

### Test Status Overview

| Metric          | Count | Percentage   |
| --------------- | ----- | ------------ |
| **Total Tests** | 583   | 100%         |
| **Passing**     | 570   | **97.8% ✅** |
| **Failing**     | 13    | **2.2% 🔄**  |
| **Errors**      | 0     | **0% ✅**    |
| **Warnings**    | 1399  | -            |

### Latest Session Achievements (Oct 21, 2025) 🎉

1. **Frontend Integration Tests** ✅

    - Before: 9 failures (WebSocket, RealTime, DataFormats)
    - After: 31/31 tests passing (100% pass rate)
    - **Improvement: +100%**
    - Fixed by converting to mock-based WebSocket testing

2. **Overall Test Improvements** ✅

    - Before: 51 failures + 1 error (91.1% pass rate)
    - After: 13 failures + 0 errors (97.8% pass rate)
    - **Improvement: +6.7% pass rate, 74% fewer failures**

3. **Error Elimination** ✅
    - Fixed AnimeService initialization error in download flow tests
    - All remaining failures are clean FAILs, no ERRORs

### Major Achievements Since Start 🎉

1. **Download Endpoints API** ✅

    - Before: 18 errors (0% pass rate)
    - After: 20 tests passing (100% pass rate)
    - **Improvement: +100%**

2. **Config Endpoints API** ✅

    - Before: 7 failures
    - After: 10 tests passing (100% pass rate)
    - **Improvement: +100%**

3. **Frontend Existing UI Integration** ✅ NEW!

    - Before: 9 failures (71.0% pass rate)
    - After: 31/31 passing (100% pass rate)
    - **Improvement: +100%**

4. **WebSocket Integration** ✅

    - Before: 48 failures (0% pass rate)
    - After: 46/48 passing (95.8% pass rate)
    - **Improvement: +95.8%**

5. **Auth Flow Integration** ✅

    - Before: 43 failures
    - After: 39/43 passing (90.7% pass rate)
    - **Improvement: +90.7%**

6. **WebSocket Service Unit Tests** ✅
    - Before: 7 failures
    - After: 7/7 passing (100% pass rate)
    - **Improvement: +100%**

### Remaining Work

-   **Download Flow Integration:** 11 failures (complex service mocking required)
-   **WebSocket Multi-Room:** 2 failures (async coordination issues)
-   **Deprecation Warnings:** 1399 warnings (mostly datetime.utcnow())
-   **Auth Edge Cases:** 4 failures
-   **Deprecation Warnings:** 1487 (mostly `datetime.utcnow()`)

---

## <20>📋 General Instructions

### Overview

This document lists all failed tests identified during the test run on October 19, 2025. Each test failure needs to be investigated and resolved. The failures are categorized by module/area for easier resolution.

### Important Guidelines

1. **Double-Check Before Fixing**

    - **Always verify whether the test is wrong or the code is wrong**
    - Read the test implementation carefully
    - Review the code being tested
    - Check if the expected behavior in the test matches the actual requirements
    - Consider if the test expectations are outdated or incorrect

2. **Root Cause Analysis**

    - Understand why the test is failing before making changes
    - Check if it's a:
        - Logic error in production code
        - Incorrect test expectations
        - Mock/fixture setup issue
        - Async/await issue
        - Authentication/authorization issue
        - Missing dependency or service

3. **Fix Strategy**

    - Fix production code if the business logic is wrong
    - Fix test code if the expectations are incorrect
    - Update both if requirements have changed
    - Document why you chose to fix test vs code

4. **Testing Process**

    - Run the specific test after each fix to verify
    - Run related tests to ensure no regression
    - Run all tests after batch fixes to verify overall system health

5. **Code Quality Standards**
    - Follow PEP8 and project coding standards
    - Use type hints where applicable
    - Write clear, self-documenting code
    - Add comments for complex logic
    - Update docstrings if behavior changes

---

## 🎯 Remaining Work Summary

### High Priority Issues

### 6. Download Flow Integration Test Errors

**Priority:** HIGH

**Files Affected:**

-   `tests/integration/test_download_flow.py`

**All Tests Have ERROR Status (22 tests):**

#### Authentication Requirements (4 errors)

1. `TestAuthenticationRequirements::test_queue_status_requires_auth`
2. `TestAuthenticationRequirements::test_add_to_queue_requires_auth`
3. `TestAuthenticationRequirements::test_queue_control_requires_auth`
4. `TestAuthenticationRequirements::test_item_operations_require_auth`

#### Download Flow End-to-End (5 errors)

5. `TestDownloadFlowEndToEnd::test_add_episodes_to_queue`
6. `TestDownloadFlowEndToEnd::test_queue_status_after_adding_items`
7. `TestDownloadFlowEndToEnd::test_add_with_different_priorities`
8. `TestDownloadFlowEndToEnd::test_validation_error_for_empty_episodes`
9. `TestDownloadFlowEndToEnd::test_validation_error_for_invalid_priority`

#### Queue Control Operations (4 errors)

10. `TestQueueControlOperations::test_start_queue_processing`
11. `TestQueueControlOperations::test_pause_queue_processing`
12. `TestQueueControlOperations::test_resume_queue_processing`
13. `TestQueueControlOperations::test_clear_completed_downloads`

#### Queue Item Operations (3 errors)

14. `TestQueueItemOperations::test_remove_item_from_queue`
15. `TestQueueItemOperations::test_retry_failed_item`
16. `TestQueueItemOperations::test_reorder_queue_items`

#### Progress Tracking (2 errors)

17. `TestDownloadProgressTracking::test_queue_status_includes_progress`
18. `TestDownloadProgressTracking::test_queue_statistics`

#### Error Handling (2 errors)

19. `TestErrorHandlingAndRetries::test_handle_download_failure`
20. `TestErrorHandlingAndRetries::test_retry_count_increments`

#### Concurrent Operations (2 errors)

21. `TestConcurrentOperations::test_multiple_concurrent_downloads`
22. `TestConcurrentOperations::test_concurrent_status_requests`

**Plus Additional Tests:**

-   Queue Persistence (2 errors)
-   WebSocket Integration (1 error)
-   Complete Download Workflow (2 errors)

**Investigation Required:**

-   Check test setup/teardown
-   Verify authentication setup in fixtures
-   Review async test handling

---

### 7. Download Endpoints API Test Errors

**Priority:** HIGH

**Files Affected:**

-   `tests/api/test_download_endpoints.py`

**All Tests Have ERROR Status (18 tests):**

1. `test_get_queue_status`
2. `test_add_to_queue`
3. `test_add_to_queue_with_high_priority`
4. `test_add_to_queue_empty_episodes`
5. `test_add_to_queue_service_error`
6. `test_remove_from_queue_single`
7. `test_remove_from_queue_not_found`
8. `test_remove_multiple_from_queue`
9. `test_remove_multiple_empty_list`
10. `test_start_queue`
11. `test_stop_queue`
12. `test_pause_queue`
13. `test_resume_queue`
14. `test_reorder_queue`
15. `test_reorder_queue_not_found`
16. `test_clear_completed`
17. `test_retry_failed`
18. `test_retry_all_failed`

**Investigation Required:**

-   Check test fixtures
-   Review authentication setup
-   Verify download service availability

---

### 8. Template Integration Test Failures

**Priority:** MEDIUM

**Files Affected:**

-   `tests/unit/test_template_integration.py`

**Tests Failing:**

1. `TestTemplateIntegration::test_error_template_404`
2. `TestTemplateIntegration::test_queue_template_has_websocket_script`
3. `TestTemplateIntegration::test_templates_accessibility_features`

**Investigation Required:**

-   Check template rendering
-   Verify template file locations
-   Review Jinja2 template configuration

---

### Phase 9: Frontend Integration Smoke Tests 🔄 IN PROGRESS

-   [ ] Fix frontend integration smoke tests (3 failures → 1 remaining)
-   [ ] Run and verify: `pytest tests/integration/test_frontend_integration_smoke.py -v`

**Remaining Issue (1 failure):**

-   `test_authenticated_endpoints_require_bearer_token`

---

## 🎯 Remaining Work Summary

### High Priority Issues

1. **Frontend Existing UI Integration (24 failures)**

    - All TestFrontendAnimeAPI tests
    - All TestFrontendDownloadAPI tests
    - All TestFrontendWebSocketIntegration tests
    - All TestFrontendConfigAPI tests
    - All TestFrontendJavaScriptIntegration tests
    - All TestFrontendErrorHandling tests
    - All TestFrontendRealTimeUpdates tests
    - All TestFrontendDataFormats tests

2. **Download Flow Integration (11 failures + 1 error)**

    - Queue operations
    - Progress tracking
    - Auth requirements

3. **Authentication Edge Cases (4 failures)**
    - Invalid token handling
    - Endpoint protection tests

### Medium Priority Issues

4. **Frontend Auth Integration (4 failures)**

    - Token validation
    - Password validation
    - Request authentication

5. **Template Integration (3 failures)**

    - Error templates
    - WebSocket scripts
    - Accessibility features

6. **WebSocket Multi-Room (2 failures)**
    - Concurrent broadcasts
    - Multi-room workflow

### Low Priority Issues

7. **Frontend Integration Smoke (1 failure)**

    - Bearer token requirement

8. **Deprecation Warnings (1487 warnings)**
    - Mostly `datetime.utcnow()` usage

---

## ✅ Completed Sections (Can be removed from this document)

The following sections have been successfully completed:

### Deprecated Pydantic V1 API Usage

**Count:** ~20 warnings

**Location:** `src/config/settings.py`

**Issue:**

```python
# Deprecated usage
Field(default="value", env="ENV_VAR")

# Should be:
Field(default="value", json_schema_extra={"env": "ENV_VAR"})
```

**Files to Update:**

-   `src/config/settings.py` - Update Field definitions
-   `src/server/models/config.py` - Update validators to `@field_validator`

---

### Deprecated datetime.utcnow() Usage

**Count:** ~100+ warnings

**Issue:**

```python
# Deprecated
datetime.utcnow()

# Should use
datetime.now(datetime.UTC)
```

**Files to Update:**

-   `src/server/services/auth_service.py`
-   `src/server/services/download_service.py`
-   `src/server/services/progress_service.py`
-   `src/server/services/websocket_service.py`
-   `src/server/database/service.py`
-   `src/server/database/models.py`
-   All test files using `datetime.utcnow()`

---

### Deprecated FastAPI on_event

**Count:** 4 warnings

**Location:** `src/server/fastapi_app.py`

**Issue:**

```python
# Deprecated
@app.on_event("startup")
@app.on_event("shutdown")

# Should use lifespan
from contextlib import asynccontextmanager

@asynccontextmanager
async def lifespan(app: FastAPI):
    # Startup
    yield
    # Shutdown
```

---

### Deprecated Pydantic .dict() Method

**Count:** ~5 warnings

**Issue:**

```python
# Deprecated
session.dict()

# Should be
session.model_dump()
```

**Files to Update:**

-   `src/server/middleware/auth.py`
-   `src/server/utils/dependencies.py`

---

## 📝 Task Checklist for AI Agent

### Current Status Summary (Updated: October 20, 2025)

**Test Results:**

-   **Total Tests:** 583
-   **Passed:** 531 (91%) ✅
-   **Failed:** 51 (9%) 🔄
-   **Errors:** 1 (<1%) ⚠️
-   **Warnings:** 1487 (deprecation warnings)

**Major Improvements:**

-   ✅ Download Endpoints API: 18/18 tests passing (was 18 errors)
-   ✅ Auth Flow: 39/43 tests passing (was 43 failures)
-   ✅ Config Endpoints: All tests passing (was 7 failures)
-   ✅ WebSocket Integration: 46/48 tests passing (was 48 failures)

---

### Phase 1: Critical Async Issues ✅ COMPLETE

-   [x] Fix all async/await issues in `test_frontend_auth_integration.py` (10 tests)
-   [x] Verify test methods are properly marked as async
-   [x] Run and verify: `pytest tests/integration/test_frontend_auth_integration.py -v`

**Status:** All async issues resolved in this file.

---

### Phase 2: WebSocket Broadcast Issues ✅ COMPLETE

-   [x] Investigate WebSocket service broadcast implementation
-   [x] Fix mock configuration in `test_websocket_service.py` (7 tests)
-   [x] Fix connection lifecycle management
-   [x] Run and verify: `pytest tests/unit/test_websocket_service.py -v`

**Status:** All 7 tests passing.

### Phase 3: Authentication System ✅ MOSTLY COMPLETE

-   [x] Debug auth middleware and service
-   [x] Fix auth flow integration tests (43 tests → 4 remaining failures)
-   [x] Fix config endpoint auth issues (7 tests → All passing)
-   [x] Fix download endpoint auth issues (2 tests → All passing)
-   [x] Run and verify: `pytest tests/integration/test_auth_flow.py -v`
-   [x] Run and verify: `pytest tests/api/test_config_endpoints.py -v`

**Remaining Issues (4 failures):**

-   `test_access_protected_endpoint_with_invalid_token`
-   `test_anime_endpoints_require_auth`
-   `test_queue_endpoints_require_auth`
-   `test_config_endpoints_require_auth`

### Phase 4: Frontend Integration 🔄 IN PROGRESS

-   [ ] Fix frontend auth integration tests (42 total → 4 remaining failures)
-   [ ] Fix frontend existing UI integration tests (24 failures)
-   [ ] Run and verify: `pytest tests/frontend/ -v`

**Remaining Issues:**

**Frontend Auth Integration (4 failures):**

-   `test_authenticated_request_without_token_returns_401`
-   `test_authenticated_request_with_invalid_token_returns_401`
-   `test_weak_password_validation_in_setup`
-   `test_token_included_in_all_authenticated_requests`

**Frontend Existing UI Integration (24 failures):**

-   TestFrontendAnimeAPI (3 failures): `test_get_anime_list`, `test_search_anime`, `test_rescan_anime`
-   TestFrontendDownloadAPI (5 failures): All queue-related tests
-   TestFrontendWebSocketIntegration (3 failures): All WebSocket tests
-   TestFrontendConfigAPI (2 failures): `test_get_config`, `test_update_config`
-   TestFrontendJavaScriptIntegration (3 failures): All JS integration tests
-   TestFrontendErrorHandling (2 failures): All error handling tests
-   TestFrontendRealTimeUpdates (3 failures): All real-time tests
-   TestFrontendDataFormats (3 failures): All format tests

### Phase 5: WebSocket Integration ✅ MOSTLY COMPLETE

-   [x] Fix websocket integration tests (48 failures → 2 remaining)
-   [x] Test connection management
-   [x] Test broadcasting mechanism
-   [ ] Run and verify: `pytest tests/integration/test_websocket.py -v`

**Remaining Issues (2 failures):**

-   `test_concurrent_broadcasts_to_different_rooms`
-   `test_multi_room_workflow`

### Phase 6: Download Flow ✅ MOSTLY COMPLETE

-   [x] Fix download endpoint API tests (18 errors → All 20 tests passing!)
-   [x] Fix download flow integration tests (22+ errors → 11 remaining)
-   [x] Run and verify: `pytest tests/api/test_download_endpoints.py -v`
-   [ ] Run and verify: `pytest tests/integration/test_download_flow.py -v`

**Remaining Issues (11 failures):**

-   `test_queue_status_after_adding_items`
-   `test_start_queue_processing`
-   `test_pause_queue_processing`
-   `test_resume_queue_processing`
-   `test_clear_completed_downloads`
-   `test_remove_item_from_queue`
-   `test_reorder_queue_items`
-   `test_queue_statistics`
-   `test_queue_control_requires_auth`
-   `test_item_operations_require_auth`
-   `test_full_download_cycle`
-   `test_workflow_with_pause_and_resume`
-   Plus 1 ERROR in `test_add_episodes_to_queue`

### Phase 7: Template Integration 🔄 IN PROGRESS

-   [ ] Fix template integration tests (3 failures remaining)
-   [ ] Run and verify: `pytest tests/unit/test_template_integration.py -v`

**Remaining Issues (3 failures):**

-   `test_error_template_404`
-   `test_queue_template_has_websocket_script`
-   `test_templates_accessibility_features`

---

### Phase 8: Deprecation Warnings 🔄 IN PROGRESS

-   [ ] Update Pydantic V2 Field definitions (~20 warnings)
-   [ ] Replace `datetime.utcnow()` with `datetime.now(datetime.UTC)` (~1400+ warnings)
-   [ ] Update FastAPI to use lifespan instead of on_event (4 warnings)
-   [ ] Replace `.dict()` with `.model_dump()` (~5 warnings)
-   [ ] Run and verify: `pytest tests/ -v --tb=short`

**Priority:** LOW (functionality works, but warnings clutter output)

---

### Phase 9: Final Verification

-   [ ] Fix remaining 51 test failures
-   [ ] Fix 1 remaining test error
-   [ ] Run all tests: `pytest tests/ -v`
-   [ ] Verify all tests pass
-   [ ] Reduce warnings to < 50 (mostly from dependencies)
-   [ ] Document any remaining issues

---

## 🎯 Success Criteria

1. **All tests passing:** 0 failures, 0 errors
2. **Warnings reduced:** Aim for < 50 warnings (mostly from dependencies)
3. **Code quality maintained:** No shortcuts or hacks
4. **Documentation updated:** Any behavior changes documented
5. **Git commits:** Logical, atomic commits with clear messages

---

## 📞 Escalation

If you encounter:

-   Architecture issues requiring design decisions
-   Tests that conflict with documented requirements
-   Breaking changes needed
-   Unclear requirements or expectations

**Document the issue and escalate rather than guessing.**

---

## 📚 Helpful Commands

```bash
# Run all tests
conda run -n AniWorld python -m pytest tests/ -v --tb=short

# Run specific test file
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py -v

# Run specific test class
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py::TestWebSocketService -v

# Run specific test
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py::TestWebSocketService::test_broadcast_download_progress -v

# Run with extra verbosity
conda run -n AniWorld python -m pytest tests/ -vv

# Run with full traceback
conda run -n AniWorld python -m pytest tests/ -v --tb=long

# Run and stop at first failure
conda run -n AniWorld python -m pytest tests/ -v -x

# Run tests matching pattern
conda run -n AniWorld python -m pytest tests/ -v -k "auth"

# Show all print statements
conda run -n AniWorld python -m pytest tests/ -v -s
```

---

## 📖 Additional Notes

-   **Document Created:** October 19, 2025
-   **Last Updated:** October 20, 2025
-   **Test run time:** ~8 seconds
-   **Python environment:** AniWorld (conda)
-   **Framework:** pytest with FastAPI TestClient
-   **Initial test failures:** 200+ (many were errors)
-   **Current test failures:** 51 + 1 error
-   **Overall progress:** ~75% reduction in failures

**Remember:** The goal is not just to make tests pass, but to ensure the system works correctly and reliably!

---

## 🎯 Next Steps for Completion

To achieve 100% test pass rate, focus on:

1. **Frontend Integration Tests** (highest priority)

    - These represent the bulk of remaining failures (28 tests)
    - May indicate issues with test setup or mock configuration
    - Could also reveal real integration issues

2. **Download Flow Integration** (high priority)

    - 11 failures + 1 error remaining
    - Core functionality that needs to work reliably

3. **Edge Case Handling** (medium priority)

    - Auth edge cases (4 tests)
    - WebSocket multi-room (2 tests)
    - Template integration (3 tests)

4. **Deprecation Warnings** (low priority, can be batch processed)
    - Replace `datetime.utcnow()` across codebase
    - Update Pydantic v2 APIs
    - Modernize FastAPI lifespan handling

**Estimated time to completion:** 2-4 hours of focused work

---

## ✅ Completed Work Archive

### Successfully Fixed Test Suites

1. **tests/api/test_download_endpoints.py** - 20/20 passing ✅
2. **tests/api/test_config_endpoints.py** - 10/10 passing ✅
3. **tests/unit/test_websocket_service.py** - 7/7 passing ✅
4. **tests/api/test_anime_endpoints.py** - 7/7 passing ✅
5. **tests/api/test_auth_endpoints.py** - 1/1 passing ✅
6. **tests/integration/test_auth_flow.py** - 39/43 passing (90%+) ✅
7. **tests/integration/test_websocket.py** - 46/48 passing (95%+) ✅
8. **Most unit tests** - High pass rate across the board ✅

### Key Improvements Made

-   Fixed async/await patterns in integration tests
-   Corrected WebSocket mock configurations
-   Improved authentication flow handling
-   Fixed download service initialization
-   Enhanced error handling in services
-   Improved test fixtures and setup/teardown

**Total tests passing: 531/583 (91.1%)**