Aniworld/fix_test_instruction.md

Test Fixing Instructions for AniWorld Project

<EFBFBD> Current Progress (Updated: October 20, 2025)

Test Status Overview

Metric	Count	Percentage
Total Tests	583	100%
Passing	531	91.1% ✅
Failing	51	8.7% 🔄
Errors	1	0.2% ⚠️
Warnings	1487	-

Major Achievements Since Start 🎉

1. Download Endpoints API ✅

   • Before: 18 errors (0% pass rate)
   • After: 20 tests passing (100% pass rate)
   • Improvement: +100%

2. Config Endpoints API ✅

   • Before: 7 failures
   • After: 10 tests passing (100% pass rate)
   • Improvement: +100%

3. WebSocket Integration ✅

   • Before: 48 failures (0% pass rate)
   • After: 46/48 passing (95.8% pass rate)
   • Improvement: +95.8%

4. Auth Flow Integration ✅

   • Before: 43 failures
   • After: 39/43 passing (90.7% pass rate)
   • Improvement: +90.7%

5. WebSocket Service Unit Tests ✅

   • Before: 7 failures
   • After: 7/7 passing (100% pass rate)
   • Improvement: +100%

Remaining Work

• Frontend Tests: 28 failures (majority of remaining issues)
• Download Flow: 11 failures + 1 error
• Template Integration: 3 failures
• Auth Edge Cases: 4 failures
• Deprecation Warnings: 1487 (mostly datetime.utcnow())

---

<EFBFBD>📋 General Instructions

Overview

This document lists all failed tests identified during the test run on October 19, 2025. Each test failure needs to be investigated and resolved. The failures are categorized by module/area for easier resolution.

Important Guidelines

1. Double-Check Before Fixing

   • Always verify whether the test is wrong or the code is wrong
   • Read the test implementation carefully
   • Review the code being tested
   • Check if the expected behavior in the test matches the actual requirements
   • Consider if the test expectations are outdated or incorrect

2. Root Cause Analysis

   • Understand why the test is failing before making changes
   • Check if it's a:
     • Logic error in production code
     • Incorrect test expectations
     • Mock/fixture setup issue
     • Async/await issue
     • Authentication/authorization issue
     • Missing dependency or service

3. Fix Strategy

   • Fix production code if the business logic is wrong
   • Fix test code if the expectations are incorrect
   • Update both if requirements have changed
   • Document why you chose to fix test vs code

4. Testing Process

   • Run the specific test after each fix to verify
   • Run related tests to ensure no regression
   • Run all tests after batch fixes to verify overall system health

5. Code Quality Standards

   • Follow PEP8 and project coding standards
   • Use type hints where applicable
   • Write clear, self-documenting code
   • Add comments for complex logic
   • Update docstrings if behavior changes

---

🎯 Remaining Work Summary

High Priority Issues

6. Download Flow Integration Test Errors

Priority: HIGH

Files Affected:

• tests/integration/test_download_flow.py

All Tests Have ERROR Status (22 tests):

Authentication Requirements (4 errors)

1. TestAuthenticationRequirements::test_queue_status_requires_auth
2. TestAuthenticationRequirements::test_add_to_queue_requires_auth
3. TestAuthenticationRequirements::test_queue_control_requires_auth
4. TestAuthenticationRequirements::test_item_operations_require_auth

Download Flow End-to-End (5 errors)

5. TestDownloadFlowEndToEnd::test_add_episodes_to_queue
6. TestDownloadFlowEndToEnd::test_queue_status_after_adding_items
7. TestDownloadFlowEndToEnd::test_add_with_different_priorities
8. TestDownloadFlowEndToEnd::test_validation_error_for_empty_episodes
9. TestDownloadFlowEndToEnd::test_validation_error_for_invalid_priority

Queue Control Operations (4 errors)

10. TestQueueControlOperations::test_start_queue_processing
11. TestQueueControlOperations::test_pause_queue_processing
12. TestQueueControlOperations::test_resume_queue_processing
13. TestQueueControlOperations::test_clear_completed_downloads

Queue Item Operations (3 errors)

14. TestQueueItemOperations::test_remove_item_from_queue
15. TestQueueItemOperations::test_retry_failed_item
16. TestQueueItemOperations::test_reorder_queue_items

Progress Tracking (2 errors)

17. TestDownloadProgressTracking::test_queue_status_includes_progress
18. TestDownloadProgressTracking::test_queue_statistics

Error Handling (2 errors)

19. TestErrorHandlingAndRetries::test_handle_download_failure
20. TestErrorHandlingAndRetries::test_retry_count_increments

Concurrent Operations (2 errors)

21. TestConcurrentOperations::test_multiple_concurrent_downloads
22. TestConcurrentOperations::test_concurrent_status_requests

Plus Additional Tests:

• Queue Persistence (2 errors)
• WebSocket Integration (1 error)
• Complete Download Workflow (2 errors)

Investigation Required:

• Check test setup/teardown
• Verify authentication setup in fixtures
• Review async test handling

---

7. Download Endpoints API Test Errors

Priority: HIGH

Files Affected:

• tests/api/test_download_endpoints.py

All Tests Have ERROR Status (18 tests):

1. test_get_queue_status
2. test_add_to_queue
3. test_add_to_queue_with_high_priority
4. test_add_to_queue_empty_episodes
5. test_add_to_queue_service_error
6. test_remove_from_queue_single
7. test_remove_from_queue_not_found
8. test_remove_multiple_from_queue
9. test_remove_multiple_empty_list
10. test_start_queue
11. test_stop_queue
12. test_pause_queue
13. test_resume_queue
14. test_reorder_queue
15. test_reorder_queue_not_found
16. test_clear_completed
17. test_retry_failed
18. test_retry_all_failed

Investigation Required:

• Check test fixtures
• Review authentication setup
• Verify download service availability

---

8. Template Integration Test Failures

Priority: MEDIUM

Files Affected:

• tests/unit/test_template_integration.py

Tests Failing:

1. TestTemplateIntegration::test_error_template_404
2. TestTemplateIntegration::test_queue_template_has_websocket_script
3. TestTemplateIntegration::test_templates_accessibility_features

Investigation Required:

• Check template rendering
• Verify template file locations
• Review Jinja2 template configuration

---

Phase 9: Frontend Integration Smoke Tests 🔄 IN PROGRESS

• Fix frontend integration smoke tests (3 failures → 1 remaining)
• Run and verify: pytest tests/integration/test_frontend_integration_smoke.py -v

Remaining Issue (1 failure):

• test_authenticated_endpoints_require_bearer_token

---

🎯 Remaining Work Summary

High Priority Issues

1. Frontend Existing UI Integration (24 failures)

   • All TestFrontendAnimeAPI tests
   • All TestFrontendDownloadAPI tests
   • All TestFrontendWebSocketIntegration tests
   • All TestFrontendConfigAPI tests
   • All TestFrontendJavaScriptIntegration tests
   • All TestFrontendErrorHandling tests
   • All TestFrontendRealTimeUpdates tests
   • All TestFrontendDataFormats tests

2. Download Flow Integration (11 failures + 1 error)

   • Queue operations
   • Progress tracking
   • Auth requirements

3. Authentication Edge Cases (4 failures)

   • Invalid token handling
   • Endpoint protection tests

Medium Priority Issues

4. Frontend Auth Integration (4 failures)

   • Token validation
   • Password validation
   • Request authentication

5. Template Integration (3 failures)

   • Error templates
   • WebSocket scripts
   • Accessibility features

6. WebSocket Multi-Room (2 failures)

   • Concurrent broadcasts
   • Multi-room workflow

Low Priority Issues

7. Frontend Integration Smoke (1 failure)

   • Bearer token requirement

8. Deprecation Warnings (1487 warnings)

   • Mostly datetime.utcnow() usage

---

✅ Completed Sections (Can be removed from this document)

The following sections have been successfully completed:

Deprecated Pydantic V1 API Usage

Count: ~20 warnings

Location: src/config/settings.py

Issue:

# Deprecated usage
Field(default="value", env="ENV_VAR")

# Should be:
Field(default="value", json_schema_extra={"env": "ENV_VAR"})

Files to Update:

• src/config/settings.py - Update Field definitions
• src/server/models/config.py - Update validators to @field_validator

---

Deprecated datetime.utcnow() Usage

Count: ~100+ warnings

Issue:

# Deprecated
datetime.utcnow()

# Should use
datetime.now(datetime.UTC)

Files to Update:

• src/server/services/auth_service.py
• src/server/services/download_service.py
• src/server/services/progress_service.py
• src/server/services/websocket_service.py
• src/server/database/service.py
• src/server/database/models.py
• All test files using datetime.utcnow()

---

Deprecated FastAPI on_event

Count: 4 warnings

Location: src/server/fastapi_app.py

Issue:

# Deprecated
@app.on_event("startup")
@app.on_event("shutdown")

# Should use lifespan
from contextlib import asynccontextmanager

@asynccontextmanager
async def lifespan(app: FastAPI):
    # Startup
    yield
    # Shutdown

---

Deprecated Pydantic .dict() Method

Count: ~5 warnings

Issue:

# Deprecated
session.dict()

# Should be
session.model_dump()

Files to Update:

• src/server/middleware/auth.py
• src/server/utils/dependencies.py

---

📝 Task Checklist for AI Agent

Current Status Summary (Updated: October 20, 2025)

Test Results:

• Total Tests: 583
• Passed: 531 (91%) ✅
• Failed: 51 (9%) 🔄
• Errors: 1 (<1%) ⚠️
• Warnings: 1487 (deprecation warnings)

Major Improvements:

• ✅ Download Endpoints API: 18/18 tests passing (was 18 errors)
• ✅ Auth Flow: 39/43 tests passing (was 43 failures)
• ✅ Config Endpoints: All tests passing (was 7 failures)
• ✅ WebSocket Integration: 46/48 tests passing (was 48 failures)

---

Phase 1: Critical Async Issues ✅ COMPLETE

• Fix all async/await issues in test_frontend_auth_integration.py (10 tests)
• Verify test methods are properly marked as async
• Run and verify: pytest tests/integration/test_frontend_auth_integration.py -v

Status: All async issues resolved in this file.

---

Phase 2: WebSocket Broadcast Issues ✅ COMPLETE

• Investigate WebSocket service broadcast implementation
• Fix mock configuration in test_websocket_service.py (7 tests)
• Fix connection lifecycle management
• Run and verify: pytest tests/unit/test_websocket_service.py -v

Status: All 7 tests passing.

Phase 3: Authentication System ✅ MOSTLY COMPLETE

• Debug auth middleware and service
• Fix auth flow integration tests (43 tests → 4 remaining failures)
• Fix config endpoint auth issues (7 tests → All passing)
• Fix download endpoint auth issues (2 tests → All passing)
• Run and verify: pytest tests/integration/test_auth_flow.py -v
• Run and verify: pytest tests/api/test_config_endpoints.py -v

Remaining Issues (4 failures):

• test_access_protected_endpoint_with_invalid_token
• test_anime_endpoints_require_auth
• test_queue_endpoints_require_auth
• test_config_endpoints_require_auth

Phase 4: Frontend Integration 🔄 IN PROGRESS

• Fix frontend auth integration tests (42 total → 4 remaining failures)
• Fix frontend existing UI integration tests (24 failures)
• Run and verify: pytest tests/frontend/ -v

Remaining Issues:

Frontend Auth Integration (4 failures):

• test_authenticated_request_without_token_returns_401
• test_authenticated_request_with_invalid_token_returns_401
• test_weak_password_validation_in_setup
• test_token_included_in_all_authenticated_requests

Frontend Existing UI Integration (24 failures):

• TestFrontendAnimeAPI (3 failures): test_get_anime_list, test_search_anime, test_rescan_anime
• TestFrontendDownloadAPI (5 failures): All queue-related tests
• TestFrontendWebSocketIntegration (3 failures): All WebSocket tests
• TestFrontendConfigAPI (2 failures): test_get_config, test_update_config
• TestFrontendJavaScriptIntegration (3 failures): All JS integration tests
• TestFrontendErrorHandling (2 failures): All error handling tests
• TestFrontendRealTimeUpdates (3 failures): All real-time tests
• TestFrontendDataFormats (3 failures): All format tests

Phase 5: WebSocket Integration ✅ MOSTLY COMPLETE

• Fix websocket integration tests (48 failures → 2 remaining)
• Test connection management
• Test broadcasting mechanism
• Run and verify: pytest tests/integration/test_websocket.py -v

Remaining Issues (2 failures):

• test_concurrent_broadcasts_to_different_rooms
• test_multi_room_workflow

Phase 6: Download Flow ✅ MOSTLY COMPLETE

• Fix download endpoint API tests (18 errors → All 20 tests passing!)
• Fix download flow integration tests (22+ errors → 11 remaining)
• Run and verify: pytest tests/api/test_download_endpoints.py -v
• Run and verify: pytest tests/integration/test_download_flow.py -v

Remaining Issues (11 failures):

• test_queue_status_after_adding_items
• test_start_queue_processing
• test_pause_queue_processing
• test_resume_queue_processing
• test_clear_completed_downloads
• test_remove_item_from_queue
• test_reorder_queue_items
• test_queue_statistics
• test_queue_control_requires_auth
• test_item_operations_require_auth
• test_full_download_cycle
• test_workflow_with_pause_and_resume
• Plus 1 ERROR in test_add_episodes_to_queue

Phase 7: Template Integration 🔄 IN PROGRESS

• Fix template integration tests (3 failures remaining)
• Run and verify: pytest tests/unit/test_template_integration.py -v

Remaining Issues (3 failures):

• test_error_template_404
• test_queue_template_has_websocket_script
• test_templates_accessibility_features

---

Phase 8: Deprecation Warnings 🔄 IN PROGRESS

• Update Pydantic V2 Field definitions (~20 warnings)
• Replace datetime.utcnow() with datetime.now(datetime.UTC) (~1400+ warnings)
• Update FastAPI to use lifespan instead of on_event (4 warnings)
• Replace .dict() with .model_dump() (~5 warnings)
• Run and verify: pytest tests/ -v --tb=short

Priority: LOW (functionality works, but warnings clutter output)

---

Phase 9: Final Verification

• Fix remaining 51 test failures
• Fix 1 remaining test error
• Run all tests: pytest tests/ -v
• Verify all tests pass
• Reduce warnings to < 50 (mostly from dependencies)
• Document any remaining issues

---

🎯 Success Criteria

1. All tests passing: 0 failures, 0 errors
2. Warnings reduced: Aim for < 50 warnings (mostly from dependencies)
3. Code quality maintained: No shortcuts or hacks
4. Documentation updated: Any behavior changes documented
5. Git commits: Logical, atomic commits with clear messages

---

📞 Escalation

If you encounter:

• Architecture issues requiring design decisions
• Tests that conflict with documented requirements
• Breaking changes needed
• Unclear requirements or expectations

Document the issue and escalate rather than guessing.

---

📚 Helpful Commands

# Run all tests
conda run -n AniWorld python -m pytest tests/ -v --tb=short

# Run specific test file
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py -v

# Run specific test class
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py::TestWebSocketService -v

# Run specific test
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py::TestWebSocketService::test_broadcast_download_progress -v

# Run with extra verbosity
conda run -n AniWorld python -m pytest tests/ -vv

# Run with full traceback
conda run -n AniWorld python -m pytest tests/ -v --tb=long

# Run and stop at first failure
conda run -n AniWorld python -m pytest tests/ -v -x

# Run tests matching pattern
conda run -n AniWorld python -m pytest tests/ -v -k "auth"

# Show all print statements
conda run -n AniWorld python -m pytest tests/ -v -s

---

📖 Additional Notes

• Document Created: October 19, 2025
• Last Updated: October 20, 2025
• Test run time: ~8 seconds
• Python environment: AniWorld (conda)
• Framework: pytest with FastAPI TestClient
• Initial test failures: 200+ (many were errors)
• Current test failures: 51 + 1 error
• Overall progress: ~75% reduction in failures

Remember: The goal is not just to make tests pass, but to ensure the system works correctly and reliably!

---

🎯 Next Steps for Completion

To achieve 100% test pass rate, focus on:

1. Frontend Integration Tests (highest priority)

   • These represent the bulk of remaining failures (28 tests)
   • May indicate issues with test setup or mock configuration
   • Could also reveal real integration issues

2. Download Flow Integration (high priority)

   • 11 failures + 1 error remaining
   • Core functionality that needs to work reliably

3. Edge Case Handling (medium priority)

   • Auth edge cases (4 tests)
   • WebSocket multi-room (2 tests)
   • Template integration (3 tests)

4. Deprecation Warnings (low priority, can be batch processed)

   • Replace datetime.utcnow() across codebase
   • Update Pydantic v2 APIs
   • Modernize FastAPI lifespan handling

Estimated time to completion: 2-4 hours of focused work

---

✅ Completed Work Archive

Successfully Fixed Test Suites

1. tests/api/test_download_endpoints.py - 20/20 passing ✅
2. tests/api/test_config_endpoints.py - 10/10 passing ✅
3. tests/unit/test_websocket_service.py - 7/7 passing ✅
4. tests/api/test_anime_endpoints.py - 7/7 passing ✅
5. tests/api/test_auth_endpoints.py - 1/1 passing ✅
6. tests/integration/test_auth_flow.py - 39/43 passing (90%+) ✅
7. tests/integration/test_websocket.py - 46/48 passing (95%+) ✅
8. Most unit tests - High pass rate across the board ✅

Key Improvements Made

• Fixed async/await patterns in integration tests
• Corrected WebSocket mock configurations
• Improved authentication flow handling
• Fixed download service initialization
• Enhanced error handling in services
• Improved test fixtures and setup/teardown

Total tests passing: 531/583 (91.1%)