Aniworld/instructions.md
Lukas 684337fd0c Add data file to database sync functionality
- Add get_all_series_from_data_files() to SeriesApp
- Sync series from data files to DB on startup
- Add unit tests for new SeriesApp method
- Add integration tests for sync functionality
- Update documentation
2025-12-13 09:32:57 +01:00

9.6 KiB

Aniworld Web Application Development Instructions

This document provides detailed tasks for AI agents to implement a modern web application for the Aniworld anime download manager. All tasks should follow the coding guidelines specified in the project's copilot instructions.

Project Overview

The goal is to create a FastAPI-based web application that provides a modern interface for the existing Aniworld anime download functionality. The core anime logic should remain in SeriesApp.py while the web layer provides REST API endpoints and a responsive UI.

Architecture Principles

  • Single Responsibility: Each file/class has one clear purpose
  • Dependency Injection: Use FastAPI's dependency system
  • Clean Separation: Web layer calls core logic, never the reverse
  • File Size Limit: Maximum 500 lines per file
  • Type Hints: Use comprehensive type annotations
  • Error Handling: Proper exception handling and logging

Additional Implementation Guidelines

Code Style and Standards

  • Type Hints: Use comprehensive type annotations throughout all modules
  • Docstrings: Follow PEP 257 for function and class documentation
  • Error Handling: Implement custom exception classes with meaningful messages
  • Logging: Use structured logging with appropriate log levels
  • Security: Validate all inputs and sanitize outputs
  • Performance: Use async/await patterns for I/O operations

📞 Escalation

If you encounter:

  • Architecture issues requiring design decisions
  • Tests that conflict with documented requirements
  • Breaking changes needed
  • Unclear requirements or expectations

Document the issue and escalate rather than guessing.


📚 Helpful Commands

# Run all tests
conda run -n AniWorld python -m pytest tests/ -v --tb=short

# Run specific test file
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py -v

# Run specific test class
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py::TestWebSocketService -v

# Run specific test
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py::TestWebSocketService::test_broadcast_download_progress -v

# Run with extra verbosity
conda run -n AniWorld python -m pytest tests/ -vv

# Run with full traceback
conda run -n AniWorld python -m pytest tests/ -v --tb=long

# Run and stop at first failure
conda run -n AniWorld python -m pytest tests/ -v -x

# Run tests matching pattern
conda run -n AniWorld python -m pytest tests/ -v -k "auth"

# Show all print statements
conda run -n AniWorld python -m pytest tests/ -v -s

#Run app
conda run -n AniWorld python -m uvicorn src.server.fastapi_app:app --host 127.0.0.1 --port 8000 --reload

Implementation Notes

  1. Incremental Development: Implement features incrementally, testing each component thoroughly before moving to the next
  2. Code Review: Review all generated code for adherence to project standards
  3. Documentation: Document all public APIs and complex logic
  4. Testing: Maintain test coverage above 80% for all new code
  5. Performance: Profile and optimize critical paths, especially download and streaming operations
  6. Security: Regular security audits and dependency updates
  7. Monitoring: Implement comprehensive monitoring and alerting
  8. Maintenance: Plan for regular maintenance and updates

Task Completion Checklist

For each task completed:

  • Implementation follows coding standards
  • Unit tests written and passing
  • Integration tests passing
  • Documentation updated
  • Error handling implemented
  • Logging added
  • Security considerations addressed
  • Performance validated
  • Code reviewed
  • Task marked as complete in instructions.md
  • Infrastructure.md updated
  • Changes committed to git; keep your messages in git short and clear
  • Take the next task

Prerequisites

  1. Server is running: conda run -n AniWorld python -m uvicorn src.server.fastapi_app:app --host 127.0.0.1 --port 8000 --reload
  2. Password: Hallo123!
  3. Login via browser at http://127.0.0.1:8000/login

Notes

  • This is a simplification that removes complexity while maintaining core functionality
  • Improves user experience with explicit manual control
  • Easier to understand, test, and maintain
  • Good foundation for future enhancements if needed

📋 TODO Tasks

Task 1: Add get_all_series_from_data_files() Method to SeriesApp

Status: [x] Completed

Description: Add a new method to SeriesApp that returns all series data found in data files from the filesystem.

File to Modify: src/core/SeriesApp.py

Requirements:

  1. Add a new method get_all_series_from_data_files() -> List[Serie] to SeriesApp
  2. This method should scan the directory_to_search for all data files
  3. Load and return all Serie objects found in data files
  4. Use the existing SerieList.load_series() pattern for file discovery
  5. Return an empty list if no data files are found
  6. Include proper logging for debugging
  7. Method should be synchronous (can be wrapped with asyncio.to_thread if needed)

Implementation Details:

def get_all_series_from_data_files(self) -> List[Serie]:
    """
    Get all series from data files in the anime directory.

    Scans the directory_to_search for all 'data' files and loads
    the Serie metadata from each file.

    Returns:
        List of Serie objects found in data files
    """
    # Use SerieList's file-based loading to get all series
    # Return list of Serie objects from self.list.keyDict.values()

Acceptance Criteria:

  • Method exists in SeriesApp
  • Method returns List[Serie]
  • Method scans filesystem for data files
  • Proper error handling for missing/corrupt files
  • Logging added for operations
  • Unit tests written and passing

Task 2: Sync Series from Data Files to Database on Setup Complete

Status: [x] Completed

Description: When the application setup is complete (anime directory configured), automatically sync all series from data files to the database.

Files to Modify:

  • src/server/fastapi_app.py (lifespan function)
  • src/server/services/ (if needed for service layer)

Requirements:

  1. After download_service.initialize() succeeds in the lifespan function
  2. Call SeriesApp.get_all_series_from_data_files() to get all series
  3. For each series, use SerieList.add_to_db() to save to database (uses existing DB schema)
  4. Skip series that already exist in database (handled by add_to_db)
  5. Log the sync progress and results
  6. Do NOT modify database model definitions

Implementation Details:

# In lifespan function, after download_service.initialize():
try:
    from src.server.database.connection import get_db_session

    # Get all series from data files using SeriesApp
    series_app = SeriesApp(settings.anime_directory)
    all_series = series_app.get_all_series_from_data_files()

    if all_series:
        async with get_db_session() as db:
            serie_list = SerieList(settings.anime_directory, db_session=db, skip_load=True)
            added_count = 0
            for serie in all_series:
                result = await serie_list.add_to_db(serie, db)
                if result:
                    added_count += 1
            await db.commit()
            logger.info("Synced %d new series to database", added_count)
except Exception as e:
    logger.warning("Failed to sync series to database: %s", e)

Acceptance Criteria:

  • Series from data files are synced to database on startup
  • Existing series in database are not duplicated
  • Database schema is NOT modified
  • Proper error handling (app continues even if sync fails)
  • Logging added for sync operations
  • Integration tests written and passing

Task 3: Validation - Verify Data File to Database Sync

Status: [x] Completed

Description: Create validation tests to ensure the data file to database sync works correctly.

File to Create: tests/integration/test_data_file_db_sync.py

Requirements:

  1. Test get_all_series_from_data_files() returns correct data
  2. Test that series are correctly added to database
  3. Test that duplicate series are not created
  4. Test that sync handles empty directories gracefully
  5. Test that sync handles corrupt data files gracefully
  6. Test end-to-end startup sync behavior

Test Cases:

class TestDataFileDbSync:
    """Test data file to database synchronization."""

    async def test_get_all_series_from_data_files_returns_list(self):
        """Test that get_all_series_from_data_files returns a list."""
        pass

    async def test_get_all_series_from_data_files_empty_directory(self):
        """Test behavior with empty anime directory."""
        pass

    async def test_series_sync_to_db_creates_records(self):
        """Test that series are correctly synced to database."""
        pass

    async def test_series_sync_to_db_no_duplicates(self):
        """Test that duplicate series are not created."""
        pass

    async def test_series_sync_handles_corrupt_files(self):
        """Test that corrupt data files don't crash the sync."""
        pass

    async def test_startup_sync_integration(self):
        """Test end-to-end startup sync behavior."""
        pass

Acceptance Criteria:

  • All test cases implemented
  • Tests use pytest async fixtures
  • Tests use temporary directories for isolation
  • Tests cover happy path and error cases
  • All tests passing
  • Code coverage > 80% for new code