Aniworld/docs/instructions.md
Lukas 1ba67357dc Add database transaction support with atomic operations
- Create transaction.py with @transactional decorator, atomic() context manager
- Add TransactionPropagation modes: REQUIRED, REQUIRES_NEW, NESTED
- Add savepoint support for nested transactions with partial rollback
- Update connection.py with TransactionManager, get_transactional_session
- Update service.py with bulk operations (bulk_mark_downloaded, bulk_delete)
- Wrap QueueRepository.save_item() and clear_all() in atomic transactions
- Add comprehensive tests (66 transaction tests, 90% coverage)
- All 1090 tests passing
2025-12-25 18:05:33 +01:00

11 KiB

Aniworld Web Application Development Instructions

This document provides detailed tasks for AI agents to implement a modern web application for the Aniworld anime download manager. All tasks should follow the coding guidelines specified in the project's copilot instructions.

Project Overview

The goal is to create a FastAPI-based web application that provides a modern interface for the existing Aniworld anime download functionality. The core anime logic should remain in SeriesApp.py while the web layer provides REST API endpoints and a responsive UI.

Architecture Principles

  • Single Responsibility: Each file/class has one clear purpose
  • Dependency Injection: Use FastAPI's dependency system
  • Clean Separation: Web layer calls core logic, never the reverse
  • File Size Limit: Maximum 500 lines per file
  • Type Hints: Use comprehensive type annotations
  • Error Handling: Proper exception handling and logging

Additional Implementation Guidelines

Code Style and Standards

  • Type Hints: Use comprehensive type annotations throughout all modules
  • Docstrings: Follow PEP 257 for function and class documentation
  • Error Handling: Implement custom exception classes with meaningful messages
  • Logging: Use structured logging with appropriate log levels
  • Security: Validate all inputs and sanitize outputs
  • Performance: Use async/await patterns for I/O operations

📞 Escalation

If you encounter:

  • Architecture issues requiring design decisions
  • Tests that conflict with documented requirements
  • Breaking changes needed
  • Unclear requirements or expectations

Document the issue and escalate rather than guessing.


📚 Helpful Commands

# Run all tests
conda run -n AniWorld python -m pytest tests/ -v --tb=short

# Run specific test file
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py -v

# Run specific test class
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py::TestWebSocketService -v

# Run specific test
conda run -n AniWorld python -m pytest tests/unit/test_websocket_service.py::TestWebSocketService::test_broadcast_download_progress -v

# Run with extra verbosity
conda run -n AniWorld python -m pytest tests/ -vv

# Run with full traceback
conda run -n AniWorld python -m pytest tests/ -v --tb=long

# Run and stop at first failure
conda run -n AniWorld python -m pytest tests/ -v -x

# Run tests matching pattern
conda run -n AniWorld python -m pytest tests/ -v -k "auth"

# Show all print statements
conda run -n AniWorld python -m pytest tests/ -v -s

#Run app
conda run -n AniWorld python -m uvicorn src.server.fastapi_app:app --host 127.0.0.1 --port 8000 --reload

Implementation Notes

  1. Incremental Development: Implement features incrementally, testing each component thoroughly before moving to the next
  2. Code Review: Review all generated code for adherence to project standards
  3. Documentation: Document all public APIs and complex logic
  4. Testing: Maintain test coverage above 80% for all new code
  5. Performance: Profile and optimize critical paths, especially download and streaming operations
  6. Security: Regular security audits and dependency updates
  7. Monitoring: Implement comprehensive monitoring and alerting
  8. Maintenance: Plan for regular maintenance and updates

Task Completion Checklist

For each task completed:

  • Implementation follows coding standards
  • Unit tests written and passing
  • Integration tests passing
  • Documentation updated
  • Error handling implemented
  • Logging added
  • Security considerations addressed
  • Performance validated
  • Code reviewed
  • Task marked as complete in instructions.md
  • Infrastructure.md updated
  • Changes committed to git; keep your messages in git short and clear
  • Take the next task

Prerequisites

  1. Server is running: conda run -n AniWorld python -m uvicorn src.server.fastapi_app:app --host 127.0.0.1 --port 8000 --reload
  2. Password: Hallo123!
  3. Login via browser at http://127.0.0.1:8000/login

Notes

  • This is a simplification that removes complexity while maintaining core functionality
  • Improves user experience with explicit manual control
  • Easier to understand, test, and maintain
  • Good foundation for future enhancements if needed


Task: Add Database Transaction Support

Objective

Implement proper transaction handling across all database write operations using SQLAlchemy's transaction support. This ensures data consistency and prevents partial writes during compound operations.

Background

Currently, the application uses SQLAlchemy sessions with auto-commit behavior through the get_db_session() generator. While individual operations are atomic, compound operations (multiple writes) can result in partial commits if an error occurs mid-operation.

Requirements

  1. All database write operations must be wrapped in explicit transactions
  2. Compound operations must be atomic - either all writes succeed or all fail
  3. Nested operations should use savepoints for partial rollback capability
  4. Existing functionality must not break - backward compatible changes only
  5. All tests must pass after implementation

Step 1: Create Transaction Utilities Module

File: src/server/database/transaction.py

Create a new module providing transaction management utilities:

  1. @transactional decorator - Wraps a function in a transaction boundary

    • Accepts a session parameter or retrieves one via dependency injection
    • Commits on success, rolls back on exception
    • Re-raises exceptions after rollback
    • Logs transaction start, commit, and rollback events
  2. TransactionContext class - Context manager for explicit transaction control

    • Supports with statement usage
    • Provides savepoint() method for nested transactions using begin_nested()
    • Handles commit/rollback automatically
  3. atomic() function - Async context manager for async operations

    • Same behavior as TransactionContext but for async code

Interface Requirements:

  • Decorator must work with both sync and async functions
  • Must handle the case where session is already in a transaction
  • Must support optional propagation parameter (REQUIRED, REQUIRES_NEW, NESTED)

Step 2: Update Connection Module

File: src/server/database/connection.py

Modify the existing session management:

  1. Add get_transactional_session() generator that does NOT auto-commit
  2. Add TransactionManager class for manual transaction control
  3. Keep get_db_session() unchanged for backward compatibility
  4. Add session state inspection utilities (is_in_transaction(), get_transaction_depth())

Step 3: Wrap Service Layer Operations

File: src/server/database/service.py

Apply transaction handling to all compound write operations:

AnimeService:

  • create_anime_with_episodes() - if exists, wrap in transaction
  • Any method that calls multiple repository methods

EpisodeService:

  • bulk_update_episodes() - if exists
  • mark_episodes_downloaded() - if handles multiple episodes

DownloadQueueService:

  • add_batch_to_queue() - if exists
  • clear_and_repopulate() - if exists
  • Any method performing multiple writes

SessionService:

  • rotate_session() - delete old + create new must be atomic
  • cleanup_expired_sessions() - bulk delete operation

Pattern to follow:

@transactional
def compound_operation(self, session: Session, data: SomeModel) -> Result:
    # Multiple write operations here
    # All succeed or all fail

Step 4: Update Queue Repository

File: src/server/services/queue_repository.py

Ensure atomic operations for:

  1. save_item() - check existence + insert/update must be atomic
  2. remove_item() - if involves multiple deletes
  3. clear_all_items() - bulk delete should be transactional
  4. reorder_queue() - multiple position updates must be atomic

Step 5: Update API Endpoints

Files: src/server/api/anime.py, src/server/api/downloads.py, src/server/api/auth.py

Review and update endpoints that perform multiple database operations:

  1. Identify endpoints calling multiple service methods
  2. Wrap in transaction boundary at the endpoint level OR ensure services handle it
  3. Prefer service-level transactions over endpoint-level for reusability

Step 6: Add Unit Tests

File: tests/unit/test_transactions.py

Create comprehensive tests:

  1. Test successful transaction commit - verify all changes persisted
  2. Test rollback on exception - verify no partial writes
  3. Test nested transaction with savepoint - verify partial rollback works
  4. Test decorator with sync function
  5. Test decorator with async function
  6. Test context manager usage
  7. Test transaction propagation modes

File: tests/unit/test_service_transactions.py

  1. Test each service's compound operations for atomicity
  2. Mock exceptions mid-operation to verify rollback
  3. Verify no orphaned data after failed operations

Step 7: Update Integration Tests

File: tests/integration/test_db_transactions.py

  1. Test real database transaction behavior
  2. Test concurrent transaction handling
  3. Test transaction isolation levels if applicable

Step 7: Update Dokumentation

  1. Check Docs folder and updated the needed files

Implementation Notes

  • SQLAlchemy Pattern: Use session.begin_nested() for savepoints
  • Error Handling: Always log transaction failures with full context
  • Performance: Transactions have overhead - don't wrap single operations unnecessarily
  • Testing: Use session.rollback() in test fixtures to ensure clean state

Files to Modify

File Action
src/server/database/transaction.py CREATE - New transaction utilities
src/server/database/connection.py MODIFY - Add transactional session support
src/server/database/service.py MODIFY - Apply @transactional decorator
src/server/services/queue_repository.py MODIFY - Ensure atomic operations
src/server/api/anime.py REVIEW - Check for multi-write endpoints
src/server/api/downloads.py REVIEW - Check for multi-write endpoints
src/server/api/auth.py REVIEW - Check for multi-write endpoints
tests/unit/test_transactions.py CREATE - Transaction unit tests
tests/unit/test_service_transactions.py CREATE - Service transaction tests
tests/integration/test_db_transactions.py CREATE - Integration tests

Acceptance Criteria

  • All database write operations use explicit transactions
  • Compound operations are atomic (all-or-nothing)
  • Exceptions trigger proper rollback
  • No partial writes occur on failures
  • All existing tests pass (1090 tests passing)
  • New transaction tests pass with >90% coverage (90% achieved)
  • Logging captures transaction lifecycle events
  • Documentation updated in DATABASE.md
  • Code follows project coding standards