Lukas
31eb0026cf
- In-memory dedup in add_to_queue() using _pending_by_episode dict - Batch-local dedup via seen_in_batch set (handles duplicates within single call) - Database unique index on episode_id via __table_args__ - 5-minute cooldown in _auto_download_missing() to prevent rapid re-triggers - Updated _add_to_pending_queue() and _remove_from_pending_queue() to track episode keys - Added TestQueueDeduplication with 4 test cases - Updated DEVELOPMENT.md and TESTING.md with queue dedup docs Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
117 lines
3.6 KiB
Markdown
117 lines
3.6 KiB
Markdown
# Development Guide
|
|
|
|
## Document Purpose
|
|
|
|
This document provides guidance for developers working on the Aniworld project.
|
|
|
|
### What This Document Contains
|
|
|
|
- **Prerequisites**: Required software and tools
|
|
- **Environment Setup**: Step-by-step local development setup
|
|
- **Project Structure**: Source code organization explanation
|
|
- **Development Workflow**: Branch strategy, commit conventions
|
|
- **Coding Standards**: Style guide, linting, formatting
|
|
- **Running the Application**: Development server, CLI usage
|
|
- **Debugging Tips**: Common debugging approaches
|
|
- **IDE Configuration**: VS Code settings, recommended extensions
|
|
- **Contributing Guidelines**: How to submit changes
|
|
- **Code Review Process**: Review checklist and expectations
|
|
|
|
### What This Document Does NOT Contain
|
|
|
|
- Production deployment (see [DEPLOYMENT.md](DEPLOYMENT.md))
|
|
- API reference (see [API.md](API.md))
|
|
- Architecture decisions (see [ARCHITECTURE.md](ARCHITECTURE.md))
|
|
- Test writing guides (see [TESTING.md](TESTING.md))
|
|
- Security guidelines (see [SECURITY.md](SECURITY.md))
|
|
|
|
### Target Audience
|
|
|
|
- New Developers joining the project
|
|
- Contributors (internal and external)
|
|
- Anyone setting up a development environment
|
|
|
|
---
|
|
|
|
## Sections to Document
|
|
|
|
1. Prerequisites
|
|
- Python version
|
|
- Conda environment
|
|
- Node.js (if applicable)
|
|
- Git
|
|
2. Getting Started
|
|
- Clone repository
|
|
- Setup conda environment
|
|
- Install dependencies
|
|
- Configuration setup
|
|
3. Project Structure Overview
|
|
4. Development Server
|
|
- Starting FastAPI server
|
|
- Hot reload configuration
|
|
- Debug mode
|
|
5. CLI Development
|
|
6. Code Style
|
|
- PEP 8 compliance
|
|
- Type hints requirements
|
|
- Docstring format
|
|
- Import organization
|
|
7. Git Workflow
|
|
- Branch naming
|
|
- Commit message format
|
|
- Pull request process
|
|
8. Common Development Tasks
|
|
|
|
### Adding Queue Deduplication
|
|
|
|
The download queue prevents duplicate entries at two levels:
|
|
|
|
**In-Memory Deduplication** (`src/server/services/download_service.py`):
|
|
- `_pending_by_episode` dict tracks pending episodes: key = `(serie_id, season, episode)`
|
|
- `_add_to_pending_queue()` updates the dict when adding items
|
|
- `add_to_queue()` checks this dict before adding episodes (includes batch-local dedup)
|
|
- `_remove_from_pending_queue()` cleans up the dict when items are removed
|
|
|
|
**Database Constraint** (`src/server/models.py`):
|
|
- `DownloadQueueItem` has a unique index on `episode_id` via `__table_args__`
|
|
- Prevents duplicate queue entries at the database level
|
|
- Unique constraint: `Index("ix_download_queue_episode_pending", "episode_id", unique=True)`
|
|
|
|
**Scheduler Cooldown** (`src/server/services/scheduler_service.py`):
|
|
- `_last_auto_download_time` tracks when auto-download last ran
|
|
- 5-minute cooldown prevents rapid re-triggers
|
|
- Checked at start of `_auto_download_missing()`
|
|
|
|
### Mocking the Download Queue
|
|
|
|
When testing components that use the download queue:
|
|
|
|
```python
|
|
# Mock repository for unit tests
|
|
class MockQueueRepository:
|
|
def __init__(self):
|
|
self._items: Dict[str, DownloadItem] = {}
|
|
|
|
async def save_item(self, item: DownloadItem) -> DownloadItem:
|
|
self._items[item.id] = item
|
|
return item
|
|
|
|
async def get_all_items(self) -> List[DownloadItem]:
|
|
return list(self._items.values())
|
|
|
|
# Use in fixture
|
|
@pytest.fixture
|
|
def mock_queue_repository():
|
|
return MockQueueRepository()
|
|
|
|
@pytest.fixture
|
|
def download_service(mock_anime_service, mock_queue_repository):
|
|
return DownloadService(
|
|
anime_service=mock_anime_service,
|
|
queue_repository=mock_queue_repository,
|
|
max_retries=3,
|
|
)
|
|
```
|
|
|
|
9. Troubleshooting Development Issues
|