feat: Complete WebSocket integration with core services
- Enhanced DownloadService broadcasts for all queue operations - Download progress, complete, and failed broadcasts with full metadata - Queue operations (add, remove, reorder, retry, clear) broadcast queue status - Queue control (start, stop, pause, resume) broadcasts state changes - AnimeService scan progress fully integrated with ProgressService - Scan lifecycle events (start, update, complete, fail) broadcasted - Progress tracking via ProgressService to scan_progress room - ProgressService WebSocket integration - Broadcast callback registered during application startup - All progress types route to appropriate rooms - Throttled broadcasts for performance (>1% changes) - Comprehensive integration tests - Test download progress and completion broadcasts - Test queue operation broadcasts - Test scan progress lifecycle - Test progress service integration - End-to-end flow testing - Updated infrastructure documentation - Detailed broadcast message formats - Room structure and subscription patterns - Production deployment considerations - Architecture benefits and scalability notes
This commit is contained in:
parent
8c8853d26e
commit
71207bc935
@ -546,9 +546,9 @@ Implemented comprehensive REST API endpoints for download queue management:
|
|||||||
- Follows same patterns as other API routers (auth, anime, config)
|
- Follows same patterns as other API routers (auth, anime, config)
|
||||||
- Full OpenAPI documentation available at `/api/docs`
|
- Full OpenAPI documentation available at `/api/docs`
|
||||||
|
|
||||||
### WebSocket Real-time Updates (October 2025)
|
### WebSocket Integration with Core Services (October 2025)
|
||||||
|
|
||||||
Implemented real-time progress tracking and WebSocket broadcasting for downloads, scans, and system events.
|
Completed comprehensive integration of WebSocket broadcasting with all core services to provide real-time updates for downloads, scans, queue operations, and progress tracking.
|
||||||
|
|
||||||
#### ProgressService
|
#### ProgressService
|
||||||
|
|
||||||
@ -697,3 +697,277 @@ Comprehensive test coverage including:
|
|||||||
- Multi-process progress synchronization (Redis/shared store)
|
- Multi-process progress synchronization (Redis/shared store)
|
||||||
- Progress event hooks for custom actions
|
- Progress event hooks for custom actions
|
||||||
- Client-side progress resumption after reconnection
|
- Client-side progress resumption after reconnection
|
||||||
|
|
||||||
|
### Core Services WebSocket Integration (October 2025)
|
||||||
|
|
||||||
|
Completed comprehensive integration of WebSocket broadcasting with all core services (DownloadService, AnimeService, ProgressService) to provide real-time updates to connected clients.
|
||||||
|
|
||||||
|
#### DownloadService WebSocket Integration
|
||||||
|
|
||||||
|
**File**: `src/server/services/download_service.py`
|
||||||
|
|
||||||
|
The download service broadcasts real-time updates for all queue and download operations:
|
||||||
|
|
||||||
|
**Download Progress Broadcasting**:
|
||||||
|
|
||||||
|
- `download_progress` - Real-time progress updates during download
|
||||||
|
- Includes: download_id, serie_name, season, episode, progress data (percent, speed, ETA)
|
||||||
|
- Sent via ProgressService which broadcasts to `download_progress` room
|
||||||
|
- Progress callback created for each download item with metadata tracking
|
||||||
|
|
||||||
|
**Download Completion/Failure Broadcasting**:
|
||||||
|
|
||||||
|
- `download_complete` - Successful download completion
|
||||||
|
- Includes: download_id, serie_name, season, episode, downloaded_mb
|
||||||
|
- Broadcast to `downloads` room
|
||||||
|
- `download_failed` - Download failure notification
|
||||||
|
- Includes: download_id, serie_name, season, episode, error, retry_count
|
||||||
|
- Broadcast to `downloads` room
|
||||||
|
|
||||||
|
**Queue Operations Broadcasting**:
|
||||||
|
All queue operations broadcast `queue_status` messages with current queue state:
|
||||||
|
|
||||||
|
- `items_added` - Items added to queue
|
||||||
|
- Data: added_ids, queue_status (complete queue state)
|
||||||
|
- `items_removed` - Items removed/cancelled
|
||||||
|
- Data: removed_ids, queue_status
|
||||||
|
- `queue_reordered` - Queue order changed
|
||||||
|
- Data: item_id, new_position, queue_status
|
||||||
|
- `items_retried` - Failed items retried
|
||||||
|
- Data: retried_ids, queue_status
|
||||||
|
- `completed_cleared` - Completed items cleared
|
||||||
|
- Data: cleared_count, queue_status
|
||||||
|
|
||||||
|
**Queue Control Broadcasting**:
|
||||||
|
|
||||||
|
- `queue_started` - Queue processor started
|
||||||
|
- Data: is_running=True, queue_status
|
||||||
|
- `queue_stopped` - Queue processor stopped
|
||||||
|
- Data: is_running=False, queue_status
|
||||||
|
- `queue_paused` - Queue processing paused
|
||||||
|
- Data: is_paused=True, queue_status
|
||||||
|
- `queue_resumed` - Queue processing resumed
|
||||||
|
- Data: is_paused=False, queue_status
|
||||||
|
|
||||||
|
**Broadcast Callback Setup**:
|
||||||
|
The download service broadcast callback is registered during dependency injection in `src/server/utils/dependencies.py`:
|
||||||
|
|
||||||
|
- Maps update types to WebSocket service methods
|
||||||
|
- Routes download_progress, download_complete, download_failed to appropriate rooms
|
||||||
|
- All queue operations broadcast complete queue status for client synchronization
|
||||||
|
|
||||||
|
#### AnimeService WebSocket Integration
|
||||||
|
|
||||||
|
**File**: `src/server/services/anime_service.py`
|
||||||
|
|
||||||
|
The anime service integrates with ProgressService for library scan operations:
|
||||||
|
|
||||||
|
**Scan Progress Broadcasting**:
|
||||||
|
|
||||||
|
- Scan operations use ProgressService for progress tracking
|
||||||
|
- Progress updates broadcast to `scan_progress` room
|
||||||
|
- Lifecycle events:
|
||||||
|
- `started` - Scan initialization
|
||||||
|
- `in_progress` - Ongoing scan with current/total file counts
|
||||||
|
- `completed` - Successful scan completion
|
||||||
|
- `failed` - Scan failure with error message
|
||||||
|
|
||||||
|
**Scan Implementation**:
|
||||||
|
|
||||||
|
- `rescan()` method wraps SeriesApp.ReScan with progress tracking
|
||||||
|
- Progress callback executed in threadpool updates ProgressService
|
||||||
|
- ProgressService automatically broadcasts to WebSocket clients
|
||||||
|
- Cache invalidation on successful scan completion
|
||||||
|
|
||||||
|
#### ProgressService WebSocket Integration
|
||||||
|
|
||||||
|
**File**: `src/server/services/progress_service.py`
|
||||||
|
|
||||||
|
Central service for tracking and broadcasting all progress operations:
|
||||||
|
|
||||||
|
**Progress Types**:
|
||||||
|
|
||||||
|
- `DOWNLOAD` - File download progress
|
||||||
|
- `SCAN` - Library scan progress
|
||||||
|
- `QUEUE` - Queue operation progress
|
||||||
|
- `SYSTEM` - System-level operations
|
||||||
|
- `ERROR` - Error notifications
|
||||||
|
|
||||||
|
**Progress Lifecycle**:
|
||||||
|
|
||||||
|
1. `start_progress()` - Initialize progress operation
|
||||||
|
- Broadcasts to room: `{progress_type}_progress`
|
||||||
|
2. `update_progress()` - Update progress values
|
||||||
|
- Calculates percentage automatically
|
||||||
|
- Broadcasts only on significant changes (>1% or forced)
|
||||||
|
3. `complete_progress()` - Mark operation complete
|
||||||
|
- Sets progress to 100%
|
||||||
|
- Moves to history
|
||||||
|
- Broadcasts completion
|
||||||
|
4. `fail_progress()` - Mark operation failed
|
||||||
|
- Captures error message
|
||||||
|
- Moves to history
|
||||||
|
- Broadcasts failure
|
||||||
|
|
||||||
|
**Broadcast Callback**:
|
||||||
|
|
||||||
|
- Callback registered during application startup in `src/server/fastapi_app.py`
|
||||||
|
- Links ProgressService to WebSocketService.manager.broadcast_to_room
|
||||||
|
- All progress updates automatically broadcast to appropriate rooms
|
||||||
|
|
||||||
|
#### WebSocket Room Structure
|
||||||
|
|
||||||
|
Clients subscribe to specific rooms to receive targeted updates:
|
||||||
|
|
||||||
|
**Room Types**:
|
||||||
|
|
||||||
|
- `downloads` - All download-related events (complete, failed, queue status)
|
||||||
|
- `download_progress` - Real-time download progress updates
|
||||||
|
- `scan_progress` - Library scan progress updates
|
||||||
|
- `queue_progress` - Queue operation progress (future use)
|
||||||
|
- `system_progress` - System-level progress (future use)
|
||||||
|
|
||||||
|
**Room Subscription**:
|
||||||
|
Clients join rooms by sending WebSocket messages:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"action": "join",
|
||||||
|
"room": "download_progress"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Message Format
|
||||||
|
|
||||||
|
All WebSocket messages follow a consistent structure:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"type": "download_progress" | "download_complete" | "queue_status" | etc.,
|
||||||
|
"timestamp": "2025-10-17T12:34:56.789Z",
|
||||||
|
"data": {
|
||||||
|
// Message-specific data
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Example: Download Progress**
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"type": "download_progress",
|
||||||
|
"timestamp": "2025-10-17T12:34:56.789Z",
|
||||||
|
"data": {
|
||||||
|
"download_id": "abc123",
|
||||||
|
"serie_name": "Attack on Titan",
|
||||||
|
"season": 1,
|
||||||
|
"episode": 5,
|
||||||
|
"progress": {
|
||||||
|
"percent": 45.2,
|
||||||
|
"downloaded_mb": 226.0,
|
||||||
|
"total_mb": 500.0,
|
||||||
|
"speed_mbps": 2.5,
|
||||||
|
"eta_seconds": 120
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Example: Queue Status**
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"type": "queue_status",
|
||||||
|
"timestamp": "2025-10-17T12:34:56.789Z",
|
||||||
|
"data": {
|
||||||
|
"action": "items_added",
|
||||||
|
"added_ids": ["item1", "item2"],
|
||||||
|
"queue_status": {
|
||||||
|
"is_running": true,
|
||||||
|
"is_paused": false,
|
||||||
|
"active_downloads": [...],
|
||||||
|
"pending_queue": [...],
|
||||||
|
"completed_downloads": [...],
|
||||||
|
"failed_downloads": [...]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Integration Testing
|
||||||
|
|
||||||
|
**File**: `tests/integration/test_websocket_integration.py`
|
||||||
|
|
||||||
|
Comprehensive integration tests verify WebSocket broadcasting:
|
||||||
|
|
||||||
|
**Test Coverage**:
|
||||||
|
|
||||||
|
- Download progress broadcasts during active downloads
|
||||||
|
- Queue operation broadcasts (add, remove, reorder, clear, retry)
|
||||||
|
- Queue control broadcasts (start, stop, pause, resume)
|
||||||
|
- Scan progress broadcasts (start, update, complete, fail)
|
||||||
|
- Progress lifecycle broadcasts for all operation types
|
||||||
|
- End-to-end flow with multiple services broadcasting
|
||||||
|
|
||||||
|
**Test Strategy**:
|
||||||
|
|
||||||
|
- Mock broadcast callbacks to capture emitted messages
|
||||||
|
- Verify message types, data structure, and content
|
||||||
|
- Test both successful and failure scenarios
|
||||||
|
- Verify proper room routing for different message types
|
||||||
|
|
||||||
|
#### Architecture Benefits
|
||||||
|
|
||||||
|
**Decoupling**:
|
||||||
|
|
||||||
|
- Services use generic broadcast callbacks without WebSocket dependencies
|
||||||
|
- ProgressService provides abstraction layer for progress tracking
|
||||||
|
- Easy to swap WebSocket implementation or add additional broadcast targets
|
||||||
|
|
||||||
|
**Consistency**:
|
||||||
|
|
||||||
|
- All services follow same broadcast patterns
|
||||||
|
- Standardized message formats across application
|
||||||
|
- Unified progress tracking via ProgressService
|
||||||
|
|
||||||
|
**Real-time UX**:
|
||||||
|
|
||||||
|
- Instant feedback on all long-running operations
|
||||||
|
- Live queue status updates
|
||||||
|
- Progress bars update smoothly without polling
|
||||||
|
- Error notifications delivered immediately
|
||||||
|
|
||||||
|
**Scalability**:
|
||||||
|
|
||||||
|
- Room-based messaging enables targeted updates
|
||||||
|
- Multiple concurrent operations supported
|
||||||
|
- Easy to add new progress types and message formats
|
||||||
|
|
||||||
|
#### Production Considerations
|
||||||
|
|
||||||
|
**Single-Process Deployment** (Current):
|
||||||
|
|
||||||
|
- In-memory connection registry in WebSocketService
|
||||||
|
- Works perfectly for single-worker deployments
|
||||||
|
- No additional infrastructure required
|
||||||
|
|
||||||
|
**Multi-Process/Multi-Host Deployment** (Future):
|
||||||
|
|
||||||
|
- Move connection registry to Redis or similar shared store
|
||||||
|
- Implement pub/sub for cross-process message broadcasting
|
||||||
|
- Add connection persistence for recovery after restarts
|
||||||
|
- Consider using sticky sessions or connection migration
|
||||||
|
|
||||||
|
**Performance**:
|
||||||
|
|
||||||
|
- Progress updates throttled to >1% changes to reduce message volume
|
||||||
|
- Broadcast operations are fire-and-forget (non-blocking)
|
||||||
|
- Failed connections automatically cleaned up
|
||||||
|
- Message serialization cached where possible
|
||||||
|
|
||||||
|
**Monitoring**:
|
||||||
|
|
||||||
|
- Structured logging for all broadcast operations
|
||||||
|
- WebSocket status available at `/ws/status` endpoint
|
||||||
|
- Connection count and room membership tracking
|
||||||
|
- Error tracking for failed broadcasts
|
||||||
|
|||||||
@ -160,13 +160,6 @@ The tasks should be completed in the following order to ensure proper dependenci
|
|||||||
|
|
||||||
### 11. Deployment and Configuration
|
### 11. Deployment and Configuration
|
||||||
|
|
||||||
#### [] Create Docker configuration
|
|
||||||
|
|
||||||
- []Create `Dockerfile`
|
|
||||||
- []Create `docker-compose.yml`
|
|
||||||
- []Add environment configuration
|
|
||||||
- []Include volume mappings for existing web assets
|
|
||||||
|
|
||||||
#### [] Create production configuration
|
#### [] Create production configuration
|
||||||
|
|
||||||
- []Create `src/server/config/production.py`
|
- []Create `src/server/config/production.py`
|
||||||
|
|||||||
@ -113,12 +113,21 @@ class DownloadService:
|
|||||||
logger.debug("Broadcast callback registered")
|
logger.debug("Broadcast callback registered")
|
||||||
|
|
||||||
async def _broadcast_update(self, update_type: str, data: dict) -> None:
|
async def _broadcast_update(self, update_type: str, data: dict) -> None:
|
||||||
"""Broadcast update to connected WebSocket clients."""
|
"""Broadcast update to connected WebSocket clients.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
update_type: Type of update (download_progress, queue_status, etc.)
|
||||||
|
data: Update data to broadcast
|
||||||
|
"""
|
||||||
if self._broadcast_callback:
|
if self._broadcast_callback:
|
||||||
try:
|
try:
|
||||||
await self._broadcast_callback(update_type, data)
|
await self._broadcast_callback(update_type, data)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.error("Failed to broadcast update", error=str(e))
|
logger.error(
|
||||||
|
"Failed to broadcast update",
|
||||||
|
update_type=update_type,
|
||||||
|
error=str(e),
|
||||||
|
)
|
||||||
|
|
||||||
def _generate_item_id(self) -> str:
|
def _generate_item_id(self) -> str:
|
||||||
"""Generate unique identifier for download items."""
|
"""Generate unique identifier for download items."""
|
||||||
@ -238,9 +247,15 @@ class DownloadService:
|
|||||||
|
|
||||||
self._save_queue()
|
self._save_queue()
|
||||||
|
|
||||||
# Broadcast update
|
# Broadcast queue status update
|
||||||
|
queue_status = await self.get_queue_status()
|
||||||
await self._broadcast_update(
|
await self._broadcast_update(
|
||||||
"queue_updated", {"added_ids": created_ids}
|
"queue_status",
|
||||||
|
{
|
||||||
|
"action": "items_added",
|
||||||
|
"added_ids": created_ids,
|
||||||
|
"queue_status": queue_status.model_dump(mode="json"),
|
||||||
|
},
|
||||||
)
|
)
|
||||||
|
|
||||||
return created_ids
|
return created_ids
|
||||||
@ -288,8 +303,15 @@ class DownloadService:
|
|||||||
|
|
||||||
if removed_ids:
|
if removed_ids:
|
||||||
self._save_queue()
|
self._save_queue()
|
||||||
|
# Broadcast queue status update
|
||||||
|
queue_status = await self.get_queue_status()
|
||||||
await self._broadcast_update(
|
await self._broadcast_update(
|
||||||
"queue_updated", {"removed_ids": removed_ids}
|
"queue_status",
|
||||||
|
{
|
||||||
|
"action": "items_removed",
|
||||||
|
"removed_ids": removed_ids,
|
||||||
|
"queue_status": queue_status.model_dump(mode="json"),
|
||||||
|
},
|
||||||
)
|
)
|
||||||
|
|
||||||
return removed_ids
|
return removed_ids
|
||||||
@ -334,9 +356,17 @@ class DownloadService:
|
|||||||
self._pending_queue = deque(queue_list)
|
self._pending_queue = deque(queue_list)
|
||||||
|
|
||||||
self._save_queue()
|
self._save_queue()
|
||||||
|
|
||||||
|
# Broadcast queue status update
|
||||||
|
queue_status = await self.get_queue_status()
|
||||||
await self._broadcast_update(
|
await self._broadcast_update(
|
||||||
"queue_reordered",
|
"queue_status",
|
||||||
{"item_id": item_id, "position": new_position}
|
{
|
||||||
|
"action": "queue_reordered",
|
||||||
|
"item_id": item_id,
|
||||||
|
"new_position": new_position,
|
||||||
|
"queue_status": queue_status.model_dump(mode="json"),
|
||||||
|
},
|
||||||
)
|
)
|
||||||
|
|
||||||
logger.info(
|
logger.info(
|
||||||
@ -410,13 +440,31 @@ class DownloadService:
|
|||||||
"""Pause download processing."""
|
"""Pause download processing."""
|
||||||
self._is_paused = True
|
self._is_paused = True
|
||||||
logger.info("Download queue paused")
|
logger.info("Download queue paused")
|
||||||
await self._broadcast_update("queue_paused", {})
|
|
||||||
|
# Broadcast queue status update
|
||||||
|
queue_status = await self.get_queue_status()
|
||||||
|
await self._broadcast_update(
|
||||||
|
"queue_paused",
|
||||||
|
{
|
||||||
|
"is_paused": True,
|
||||||
|
"queue_status": queue_status.model_dump(mode="json"),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
async def resume_queue(self) -> None:
|
async def resume_queue(self) -> None:
|
||||||
"""Resume download processing."""
|
"""Resume download processing."""
|
||||||
self._is_paused = False
|
self._is_paused = False
|
||||||
logger.info("Download queue resumed")
|
logger.info("Download queue resumed")
|
||||||
await self._broadcast_update("queue_resumed", {})
|
|
||||||
|
# Broadcast queue status update
|
||||||
|
queue_status = await self.get_queue_status()
|
||||||
|
await self._broadcast_update(
|
||||||
|
"queue_resumed",
|
||||||
|
{
|
||||||
|
"is_paused": False,
|
||||||
|
"queue_status": queue_status.model_dump(mode="json"),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
async def clear_completed(self) -> int:
|
async def clear_completed(self) -> int:
|
||||||
"""Clear completed downloads from history.
|
"""Clear completed downloads from history.
|
||||||
@ -427,6 +475,19 @@ class DownloadService:
|
|||||||
count = len(self._completed_items)
|
count = len(self._completed_items)
|
||||||
self._completed_items.clear()
|
self._completed_items.clear()
|
||||||
logger.info("Cleared completed items", count=count)
|
logger.info("Cleared completed items", count=count)
|
||||||
|
|
||||||
|
# Broadcast queue status update
|
||||||
|
if count > 0:
|
||||||
|
queue_status = await self.get_queue_status()
|
||||||
|
await self._broadcast_update(
|
||||||
|
"queue_status",
|
||||||
|
{
|
||||||
|
"action": "completed_cleared",
|
||||||
|
"cleared_count": count,
|
||||||
|
"queue_status": queue_status.model_dump(mode="json"),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
return count
|
return count
|
||||||
|
|
||||||
async def retry_failed(
|
async def retry_failed(
|
||||||
@ -471,8 +532,15 @@ class DownloadService:
|
|||||||
|
|
||||||
if retried_ids:
|
if retried_ids:
|
||||||
self._save_queue()
|
self._save_queue()
|
||||||
|
# Broadcast queue status update
|
||||||
|
queue_status = await self.get_queue_status()
|
||||||
await self._broadcast_update(
|
await self._broadcast_update(
|
||||||
"items_retried", {"item_ids": retried_ids}
|
"queue_status",
|
||||||
|
{
|
||||||
|
"action": "items_retried",
|
||||||
|
"retried_ids": retried_ids,
|
||||||
|
"queue_status": queue_status.model_dump(mode="json"),
|
||||||
|
},
|
||||||
)
|
)
|
||||||
|
|
||||||
return retried_ids
|
return retried_ids
|
||||||
@ -530,7 +598,11 @@ class DownloadService:
|
|||||||
self._broadcast_update(
|
self._broadcast_update(
|
||||||
"download_progress",
|
"download_progress",
|
||||||
{
|
{
|
||||||
|
"download_id": item.id,
|
||||||
"item_id": item.id,
|
"item_id": item.id,
|
||||||
|
"serie_name": item.serie_name,
|
||||||
|
"season": item.episode.season,
|
||||||
|
"episode": item.episode.episode,
|
||||||
"progress": item.progress.model_dump(mode="json"),
|
"progress": item.progress.model_dump(mode="json"),
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
@ -615,7 +687,17 @@ class DownloadService:
|
|||||||
)
|
)
|
||||||
|
|
||||||
await self._broadcast_update(
|
await self._broadcast_update(
|
||||||
"download_completed", {"item_id": item.id}
|
"download_complete",
|
||||||
|
{
|
||||||
|
"download_id": item.id,
|
||||||
|
"item_id": item.id,
|
||||||
|
"serie_name": item.serie_name,
|
||||||
|
"season": item.episode.season,
|
||||||
|
"episode": item.episode.episode,
|
||||||
|
"downloaded_mb": item.progress.downloaded_mb
|
||||||
|
if item.progress
|
||||||
|
else 0,
|
||||||
|
},
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
raise AnimeServiceError("Download returned False")
|
raise AnimeServiceError("Download returned False")
|
||||||
@ -643,7 +725,15 @@ class DownloadService:
|
|||||||
|
|
||||||
await self._broadcast_update(
|
await self._broadcast_update(
|
||||||
"download_failed",
|
"download_failed",
|
||||||
{"item_id": item.id, "error": item.error},
|
{
|
||||||
|
"download_id": item.id,
|
||||||
|
"item_id": item.id,
|
||||||
|
"serie_name": item.serie_name,
|
||||||
|
"season": item.episode.season,
|
||||||
|
"episode": item.episode.episode,
|
||||||
|
"error": item.error,
|
||||||
|
"retry_count": item.retry_count,
|
||||||
|
},
|
||||||
)
|
)
|
||||||
|
|
||||||
finally:
|
finally:
|
||||||
@ -698,6 +788,16 @@ class DownloadService:
|
|||||||
asyncio.create_task(self._queue_processor())
|
asyncio.create_task(self._queue_processor())
|
||||||
|
|
||||||
logger.info("Download queue service started")
|
logger.info("Download queue service started")
|
||||||
|
|
||||||
|
# Broadcast queue started event
|
||||||
|
queue_status = await self.get_queue_status()
|
||||||
|
await self._broadcast_update(
|
||||||
|
"queue_started",
|
||||||
|
{
|
||||||
|
"is_running": True,
|
||||||
|
"queue_status": queue_status.model_dump(mode="json"),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
async def stop(self) -> None:
|
async def stop(self) -> None:
|
||||||
"""Stop the download queue processor."""
|
"""Stop the download queue processor."""
|
||||||
@ -726,6 +826,16 @@ class DownloadService:
|
|||||||
self._executor.shutdown(wait=True)
|
self._executor.shutdown(wait=True)
|
||||||
|
|
||||||
logger.info("Download queue service stopped")
|
logger.info("Download queue service stopped")
|
||||||
|
|
||||||
|
# Broadcast queue stopped event
|
||||||
|
queue_status = await self.get_queue_status()
|
||||||
|
await self._broadcast_update(
|
||||||
|
"queue_stopped",
|
||||||
|
{
|
||||||
|
"is_running": False,
|
||||||
|
"queue_status": queue_status.model_dump(mode="json"),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
# Singleton instance
|
# Singleton instance
|
||||||
|
|||||||
470
tests/integration/test_websocket_integration.py
Normal file
470
tests/integration/test_websocket_integration.py
Normal file
@ -0,0 +1,470 @@
|
|||||||
|
"""Integration tests for WebSocket integration with core services.
|
||||||
|
|
||||||
|
This module tests the integration between WebSocket broadcasting and
|
||||||
|
core services (DownloadService, AnimeService, ProgressService) to ensure
|
||||||
|
real-time updates are properly broadcasted to connected clients.
|
||||||
|
"""
|
||||||
|
import asyncio
|
||||||
|
from typing import Any, Dict, List
|
||||||
|
from unittest.mock import Mock, patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from src.server.models.download import (
|
||||||
|
DownloadPriority,
|
||||||
|
DownloadStatus,
|
||||||
|
EpisodeIdentifier,
|
||||||
|
)
|
||||||
|
from src.server.services.anime_service import AnimeService
|
||||||
|
from src.server.services.download_service import DownloadService
|
||||||
|
from src.server.services.progress_service import ProgressService, ProgressType
|
||||||
|
from src.server.services.websocket_service import WebSocketService
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def mock_series_app():
|
||||||
|
"""Mock SeriesApp for testing."""
|
||||||
|
app = Mock()
|
||||||
|
app.series_list = []
|
||||||
|
app.search = Mock(return_value=[])
|
||||||
|
app.ReScan = Mock()
|
||||||
|
app.download = Mock(return_value=True)
|
||||||
|
return app
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def progress_service():
|
||||||
|
"""Create a ProgressService instance for testing."""
|
||||||
|
return ProgressService()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def websocket_service():
|
||||||
|
"""Create a WebSocketService instance for testing."""
|
||||||
|
return WebSocketService()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
async def anime_service(mock_series_app, progress_service):
|
||||||
|
"""Create an AnimeService with mocked dependencies."""
|
||||||
|
with patch("src.server.services.anime_service.SeriesApp", return_value=mock_series_app):
|
||||||
|
service = AnimeService(
|
||||||
|
directory="/test/anime",
|
||||||
|
progress_service=progress_service,
|
||||||
|
)
|
||||||
|
yield service
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
async def download_service(anime_service, progress_service):
|
||||||
|
"""Create a DownloadService with dependencies."""
|
||||||
|
service = DownloadService(
|
||||||
|
anime_service=anime_service,
|
||||||
|
max_concurrent_downloads=2,
|
||||||
|
progress_service=progress_service,
|
||||||
|
persistence_path="/tmp/test_queue.json",
|
||||||
|
)
|
||||||
|
yield service
|
||||||
|
await service.stop()
|
||||||
|
|
||||||
|
|
||||||
|
class TestWebSocketDownloadIntegration:
|
||||||
|
"""Test WebSocket integration with DownloadService."""
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_download_progress_broadcast(
|
||||||
|
self, download_service, websocket_service
|
||||||
|
):
|
||||||
|
"""Test that download progress updates are broadcasted."""
|
||||||
|
broadcasts: List[Dict[str, Any]] = []
|
||||||
|
|
||||||
|
async def mock_broadcast(update_type: str, data: dict):
|
||||||
|
"""Capture broadcast calls."""
|
||||||
|
broadcasts.append({"type": update_type, "data": data})
|
||||||
|
|
||||||
|
download_service.set_broadcast_callback(mock_broadcast)
|
||||||
|
|
||||||
|
# Add item to queue
|
||||||
|
item_ids = await download_service.add_to_queue(
|
||||||
|
serie_id="test_serie",
|
||||||
|
serie_name="Test Anime",
|
||||||
|
episodes=[EpisodeIdentifier(season=1, episode=1)],
|
||||||
|
priority=DownloadPriority.HIGH,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert len(item_ids) == 1
|
||||||
|
assert len(broadcasts) == 1
|
||||||
|
assert broadcasts[0]["type"] == "queue_status"
|
||||||
|
assert broadcasts[0]["data"]["action"] == "items_added"
|
||||||
|
assert item_ids[0] in broadcasts[0]["data"]["added_ids"]
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_queue_operations_broadcast(
|
||||||
|
self, download_service
|
||||||
|
):
|
||||||
|
"""Test that queue operations broadcast status updates."""
|
||||||
|
broadcasts: List[Dict[str, Any]] = []
|
||||||
|
|
||||||
|
async def mock_broadcast(update_type: str, data: dict):
|
||||||
|
broadcasts.append({"type": update_type, "data": data})
|
||||||
|
|
||||||
|
download_service.set_broadcast_callback(mock_broadcast)
|
||||||
|
|
||||||
|
# Add items
|
||||||
|
item_ids = await download_service.add_to_queue(
|
||||||
|
serie_id="test",
|
||||||
|
serie_name="Test",
|
||||||
|
episodes=[EpisodeIdentifier(season=1, episode=i) for i in range(1, 4)],
|
||||||
|
priority=DownloadPriority.NORMAL,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Remove items
|
||||||
|
removed = await download_service.remove_from_queue([item_ids[0]])
|
||||||
|
assert len(removed) == 1
|
||||||
|
|
||||||
|
# Check broadcasts
|
||||||
|
add_broadcast = next(
|
||||||
|
b for b in broadcasts
|
||||||
|
if b["data"].get("action") == "items_added"
|
||||||
|
)
|
||||||
|
remove_broadcast = next(
|
||||||
|
b for b in broadcasts
|
||||||
|
if b["data"].get("action") == "items_removed"
|
||||||
|
)
|
||||||
|
|
||||||
|
assert add_broadcast["type"] == "queue_status"
|
||||||
|
assert len(add_broadcast["data"]["added_ids"]) == 3
|
||||||
|
|
||||||
|
assert remove_broadcast["type"] == "queue_status"
|
||||||
|
assert item_ids[0] in remove_broadcast["data"]["removed_ids"]
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_queue_start_stop_broadcast(
|
||||||
|
self, download_service
|
||||||
|
):
|
||||||
|
"""Test that start/stop operations broadcast updates."""
|
||||||
|
broadcasts: List[Dict[str, Any]] = []
|
||||||
|
|
||||||
|
async def mock_broadcast(update_type: str, data: dict):
|
||||||
|
broadcasts.append({"type": update_type, "data": data})
|
||||||
|
|
||||||
|
download_service.set_broadcast_callback(mock_broadcast)
|
||||||
|
|
||||||
|
# Start queue
|
||||||
|
await download_service.start()
|
||||||
|
await asyncio.sleep(0.1)
|
||||||
|
|
||||||
|
# Stop queue
|
||||||
|
await download_service.stop()
|
||||||
|
|
||||||
|
# Find start/stop broadcasts
|
||||||
|
start_broadcast = next(
|
||||||
|
(b for b in broadcasts if b["type"] == "queue_started"),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
stop_broadcast = next(
|
||||||
|
(b for b in broadcasts if b["type"] == "queue_stopped"),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert start_broadcast is not None
|
||||||
|
assert start_broadcast["data"]["is_running"] is True
|
||||||
|
|
||||||
|
assert stop_broadcast is not None
|
||||||
|
assert stop_broadcast["data"]["is_running"] is False
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_queue_pause_resume_broadcast(
|
||||||
|
self, download_service
|
||||||
|
):
|
||||||
|
"""Test that pause/resume operations broadcast updates."""
|
||||||
|
broadcasts: List[Dict[str, Any]] = []
|
||||||
|
|
||||||
|
async def mock_broadcast(update_type: str, data: dict):
|
||||||
|
broadcasts.append({"type": update_type, "data": data})
|
||||||
|
|
||||||
|
download_service.set_broadcast_callback(mock_broadcast)
|
||||||
|
|
||||||
|
# Pause queue
|
||||||
|
await download_service.pause_queue()
|
||||||
|
|
||||||
|
# Resume queue
|
||||||
|
await download_service.resume_queue()
|
||||||
|
|
||||||
|
# Find pause/resume broadcasts
|
||||||
|
pause_broadcast = next(
|
||||||
|
(b for b in broadcasts if b["type"] == "queue_paused"),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
resume_broadcast = next(
|
||||||
|
(b for b in broadcasts if b["type"] == "queue_resumed"),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert pause_broadcast is not None
|
||||||
|
assert pause_broadcast["data"]["is_paused"] is True
|
||||||
|
|
||||||
|
assert resume_broadcast is not None
|
||||||
|
assert resume_broadcast["data"]["is_paused"] is False
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_clear_completed_broadcast(
|
||||||
|
self, download_service
|
||||||
|
):
|
||||||
|
"""Test that clearing completed items broadcasts update."""
|
||||||
|
broadcasts: List[Dict[str, Any]] = []
|
||||||
|
|
||||||
|
async def mock_broadcast(update_type: str, data: dict):
|
||||||
|
broadcasts.append({"type": update_type, "data": data})
|
||||||
|
|
||||||
|
download_service.set_broadcast_callback(mock_broadcast)
|
||||||
|
|
||||||
|
# Manually add a completed item to test
|
||||||
|
from datetime import datetime
|
||||||
|
|
||||||
|
from src.server.models.download import DownloadItem
|
||||||
|
|
||||||
|
completed_item = DownloadItem(
|
||||||
|
id="test_completed",
|
||||||
|
serie_id="test",
|
||||||
|
serie_name="Test",
|
||||||
|
episode=EpisodeIdentifier(season=1, episode=1),
|
||||||
|
status=DownloadStatus.COMPLETED,
|
||||||
|
priority=DownloadPriority.NORMAL,
|
||||||
|
added_at=datetime.utcnow(),
|
||||||
|
)
|
||||||
|
download_service._completed_items.append(completed_item)
|
||||||
|
|
||||||
|
# Clear completed
|
||||||
|
count = await download_service.clear_completed()
|
||||||
|
|
||||||
|
assert count == 1
|
||||||
|
|
||||||
|
# Find clear broadcast
|
||||||
|
clear_broadcast = next(
|
||||||
|
(
|
||||||
|
b for b in broadcasts
|
||||||
|
if b["data"].get("action") == "completed_cleared"
|
||||||
|
),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert clear_broadcast is not None
|
||||||
|
assert clear_broadcast["data"]["cleared_count"] == 1
|
||||||
|
|
||||||
|
|
||||||
|
class TestWebSocketScanIntegration:
|
||||||
|
"""Test WebSocket integration with AnimeService scan operations."""
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_scan_progress_broadcast(
|
||||||
|
self, anime_service, progress_service, mock_series_app
|
||||||
|
):
|
||||||
|
"""Test that scan progress updates are broadcasted."""
|
||||||
|
broadcasts: List[Dict[str, Any]] = []
|
||||||
|
|
||||||
|
async def mock_broadcast(message_type: str, data: dict, room: str):
|
||||||
|
"""Capture broadcast calls."""
|
||||||
|
broadcasts.append({
|
||||||
|
"type": message_type,
|
||||||
|
"data": data,
|
||||||
|
"room": room,
|
||||||
|
})
|
||||||
|
|
||||||
|
progress_service.set_broadcast_callback(mock_broadcast)
|
||||||
|
|
||||||
|
# Mock scan callback to simulate progress
|
||||||
|
def mock_scan_callback(callback):
|
||||||
|
"""Simulate scan progress."""
|
||||||
|
if callback:
|
||||||
|
callback({"current": 5, "total": 10, "message": "Scanning..."})
|
||||||
|
callback({"current": 10, "total": 10, "message": "Complete"})
|
||||||
|
|
||||||
|
mock_series_app.ReScan = mock_scan_callback
|
||||||
|
|
||||||
|
# Run scan
|
||||||
|
await anime_service.rescan()
|
||||||
|
|
||||||
|
# Verify broadcasts were made
|
||||||
|
assert len(broadcasts) >= 2 # At least start and complete
|
||||||
|
|
||||||
|
# Check for scan progress broadcasts
|
||||||
|
scan_broadcasts = [
|
||||||
|
b for b in broadcasts if b["room"] == "scan_progress"
|
||||||
|
]
|
||||||
|
assert len(scan_broadcasts) >= 2
|
||||||
|
|
||||||
|
# Verify start broadcast
|
||||||
|
start_broadcast = scan_broadcasts[0]
|
||||||
|
assert start_broadcast["data"]["status"] == "started"
|
||||||
|
assert start_broadcast["data"]["type"] == ProgressType.SCAN.value
|
||||||
|
|
||||||
|
# Verify completion broadcast
|
||||||
|
complete_broadcast = scan_broadcasts[-1]
|
||||||
|
assert complete_broadcast["data"]["status"] == "completed"
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_scan_failure_broadcast(
|
||||||
|
self, anime_service, progress_service, mock_series_app
|
||||||
|
):
|
||||||
|
"""Test that scan failures are broadcasted."""
|
||||||
|
broadcasts: List[Dict[str, Any]] = []
|
||||||
|
|
||||||
|
async def mock_broadcast(message_type: str, data: dict, room: str):
|
||||||
|
broadcasts.append({
|
||||||
|
"type": message_type,
|
||||||
|
"data": data,
|
||||||
|
"room": room,
|
||||||
|
})
|
||||||
|
|
||||||
|
progress_service.set_broadcast_callback(mock_broadcast)
|
||||||
|
|
||||||
|
# Mock scan to raise error
|
||||||
|
def mock_scan_error(callback):
|
||||||
|
raise RuntimeError("Scan failed")
|
||||||
|
|
||||||
|
mock_series_app.ReScan = mock_scan_error
|
||||||
|
|
||||||
|
# Run scan (should fail)
|
||||||
|
with pytest.raises(Exception):
|
||||||
|
await anime_service.rescan()
|
||||||
|
|
||||||
|
# Verify failure broadcast
|
||||||
|
scan_broadcasts = [
|
||||||
|
b for b in broadcasts if b["room"] == "scan_progress"
|
||||||
|
]
|
||||||
|
assert len(scan_broadcasts) >= 2 # Start and fail
|
||||||
|
|
||||||
|
# Verify failure broadcast
|
||||||
|
fail_broadcast = scan_broadcasts[-1]
|
||||||
|
assert fail_broadcast["data"]["status"] == "failed"
|
||||||
|
# Verify error message or failed status
|
||||||
|
is_error = "error" in fail_broadcast["data"]["message"].lower()
|
||||||
|
is_failed = fail_broadcast["data"]["status"] == "failed"
|
||||||
|
assert is_error or is_failed
|
||||||
|
|
||||||
|
|
||||||
|
class TestWebSocketProgressIntegration:
|
||||||
|
"""Test WebSocket integration with ProgressService."""
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_progress_lifecycle_broadcast(
|
||||||
|
self, progress_service
|
||||||
|
):
|
||||||
|
"""Test that progress lifecycle events are broadcasted."""
|
||||||
|
broadcasts: List[Dict[str, Any]] = []
|
||||||
|
|
||||||
|
async def mock_broadcast(message_type: str, data: dict, room: str):
|
||||||
|
broadcasts.append({
|
||||||
|
"type": message_type,
|
||||||
|
"data": data,
|
||||||
|
"room": room,
|
||||||
|
})
|
||||||
|
|
||||||
|
progress_service.set_broadcast_callback(mock_broadcast)
|
||||||
|
|
||||||
|
# Start progress
|
||||||
|
await progress_service.start_progress(
|
||||||
|
progress_id="test_progress",
|
||||||
|
progress_type=ProgressType.DOWNLOAD,
|
||||||
|
title="Test Download",
|
||||||
|
total=100,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Update progress
|
||||||
|
await progress_service.update_progress(
|
||||||
|
progress_id="test_progress",
|
||||||
|
current=50,
|
||||||
|
force_broadcast=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Complete progress
|
||||||
|
await progress_service.complete_progress(
|
||||||
|
progress_id="test_progress",
|
||||||
|
message="Download complete",
|
||||||
|
)
|
||||||
|
|
||||||
|
# Verify broadcasts
|
||||||
|
assert len(broadcasts) == 3
|
||||||
|
|
||||||
|
start_broadcast = broadcasts[0]
|
||||||
|
assert start_broadcast["data"]["status"] == "started"
|
||||||
|
assert start_broadcast["room"] == "download_progress"
|
||||||
|
|
||||||
|
update_broadcast = broadcasts[1]
|
||||||
|
assert update_broadcast["data"]["status"] == "in_progress"
|
||||||
|
assert update_broadcast["data"]["percent"] == 50.0
|
||||||
|
|
||||||
|
complete_broadcast = broadcasts[2]
|
||||||
|
assert complete_broadcast["data"]["status"] == "completed"
|
||||||
|
assert complete_broadcast["data"]["percent"] == 100.0
|
||||||
|
|
||||||
|
|
||||||
|
class TestWebSocketEndToEnd:
|
||||||
|
"""End-to-end integration tests with all services."""
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_complete_download_flow_with_broadcasts(
|
||||||
|
self, download_service, anime_service, progress_service
|
||||||
|
):
|
||||||
|
"""Test complete download flow with all broadcasts."""
|
||||||
|
all_broadcasts: List[Dict[str, Any]] = []
|
||||||
|
|
||||||
|
async def capture_download_broadcast(update_type: str, data: dict):
|
||||||
|
all_broadcasts.append({
|
||||||
|
"source": "download",
|
||||||
|
"type": update_type,
|
||||||
|
"data": data,
|
||||||
|
})
|
||||||
|
|
||||||
|
async def capture_progress_broadcast(
|
||||||
|
message_type: str, data: dict, room: str
|
||||||
|
):
|
||||||
|
all_broadcasts.append({
|
||||||
|
"source": "progress",
|
||||||
|
"type": message_type,
|
||||||
|
"data": data,
|
||||||
|
"room": room,
|
||||||
|
})
|
||||||
|
|
||||||
|
download_service.set_broadcast_callback(capture_download_broadcast)
|
||||||
|
progress_service.set_broadcast_callback(capture_progress_broadcast)
|
||||||
|
|
||||||
|
# Add items to queue
|
||||||
|
item_ids = await download_service.add_to_queue(
|
||||||
|
serie_id="test",
|
||||||
|
serie_name="Test Anime",
|
||||||
|
episodes=[EpisodeIdentifier(season=1, episode=1)],
|
||||||
|
priority=DownloadPriority.HIGH,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Start queue
|
||||||
|
await download_service.start()
|
||||||
|
await asyncio.sleep(0.1)
|
||||||
|
|
||||||
|
# Pause queue
|
||||||
|
await download_service.pause_queue()
|
||||||
|
|
||||||
|
# Resume queue
|
||||||
|
await download_service.resume_queue()
|
||||||
|
|
||||||
|
# Stop queue
|
||||||
|
await download_service.stop()
|
||||||
|
|
||||||
|
# Verify we received broadcasts from both services
|
||||||
|
download_broadcasts = [
|
||||||
|
b for b in all_broadcasts if b["source"] == "download"
|
||||||
|
]
|
||||||
|
|
||||||
|
assert len(download_broadcasts) >= 4 # add, start, pause, resume, stop
|
||||||
|
assert len(item_ids) == 1
|
||||||
|
|
||||||
|
# Verify queue status broadcasts
|
||||||
|
queue_status_broadcasts = [
|
||||||
|
b for b in download_broadcasts if b["type"] == "queue_status"
|
||||||
|
]
|
||||||
|
assert len(queue_status_broadcasts) >= 1
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
pytest.main([__file__, "-v"])
|
||||||
Loading…
x
Reference in New Issue
Block a user