Complete download queue SQLite migration: documentation and cleanup

- Updated infrastructure.md with queue database schema and storage details
- Updated instructions.md to mark migration task as completed
- No deprecated JSON code remains in codebase
This commit is contained in:
Lukas 2025-12-02 16:08:37 +01:00
parent b0f3b643c7
commit 3b516c0e24
3 changed files with 71 additions and 55 deletions

View File

@ -164,20 +164,68 @@ All series-related WebSocket events include `key` as the primary identifier in t
- `AnimeSeriesService.get_by_id(id)` - Internal lookup by database ID - `AnimeSeriesService.get_by_id(id)` - Internal lookup by database ID
- No `get_by_folder()` method exists - folder is never used for lookups - No `get_by_folder()` method exists - folder is never used for lookups
### DownloadQueueItem Fields
| Field | Type | Purpose |
| -------------- | ----------- | --------------------------------------------- |
| `id` | String (PK) | UUID for the queue item |
| `serie_id` | String | Series key for identification |
| `serie_folder` | String | Filesystem folder path |
| `serie_name` | String | Display name for the series |
| `season` | Integer | Season number |
| `episode` | Integer | Episode number |
| `status` | Enum | pending, downloading, completed, failed |
| `priority` | Enum | low, normal, high |
| `progress` | Float | Download progress percentage (0.0-100.0) |
| `error` | String | Error message if failed |
| `retry_count` | Integer | Number of retry attempts |
| `added_at` | DateTime | When item was added to queue |
| `started_at` | DateTime | When download started (nullable) |
| `completed_at` | DateTime | When download completed/failed (nullable) |
## Data Storage ## Data Storage
### Storage Architecture ### Storage Architecture
The application uses **SQLite database** as the primary storage for anime series metadata. This replaces the legacy file-based storage system. The application uses **SQLite database** as the primary storage for all application data.
| Storage Method | Status | Location | Purpose | | Data Type | Storage Location | Service |
| -------------- | --------------------- | -------------------- | ----------------------------- | | --------------- | ------------------ | ---------------------------- |
| SQLite DB | **Primary (Current)** | `data/aniworld.db` | All series metadata and state | | Anime Series | `data/aniworld.db` | `AnimeSeriesService` |
| Data Files | **Deprecated** | `{anime_dir}/*/data` | Legacy per-series JSON files | | Episodes | `data/aniworld.db` | `AnimeSeriesService` |
| Download Queue | `data/aniworld.db` | `DownloadService` via `QueueRepository` |
| User Sessions | `data/aniworld.db` | `AuthService` |
| Configuration | `data/config.json` | `ConfigService` |
### Database Storage (Recommended) ### Download Queue Storage
All new series are stored in the SQLite database via `AnimeSeriesService`: The download queue is stored in SQLite via `QueueRepository`, which wraps `DownloadQueueService`:
```python
# QueueRepository provides async operations for queue items
repository = QueueRepository(session_factory)
# Save item to database
saved_item = await repository.save_item(download_item)
# Get pending items (ordered by priority and add time)
pending = await repository.get_pending_items()
# Update item status
await repository.update_status(item_id, DownloadStatus.COMPLETED)
# Update download progress
await repository.update_progress(item_id, progress=45.5, downloaded=450, total=1000, speed=2.5)
```
**Queue Persistence Features:**
- Queue state survives server restarts
- Items in `downloading` status are reset to `pending` on startup
- Failed items within retry limit are automatically re-queued
- Completed and failed history is preserved (with limits)
- Real-time progress updates are persisted to database
### Anime Series Database Storage
```python ```python
# Add series to database # Add series to database

View File

@ -121,53 +121,24 @@ For each task completed:
--- ---
## 🗄️ Task: Migrate Download Queue from JSON to SQLite Database ## ✅ Completed: Download Queue Migration to SQLite Database
### Background The download queue has been successfully migrated from JSON file to SQLite database:
The project currently has a **hybrid data persistence approach**: | Component | Status | Description |
| ---------------------- | --------- | ------------------------------------------------ |
| QueueRepository | ✅ Done | `src/server/services/queue_repository.py` |
| DownloadService | ✅ Done | Refactored to use repository pattern |
| Application Startup | ✅ Done | Queue restored from database on startup |
| API Endpoints | ✅ Done | All endpoints work with database-backed queue |
| Tests Updated | ✅ Done | All 1104 tests passing with MockQueueRepository |
| Documentation Updated | ✅ Done | `infrastructure.md` updated with new architecture|
| Data Type | Current Storage | Target Storage | **Key Changes:**
| ------------------ | ----------------- | -------------- | - `DownloadService` no longer uses `persistence_path` parameter
| Anime Series | SQLite Database | ✅ Done | - Queue state is persisted to SQLite via `QueueRepository`
| Episodes | SQLite Database | ✅ Done | - In-memory cache maintained for performance
| User Sessions | SQLite Database | ✅ Done | - All tests use `MockQueueRepository` fixture
| **Download Queue** | SQLite Database | ✅ Done |
The database infrastructure exists in `src/server/database/`:
- `DownloadQueueItem` model in `models.py`
- `DownloadQueueService` with full CRUD operations in `service.py`
- `DownloadStatus` and `DownloadPriority` enums ✅
The `DownloadService` now uses SQLite via `QueueRepository` for queue persistence.
### ✅ Completed Tasks
- **Task 1**: Created `QueueRepository` adapter in `src/server/services/queue_repository.py`
- **Task 2**: Refactored `DownloadService` to use repository pattern
- **Task 3**: Updated dependency injection and application startup
- **Task 4**: All API endpoints work with database-backed queue
---
### Task 5: Cleanup and Documentation
**Objective:** Remove deprecated code and update documentation.
**Requirements:**
- [ ] Remove deprecated JSON persistence code from codebase
- [ ] Delete `data/download_queue.json` if it exists
- [ ] Update `infrastructure.md` with new queue architecture
- [ ] Update API documentation if needed
- [ ] Add database schema documentation for download_queue table
- [ ] Update configuration documentation (remove JSON path config)
**Acceptance Criteria:**
- No dead code remains
- Documentation accurately reflects new architecture
--- ---

View File

@ -19,10 +19,7 @@ from src.server.models.download import (
EpisodeIdentifier, EpisodeIdentifier,
) )
from src.server.services.anime_service import AnimeService from src.server.services.anime_service import AnimeService
from src.server.services.download_service import ( from src.server.services.download_service import DownloadService, DownloadServiceError
DownloadService,
DownloadServiceError,
)
class MockQueueRepository: class MockQueueRepository: