Implement initial scan tracking for one-time setup
- Add SystemSettings model to track setup completion status - Create SystemSettingsService for managing setup flags - Modify fastapi_app startup to check and set initial_scan_completed flag - Anime folder scanning now only runs on first startup - Update DATABASE.md with new system_settings table documentation - Add unit test for SystemSettingsService functionality This ensures expensive one-time operations like scanning the entire anime directory only occur during initial setup, not on every application restart.
This commit is contained in:
@@ -33,31 +33,55 @@ Source: [src/server/database/connection.py](../src/server/database/connection.py
|
||||
## 2. Entity Relationship Diagram
|
||||
|
||||
```
|
||||
+-------------------+ +-------------------+ +------------------------+
|
||||
| anime_series | | episodes | | download_queue_item |
|
||||
+-------------------+ +-------------------+ +------------------------+
|
||||
| id (PK) |<--+ | id (PK) | +-->| id (PK, VARCHAR) |
|
||||
| key (UNIQUE) | | | series_id (FK)----+---+ | series_id (FK)---------+
|
||||
| name | +---| | | status |
|
||||
| site | | season | | priority |
|
||||
| folder | | episode_number | | season |
|
||||
| created_at | | title | | episode |
|
||||
| updated_at | | file_path | | progress_percent |
|
||||
+-------------------+ | is_downloaded | | error_message |
|
||||
| created_at | | retry_count |
|
||||
| updated_at | | added_at |
|
||||
+-------------------+ | started_at |
|
||||
| completed_at |
|
||||
| created_at |
|
||||
| updated_at |
|
||||
+------------------------+
|
||||
+---------------------+ +-------------------+ +-------------------+ +------------------------+
|
||||
| system_settings | | anime_series | | episodes | | download_queue_item |
|
||||
+---------------------+ +-------------------+ +-------------------+ +------------------------+
|
||||
| id (PK) | | id (PK) |<--+ | id (PK) | +-->| id (PK, VARCHAR) |
|
||||
| initial_scan_... | | key (UNIQUE) | | | series_id (FK)----+---+ | series_id (FK)---------+
|
||||
| initial_nfo_scan... | | name | +---| | | status |
|
||||
| initial_media_... | | site | | season | | priority |
|
||||
| last_scan_timestamp | | folder | | episode_number | | season |
|
||||
| created_at | | created_at | | title | | episode |
|
||||
| updated_at | | updated_at | | file_path | | progress_percent |
|
||||
+---------------------+ +-------------------+ | is_downloaded | | error_message |
|
||||
| created_at | | retry_count |
|
||||
| updated_at | | added_at |
|
||||
+-------------------+ | started_at |
|
||||
| completed_at |
|
||||
| created_at |
|
||||
| updated_at |
|
||||
+------------------------+
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Table Schemas
|
||||
|
||||
### 3.1 anime_series
|
||||
### 3.1 system_settings
|
||||
|
||||
Stores application-wide system settings and initialization state.
|
||||
|
||||
| Column | Type | Constraints | Description |
|
||||
| ------------------------------ | -------- | -------------------------- | --------------------------------------------- |
|
||||
| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Internal database ID (only one row) |
|
||||
| `initial_scan_completed` | BOOLEAN | NOT NULL, DEFAULT FALSE | Whether initial anime folder scan is complete |
|
||||
| `initial_nfo_scan_completed` | BOOLEAN | NOT NULL, DEFAULT FALSE | Whether initial NFO scan is complete |
|
||||
| `initial_media_scan_completed` | BOOLEAN | NOT NULL, DEFAULT FALSE | Whether initial media scan is complete |
|
||||
| `last_scan_timestamp` | DATETIME | NULLABLE | Timestamp of last completed scan |
|
||||
| `created_at` | DATETIME | NOT NULL, DEFAULT NOW | Record creation timestamp |
|
||||
| `updated_at` | DATETIME | NOT NULL, ON UPDATE NOW | Last update timestamp |
|
||||
|
||||
**Purpose:**
|
||||
|
||||
This table tracks the initialization status of the application to ensure that expensive one-time setup operations (like scanning the entire anime directory) only run on the first startup, not on every restart.
|
||||
|
||||
- Only one row exists in this table
|
||||
- The `initial_scan_completed` flag prevents redundant full directory scans on each startup
|
||||
- The NFO and media scan flags similarly track completion of those setup tasks
|
||||
|
||||
Source: [src/server/database/models.py](../src/server/database/models.py), [src/server/database/system_settings_service.py](../src/server/database/system_settings_service.py)
|
||||
|
||||
### 3.2 anime_series
|
||||
|
||||
Stores anime series metadata.
|
||||
|
||||
@@ -79,7 +103,7 @@ Stores anime series metadata.
|
||||
|
||||
Source: [src/server/database/models.py](../src/server/database/models.py#L23-L87)
|
||||
|
||||
### 3.2 episodes
|
||||
### 3.3 episodes
|
||||
|
||||
Stores **missing episodes** that need to be downloaded. Episodes are automatically managed during scans:
|
||||
|
||||
@@ -105,7 +129,7 @@ Stores **missing episodes** that need to be downloaded. Episodes are automatical
|
||||
|
||||
Source: [src/server/database/models.py](../src/server/database/models.py#L122-L181)
|
||||
|
||||
### 3.3 download_queue_item
|
||||
### 3.4 download_queue_item
|
||||
|
||||
Stores download queue items with status tracking.
|
||||
|
||||
@@ -143,6 +167,7 @@ Source: [src/server/database/models.py](../src/server/database/models.py#L200-L3
|
||||
|
||||
| Table | Index Name | Columns | Purpose |
|
||||
| --------------------- | ----------------------- | ----------- | --------------------------------- |
|
||||
| `system_settings` | N/A (single row) | N/A | Only one row, no indexes needed |
|
||||
| `anime_series` | `ix_anime_series_key` | `key` | Fast lookup by primary identifier |
|
||||
| `anime_series` | `ix_anime_series_name` | `name` | Search by name |
|
||||
| `episodes` | `ix_episodes_series_id` | `series_id` | Join with series |
|
||||
|
||||
@@ -119,23 +119,18 @@ For each task completed:
|
||||
|
||||
## TODO List:
|
||||
|
||||
✅ **FIXED:** Anime list endpoint now correctly returns anime data after server startup.
|
||||
Make sure you do not produce doublicate code. the function below is mostly implemented.
|
||||
make sure you maintain the function on one location
|
||||
|
||||
**Root Cause:** The anime list was empty because:
|
||||
1. The `SeriesApp.list` was initialized with `skip_load=True` to avoid loading from filesystem during initialization
|
||||
2. Series data is synced from filesystem data files to the database during server startup
|
||||
3. Series are then loaded from the database into `SeriesApp` memory via `anime_service._load_series_from_db()`
|
||||
4. The server needed to be restarted to complete this initialization process
|
||||
1. scanning anime from folder
|
||||
make sure that scanning anime from folder only runs on setup and not on each start
|
||||
|
||||
**Solution:** The existing startup process in [fastapi_app.py](../src/server/fastapi_app.py) correctly:
|
||||
- Syncs series from data files to database via `sync_series_from_data_files()`
|
||||
- Loads series from database into memory via `anime_service._load_series_from_db()`
|
||||
2. Nfo scan
|
||||
make sure nfo scan runs only on setup and not on each start
|
||||
|
||||
The issue was resolved by restarting the server to allow the full initialization process to complete.
|
||||
3. nfo data
|
||||
during nfo scan read tmdb id from nfo file and write it in db.
|
||||
during nfo scan read tvdb id from nfo file and write it in db.
|
||||
|
||||
**Verified:** GET `/api/anime` now returns 192 anime series with complete metadata including:
|
||||
- Unique key (primary identifier)
|
||||
- Name and folder
|
||||
- Missing episodes tracking
|
||||
- NFO metadata status
|
||||
- TMDB/TVDB IDs when available
|
||||
4. Media scan
|
||||
make sure media scan runs only on setup and not on each start
|
||||
|
||||
Reference in New Issue
Block a user