fix(episodes): prevent duplicate-row accumulation at schema + write sites
Followup to commitc84f968(read-boundary dedup) and commitf75d591(cleanup CLI). The read boundary filters duplicates out of the in-memory episodeDict and the CLI cleans up historical duplicates in the DB, but the underlying pathology — duplicate rows being created in the first place — was still active on every rescan. Two layered prevention fixes: 1. Schema-level guard: add UNIQUE(series_id, season, episode_number) to the episodes table. SQLite's CREATE UNIQUE INDEX requires no existing duplicates, but the cleanup CLI fromf75d591has already been run (or is a one-shot prerequisite for users on older DBs). Future duplicate rows are rejected at the DB layer. 2. Write-site guard: SerieScanner.scan_single_series used to `extend` the in-memory episodeDict on every rescan of a series already in keyDict — across N rescans, the same missing list was appended N times, growing the dict with duplicates that then flowed through _update_series_in_db into the episodes table. The fix replaces the cache with the latest scan result instead of extending, and dedupes within a single call as defense in depth against a buggy upstream loader. Defensive dedup is layered three deep: - schema constraint (this commit, primary) - scan_single_series replace-not-extend (this commit, secondary) - episodeDict property read-boundary dedup (commitc84f968, tertiary — covers legacy DBs that predate the constraint) Tests: - Updated test_serie_scanner.test_scan_single_series_existing_entry to assert the new replace-not-merge behavior (the old assertion encoded the buggy extend behavior). - New test_serie_scanner_scan_dedup.py covers the regression directly: two rescans of the same series with the same missing list must yield a canonical dict, not an accumulated one. - test_database_models and test_clean_duplicate_episodes_cli now use a legacy_engine fixture that drops the UNIQUE constraint, so the duplicate-row scenarios they exercise (the read-boundary dedup and the cleanup tool, both meant to defend against pre-migration state) can still be tested under the new schema. Verified manually: clean_duplicate_episodes --apply on the user's backup DB still removes all 633 duplicate rows under the new schema (the CLI doesn't depend on the UNIQUE constraint — it operates on whatever rows already exist).
This commit is contained in:
@@ -194,15 +194,23 @@ class TestSerieScannerSingleSeries:
|
||||
def test_scan_single_series_existing_entry(
|
||||
self, temp_directory, mock_loader, sample_serie
|
||||
):
|
||||
"""Test scan_single_series updates existing entry in keyDict."""
|
||||
"""Test scan_single_series replaces the existing entry's
|
||||
``episodeDict`` with the new scan's missing-episode list.
|
||||
|
||||
Note: the previous implementation ``extend````ed the
|
||||
existing list with the new one, which accumulated
|
||||
duplicates across rescans. The fix is to replace, not
|
||||
extend — see ``test_serie_scanner_scan_dedup.py`` for the
|
||||
regression tests for that specific bug.
|
||||
"""
|
||||
scanner = SerieScanner(temp_directory, mock_loader)
|
||||
|
||||
|
||||
# Pre-populate keyDict
|
||||
scanner.keyDict[sample_serie.key] = sample_serie
|
||||
# Use deepcopy because episodeDict is modified in-place
|
||||
# Use deepcopy because episodeDict is mutated by the scanner.
|
||||
import copy
|
||||
old_episode_dict = copy.deepcopy(sample_serie.episodeDict)
|
||||
|
||||
|
||||
with patch.object(
|
||||
scanner,
|
||||
'_SerieScanner__get_missing_episodes_and_season',
|
||||
@@ -212,11 +220,16 @@ class TestSerieScannerSingleSeries:
|
||||
key=sample_serie.key,
|
||||
folder=sample_serie.folder
|
||||
)
|
||||
|
||||
# Verify existing entry was updated - episodeDict is merged (not replaced)
|
||||
# Old episodes [2, 3, 4] + new episodes [10, 11, 12] = merged result
|
||||
assert scanner.keyDict[sample_serie.key].episodeDict != old_episode_dict
|
||||
assert scanner.keyDict[sample_serie.key].episodeDict == {1: [2, 3, 4, 10, 11, 12]}
|
||||
|
||||
# The cached episodeDict is REPLACED with the latest
|
||||
# scan's missing-episode list — not merged. Old entries
|
||||
# ([2, 3, 4]) are dropped because the latest scan
|
||||
# reports only [10, 11, 12] as still missing.
|
||||
new_episode_dict = scanner.keyDict[
|
||||
sample_serie.key
|
||||
].episodeDict
|
||||
assert new_episode_dict != old_episode_dict
|
||||
assert new_episode_dict == {1: [10, 11, 12]}
|
||||
|
||||
def test_scan_single_series_empty_key_raises_error(
|
||||
self, temp_directory, mock_loader
|
||||
|
||||
Reference in New Issue
Block a user