fix: process image loading batch sequentially to avoid concurrent AsyncSession use
SQLAlchemy async sessions are not safe for concurrent operations. load_series_images_batch was using asyncio.gather to run multiple load_series_images calls concurrently, all sharing the same db session. This caused 'session is provisioning a new connection; concurrent operations are not permitted' errors for every series in the batch. Fix by processing each batch sequentially instead of concurrently. BATCH_SIZE=10 still paces TMDB requests as intended.
This commit is contained in:
@@ -162,17 +162,18 @@ class ImageLoadingService:
|
|||||||
for i in range(0, len(series_list), self.BATCH_SIZE):
|
for i in range(0, len(series_list), self.BATCH_SIZE):
|
||||||
batch = series_list[i : i + self.BATCH_SIZE]
|
batch = series_list[i : i + self.BATCH_SIZE]
|
||||||
|
|
||||||
tasks = [
|
# Process each series sequentially to avoid concurrent use of the
|
||||||
self.load_series_images(
|
# same AsyncSession (SQLAlchemy async sessions are not thread-safe
|
||||||
|
# for concurrent operations). BATCH_SIZE still paces TMDB requests.
|
||||||
|
results: List[Dict[str, Any] | Exception] = []
|
||||||
|
for series in batch:
|
||||||
|
result = await self.load_series_images(
|
||||||
key=series["key"],
|
key=series["key"],
|
||||||
folder=series["folder"],
|
folder=series["folder"],
|
||||||
anime_directory=anime_directory,
|
anime_directory=anime_directory,
|
||||||
db=db,
|
db=db,
|
||||||
)
|
)
|
||||||
for series in batch
|
results.append(result)
|
||||||
]
|
|
||||||
|
|
||||||
results = await asyncio.gather(*tasks, return_exceptions=True)
|
|
||||||
|
|
||||||
for series, result in zip(batch, results):
|
for series, result in zip(batch, results):
|
||||||
if isinstance(result, Exception):
|
if isinstance(result, Exception):
|
||||||
|
|||||||
Reference in New Issue
Block a user