Add scheduled cleanup for rate limiter (#32)
Implement periodic cleanup of expired rate-limiter entries to prevent unbounded memory growth during long runtimes. Changes: - Create rate_limiter_cleanup task that calls cleanup_expired() every 30 minutes - Register the task in the startup DAG alongside other background jobs - Update rate_limiter module documentation with operational notes about the cleanup lifecycle and memory management strategy The cleanup is conservative and only removes IPs with no recent attempts (all timestamps outside the rate-limit window), so active IPs are preserved. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
@@ -11,6 +11,23 @@ attacks to a single worker.
|
||||
The penalty strategy for failed login attempts is also managed here:
|
||||
record_failure() records a failure timestamp and returns the penalty delay
|
||||
to apply, enabling progressive back-off without exhausting request capacity.
|
||||
|
||||
Operational Notes
|
||||
-----------------
|
||||
|
||||
**Cleanup Lifecycle**: The rate limiter state (_attempts, _failures, _lock_counts)
|
||||
grows as IPs interact with the system. To prevent unbounded memory growth during
|
||||
long runtimes, a scheduled background task (rate_limiter_cleanup) calls the
|
||||
cleanup_expired() method every 30 minutes. This is safe because:
|
||||
|
||||
- cleanup_expired() only removes IPs with no recent attempts (all timestamps
|
||||
outside the rate-limit window), so active IPs are never disrupted.
|
||||
- The cleanup is non-blocking and logged for observability.
|
||||
- Individual requests already prune old timestamps from each IP's deque during
|
||||
is_allowed() and record_failure(), so cleanup primarily handles dormant IPs.
|
||||
|
||||
For monitoring, check logs for "rate_limiter_cleanup" events to observe how
|
||||
many IPs are being retired from memory each cleanup cycle.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
Reference in New Issue
Block a user