_job_bulk_dedup_performers called run_bulk_dedup(strategy="performers") without the cross_source_only guard whose docstring exists precisely to prevent this OOM. At current catalog scale the unguarded path materializes N²/2 pairs per prolific performer into a list → worker hit 6GB RSS and was OOM-killed every 12h (05:00/ 17:00), taking down concurrent tpdb/stashdb/movie ingests as killed_by_restart (0 new movies). Verified in prod: 05:00 run now completes (885k pairs scored, no OOM) and ingests succeed (stashdb +241, tpdb +175). Also wrap in _run_with_timeout like tpdb/stashdb (job had no hard-timeout). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| browse_latest.py | ||
| bulk_dedup.py | ||
| deep_crawl.py | ||
| jobs.py | ||
| performer_driven.py | ||
| taxonomy_counts.py | ||
| worker.py | ||