goon/app/scheduler
jtrzupek 0424cb9138 feat(scheduler): per-origin ingest freshness watchdog -> Sentry
The global source monitor can't catch a single stalled tube because every tube scraper shares one Source row (tube-scraper), so an aggregate run still reports success while one origin freezes (freshporno browsing the rotating KVS homepage root, report 14f3a655). New watchdog checks max(created_at) per active browse-scraper origin (tube:<sitetag>); if a tube with history hasn't produced a new scene in > max_age_hours it fires a Sentry message with a stable per-origin fingerprint (age in extras, not the title, so it stays one grouped issue). Runs every 6h, 48h threshold, both env-tunable (GOON_SCHED_INGEST_WATCHDOG_HOURS / GOON_INGEST_WATCHDOG_MAX_AGE_HOURS). Verified: 0 stale at 48h post-fix, detects neporn at a strict 12h threshold.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-15 10:26:25 +02:00
..
__init__.py Initial commit 2026-05-20 10:10:22 +02:00
browse_latest.py refactor(ingest): rename scraper Source name "pornapp" -> "tube-scraper" 2026-06-07 16:54:55 +02:00
bulk_dedup.py fix(dedup): tighten cross-source candidate prefilter — kill 1800s hang (GOON-V) 2026-06-08 10:03:33 +02:00
deep_crawl.py refactor(ingest): rename scraper Source name "pornapp" -> "tube-scraper" 2026-06-07 16:54:55 +02:00
ingest_watchdog.py feat(scheduler): per-origin ingest freshness watchdog -> Sentry 2026-06-15 10:26:25 +02:00
jobs.py feat(scheduler): per-origin ingest freshness watchdog -> Sentry 2026-06-15 10:26:25 +02:00
performer_driven.py refactor(ingest): rename scraper Source name "pornapp" -> "tube-scraper" 2026-06-07 16:54:55 +02:00
taxonomy_counts.py fix(scenes): propagate playback duration to Scene + duration-consistent counts 2026-06-01 21:31:01 +02:00
thumb_dedup.py feat(scheduler): periodic thumb-asset dedup (hdporn.gg/fullmovies.xxx) 2026-06-14 14:56:45 +02:00
worker.py feat(scheduler): per-origin ingest freshness watchdog -> Sentry 2026-06-15 10:26:25 +02:00