goon/scripts
jtrzupek da7fcda132 feat(ingest): SQL phash match, tag inference + backfill, clip-store skip, browse tubes, watchdog
Resolver/perf:
- find_by_phash_within: nearest match via Postgres bit_count over bit(64) XOR
  instead of Python scan of all phash fingerprints (~20x faster per scene;
  unblocks long delta runs that were killed mid-run before since advanced).

Scheduler/reliability:
- reap ingest_runs stuck in 'running' on worker startup (killed_by_restart).
- smoke_test: per-source ingest health, stuck-run and browse-freshness checks
  -> Sentry; exclude killed_by_restart from the failed-run alarm.

Tags (ingest with tags + fill blanks):
- wire infer_tag_slugs into normalize_scene so tube scenes get title-inferred
  tags (was dead code); union with connector tags.
- scripts/backfill_inferred_tags.py: keyset/batched/idempotent backfill for
  existing tagless scenes (playable tag coverage 16% -> ~52%).

Clip-store:
- skip ManyVids/IWantClips/Clips4Sale/... from canonical sources at ingest
  (GOON_SKIP_CLIP_STORE, default on) — permanent orphans, ~56% of canonical
  ingest, never have a free-tube playback source.

Browse tubes:
- enable fullmovies + hdporn.gg: studio parsed from title prefix instead of
  the /networks/ sidebar (which always yielded the first listed network);
  drop phash compute (pilot: 0% canonical hit within Hamming 5 — auto-screenshots),
  matching relies on title/performer/duration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 15:07:35 +02:00
..
sql Initial commit 2026-05-20 10:10:22 +02:00
_extract_apk_sig_hash.py session work: bug-report fixes + WIP cleanup 2026-05-25 22:02:52 +02:00
_patch_manifest.py session work: bug-report fixes + WIP cleanup 2026-05-25 22:02:52 +02:00
add_performer_tpdb_ref.py Initial commit 2026-05-20 10:10:22 +02:00
audit_false_merges.py scripts: add gated --fix to false-merge audit (short-clip outliers) 2026-06-01 11:30:23 +02:00
auto_merge_freshporno_to_canonical.py session work: bug-report fixes + WIP cleanup 2026-05-25 22:02:52 +02:00
backfill_durations.py Initial commit 2026-05-20 10:10:22 +02:00
backfill_freshporno_dates.py session work: bug-report fixes + WIP cleanup 2026-05-25 22:02:52 +02:00
backfill_freshporno_titles.py session work: bug-report fixes + WIP cleanup 2026-05-25 22:02:52 +02:00
backfill_inferred_tags.py feat(ingest): SQL phash match, tag inference + backfill, clip-store skip, browse tubes, watchdog 2026-06-01 15:07:35 +02:00
backfill_paradisehill_tags.py session work: bug-report fixes + WIP cleanup 2026-05-25 22:02:52 +02:00
backfill_phash_tube.py Initial commit 2026-05-20 10:10:22 +02:00
backfill_scene_thumbnails.py Initial commit 2026-05-20 10:10:22 +02:00
bulk_auto_merge.py Initial commit 2026-05-20 10:10:22 +02:00
bulk_rescrape_hqporner.py Initial commit 2026-05-20 10:10:22 +02:00
bulk_resolve_merges.py Initial commit 2026-05-20 10:10:22 +02:00
check_all_hosters.py Mobile 0.1.9: OTA enable, WebView cookie-dismiss fix, porndoe connector 2026-05-22 11:20:57 +02:00
check_hetzner_traffic.py Initial commit 2026-05-20 10:10:22 +02:00
check_series_detector.py Mobile 0.1.9: OTA enable, WebView cookie-dismiss fix, porndoe connector 2026-05-22 11:20:57 +02:00
compare_performer_canon.py Initial commit 2026-05-20 10:10:22 +02:00
debug_pornxp_listing.py Initial commit 2026-05-20 10:10:22 +02:00
debug_tpdb_performer.py Initial commit 2026-05-20 10:10:22 +02:00
dedup_favorite_performers.py Initial commit 2026-05-20 10:10:22 +02:00
dump_pornapp_sites.ps1 Initial commit 2026-05-20 10:10:22 +02:00
fill_tpdb_refs_batch.py Initial commit 2026-05-20 10:10:22 +02:00
find_underfilled_performers.py Initial commit 2026-05-20 10:10:22 +02:00
generate_icons.py Initial commit 2026-05-20 10:10:22 +02:00
goon_debug_proxy.py Mobile 0.1.9: OTA enable, WebView cookie-dismiss fix, porndoe connector 2026-05-22 11:20:57 +02:00
killall_bulk_rescrape.py Initial commit 2026-05-20 10:10:22 +02:00
merge_tags.py Initial commit 2026-05-20 10:10:22 +02:00
migrate_paradisehill_to_movies.py Initial commit 2026-05-20 10:10:22 +02:00
phash_benchmark.py Initial commit 2026-05-20 10:10:22 +02:00
phash_dedup_scenes.py Initial commit 2026-05-20 10:10:22 +02:00
pilot_browse_scrapers.py Initial commit 2026-05-20 10:10:22 +02:00
probe_all_extract.py Initial commit 2026-05-20 10:10:22 +02:00
probe_browse_scraper.py Initial commit 2026-05-20 10:10:22 +02:00
probe_lpv_extract.py Initial commit 2026-05-20 10:10:22 +02:00
probe_mangoporn_hosters.py Initial commit 2026-05-20 10:10:22 +02:00
probe_perverzija.py Initial commit 2026-05-20 10:10:22 +02:00
probe_pv_extract.py Initial commit 2026-05-20 10:10:22 +02:00
probe_scene.py Initial commit 2026-05-20 10:10:22 +02:00
publish_update.py fix(ota+mobile): strip expo-font from bundle, runtime back to 1.0 2026-05-31 11:41:29 +02:00
reingest_pandamovies_hosts.py Initial commit 2026-05-20 10:10:22 +02:00
repair_dooplay_movies.py Initial commit 2026-05-20 10:10:22 +02:00
repair_truncated_titles.py Initial commit 2026-05-20 10:10:22 +02:00
reresolve_freshporno_orphans.py Initial commit 2026-05-20 10:10:22 +02:00
restore_canonical_titles.py Initial commit 2026-05-20 10:10:22 +02:00
smoke_test.py feat(ingest): SQL phash match, tag inference + backfill, clip-store skip, browse tubes, watchdog 2026-06-01 15:07:35 +02:00
stashdb_studio_backfill.py Initial commit 2026-05-20 10:10:22 +02:00
status_tubes.py Initial commit 2026-05-20 10:10:22 +02:00
studio_retrofix.py Initial commit 2026-05-20 10:10:22 +02:00
test_cross_ip.py Initial commit 2026-05-20 10:10:22 +02:00
test_porndoe_scraper.py Mobile 0.1.9: OTA enable, WebView cookie-dismiss fix, porndoe connector 2026-05-22 11:20:57 +02:00
test_resolve_endpoint.py Initial commit 2026-05-20 10:10:22 +02:00
theporndude_coverage_check.py Mobile 0.1.9: OTA enable, WebView cookie-dismiss fix, porndoe connector 2026-05-22 11:20:57 +02:00
theporndude_coverage_match.py Mobile 0.1.9: OTA enable, WebView cookie-dismiss fix, porndoe connector 2026-05-22 11:20:57 +02:00
theporndude_curl_triage.py Mobile 0.1.9: OTA enable, WebView cookie-dismiss fix, porndoe connector 2026-05-22 11:20:57 +02:00
theporndude_movies_pipeline.py Mobile 0.1.9: OTA enable, WebView cookie-dismiss fix, porndoe connector 2026-05-22 11:20:57 +02:00
theporndude_resolve_domains.py Mobile 0.1.9: OTA enable, WebView cookie-dismiss fix, porndoe connector 2026-05-22 11:20:57 +02:00
theporndude_scorecard.py Mobile 0.1.9: OTA enable, WebView cookie-dismiss fix, porndoe connector 2026-05-22 11:20:57 +02:00
title_levenshtein_benchmark.py Initial commit 2026-05-20 10:10:22 +02:00
tpdb_backfill.py Initial commit 2026-05-20 10:10:22 +02:00
tpdb_backfill_status.py Initial commit 2026-05-20 10:10:22 +02:00
tpdb_studio_backfill.py Initial commit 2026-05-20 10:10:22 +02:00