goon-foss/goon
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
goon/scripts
History
Exact
goon-foss ef8c255e1f perf(scenes): indeks trigramowy na title + ranking kandydatow do przegladu 
Kazde szukanie slowa w tytulach bylo sekwencyjnym skanem 3,46 mln wierszy. Blokowalo
to ranking over-attribution: metryka "ile scen ma te nazwe w tytule, ale NIE jest do
niej przypisanych" wymaga jednego skanu NA KANDYDATA, a kandydatow jest ~800. Proba
zostala ubita po 10 minutach.

GIN + gin_trgm_ops obsluguje LIKE, ILIKE i regexy, a pg_trgm sam normalizuje wielkosc
liter, wiec indeks na surowym title wystarcza. Efekt: 26 ms zamiast sekund na
zapytanie, ranking 800 kandydatow w 48 sekund zamiast godzin. Indeks 315 MB.

Zbudowany CONCURRENTLY w autocommit_block: zwykle CREATE INDEX trzymaloby lock
blokujacy zapisy na scenes przez cala budowe i zatrzymalo ingest.

Ranking od razu pokazuje dwie rozne rzeczy. Czyste smieci ("Pornhub" 105.9, "Monster"
92, "Pretty" 63, "Nice" 42) oraz SAME NAZWISKA (Devine, Starr, Dior, Cruz, West,
Knight, Johnson, James) - wpisy powstale z rozbicia imienia i nazwiska, ktore potem
lapia kazda scene z tym nazwiskiem, nalezaca do zupelnie innej osoby.

Usuniete przy okazji: "Pornhub" (108 scen, 0 z kanonu, wszystkie to "full video on
pornhub") i "Precious" (119 scen, 0 z kanonu, sam przymiotnik).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-27 15:59:52 +02:00
..
sql Initial commit 2026-05-20 10:10:22 +02:00
_extract_apk_sig_hash.py session work: bug-report fixes + WIP cleanup 2026-05-25 22:02:52 +02:00
_patch_manifest.py session work: bug-report fixes + WIP cleanup 2026-05-25 22:02:52 +02:00
_tube_survey.py chore(scripts): tube SSR-richness survey probe 2026-06-03 11:23:49 +02:00
add_performer_tpdb_ref.py Initial commit 2026-05-20 10:10:22 +02:00
audit_false_merges.py scripts: add gated --fix to false-merge audit (short-clip outliers) 2026-06-01 11:30:23 +02:00
audit_teaser_only.py feat(scripts): audit_teaser_only — hide scenes whose only source is a teaser 2026-06-07 19:52:44 +02:00
auto_merge_freshporno_to_canonical.py session work: bug-report fixes + WIP cleanup 2026-05-25 22:02:52 +02:00
backfill_durations.py Initial commit 2026-05-20 10:10:22 +02:00
backfill_freshporno_dates.py session work: bug-report fixes + WIP cleanup 2026-05-25 22:02:52 +02:00
backfill_freshporno_titles.py session work: bug-report fixes + WIP cleanup 2026-05-25 22:02:52 +02:00
backfill_inferred_tags.py feat(ingest): SQL phash match, tag inference + backfill, clip-store skip, browse tubes, watchdog 2026-06-01 15:07:35 +02:00
backfill_paradisehill_movies.py fix(movies): paradisehill delta date-granularity + browse cadence docs 2026-06-01 17:00:10 +02:00
backfill_paradisehill_tags.py session work: bug-report fixes + WIP cleanup 2026-05-25 22:02:52 +02:00
backfill_phash_tube.py Initial commit 2026-05-20 10:10:22 +02:00
backfill_scene_duration_from_playback.py fix(scenes): propagate playback duration to Scene + duration-consistent counts 2026-06-01 21:31:01 +02:00
backfill_scene_performer_source.py feat(dedup): provenance przypisan performerow do scen (scene_performers.source_id) 2026-07-27 15:04:10 +02:00
backfill_scene_thumbnails.py Initial commit 2026-05-20 10:10:22 +02:00
backfill_xvideos_performers.py fix(scripts): backfill arg parser consumed --workers value as LIMIT 2026-06-08 10:15:09 +02:00
bulk_auto_merge.py Initial commit 2026-05-20 10:10:22 +02:00
bulk_rescrape_hqporner.py Initial commit 2026-05-20 10:10:22 +02:00
bulk_resolve_merges.py Initial commit 2026-05-20 10:10:22 +02:00
check_all_hosters.py Mobile 0.1.9: OTA enable, WebView cookie-dismiss fix, porndoe connector 2026-05-22 11:20:57 +02:00
check_hetzner_traffic.py Initial commit 2026-05-20 10:10:22 +02:00
check_series_detector.py Mobile 0.1.9: OTA enable, WebView cookie-dismiss fix, porndoe connector 2026-05-22 11:20:57 +02:00
compare_performer_canon.py Initial commit 2026-05-20 10:10:22 +02:00
debug_pornxp_listing.py Initial commit 2026-05-20 10:10:22 +02:00
debug_tpdb_performer.py Initial commit 2026-05-20 10:10:22 +02:00
dedup_favorite_performers.py Initial commit 2026-05-20 10:10:22 +02:00
dedup_n2_canonical.py feat(scripts): dedup_n2_canonical — resolve n=2 false-merges via canonical duration 2026-06-07 20:25:10 +02:00
dump_pornapp_sites.ps1 Initial commit 2026-05-20 10:10:22 +02:00
fill_tpdb_refs_batch.py Initial commit 2026-05-20 10:10:22 +02:00
find_underfilled_performers.py Initial commit 2026-05-20 10:10:22 +02:00
generate_icons.py Initial commit 2026-05-20 10:10:22 +02:00
goon_debug_proxy.py Mobile 0.1.9: OTA enable, WebView cookie-dismiss fix, porndoe connector 2026-05-22 11:20:57 +02:00
killall_bulk_rescrape.py Initial commit 2026-05-20 10:10:22 +02:00
merge_altkey_tags.py fix(tags): collision-safe + FK-safe alnum tag merge 2026-07-21 16:24:50 +02:00
merge_dup2_tags.py fix(tags): merge <base>2 numbered-duplicate tags + prevent regeneration 2026-06-06 23:18:44 +02:00
merge_dupe_thumb_asset.py feat(scheduler): periodic thumb-asset dedup (hdporn.gg/fullmovies.xxx) 2026-06-14 14:56:45 +02:00
merge_exact_title_duration.py feat(scheduler): periodic title+duration dedup (missing-merge tube dupes) 2026-06-19 11:20:48 +02:00
merge_phash_exact_dupes.py fix(dedup): bezpiecznik na pary drugiego rzedu w merge_phash_exact_dupes 2026-07-27 12:53:54 +02:00
merge_tags.py Initial commit 2026-05-20 10:10:22 +02:00
migrate_paradisehill_to_movies.py Initial commit 2026-05-20 10:10:22 +02:00
phash_benchmark.py Initial commit 2026-05-20 10:10:22 +02:00
phash_dedup_scenes.py Initial commit 2026-05-20 10:10:22 +02:00
pilot_browse_scrapers.py Initial commit 2026-05-20 10:10:22 +02:00
pilot_porndoe_deepcrawl.py fix(review): player recovery deadlocks, quick-play re-fire, backfill gaps, em-dashes 2026-07-02 09:20:40 +02:00
probe_all_extract.py Initial commit 2026-05-20 10:10:22 +02:00
probe_browse_scraper.py Initial commit 2026-05-20 10:10:22 +02:00
probe_lpv_extract.py Initial commit 2026-05-20 10:10:22 +02:00
probe_mangoporn_hosters.py Initial commit 2026-05-20 10:10:22 +02:00
probe_perverzija.py Initial commit 2026-05-20 10:10:22 +02:00
probe_pv_extract.py Initial commit 2026-05-20 10:10:22 +02:00
probe_scene.py Initial commit 2026-05-20 10:10:22 +02:00
publish_update.py fix(scripts): force UTF-8 stdout in publish_update — stop false exit-1 2026-06-08 11:58:43 +02:00
refresh_sxyprn_thumbs.py feat(sxyprn): mark dead posts during thumbnail refresh sweep 2026-06-10 19:20:28 +02:00
reingest_pandamovies_hosts.py Initial commit 2026-05-20 10:10:22 +02:00
repair_dooplay_movies.py Initial commit 2026-05-20 10:10:22 +02:00
repair_truncated_titles.py Initial commit 2026-05-20 10:10:22 +02:00
reresolve_freshporno_orphans.py Initial commit 2026-05-20 10:10:22 +02:00
restore_canonical_titles.py Initial commit 2026-05-20 10:10:22 +02:00
review_performer_attributions.py perf(scenes): indeks trigramowy na title + ranking kandydatow do przegladu 2026-07-27 15:59:52 +02:00
smoke_test.py feat(ingest): SQL phash match, tag inference + backfill, clip-store skip, browse tubes, watchdog 2026-06-01 15:07:35 +02:00
stashdb_studio_backfill.py Initial commit 2026-05-20 10:10:22 +02:00
status_tubes.py Initial commit 2026-05-20 10:10:22 +02:00
studio_retrofix.py refactor(ingest): rename scraper Source name "pornapp" -> "tube-scraper" 2026-06-07 16:54:55 +02:00
test_cross_ip.py Initial commit 2026-05-20 10:10:22 +02:00
test_porndoe_scraper.py Mobile 0.1.9: OTA enable, WebView cookie-dismiss fix, porndoe connector 2026-05-22 11:20:57 +02:00
test_resolve_endpoint.py Initial commit 2026-05-20 10:10:22 +02:00
theporndude_coverage_check.py Mobile 0.1.9: OTA enable, WebView cookie-dismiss fix, porndoe connector 2026-05-22 11:20:57 +02:00
theporndude_coverage_match.py Mobile 0.1.9: OTA enable, WebView cookie-dismiss fix, porndoe connector 2026-05-22 11:20:57 +02:00
theporndude_curl_triage.py fix(triage): theporndude candidate audit had two false-negative bugs 2026-07-27 09:37:51 +02:00
theporndude_movies_pipeline.py Mobile 0.1.9: OTA enable, WebView cookie-dismiss fix, porndoe connector 2026-05-22 11:20:57 +02:00
theporndude_resolve_domains.py Mobile 0.1.9: OTA enable, WebView cookie-dismiss fix, porndoe connector 2026-05-22 11:20:57 +02:00
theporndude_scorecard.py Mobile 0.1.9: OTA enable, WebView cookie-dismiss fix, porndoe connector 2026-05-22 11:20:57 +02:00
title_levenshtein_benchmark.py Initial commit 2026-05-20 10:10:22 +02:00
tpdb_backfill.py Initial commit 2026-05-20 10:10:22 +02:00
tpdb_backfill_status.py Initial commit 2026-05-20 10:10:22 +02:00
tpdb_studio_backfill.py Initial commit 2026-05-20 10:10:22 +02:00