goon/alembic/versions
goon-foss 544de9c5ba feat(ingest): backfill brakujacej dlugosci scen
1 755 421 scen nie ma dlugosci ani na scenie, ani na zrodle, wiec wypadaja z
feedu: w SQL "duration_sec >= 60" nie jest prawda dla NULL, przez co scena o
nieznanej dlugosci znika tak samo jak dziesieciosekundowy smiec. Filtr zostaje,
bo odcina smieci, uzupelniamy dane.

Swiadomie waskie originy. Uniwersalny regex trafia w smieci: porntrex ma w HTML
"duration: 5" reklamy, mypornerleak dlugosc ciastka 720, a xhamster i porntrex
wypisuja dlugosci filmow z paska obok (naiwne dopasowanie dalo 198s zamiast 240s).
Zla dlugosc jest gorsza niz brak, bo dedup opiera sie na niej najmocniej.

- xvideos, xnxx: VideoObject LD-JSON, 5/5 na probce
- xhamster: videoEntity z kontrola zgodnosci id z URL-a
- porntrex pominiety: strona szczegolow nie podaje wlasnej dlugosci

Kolejka jak przy phashu (plan raz, mielenie paczkami, przezywa restart):
845 899 scen, 500 stron co 5 min, czyli ok. 6 dni. Plan powtarza sie co 24h,
zeby swieze sceny bez dlugosci tez trafialy do kolejki.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 12:10:22 +02:00
..
20260502_0001_initial.py Initial commit 2026-05-20 10:10:22 +02:00
20260502_0002_playback_sources.py Initial commit 2026-05-20 10:10:22 +02:00
20260503_0003_playback_dead.py Initial commit 2026-05-20 10:10:22 +02:00
20260504_0004_animated_thumbnail.py Initial commit 2026-05-20 10:10:22 +02:00
20260504_0005_favorite_performers.py Initial commit 2026-05-20 10:10:22 +02:00
20260504_0006_blacklists.py Initial commit 2026-05-20 10:10:22 +02:00
20260504_0007_play_progress.py Initial commit 2026-05-20 10:10:22 +02:00
20260506_0008_performer_search_meta.py Initial commit 2026-05-20 10:10:22 +02:00
20260506_0009_movies.py Initial commit 2026-05-20 10:10:22 +02:00
20260506_0010_favorite_scenes.py Initial commit 2026-05-20 10:10:22 +02:00
20260507_0011_origin_pornapp_to_tube.py Initial commit 2026-05-20 10:10:22 +02:00
20260508_0012_favorite_studios.py Initial commit 2026-05-20 10:10:22 +02:00
20260509_0013_bug_reports.py Initial commit 2026-05-20 10:10:22 +02:00
20260509_0014_favorite_movies.py Initial commit 2026-05-20 10:10:22 +02:00
20260510_0015_bug_reports_movie_id.py Initial commit 2026-05-20 10:10:22 +02:00
20260512_0016_realdebrid_cache.py Initial commit 2026-05-20 10:10:22 +02:00
20260512_0017_drop_realdebrid_cache.py Initial commit 2026-05-20 10:10:22 +02:00
20260528_0018_movie_play_progress.py feat(movies): watched/continue-watching tracking end-to-end 2026-05-28 23:24:06 +02:00
20260531_0019_taxonomy_scene_counts.py perf(taxonomy): denormalize scene_count for tags/performers/studios 2026-05-31 17:53:48 +02:00
20260602_0020_scene_tags_tag_id_index.py perf(scenes): drop exact count on filtered lists; index scene_tags(tag_id) 2026-06-02 12:00:36 +02:00
20260607_0021_scene_tags_tag_id_stats.py migration(0021): raise scene_tags.tag_id statistics target to 1000 2026-06-07 21:12:22 +02:00
20260608_0022_device_scoped_user_state.py feat(api): device-scope user state (favorites/progress/blacklists) 2026-06-10 08:58:01 +02:00
20260612_0023_bug_report_replies.py feat(bug-reports): two-way replies (device-scoped) + admin reply endpoint 2026-06-12 11:35:44 +02:00
20260616_0024_saved_searches.py feat(api): per-device saved searches (keyword favorites) 2026-06-16 13:52:18 +02:00
20260622_0025_source_ranking.py feat(sources): 0-5★ ranking on Sites (freshness/metadata/plays) + playback telemetry 2026-06-22 10:00:59 +02:00
20260702_0026_scene_backfill_flag.py fix(review): player recovery deadlocks, quick-play re-fire, backfill gaps, em-dashes 2026-07-02 09:20:40 +02:00
20260705_0027_playback_event_error_detail.py fix(watchporn): re-resolve stale CDN URL in-player, drop WebView page fallback 2026-07-05 01:18:08 +02:00
20260721_0028_tags_name_altkey_index.py fix(tags): collapse spelling-variant duplicate tags + prevent regrowth 2026-07-21 15:59:33 +02:00
20260727_0025_phash_blacklist.py fix(dedup): blacklista zdegenerowanych phashy + scalenie 4647 duplikatow 2026-07-27 12:47:20 +02:00
20260727_0026_scene_performer_source.py feat(dedup): provenance przypisan performerow do scen (scene_performers.source_id) 2026-07-27 15:04:10 +02:00
20260727_0027_scenes_title_trgm.py perf(scenes): indeks trigramowy na title + ranking kandydatow do przegladu 2026-07-27 15:59:52 +02:00
20260813_0028_phash_backfill_queue.py feat(dedup): backfill phasha z zapisanych miniaturek 2026-08-13 10:49:12 +02:00
20260818_0029_scenes_max_quality.py feat(filtr): minimalna jakosc w filtrze scen (720p/1080p/4K) 2026-08-18 11:06:34 +02:00
20260818_0030_scenes_quality_threshold_idx.py feat(filtr): minimalna jakosc w filtrze scen (720p/1080p/4K) 2026-08-18 11:06:34 +02:00
20260820_0031_duration_backfill_queue.py feat(ingest): backfill brakujacej dlugosci scen 2026-08-20 12:10:22 +02:00