tubepornclassic.com to SPA sieci TXXX (ten sam silnik co vjav): HTML ma pusty
<title> i zero linkow do scen, wszystko idzie z JSON API. Listing
/api/json/videos2/<cache>/str/latest-updates/60/..<page>.all...json oddaje
60 rekordow z pelnymi metadanymi (tytul, slug, dlugosc, post_date, obsada,
kategorie+tagi, miniatura), wiec nie ma fetchu per scena. ~168 tys. wideo,
wgrania codziennie. Koniec katalogu = 200 z code=videos_not_found.
Stream: videofile.php -> dekod video_url -> get_file -> f=video.m3u8 -> HLS
na ahcdn (klucz time-bound, nie IP-bound: adres wybity z VPS gra z lacza
domowego). Ten tor zyl dotad w tubes/vjav.py; wyniesiony do
_txxx.extract_hls, vjav i tubepornclassic wolaja to samo. HLS zamiast mp4,
bo mp4 z tego samego get_file ma w URL CDN speed=... (~1,2 MB/s), a segmenty
HLS leca ~3,6 MB/s. Wyzwanie 429 (ciasteczko w ciele) obsluzone w
ekstraktorze, scraperze i juz wczesniej w /proxy/hls.
Bramka: 38% tytulow to generator SEO ("Fabulous Sex Movie Vintage Try To
Watch For Ever Seen"). Z doczepionym nazwiskiem scena jest odnajdywalna
i zostaje; bez obsady (22%) nie wchodzi, bo nie ma po czym jej znalezc
ani z czym zdeduplikowac. Obsada u 46% scen, studiow brak.
Pilot: 2 strony, 120 scen, 111 nowych, 9 juz w bazie, wszystkie z dlugoscia.
Odtworzenie na emulatorze: playback_events success (ttff 3,3 s), AudioTrack
started. vjav po refaktorze: ten sam wynik ekstraktora co przed.
fullvideosporn.com is sextu.com rebranded, not a clone of fullmovies.xxx
(different engine, different catalog, 0/140 title overlap with our
fullmoviesxxx corpus).
The site is only worth ingesting behind a gate: 65-75% of its catalog has
no performer at all, and those scenes also carry SEO-spun titles, so they
would never match canonical and would land as empty orphans. So we ingest
only scenes with at least one performer. That keeps ~25-35% of the catalog
(verified: 19 of 60 on page one) where the signal is good, since 88-89% of
the performer names in the research sample already resolve to a canonical
performer in our DB.
Three site-specific traps, all handled:
- Titles come from the player's vit:"..." field, not og:title/h1, which are
sometimes an AI SEO rewrite rather than the real scene title.
- Cast is read only from the <h3>Porn-stars:</h3> section; the page carries
~22 videos.php?q= links overall but only 1-2 real performers, the same
pollution that got xxxfiles rejected. Porn Site / Porn Categories are
separate h3 blocks and are parsed per-section so they don't bleed.
- Every fetch passes a cookie gate: a fresh session gets HTTP 429 plus a
small JS challenge, so we read the cookie out of it and retry on the same
session. Hence the custom crawl_page instead of the base browser_get.
The TXXX video_url decoder moved out of vjav into _txxx.py since both tubes
share the engine; vjav keeps an alias and was re-verified after the move.
Playback resolves videofile.php -> decode -> get_file -> 302 -> znvcdn, and
the final CDN URL is portable cross-IP so the phone streams it directly.
Note for future debugging: the VPS itself gets 429 from that CDN because of
datacenter IP reputation, so playback health-checks run from the VPS will
be falsely negative.
Pilot ingest of 2 pages: 39 seen, 11 merged into existing scenes, 28 new,
0 errors; 0/19 without cast or duration on the sampled page.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
