Resolver/perf:
- find_by_phash_within: nearest match via Postgres bit_count over bit(64) XOR
instead of Python scan of all phash fingerprints (~20x faster per scene;
unblocks long delta runs that were killed mid-run before since advanced).
Scheduler/reliability:
- reap ingest_runs stuck in 'running' on worker startup (killed_by_restart).
- smoke_test: per-source ingest health, stuck-run and browse-freshness checks
-> Sentry; exclude killed_by_restart from the failed-run alarm.
Tags (ingest with tags + fill blanks):
- wire infer_tag_slugs into normalize_scene so tube scenes get title-inferred
tags (was dead code); union with connector tags.
- scripts/backfill_inferred_tags.py: keyset/batched/idempotent backfill for
existing tagless scenes (playable tag coverage 16% -> ~52%).
Clip-store:
- skip ManyVids/IWantClips/Clips4Sale/... from canonical sources at ingest
(GOON_SKIP_CLIP_STORE, default on) — permanent orphans, ~56% of canonical
ingest, never have a free-tube playback source.
Browse tubes:
- enable fullmovies + hdporn.gg: studio parsed from title prefix instead of
the /networks/ sidebar (which always yielded the first listed network);
drop phash compute (pilot: 0% canonical hit within Hamming 5 — auto-screenshots),
matching relies on title/performer/duration.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Re-checked whether these four KVS tubes could move to server-side resolve
like yespornvip/pornditt/porntrex. All four are reachable from the backend,
but cross-IP testing showed their final CDN URLs are IP-bound to the
resolving host (403 / connection refused from a different IP; fpo.xxx even
embeds the resolver IP in its acctoken). Unlike the portable cdn5/twa CDNs,
backend resolve cannot produce a mobile-playable URL here without a proxy,
which is out of scope for the public app.
- porn00: was using force_proxy resolve (violated the no-proxy stance);
switched to the WebView fallback like its siblings. The ad exposure that
originally motivated the proxy path is mitigated by the recent ad-filter
work (AD_HOSTS + cover overlay + injected-JS ad-CDN skipping).
- freshporno/pornxp/fpoxxx already on WebView fallback; comments updated
with the cross-IP findings so this isn't re-investigated.
- Dropped the now-unused tube extractor imports (F401).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
pornditt is the same kt_player KVS engine as yespornvip: flashvars carry
function/0/-obfuscated get_file urls + license_code, and the VPS reaches it
(HTTP 200). It was on _vps_blocked_fallback (WebView), where the scrape grabbed
the VAST preroll ad (trafostatic) instead of content (user bug "pornditt łapie
reklamę zamiast video").
Extracted the verified yespornvip logic into app/extractors/tubes/_kvs.py
(resolve_kvs: fetch page → decode function/0 get_file via kt_player algo → follow
302 in-session → portable CDN, multi-quality). yespornvip.py and new pornditt.py
are now thin wrappers. Registry: porndittcom _vps_blocked_fallback → pornditt.extract.
Verified on prod: pornditt → 720p/480p on twa.tgprn.com (portable, fresh-session
206 video/mp4); yespornvip still → 1080/720/480p on cdn5 (refactor intact).
Backend-only, no OTA — mobile plays mp4+mobile_direct_ok natively with quality
picker, zero WebView/ads.
Note: a real-browser residential load shows MEDIA_ERR on the content (the page's
own player flow / ad gating); server-side decode+follow sidesteps the player
entirely, which is why it resolves cleanly. The original bug scene (40f118e1) has
its video deleted on pornditt — verified on a live scene (156091).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
yespornvip was on the WebView fallback, which loaded the ad-heavy host page; the
INJECTED_JS scrape grabbed the preroll ad video (bkcdn.net, ~30s) instead of the
content, so the native player showed a 30s ad. The get_file content url is also
session/cookie-bound (410 for a cookieless ExoPlayer request).
Key finding: the VPS now reaches yesporn.vip (HTTP 200 — unblocked, same as
porntrex got 2026-05-22), so we can resolve server-side like porntrex instead of
relying on the browser. KVS flashvars carry function/0/-obfuscated get_file urls +
license_code; decode the hash with the kt_player algorithm (yt-dlp KVS algo,
verified to reproduce kt_player's output), then follow each quality's get_file 302
in the same curl_cffi session → final cdn5 url. That url is time-bound signed but
NOT IP/cookie-bound — verified portable cross-IP (VPS-resolved url fetched from a
different IP → 206 video/mp4).
New app/extractors/tubes/yespornvip.py returns 480p/720p/1080p portable CDN urls;
registry switched from _vps_blocked_fallback → yespornvip.extract. Mobile plays
direct natively with a working quality picker — zero WebView, zero ads, zero proxy.
Verified on prod (3 cdn5 sources) and emulator (quality picker → 1080p native
decode at 1920px, no WebView, no ad). Backend-only; no OTA needed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
User bug: porntrex plays slowly, no quality picker, reload flicker — suspected
VPS proxy. Root cause: porntrex KVS get_file tokens are cookie/session-bound, not
just time-bound as previously assumed. The extractor handed mobile the raw
get_file url; ExoPlayer's cookieless request → 410 → mobile fell back to the VPS
proxy (slow + nav.replace flicker).
Verified: following get_file in the same curl_cffi session that fetched the page
→ 200 (streams video); a fresh session → 410. The final CDN url after the 302
(cdn.pcdn.cloudswitches.com/...?expires=&md5=) is portable — fresh session → 206.
Fix: extract() now uses one curl_cffi Session for page + get_file, follows each
quality's 302 (stream + Range, no body download) and returns the resolved CDN url.
Mobile plays direct, multi-quality picker works, zero proxy bandwidth. Falls back
to the raw get_file url if a resolve fails. Verified on prod: both 720p/480p now
resolve to cloudswitches CDN.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Filtered /scenes (tag/origin/q/studio/performer) ran exhaustive COUNT with
stub-filter EXISTS over 1.7M rows: TAG 5.1s, ORIGIN 4.9s, SEARCH 3.1s.
Mobile relied on `loaded < total` for infinite-scroll, making exact count
mandatory and ruling out approximate shortcuts.
Backend:
- SceneListOut gains has_more (bool) and total_capped (bool), both optional
for backward compat with old mobile
- Filtered count uses LIMIT _COUNT_CAP+1 (1000) subquery — cost is
O(min(matches, cap)) instead of O(all). Measured: TAG 5.1s→664ms,
SEARCH 3.1s→138ms, ORIGIN 4.9s→1.07s (also fixes SiteScenes showing
global count ~1M instead of per-site count)
- has_more from fetching per_page+1 rows (essentially free); extra row
stripped before serialisation
- Pure-default list (no filters at all) keeps TTL-cached full count
Mobile:
- getNextPageParam uses has_more ?? fallback to loaded<total
- Display shows "{total}+" when total_capped=true (5 screens)
Verified on emulator: tag "Big Tits" → "1000 scenes" loaded, no 500s,
backward compat confirmed (old APK works against new backend).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Counts for /tags, /performers, /studios and /favorites were computed live
per-request by aggregating scene_tags / scene_performers with an EXISTS to
playback_sources. As the catalog grew to ~1.7M scenes (6.3M scene_tags) this
ran ~4.3s for /tags?order=popular (x2 incl. the total count) and ~950ms for
the default /scenes count, making those screens load in several seconds.
- migration 0019: add scene_count (+ DESC index) to tags/performers/studios
- background job _job_refresh_taxonomy_counts (every 3h) recomputes the counts
in one UPDATE..FROM each (IS DISTINCT FROM to skip unchanged rows)
- /tags, /performers, /studios scenes path now read the column + ORDER BY the
indexed scene_count; for_movies paths keep live aggregation (small tables)
- favorites read denormalized scene_count instead of a grouped EXISTS aggregate
- /scenes default count: 10-min in-process TTL cache (header is approximate)
Measured: /tags?order=popular&per_page=500 ~8s -> 66ms incl. serialization.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- app/api/seo.py (+ app/templates/seo/*): publiczny HTML SEO router (programmatic
entity long-tail: performer/studio/scene/landing/2257), bez api-key. Importowany
przez main.py — wymagany do uruchomienia, dotąd untracked. Opsec-clean (brak
VPS IP/sekretów).
- CLAUDE.md: instrukcje projektu (dotąd untracked).
- .gitignore: .nimbalyst/ (lokalne tracker-tooling, nie dla OSS repo).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
INSTALL BUG ("klikam Install i nic się nie dzieje"): ApkInstallerModule
commitował PackageInstaller.Session z PendingIntent → MainActivity, ale Android
po commit odsyła STATUS_PENDING_USER_ACTION + EXTRA_INTENT który MUSI być
startActivity()-owany żeby pokazać systemowy dialog "Install update?". Nic tego
nie obsługiwało → download OK, sesja commit OK, ale dialog NIGDY się nie
pokazywał. Fix: getBroadcast + runtime BroadcastReceiver → na PENDING_USER_ACTION
launchuje EXTRA_INTENT (FLAG_ACTIVITY_NEW_TASK), unregister na terminal status.
(Native — działa dla 0.2.1→przyszłe; do 0.2.1 user sideload z goon-foss.org.)
LAUNCHER ICON: regenerowane mipmapy (oo logo) z assets/icon.png przez PIL —
ic_launcher / ic_launcher_round / ic_launcher_foreground we wszystkich 5
densities (webp). iconBackground #0E1018 (stary navy) → #15110D (warm charcoal).
version 0.2.1 / versionCode 11. Build verified: podpis SHA-256 == ALLOWED,
Running "main" bez crasha. Deployed: /version=0.2.1, /static + webroot + landing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Option B (rebuild APK) — odblokowuje custom fonty na stałe + sprawia że
przyszłe font-OTA nie crashują.
- runtime 1.0 → 1.1 (app.json + AndroidManifest EXPO_RUNTIME_VERSION): nowy APK
ma native ExpoFontLoader, więc MUSI mieć inny runtime niż stare instalacje 1.0
(inaczej font-OTA crashnęłoby stare). 1.0 channel zostaje na d5b87e5c
(font-stripped) dla starych, 1.1 = nowy APK z fontami.
- version 0.2.0 / versionCode 10 (build.gradle) — in-app updater (/version=0.2.0)
zaoferuje install starym 0.1.9.
- Fonty przywrócone (useFonts, theme.fonts realne, SceneTile/MoviePosterCard/
navigation/GoonWordmark fontFamily) — działają bo native jest w APK.
- Build: gradlew assembleRelease (autolinking expo-font, BEZ prebuild — zachowane
custom native AntiTamper/ApkInstaller), Sentry source-map upload wyłączony
(SENTRY_DISABLE_AUTO_UPLOAD, brak org/auth — krok poboczny).
- app/main.py /version 0.1.9 → 0.2.0.
ZWERYFIKOWANE na emulatorze: podpis SHA-256 == ALLOWED_APP_SIG_HASH (anti-tamper
OK), ExpoFontLoader w classes3.dex, `ReactNativeJS: Running "main"` bez crasha.
APK live: /static/app-release.apk + goon-v0.2.0.apk + landing webroot.
UWAGA: launcher-icon (native mipmaps) NIE zmienione w tym buildzie — nadal stara
ikona. Nowy oo-icon wymaga regeneracji res/mipmap-* + rebuild (follow-up).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bug-report 2026-05-30 "ingest znów się zawiesił". streamporn/pandamovies
wieszały się intermittentnie mid-run (zależnie od live-contentu danego dnia),
blokując sekwencyjny _job_movie_ingest → mangoporn (jedyny mirror z realnym
new-content: 72 nowych 05-28) nigdy nie startował. try/except chronił przed
wyjątkiem, NIE przed hangiem.
Fix:
- _job_movie_ingest: każdy connector w ThreadPoolExecutor z future.result
(timeout=360s). Hang jednego źródła → log + shutdown(wait=False) + kolejka
leci dalej. Healthy run ~50s, cap 6min = zapas.
- get_movie_connectors: reorder paradisehill, MANGOPORN, streamporn, pandamovies
— mangoporn zaraz po canonical primary, przed wolniejszymi/wieszającymi się.
Zweryfikowane: pełny _job_movie_ingest przeszedł wszystkie 4 success w nowej
kolejności (mangoporn 2nd, 23s). 33 osierocone "running" rows (worker ubity
mid-run przy deployach) wyczyszczone osobno.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Previous attempts to extract direct stream URL from yesporn.vip flashvars both
turned out broken (verified 2026-05-30):
- video_url ('/get_file/7/'): requires PHPSESSID cookie from embed page session;
standalone mobile fetch returns 404
- event_reporting2 ('/get_file/1/'): returns HTTP 200 but Content-Type: image/gif
(1x1 analytics tracker pixel, not video)
Switch yespornvip -> _vps_blocked_fallback.extract. Mobile loads embed in WebView
with phone IP; kt_player JS decodes URL inside browser context (cookies + session
set properly); INJECTED_JS scrapes <video>.src and posts to ExoPlayer. UX flicker
(page renders before video) is the trade-off but aligns with no-video-proxy policy
(public-app bandwidth/anonymity priority).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Verify 2026-05-29: extractor zwracal video_url server `/get_file/7/...?embed=true`
ktore 404-uje na direct fetch nawet ze swiezym tokenem - URL wymaga PHPSESSID
z embed page session (cookies jar mobile-side nie matchuje VPS-side ekstraktora).
Switch na `event_reporting2` ktore wskazuje na `/get_file/1/...` - standalone
time-bound signed URL, 200 OK direct fetch z UA+Referer. Quality label
zachowany z `video_url_text`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bug-report 2026-05-28 ("od wczoraj nie ma nowych filmow"). DooplayConnector
.fetch_movies mial `while True` po stronach bez bound; streamporn (>2k filmow)
wisial godzinami az do dailowego killa schedulera, blokujac kolejke mangoporn
+ pandamovies. Watermark zamrozony, dziennie 0 nowych filmow.
Fix: cap _MAX_PAGES_DELTA=3 (since-driven runs, ~144 najnowszych pozycji)
i _MAX_PAGES_FULL=50 (full backfill gdy since=None). Wczesniejsza proba
filtrowania przez release_date odrzucona - release_date to data wydania filmu
(np. 2013), nie data uploadu na strone, wiec sortowanie listing nie matchuje.
Po deployu manualne re-run: streamporn 144/46s, pandamovies 120/47s,
mangoporn 108 z 72 NEW filmow w 58s. Scheduler queue unblocked.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace SAVEPOINT + IntegrityError fallback in resolve_tag with
postgres INSERT ... ON CONFLICT (slug) DO NOTHING + re-SELECT.
Postgres serializes on the unique index, so concurrent inserts of
the same slug no longer race on lookup→insert and the second caller
no longer raises uq_tags_slug. Mirrors the on_conflict pattern
already used for SceneTag/MovieTag inserts.
filemoon (+ mirrory kerapoxy/lvturbo/emturbovid/bysezoxexe/bysezejataos)
nie umarł — ~2026-05 zrobił rebrand na Vite SPA "Byse Frontend". Stary
P.A.C.K.E.R.-JWPlayer embed zniknął, więc backend uznał go za martwego i
wpisał na DEAD_HOSTER_RE. RE bundla index-ChwZgmXV.js (2026-05-22):
POST /api/videos/<code>/embed/playback body {"fingerprint":{}}
→ {"playback":{"key_parts":[..],"iv":..,"payload":..}}
→ key=concat(b64url(key_parts)); AES-256-GCM(key,iv,payload) → JSON
→ sources[*].url = HLS master.m3u8
Browser-attestation jest opcjonalny — pusty fingerprint wystarcza.
Stream URL jest IP-bound (token wiąże się z IP requestera), więc resolve
musi iść z urządzenia użytkownika (jak doodstream.ts / packerHoster.ts).
- mobile/src/lib/aesGcm.ts — pure-JS AES-256-GCM decrypt (RN/Hermes nie
ma Web Crypto); S-box liczony z GF(2^8), GHASH weryfikuje tag.
Zweryfikowane przeciw cryptography (Python) na 2 payloadach.
- mobile/src/lib/filemoonHoster.ts — resolver: POST playback → decrypt →
pick best source. E2E test: filemoon.to/e + /d + bysezoxexe.com mirror.
- PlayerScreen: filemoon w resolve useEffect obok doodstream/packer.
- backend: filemoon poza DEAD_HOSTER_RE; hoster.py early-return → przelot
jako type='hoster' do mobile resolvera (server-side resolve bezcelowy,
bo URL IP-bound do VPS).
- direct_scrapers: poprawiony błędny komentarz "filemoon shutdown".
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Dotąd nie dało się docelować sceny konkretnego hostera — search faworyzuje
xnxx/xvideos (dominują bazę), brak filtra po źródle. Diagnostyka per-hoster
(test cookie-fix, luluvid, porntrex) wymagała trafienia sceny danego tube'a.
- /scenes?origin=<substr> — exists() na PlaybackSource.origin ilike, np.
'hqporner' łapie tube:hqpornercom
- ScenesFilterModal: sekcja "Source / hoster" (TextInput) w FilterState.origin
- ScenesScreen: filter.origin → listScenes; liczone do activeCount
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
xtremestream (perverzija):
- extract_stream_from_hoster special-case: embed /player/index.php?data=<H>
→ m3u8 master = /player/xs1.php?data=<H> (z inline JS m3u8_loader_url)
- Wcześniej brak packera/file w videojs HTML → WebView fallback
porntrex (KVS) — VPS znów ma dostęp 2026-05-22:
- Nowy app/extractors/tubes/porntrex.py — flashvars video_url/_alt_url
→ get_file URLs (480/720/1080p)
- get_file 302 → CDN time-bound signed (expires+md5, NIE IP-bound)
→ mobile_direct_ok=True, mobile gra direct, zero VPS bandwidth
- _REGISTRY: porntrexcom _vps_blocked_fallback → porntrex.extract
bysezoxexe (latestpornvideo 2nd embed) — filemoon-rebrand Vite SPA,
wymaga osobnego RE; latestpornvideo i tak działa przez luluvid.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Goon — self-hosted aggregator for adult-content scene metadata.
Indexes scenes from TPDB, StashDB, and 30+ public adult tube sites.
Cross-source deduplication via perceptual hash + Levenshtein distance.
FastAPI backend + APScheduler worker + React Native (Expo) mobile client.
FOSS, ad-free, donation-funded. See README for details.
