Browse over /best/<YYYY-MM>/<page> (SSR; xnxx has no clean /new/ and its homepage is
JS-rendered) for a latest-feed freshness signal next to the performer-driven search
scraper. JSON-LD VideoObject only — xnxx detail (unlike its xvideos twin) doesn't
expose /models/ or /tags/ in SSR, so performers/tags come via canonical merge + the
search scraper. Title is html.unescaped (JSON-LD ships ,/! entities).
xhamster and sxyprn intentionally left search-only: xhamster Cloudflare-blocks the
VPS on listing pages (1KB challenge), sxyprn has no clean SSR listing (IP-bound) —
a flaky browse scraper would be worse than the working search + 168h watchdog.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Browse over /browse/time/?page=<n> (SSR) for guaranteed latest-feed freshness next to
the existing performer-driven search scraper. JSON-LD VideoObject only (title /
duration / uploadDate / thumbnail) — deliberately NOT scraping performers/tags from
the detail page: JSON-LD has no actor field and the /pornstar//category links are
sidebar-polluted with no scene-scoped container, so a naive regex attached the same
2 pornstars to every scene. Performers/tags come via canonical merge + the search
scraper instead.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both were search-only — fresh only as long as the performer queue cycles and the
site search keeps working. Added browse scrapers next to the existing search ones
(xvideos/eporner pattern: search keeps performer back-catalog coverage, browse
guarantees latest-feed freshness → watchdog 48h instead of 168h):
- porntrex: KVS /latest-updates/<n>/ (title + thumb + phash)
- mypornerleak: WP REST /wp-json/wp/v2/posts?_embed=1 (title + date + studio from
category + performers from the actors taxonomy)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Watchdog flagged porndish as frozen (search ?s= stopped yielding new scenes
2026-05-07, 1151h). It's WordPress and the VPS can reach it, so converted to a browse
scraper over the WP REST API (/wp-json/wp/v2/posts?_embed=1), same pattern as
perverzija: title, date, featured thumbnail, studio (category — FreeUseFantasy /
I Have A Wife / … paysite content) and tags. Performers via canonical merge. Playback
unchanged (embed iframe → phone-side). 60 fresh scenes on first crawl.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Search (?s=) started returning 429 and the homepage is JS-rendered (no post links in
raw HTML), so the old search scraper got 0 (frozen since 2026-05-07). perverzija is
WordPress and the VPS can reach it (200, not CF-blocked), so converted to a browse
scraper over the WP REST API (/wp-json/wp/v2/posts?_embed=1): one structured call per
page gives title, date, featured thumbnail, studio (category — DadCrush/FamilyStrokes/
… TeamSkeet-family paysite re-ups) and genre tags. Performers via canonical merge
(stars taxonomy isn't REST-exposed; title carries names). Playback unchanged (embed
iframe → phone-side). 15 fresh + 45 refreshed on first crawl.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three orphan-factory tubes (0–0.2% canonical match — auto-screenshot thumbs and
slug titles that never match TPDB/StashDB) — to be replaced by better sources.
Removed scrapers (files + imports), extractors (registry + modules), the pornhat
entry from tag-enrichment priority lists and the 0dayxx display override, and purged
the DB (19,003 playback_sources + 9,904 solo-orphan scenes; shared mirror scenes keep
their other sources). The pornhat-based enrich_studio endpoint stays as a graceful
no-op (no pornhat sources → returns no studio).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
fpo.xxx is a KVS site, not WordPress, so the old `?s=` search scraper matched
nothing (frozen since 2026-05-07). Converted to a browse scraper reading /new-<n>/
(title + duration + thumbnail + phash from the listing tile; performers via canonical
merge). Playback was already phone-side (KVS). 32 fresh scenes on first crawl.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both scrapers were disabled since 2026-05-12 (~0.4% canonical match — mostly short
amateur clips that never match studio content); their data sat frozen. Removed for
good: deleted the extractor registry entries, scraper files and imports, dropped them
from the tag-enrichment priority lists, and purged the DB (17,906 playback_sources +
122 scenes that had no other source; mirror scenes shared with other tubes just lost
the ph/rt link).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rates each source on three axes the user asked for:
- freshness: how recently/often new content arrives (newest age + 7d volume)
- richness: metadata coverage (thumbnail/tags/performers/description/studio/duration)
- plays: does it actually play — from real playback telemetry when available,
else a proxy from the resolve mechanism. 0★ = offline (gates the overall stars,
so a fresh+rich source that doesn't play still ranks bottom — the hqfap/4k69 case)
Backend:
- playback_events: fire-and-forget telemetry POST from the app per playback attempt
(origin + success/error + time-to-first-frame), append-only, 30d retention
- source_stats: per-origin computed scores, refreshed by a scheduler job (6h);
/sources joins it and sorts by stars
- models + local migration 0025; new GOON_SCHED_SOURCE_STATS_HOURS setting
Mobile:
- Sites rows show ★ rating; tap the stars for a breakdown (axes + metadata %, plus
whether "plays" is measured or estimated)
- PlayerScreen reports playback success/failure per source (native path only —
symmetric, conservative); origin threaded through Scene/Movie play callsites
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- hqfap + 4k69: both ingested fresh but playback is dead (hqfap serves a fixed
~3MB "server down" stub for every scene; 4k69 resolves no playable URL).
Removed from ALL_BROWSE_SCRAPERS so no new dead sources get ingested; existing
live playback_sources marked dead in prod (scenes drop out of has_playback /
Sites). Extractors kept in registry for easy re-enable if the hosts recover.
- latestpornvideo: was a performer-search scraper, so it never picked up the
site's "latest" feed — users saw a stale set. Converted to a browse scraper
reading /page/N/ (studio+date from title/thumb, category tags; performers via
canonical merge). Moved DIRECT → BROWSE list.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
hqfap migrated its JSON-LD contentUrl (and the *.workers.dev mirror) to /upload/videos/video_down.mp4, which serves a FIXED ~3.04MB file for EVERY scene regardless of declared length (verified 5/5 scenes at 14-47min all = 3.04MB, 2026-06-21). It is a placeholder/'server down' clip, not the content — the browser's own player streamed the same stub via MediaSource. We were handing users that 3MB stub (reports c382d441/ef10b946). Now reject the video_down.mp4 contentUrl and return no source, so scenes fall through to other sources or show no playback instead of a fake clip. Real older scenes (cdnde.com / okcdn.ru direct mp4) still resolve. This also makes the proxy-fallback question moot — there is no source to proxy.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Movie detail showed ~100 playback links (report 41ca1fa4) because the 3 dooplay mirrors (mangoporn/pandamovies/streamporn) each record the SAME hoster embed as a separate row (e.g. luluvid/e/X from all three). Dedup by real target (embed_url/stream_url/page_url) after the priority sort, keeping the highest-priority copy — one verified movie drops 101 -> 58 unique.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
siska's ?s= search ignores the query (returns latest regardless), so the performer-driven search scraper always yielded 0 and was disabled. Rewrote SiskaScraper as a latest-browse scraper (BaseBrowseScraper, /page/<n>/) and moved it to ALL_BROWSE_SCRAPERS. The listing tile carries everything (no detail fetch): title, duration (MM:SS span), thumbnail (img data-src), performer + studio (img alt 'Performer - Title - Studio'), category (thumbnail path). Playback unchanged: fresh videos embed playmogo + luluvid, resolved phone-side via _embed_iframe. Verified ingest: 26 seen / 11 new / 15 updated / 0 errors — and 15 updated means siska scenes match existing canonical scenes, adding playback coverage rather than orphans. Now covered by the browse ingest-watchdog (48h) and the 6h browse-latest + deep-crawl jobs. Old self-player videos (player.siska.video -> cfglobalcdn, ~2018) are dead and age out.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Revisited siska re-enable (user fa4083a2). Findings: (1) fresh siska videos (videoID 227xxx) embed playmogo + luluvid and ARE phone-resolvable; updated siska.py scene regex + extractor path to the current video.php?videoID= format (old /<slug>/ format is gone). (2) BUT siska's ?s=<query> search is broken site-side — it returns the latest videos regardless of query (angela white == riley reid == homepage), so as a performer-driven BaseSearchScraper it always yields 0 (title token filter rejects everything). Reviving siska would require converting it to a browse/latest scraper (changes ingest character) — left as a decision. Old self-player videos (player.siska.video -> cfglobalcdn) are dead. Scraper stays disabled.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Missing-merge duplicates (same performer + identical normalized title + identical duration-to-the-second) that bulk_dedup misses — tube re-scrapes and cross-tube re-ingests like porn00 pulling a video already present from xnxx (reports 28fe8181/32df33b1). Extracted the proven merge_exact_title_duration logic into app/scheduler/title_duration_dedup.py (script now a thin wrapper), wired a 12h scheduler job (playback-only = what users actually see, GOON_SCHED_TITLE_DEDUP_HOURS). Signal is near-certain (two different videos don't share byte-identical title AND exact duration); no shared performer = not merged (over-match guard). Verified: job registers (jobs=14), backlog currently 0 after the one-shot global merge.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
scene_resolver._sync_tags used check-then-insert (select existing -> add if None), which races under concurrent ingest of the same scene: two runs both see existing=None, both add, flush -> IntegrityError pk_scene_tags (Sentry GOON-M, 4 events). Switched to pg_insert(...).on_conflict_do_nothing(index_elements=[scene_id, tag_id]) + in-batch dedup, identical to movie_resolver._sync_tags.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two observability additions to the worker scheduler (intertwined in the same files): (1) ingest-watchdog now also covers performer-driven search scrapers (ALL_DIRECT_SCRAPERS) with a separate 7d threshold, not just browse tubes at 48h — several search tubes (perverzija, fpoxxx, porndish, ...) had frozen silently for weeks. (2) New Hetzner Cloud bandwidth monitor (app/scheduler/hetzner_monitor.py): polls outgoing_traffic vs included_traffic and fires a Sentry message at info/warning/error % thresholds with a per-level fingerprint. The config fields existed for ages but the monitor was never implemented. No-op until HETZNER_API_TOKEN + HETZNER_SERVER_ID are set in .env (verified: returns {enabled: False}, job registers as 'hetzner-monitor every 6h', jobs=13).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Old regex matched junk (/wp-json etc.), not scenes (scenes are /<post_id>/).
Frozen since 06-13. Rewrote search() to scrape the /actor/<slug>/ listing
and parse <article> cards: scene URL, title, performers + tags from the
class (actors-*/tag-*/category-*, dropping performer-name fragment tags),
thumbnail. Studio + release date parsed from the "<Studio>-YYYY-MM-DD"
thumbnail filename, with a title-prefix "<Studio> YY MM DD" fallback.
Multi-performer works; no duration in listing; playback unchanged (hoster).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Content moved to the w8.mypornerleak.com (wN) load-balancer subdomain, so
the old bare-domain scene regex matched nothing (frozen since 05-07).
Rewrote search() to scrape the canonical /actor/<slug>/ listing: scene
URL (wN host normalized to canonical for stable dedup), title, duration,
performers and category-tags from the <article> class (actors-*/category-*),
thumbnail. No studio (OnlyFans/amateur leaks have none). Multi-performer
works; playback unchanged (hoster, phone-side).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
sxyland dropped the /<numeric_id>/<slug>/ scene URL format for /<slug>/,
so the old regex matched nothing (frozen since 06-07). Rewrote search()
to use the performer page /actor/<slug>/ and fetch each scene for full
metadata: all performers (with co-stars, from /actor/ links), tags
(scoped to the scene's tags-list, not the sidebar), duration + upload
date (itemprop), studio from the title prefix (BraZZers/MilfCoach/... ,
guarded so a performer-name prefix isn't mistaken for a studio). Junk
nav pages (Terms of Use etc.) are dropped via a no-duration-and-no-tags
guard. Verified: clean studio/performers/tags in DB, 0 errors.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
sxyprn ingest was frozen since 05-07: the old ?type=videos&query= endpoint
returns trending (not performer-filtered), so the strict token filter
correctly dropped everything -> 0 ingest. Real "search" is the performer
page /<First-Last>.html. Rewrote search() to scrape those cards: clean
performer (the query, avoids sxyprn's Dallas/Rae name fragmentation),
studio (channel subcat), tags (#hashtags), duration, thumbnail. Token
filter now runs on the card title so only genuine matches attach the
performer. Verified: Lana Rhoades/Riley Reid/Angela White return results,
metadata persists in DB (studio e.g. Vixen, 10-31 tags/scene), playback
mp4 206.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
User-report (mobilism): scenes are often poorly titled, so saved keyword queries are a useful extra retrieval strategy. New saved_searches table (device-scoped via X-Device-Id, unique per device+query, 50/device cap) + GET/POST/DELETE /saved-searches. Migration 0024. Verified CRUD on prod: add trims+dedups idempotently, empty rejected 422, delete idempotent.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The global source monitor can't catch a single stalled tube because every tube scraper shares one Source row (tube-scraper), so an aggregate run still reports success while one origin freezes (freshporno browsing the rotating KVS homepage root, report 14f3a655). New watchdog checks max(created_at) per active browse-scraper origin (tube:<sitetag>); if a tube with history hasn't produced a new scene in > max_age_hours it fires a Sentry message with a stable per-origin fingerprint (age in extras, not the title, so it stays one grouped issue). Runs every 6h, 48h threshold, both env-tunable (GOON_SCHED_INGEST_WATCHDOG_HOURS / GOON_INGEST_WATCHDOG_MAX_AGE_HOURS). Verified: 0 stale at 48h post-fix, detects neporn at a strict 12h threshold.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The homepage root / is a KVS page with cache-control: no-store and a fresh PHPSESSID per request; the server rotates its featured block and on a cold session can serve an old set instead of the newest scenes. Result: browse-latest skipped everything for 3 days (root served 20 May content), no new freshporno scenes since 12 Jun (user report). Switch _listing_url to the explicit date-sorted /latest-updates/ feed (pagination /latest-updates/N/), which is not subject to that rotation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The one-off cleanup merged ~13.5k same-video-different-title dupes, but they regrow as
these sibling tubes re-ingest under new titles. Wire the asset-id+duration merge into
the scheduler (every 12h, GOON_SCHED_THUMB_DEDUP_HOURS, 0=off) so it stays clean.
Shared logic lives in app/scheduler/thumb_dedup.py (run_thumb_asset_dedup); the one-shot
script now imports it. Same tight signature as the cleanup: family hosts only + identical
duration (the bare asset-id number is reused across unrelated CDNs, so cross-host/diff-
duration grouping is excluded). Reports 205b17d9 / 5a2944cb.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reverse-engineered the migrated 4k69 player: jwplayer now serves OK.ru CDN (okcdn.ru)
mp4s. The static page (SSR behind Cloudflare, fetched via proxy) carries "file"+"label"
pairs for every quality. okcdn's srcIp param is NOT enforced (cross-IP test 2026-06-14:
206 video/mp4 from a residential IP != srcIp), so the URL plays from any IP. Parse the
okcdn sources server-side and return them mobile_direct_ok — the phone plays the direct
video, no WebView, no VAST preroll, no age-gate, zero VPS proxy. Skips 4K/2K. Reverts
the brief _vps_blocked_fallback routing (WebView grabbed the preroll ad, not content).
Verified on emulator: native player streams the actual scene (report 5de3fbc5).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
4k69 swapped its player from get_file (4kporno.xxx) to jwplayer + okcdn.ru, whose token
carries srcIp= (IP-bound); the site is also behind Cloudflare (VPS fetch only via proxy).
The native get_file extractor matched nothing and returned None, surfacing as a "host
problem" error even though the video plays fine (report 5de3fbc5). Switch 4k69com to
_vps_blocked_fallback: the on-device WebView (residential IP) clears Cloudflare, the
okcdn token binds to the phone IP, and INJECTED_JS hands the jwplayer source to ExoPlayer.
fourk69.extract stays in the module in case the site reverts to get_file.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
enrich-thumbnail was fill-only (skipped scenes that already had a thumbnail), so a
broken or stale preview (rotting sxyprn/trafficdeposit) could not be refreshed. Add a
force flag that re-fetches the source page and overwrites the existing thumbnail.
Backs the new "Refresh thumbnail" button (report d3376a71).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Porntrex soft-deletes: a removed video returns HTTP 200 with a "this video was deleted"
message instead of a player, so extract returned [] (transient) and the source was never
marked dead, leaving users on a permanently broken link (report 75dbf53e). Match the
deletion message and raise HosterDead so resolve marks the source dead.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Some sources (sexlikereal) build a giant `code`/`director` from a multi-performer
compilation title, overflowing scenes.code varchar(128) -> StringDataRightTruncation,
and the scene silently dropped from ingest. Cap both at the column limit in
_create_canonical and the fill path; code/director are stored metadata, not match keys,
so truncation is safe.
Fixes GOON-J
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reports were anonymous and one-way. Tie each report to the submitting device
(X-Device-Id), add an admin response back-channel, and let the app fetch replies for
its own device:
- migration 0023: bug_reports gains device_id, response, responded_at, response_seen.
- create_bug_report captures device_id.
- GET /bug-reports/mine (device-scoped) returns this device's reports + unseen count.
- POST /bug-reports/mine/seen clears the unseen flag.
- POST /bug-reports/{id}/reply sets the admin response (authored during triage).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A source (TPDB) returned a performer alias containing a literal U+0000 ("Ramon..").
Postgres cannot store in JSONB or text, so the external_records JSONB insert in
_upsert_external_record failed with UntranslatableCharacter and the scene never ingested
(GOON-Z). Recursively strip NUL from the raw payload (-> external_records.raw) and, when
present, also re-validate the RawScene/RawMovie so normalize -> typed text columns get
clean data too. Gated by a cheap _has_nul scan so clean records (the overwhelming
majority) pay no extra cost.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Time-bound HLS hosters whose manifest URL lacks a .m3u8 extension (e.g. pornhat's
"...mp4,?..." path) were mis-detected by ExoPlayer as progressive MP4 and failed,
forcing a full proxy fallback that streamed the whole video through the server. Serve
such manifests via /proxy/hls/<token>/play.m3u8 with child URLs left absolute on the
CDN, so the device fetches variant+segments directly and only the ~1KB manifest is
proxied. Routed only for mobile_direct_ok (time-bound) HLS without a .m3u8 path.
Also swallow httpx.TransportError in the stream proxy body generator: an upstream CDN
closing the connection mid-stream is benign (client just retries a range) and should
not surface as an unhandled error.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
It is an ISP proxy (static ISP IPs, flat billing), not residential —
so HTML-ingest bandwidth is free and the full deep-crawl is fine.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
superporn hard-blocks the VPS IP with Cloudflare 403 on every TLS
impersonation, so HTML ingest routes through Bright Data residential
(BRIGHTDATA_PROXY_URL, parsed in config). First scraper to use a proxy:
optional _proxy on the browse base, threaded into browser_get.
JSON-LD VideoObject (title/desc/uploadDate/thumb/duration) + pornstar
and category chips; superporn double-encodes HTML entities so titles
are unescaped twice. Thumbnails fetch fine from the VPS (no proxy).
Playback stays off-proxy: the <source> mp4 token is IP-bound to the
fetcher, so resolve is phone-side via WebView (extractor superporncom
-> _vps_blocked_fallback), same as porndoe.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
4k69.com (~65k scenes): same PlayTube CMS as hqfap - common logic moved
to _playtube.py (sitemap catalog, JSON-LD, pills). Studio classified by
matching category pills against the studios index page. Streams are
get_file (fullmovies family) returned unresolved with mobile_direct,
2160p skipped.
neporn.com: KVS engine, latest-updates listing, JSON-LD + video:duration
meta, performers from models links with flashvars video_tags fallback
for fresh uploads. Resolve via _kvs; final URL portable cross-IP.
superporn.com rejected: Cloudflare 403 from VPS on all TLS impersonations.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
PlayTube CMS. Sitemap-based pagination (listing has no GET paging),
JSON-LD VideoObject metadata, pornstar/category pills, " Clips"
categories mapped to studio. Direct mp4 (cdnde.com/okcdn.ru), tokens
time-bound and portable cross-IP, so mobile plays direct.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
trafficdeposit poster tokens live ~1h (hour-bucketed), so stored URLs can't persist.
New GET /proxy/sxyprn-thumb/{post_id}: resolves the current og:image from the live
/post/<id> page (cache resolved poster URL ~40min), streams bytes with Referer +
long client Cache-Control (URL is stable per post_id → client disk-caches the image,
backend fetches each post ~once). Deleted posts ("Post Not Found") → 404.
Scene grid now emits /proxy/sxyprn-thumb/<id> for sxyprn sources (derived from
page_url) instead of the dead stored trafficdeposit URL. Verified: live post → 200
image, deleted → 404, grid emits resolver URL. Backend-only, no OTA.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CORRECTION: trafficdeposit thumbnail tokens are hour-bucketed and valid only ~1h
(verified 2026-06-10: stored ts=11:00 dead at 12:27, current ts=13:00 loads). Earlier
"~weekly rot" read was wrong. Storing/periodically-refreshing sxyprn thumbnail URLs
is futile — they expire within the hour. Default the refresh job OFF (kept in code).
The dead-marking sweep (Post Not Found → dead_at) it performed was still valid. Live
sxyprn thumbnails need on-demand resolution at serve time (future work).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CORRECTION to earlier "unrecoverable" call: the /post/<id> page is alive (200) and
DOES expose the scene's own fresh-signed poster via og:image / <video poster>
(post-id embedded, current timestamp) — only the STORED thumbnail URL had rotted.
Search/listings don't re-surface old posts (0 overlap), but per-post fetch works.
scripts/refresh_sxyprn_thumbs.py: iterate live sxyprn sources, fetch post page,
extract fresh og:image, UPDATE thumbnail_url (verified: refreshed URLs return 200).
_job_refresh_sxyprn_thumbs: every 12h refresh the 1200 least-recently-updated sources
(cycles the ~19k catalog within the expiry window). Pairs with the scene_resolver
overwrite fix so refreshed thumbnails stick.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
_upsert_playback_sources only set thumbnail_url when the existing value was NULL,
so signed CDN thumbnails that ROT (sxyprn/trafficdeposit tokens expire ~weekly →
404) were never replaced even when a fresh re-scrape captured a valid URL — making
the rot permanent (bug 2026-06-10). Always overwrite thumbnail_url/animated_thumbnail_url
with the freshly-scraped value when present; other fields keep fill-if-null. Lets
the regular performer-driven ingest self-heal thumbnails for re-crawled scenes.
(Note: old sxyprn backlog can't be bulk-refreshed — search/listings don't re-surface
those posts, verified 0 overlap — so it's forward-looking; old sxyprn-only scenes
fall back to the clean placeholder.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
sxyprn thumbnails are time-signed on trafficdeposit CDN and ROT — the signed asset
404s after ~weeks and can't be re-signed/refreshed server-side (bug 2026-06-10,
~15k sxyprn-only scenes showed broken thumbs). In the light-list slim-thumbnail pick,
prefer a thumbnail from any non-trafficdeposit source; fall back to sxyprn only when
it's the scene's sole thumbnail (recent ones still load; dead ones now render a clean
placeholder client-side instead of a broken image).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Public instance has no accounts, so all user state was GLOBAL in DB — new users
saw/overwrote each other's (and Jan's) favorites, watched badges and blacklists
(bug 2026-06-10). Add device_id (VARCHAR 64) to 9 state tables with composite PK
(device_id, entity_id); app sends X-Device-Id header (get_device_id dep). All
favorites/scene-favorites/blacklist/watch + scene&movie list/detail (is_favorite,
watched, blacklist-hide) now filter by device. Existing rows backfilled to
'legacy-shared'; POST /me/adopt-legacy reassigns them to the caller once. Old
clients (no header) map to legacy-shared so they keep working until OTA updates.
Migration 0022: add col, backfill, composite PK. Verified on prod: 967 progress
rows preserved, device isolation holds (new device sees none of legacy state).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The hold-to-preview gesture is being removed (did nothing useful), and no client
sends this filter. Remove the Query param, its EXISTS filter, and the pure-default
count guard reference.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
POST /scenes/{id}/hide — marks all playback_sources dead so the scene drops out
of has_playback lists (reversible via dead_at; row kept for dedup/refs).
POST /scenes/{keep_id}/merge/{drop_id} — merges drop into keep via scene_merge
(moves refs/performers/tags/fingerprints/playback). Backs the new tile long-press
menu (hide / mark-duplicate) replacing the dead animated-preview gesture.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
make_token embedded the current timestamp in the expiry, so every /scenes fetch
produced a DIFFERENT proxied URL for the same thumbnail → expo-image (keyed by URI)
cache-missed and re-downloaded every list load / app launch. Add stable_bucket_sec:
quantize the expiry base to a window so the URL is identical across requests.
_wrap_image_proxy uses a 7-day bucket → thumbnails disk-cache for a week instead of
re-fetching constantly. Answers "czy miniatury są cache'owane" — now yes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
xhamster: move from WebView fallback to server-side native HLS. The scene page
is fetchable server-side and the xhcdn master m3u8 (variants + segments) is
time-bound, not IP-bound (verified cross-IP), so mobile plays the HLS direct
with zero proxy bandwidth. New tubes/xhamster.py pulls the master m3u8 from
SSR HTML and returns type='m3u8' mobile_direct; registry remaps xhamstercom
off _vps_blocked_fallback.
hqporner: add flyflv to the player-iframe host whitelist. hqporner rotated
some players to flyflv.com; the CDN host was already whitelisted but the iframe
host was not, so those scenes returned no stream.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
bug-report 1a4bf258: "Re-scrape mógłby zniknąć, za to tagi/kategorie by mogły".
Re-scrape was a dev-only bulk thumbnail/tag enrich — noise on the performer page
(per-scene enrich already happens on SceneDetail). Removed it; kept Search.
New GET /performers/{id}/tags aggregates scene_tags across the performer's
live-playback scenes (top N). PerformerScenes renders them as chips → tap navigates
to TagScenes. Search button widened to full row.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A merged scene often aggregates several uploads from ONE tube (re-encodes / 4K
dups). bug-report aa79a995 "why 2 links, both porntrex?" = same scene std + 4K
(porntrex 2591377 + 2593449 "...in 4K"). In the UI these are indistinguishable
links to one hoster (same extractor). Keep one best per origin: prefer duration
matching the scene → any duration → first (origin-asc stable). Dead already filtered.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Chrome-DevTools investigation of bug-report 827a50a1 (sxyland "long loading,
then webview, no autoplay") showed sxyland embeds playmogo.com/e/<id> — a
DoodStream clone (doodcdn.io infra, pass_md5 protocol, get_slides) behind an
INVISIBLE Cloudflare Turnstile (not an interactive CAPTCHA; auto-passes in a
real browser/WebView from a residential IP). The sxyland page itself is NOT
Turnstile-gated — VPS curl pulls the playmogo iframe URL straight from the HTML.
sxylandcom was wired to _vps_blocked_fallback → phone loaded the entire sxyland
page in WebView (ads, click-to-play, no autoplay = the reported symptom), and the
playmogo embed never reached the phone's dood resolver. _embed_iframe (which
already lists sxyland in its docstring) extracts the playmogo embed and emits it
as type='hoster' → PlayerScreen routes playmogo URLs to doodstream.ts (resolveDoodStream),
which resolves phone-side (phone IP passes invisible Turnstile) → direct mp4 → autoplay.
Mobile unchanged (hoster→dood path already exists for xmoviesforyou/siska). Backend-only.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
