goon-foss/goon
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(thumbnails): watchporn thumbs at ingest + backfill; SceneDetail uses live sxyprn thumb

Browse source 
Two bug-report clusters about missing thumbnails:

1. watchporn scenes had no thumbnail on the list, only appearing after opening
   SceneDetail (which auto-enriches og:image). Coverage was 1.3% (463/36,915).
   The browse scraper never captured a thumbnail. KVS stores the poster at a fixed
   derivable path (contents/videos_screenshots/<id//1000*1000>/<id>/preview.jpg,
   verified 8/8 loading). Scraper now sets thumbnail_url (og:image, else derived);
   backfilled 36,687 existing rows -> 100% coverage.

2. SceneDetail showed no thumb where the list showed one: the mobile detail picks
   the first source with a thumbnail_url (origin ASC often puts sxyprncom first),
   and sxyprn/trafficdeposit stored thumbs rot to 404. The list already swaps those
   for a live resolver (/proxy/sxyprn-thumb/), but the detail builder did not. It
   now applies the same live-resolver swap and nulls other rotting thumbs so the
   detail lands on a working image.

Both backend-only, no OTA needed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
goon-foss 2026-07-07 23:21:59 +02:00
parent 8ea8b94b4a
commit cb22174657
2 changed files with 30 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

10
app/api/scenes.py
View file

@ -647,6 +647,16 @@ def _build_scenes_out_batch(
).scalars().all()
for p in pb_rows:
out = PlaybackSourceOut.model_validate(p)
# sxyprn/trafficdeposit stored thumb rotuje (404 po ~tygodniach). SceneDetail
# bierze PIERWSZE źródło z thumbem (origin ASC → sxyprncom często pierwsze), więc
# bez tego pokazywał martwy obraz mimo że LISTA używała żywej wersji (report:
# "na liście jest miniaturka, w scenie nie"). Podmieniamy na żywy resolver;
# inne martwe rotting-thumby zerujemy → mobile bierze kolejne źródło / placeholder.
sxy = _sxyprn_thumb_url(p.page_url)
if sxy:
out.thumbnail_url = sxy
elif out.thumbnail_url and _is_rotting_thumb(out.thumbnail_url):
out.thumbnail_url = None
if out.thumbnail_url and _needs_proxy(out.thumbnail_url):
out.thumbnail_url = _wrap_image_proxy(out.thumbnail_url, p.page_url)
if out.animated_thumbnail_url and _needs_proxy(out.animated_thumbnail_url):

21
app/connectors/direct_scrapers/watchporn.py
View file

@ -34,6 +34,18 @@ _TAG_RE = re.compile(r'href="https://watchporn\.to/tags/([a-z0-9\-]+)/"[^>]*>([^
_CAT_RE = re.compile(r'href="https://watchporn\.to/categories/([a-z0-9\-]+)/"[^>]*>([^<]+)', re.IGNORECASE)
_DUR_RE = re.compile(r'"duration"\s*:\s*"([^"]+)"')
_UPLOAD_RE = re.compile(r'"uploadDate"\s*:\s*"([^"]+)"')
_VIDEO_ID_RE = re.compile(r"/video/(\d+)/")
def _derive_thumb(scene_url: str) -> str | None:
"""KVS trzyma poster pod stałym wzorem `contents/videos_screenshots/<id//1000*1000>/
<id>/preview.jpg` (zweryfikowane). Fallback gdy detail page nie ma og:image, żeby
kafelek na liście miał miniaturkę OD RAZU (bez czekania na auto-enrich w SceneDetail)."""
m = _VIDEO_ID_RE.search(scene_url)
if not m:
return None
vid = int(m.group(1))
return f"{_BASE}/contents/videos_screenshots/{vid // 1000 * 1000}/{vid}/preview.jpg"
class WatchPornScraper(BaseBrowseScraper):
@ -82,6 +94,10 @@ class WatchPornScraper(BaseBrowseScraper):
seen_t.add(slug)
tags.append(RawTag(external_id=f"{self.sitetag}:tag:{slug}", name=name, slug=slug))
thumbnail_url = (
meta_content(detail_html, property="og:image") or _derive_thumb(scene_url)
)
studio: RawStudio | None = None
cm = _CAT_RE.search(detail_html)
if cm:
@ -102,7 +118,10 @@ class WatchPornScraper(BaseBrowseScraper):
tags=tags,
playback_sources=[
RawPlaybackSource(
origin=f"tube:{self.sitetag}", page_url=scene_url, duration_sec=duration_sec
origin=f"tube:{self.sitetag}",
page_url=scene_url,
duration_sec=duration_sec,
thumbnail_url=thumbnail_url,
)
],
)