28f56d646c
Some checks are pending
Backend tests / test (push) Waiting to run
hqfap was hard-removed 2026-06-25 when its entire CDN library started serving a fixed /upload/videos/video_down.mp4 "server down" stub for every scene. The site is back and now serves real video from a new CDN (d*.vstor.top/whlvid/...). Probe of the live sitemap: 5/6 sampled scenes return a real mp4 (0 stubs), and the CDN is portable cross-IP (HTTP 206 from a residential IP and from the VPS with the hqfap referer). Restores the PlayTube CMS browse scraper (HQFapScraper, sitetag hqfapcom) and the JSON-LD contentUrl extractor, re-registered in both registries. The extractor now sets mobile_direct_ok=True (vstor.top is not matched by _TIME_BOUND_CDN_RE, but is verified portable) so mobile plays direct with zero proxy. The video_down.mp4 stub guard is kept defensively. 4k69 stays removed (not re-checked). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
84 lines
3.2 KiB
Python
84 lines
3.2 KiB
Python
"""hqfap.com — direct stream extractor.
|
|
|
|
Scene page (SSR, za Cloudflare → curl_cffi w fetch_tube_html) ma JSON-LD
|
|
VideoObject z `contentUrl` = direct mp4. Generacje hostingu w katalogu:
|
|
|
|
- obecne sceny (2026-07-07): `d*.vstor.top/whlvid/<epoch>/<token>/.../<id>_<q>p.mp4`
|
|
— token time-bound, PORTABLE cross-IP (206 z residential ISP i VPS Hetzner,
|
|
weryfikacja 2026-07-07 po powrocie strony),
|
|
- starsze: `v4.cdnde.com/...?video=<b64>&time=<epoch>&ip=<addr>` (`ip` nieegzekwowany)
|
|
oraz `vd*.okcdn.ru/?expires=...&srcIp=...` (ok.ru) — również portable cross-IP.
|
|
|
|
Wszystkie generacje grają direct z telefonu → StreamSource ma `mobile_direct_ok`
|
|
(vstor.top nie łapie się w `_TIME_BOUND_CDN_RE` w playback.py, więc flagujemy jawnie),
|
|
zero proxy/WebView.
|
|
|
|
Historia: wyłączony 2026-06-22 i USUNIĘTY 2026-06-25 gdy CAŁA biblioteka CDN
|
|
serwowała stały `/upload/videos/video_down.mp4` stub. Strona wróciła na nowy CDN
|
|
(vstor.top) z realnymi plikami → przywrócony 2026-07-07 (user request). Guard na
|
|
stub zostaje defensywnie.
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
import logging
|
|
import re
|
|
|
|
from app.extractors._fetch import fetch_tube_html
|
|
from app.extractors._models import StreamSource
|
|
|
|
log = logging.getLogger(__name__)
|
|
|
|
_JSONLD_RE = re.compile(
|
|
r'<script[^>]+type=["\']application/ld\+json["\'][^>]*>(.*?)</script>',
|
|
re.IGNORECASE | re.DOTALL,
|
|
)
|
|
# Fallback gdy JSON-LD nie parsuje się jako JSON (trailing comma itp.).
|
|
_CONTENT_URL_RE = re.compile(r'"contentUrl"\s*:\s*"([^"]+)"')
|
|
_QUALITY_RE = re.compile(r"_(\d{3,4})p\.mp4", re.IGNORECASE)
|
|
|
|
|
|
def extract(page_url: str, *, timeout: float = 60.0) -> list[StreamSource] | None:
|
|
html = fetch_tube_html(page_url, timeout=timeout)
|
|
|
|
content_url: str | None = None
|
|
for m in _JSONLD_RE.finditer(html):
|
|
raw = m.group(1).strip()
|
|
if not raw:
|
|
continue
|
|
try:
|
|
data = json.loads(raw)
|
|
except (json.JSONDecodeError, ValueError):
|
|
continue
|
|
items = data if isinstance(data, list) else [data]
|
|
for obj in items:
|
|
if isinstance(obj, dict) and obj.get("@type") == "VideoObject":
|
|
content_url = (obj.get("contentUrl") or "").strip() or None
|
|
break
|
|
if content_url:
|
|
break
|
|
if not content_url:
|
|
rm = _CONTENT_URL_RE.search(html)
|
|
content_url = rm.group(1).strip() if rm else None
|
|
if not content_url or not content_url.startswith("http"):
|
|
log.warning("hqfap: no contentUrl in JSON-LD for %s", page_url)
|
|
return None
|
|
|
|
# Guard: `/upload/videos/video_down.mp4` (+ mirror *.workers.dev) to stały ~3MB
|
|
# "server down" placeholder (powód usunięcia 2026-06-25). Jeśli wróci — traktuj
|
|
# jak brak źródła (lepiej żadne niż stub).
|
|
if "/upload/videos/video_down.mp4" in content_url:
|
|
log.info("hqfap: stub video_down.mp4 (placeholder, no real video) on %s", page_url)
|
|
return None
|
|
|
|
qm = _QUALITY_RE.search(content_url)
|
|
quality = f"{qm.group(1)}p" if qm else None
|
|
return [
|
|
StreamSource(
|
|
link=content_url,
|
|
quality=quality,
|
|
type="mp4",
|
|
referer="https://hqfap.com/",
|
|
raw={"mobile_direct_ok": True},
|
|
)
|
|
]
|