d716b0c75d
pornbusy.com exposes every field as a discrete itemprop node (title, duration, uploadDate, thumbnail, description) plus a dedicated <div id="video-actors"> cast block with no sidebar pollution. Measured orphan risk is low: 80% of a 100-scene sample have a performer that already carries a tpdb/stashdb ref, and 21% of titles strongly match scenes we already hold. No studio field exists on the site. Homepage pagination is broken (pages 1/2/3 return an identical set), so the listing is driven off sitemap_index.xml -> 10 post-sitemaps sorted by lastmod, chunked in crawl_page the same way the PlayTube base does it. Playback dispatches on embedURL and covers ~69% of the catalog: the seekplayer family (already handled by the engine after the earlier host regex widening) and zpi.cx (the embedURL is the file itself), plus loadvid at ~41%, which needed new plumbing. loadvid hands back the CONTENT of an m3u8 over POST /videos/resolve-token (CSRF + videoToken from the embed page) and has no manifest URL at all: GET on that endpoint is 405 and the guessable .m3u8 paths are 404. So make_token grew an optional `producer` marker and /proxy/hls calls the producer instead of GETting a URL. Segments in the returned manifest are absolute and fully portable (verified: 206 on a Range request with no headers, no referer, no token), so the phone still pulls them straight from the CDN and only the manifest travels through the VPS. upload18 (~12%) and the tail are deliberately left unresolved: their token embeds the fetcher's /24, so resolving server-side would force the whole video through the VPS. Verified: 19 scenes/page, 19/19 with duration, 17/19 with cast, 19/19 with tags (names read from the title attribute since an icon element precedes the anchor text), playback 8/8 via loadvid, and /proxy/hls returning a 116KB manifest with 869 absolute segments. Pilot ingest of 3 pages: 59 seen, 35 merged into existing scenes, 24 new, 0 errors. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
82 lines
3 KiB
Python
82 lines
3 KiB
Python
"""pornbusy.com — dispatch po hosterze z `itemprop="embedURL"`.
|
|
|
|
Katalog jest rozproszony po hosterach (próbka 120 scen): ~41% loadvid,
|
|
~20% rodzina seekplayer (av.ezplayer.me / av.seeks.cloud / 4k.upn.one /
|
|
4k.player4me.vip — objęte poszerzonym `_HOST_RE` silnika), ~8% zpi.cx,
|
|
~12% upload18 (token z wbitym `i=<ip>/24` = IP-bound), reszta ogon.
|
|
|
|
Obsługujemy trzy pierwsze (~69% katalogu):
|
|
- loadvid → manifest przez POST, marker `manifest_producer` (patrz hosters/loadvid.py)
|
|
- seekplayer → istniejący silnik (ten sam AES key/IV)
|
|
- zpi.cx → `embedURL` JEST plikiem wideo (mimo `.webm` w nazwie to mp4);
|
|
zweryfikowane: Range 206 `video/mp4`, bez Referera, bez tokena
|
|
|
|
upload18 i ogon zwracamy jako None — resolve na VPS wymusiłby proxowanie CAŁEGO
|
|
wideo (token IP-bound), co łamie zasadę no-video-proxy.
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import logging
|
|
import re
|
|
from urllib.parse import urlparse
|
|
|
|
from app.extractors._fetch import fetch_tube_html
|
|
from app.extractors._models import StreamSource
|
|
from app.extractors.hosters import loadvid, seekplayer_engine
|
|
|
|
log = logging.getLogger(__name__)
|
|
|
|
_BASE = "https://pornbusy.com"
|
|
_EMBED_RE = re.compile(r'itemprop="embedURL"\s+content="([^"]+)"', re.IGNORECASE)
|
|
_IFRAME_RE = re.compile(r'<iframe[^>]+src="([^"]+)"', re.IGNORECASE)
|
|
_ZPI_HOST_RE = re.compile(r"^(?:[a-z0-9]+\.)?zpi\.cx$", re.IGNORECASE)
|
|
|
|
|
|
def extract(page_url: str, *, timeout: float = 60.0) -> list[StreamSource] | None:
|
|
html_text = fetch_tube_html(page_url, timeout=timeout)
|
|
m = _EMBED_RE.search(html_text) or _IFRAME_RE.search(html_text)
|
|
if not m:
|
|
log.info("pornbusy: no embedURL/iframe on %s", page_url)
|
|
return None
|
|
embed = m.group(1).strip()
|
|
if embed.startswith("//"):
|
|
embed = "https:" + embed
|
|
|
|
if loadvid.matches(embed):
|
|
return loadvid.extract(embed, timeout=timeout)
|
|
|
|
if seekplayer_engine.matches(embed):
|
|
sources = seekplayer_engine.extract(embed, timeout=timeout)
|
|
if not sources:
|
|
return None
|
|
player_origin = f"https://{urlparse(embed).hostname}/"
|
|
out: list[StreamSource] = []
|
|
for s in sources:
|
|
raw = dict(s.raw or {})
|
|
raw["proxy_no_verify"] = True
|
|
out.append(
|
|
StreamSource(
|
|
link=s.link,
|
|
type=s.type or "m3u8",
|
|
quality=s.quality,
|
|
# Referer = origin PLAYERA, nie strony (jak przy galaxyporn:
|
|
# z Refererem strony CDN zwraca 403).
|
|
referer=s.referer or player_origin,
|
|
raw=raw,
|
|
)
|
|
)
|
|
return out
|
|
|
|
host = (urlparse(embed).hostname or "").lower()
|
|
if _ZPI_HOST_RE.match(host):
|
|
# embedURL to bezpośrednio plik (rozszerzenie `.webm` myli — to mp4).
|
|
return [
|
|
StreamSource(
|
|
link=embed,
|
|
type="mp4",
|
|
raw={"mobile_direct_ok": True},
|
|
)
|
|
]
|
|
|
|
log.info("pornbusy: unsupported hoster %s (%s)", host, page_url)
|
|
return None
|