goon/app/extractors/tubes
goon-foss b3e1092175 feat(pornmike): browse scraper behind a cast gate + direct-mp4 extractor
pornmike.com puts everything in JSON-LD @graph -> ItemPage.mainEntity
(name, duration, uploadDate, actor[], genre[], keywords, description,
thumbnail), so the parser reads one JSON blob instead of scraping markup.
Cast is clean: the whole scene page carries exactly as many /pornstar/
links as there are actors, none of the sidebar pollution that got xxxfiles
rejected. 82% of sampled performers already carry a tpdb/stashdb ref and
19-20 of 20 channels are studios we already know.

Ingest is gated on a non-empty actor[]: 23% of the catalog has no cast and
those scenes could neither be attributed nor deduped. Tags and categories
come only from JSON-LD (keywords + genre), never from the HTML, which
carries 38 /category/ and 22 /tag/ nav and sidebar links per page.
Pagination is ?p=N only: the /N/ form 404s and ?page=N is silently ignored,
returning page 1.

Playback is the simplest in the portfolio: a plain <source> mp4 on twincdn
with no token, no query string and no expiry. Verified 206 on a Range
request from the VPS and from another machine in another country, both
without a Referer, so it is neither hotlink-guarded nor IP-bound and the
phone streams it directly.

Two honest caveats recorded in the module docstring: these are 5-12 minute
clips (median ~487s against 1800-2400s for the tubes we accepted), so for
the ~20% of the catalog from Tushy/Blacked Raw/Milfy/Anilos they will sit
as a short shadow next to full canonical scenes; and uploadDate is the
tube's import date, not the studio release, so pages past the second are
marked backfill.

Pilot ingest of 3 pages: 127 seen, 49 merged into existing scenes, 66 new,
0 errors; on page one the gate passed 42 of 62 links with zero scenes
missing cast, duration or studio.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-07-27 10:42:39 +02:00
..
__init__.py Initial commit 2026-05-20 10:10:22 +02:00
_embed_iframe.py feat(jav): javflix.cc scraper + extractor (first JAV source, gated off main feed) 2026-07-10 10:58:34 +02:00
_kvs.py fix(porn00): backend KVS resolve (portable CDN, no proxy) — corrects #20 2026-06-05 21:15:19 +02:00
_kvs_source.py Initial commit 2026-05-20 10:10:22 +02:00
_source_getfile.py fix(hdporngg+fullmovies): native get_file, skip broken 4K — "loading forever" 2026-06-05 22:48:55 +02:00
_txxx.py feat(fullvideosporn): browse scraper behind a cast gate + TXXX extractor 2026-07-27 08:56:23 +02:00
_vps_blocked_fallback.py Initial commit 2026-05-20 10:10:22 +02:00
_ytdlp.py Initial commit 2026-05-20 10:10:22 +02:00
eporner.py Initial commit 2026-05-20 10:10:22 +02:00
freshporno.py fix(freshporno): backend KVS resolve (portable CDN) — corrects #20 2026-06-05 21:12:17 +02:00
fullmovies.py fix(hdporngg+fullmovies): native get_file, skip broken 4K — "loading forever" 2026-06-05 22:48:55 +02:00
fullvideosporn.py feat(fullvideosporn): browse scraper behind a cast gate + TXXX extractor 2026-07-27 08:56:23 +02:00
galaxyporn.py feat(galaxyporn): browse scraper + extractor; widen seekplayer host regex 2026-07-26 18:23:13 +02:00
hdporngg.py fix(hdporngg+fullmovies): native get_file, skip broken 4K — "loading forever" 2026-06-05 22:48:55 +02:00
hqfap.py feat(hqfap): restore hqfap.com scraper + extractor (site back on new CDN) 2026-07-07 23:53:43 +02:00
hqporner.py feat(extractors): native HLS for xhamster; hqporner flyflv player 2026-06-09 09:35:58 +02:00
javflix.py feat(jav): javflix.cc scraper + extractor (first JAV source, gated off main feed) 2026-07-10 10:58:34 +02:00
latestpornvideo.py Initial commit 2026-05-20 10:10:22 +02:00
neporn.py feat(tubes): add 4k69 + neporn browse scrapers, shared PlayTube base 2026-06-10 18:15:13 +02:00
paradisehill.py Initial commit 2026-05-20 10:10:22 +02:00
porn00.py fix(porn00): backend KVS resolve (portable CDN, no proxy) — corrects #20 2026-06-05 21:15:19 +02:00
pornbusy.py feat(pornbusy): browse scraper + loadvid POST-manifest producer 2026-07-27 08:45:00 +02:00
pornmike.py feat(pornmike): browse scraper behind a cast gate + direct-mp4 extractor 2026-07-27 10:42:39 +02:00
porntrex.py feat(extractors): detect deleted porntrex videos and mark dead 2026-06-13 19:04:10 +02:00
pornxp.py Initial commit 2026-05-20 10:10:22 +02:00
supjav.py feat(supjav): resolve voe hoster to direct stream instead of WebView 2026-07-11 19:52:26 +02:00
sxyprn.py fix(playback): mark deleted sxyprn posts dead + rank native sources first 2026-06-07 14:09:01 +02:00
vjav.py feat(fullvideosporn): browse scraper behind a cast gate + TXXX extractor 2026-07-27 08:56:23 +02:00
watchporn.py feat(watchporn): re-enable as KVS browse scraper + native mp4 extractor 2026-07-02 14:32:36 +02:00
xhamster.py feat(extractors): native HLS for xhamster; hqporner flyflv player 2026-06-09 09:35:58 +02:00
yespornvip.py fix(pornditt): server-side KVS resolve; extract shared _kvs helper 2026-06-01 10:36:33 +02:00
youperv.py feat(youperv): browse scraper + direct-mp4 extractor 2026-07-26 17:57:24 +02:00