goon/app/connectors
goon-foss b3e1092175 feat(pornmike): browse scraper behind a cast gate + direct-mp4 extractor
pornmike.com puts everything in JSON-LD @graph -> ItemPage.mainEntity
(name, duration, uploadDate, actor[], genre[], keywords, description,
thumbnail), so the parser reads one JSON blob instead of scraping markup.
Cast is clean: the whole scene page carries exactly as many /pornstar/
links as there are actors, none of the sidebar pollution that got xxxfiles
rejected. 82% of sampled performers already carry a tpdb/stashdb ref and
19-20 of 20 channels are studios we already know.

Ingest is gated on a non-empty actor[]: 23% of the catalog has no cast and
those scenes could neither be attributed nor deduped. Tags and categories
come only from JSON-LD (keywords + genre), never from the HTML, which
carries 38 /category/ and 22 /tag/ nav and sidebar links per page.
Pagination is ?p=N only: the /N/ form 404s and ?page=N is silently ignored,
returning page 1.

Playback is the simplest in the portfolio: a plain <source> mp4 on twincdn
with no token, no query string and no expiry. Verified 206 on a Range
request from the VPS and from another machine in another country, both
without a Referer, so it is neither hotlink-guarded nor IP-bound and the
phone streams it directly.

Two honest caveats recorded in the module docstring: these are 5-12 minute
clips (median ~487s against 1800-2400s for the tubes we accepted), so for
the ~20% of the catalog from Tushy/Blacked Raw/Milfy/Anilos they will sit
as a short shadow next to full canonical scenes; and uploadDate is the
tube's import date, not the studio release, so pages past the second are
marked backfill.

Pilot ingest of 3 pages: 127 seen, 49 merged into existing scenes, 66 new,
0 errors; on page one the gate passed 42 of 62 links with zero scenes
missing cast, duration or studio.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-07-27 10:42:39 +02:00
..
direct_scrapers feat(pornmike): browse scraper behind a cast gate + direct-mp4 extractor 2026-07-27 10:42:39 +02:00
__init__.py feat(movies): add freeomovie.to as a movie mirror 2026-07-02 14:39:42 +02:00
base.py Initial commit 2026-05-20 10:10:22 +02:00
dooplay.py feat(movies): add streamporn.vip mirror, retire dead streamporn.nl 2026-07-02 14:27:21 +02:00
freeomovie.py fix(feedback): freeomovie posters + dead hoster + myvidplay; watchporn _embed_iframe guard 2026-07-03 13:02:47 +02:00
paradisehill.py fix(movies): paradisehill delta date-granularity + browse cadence docs 2026-06-01 17:00:10 +02:00
stashdb.py Initial commit 2026-05-20 10:10:22 +02:00
tpdb.py feat(movies): TPDB movie enrichment + dedup 2026-07-02 11:36:36 +02:00