goon/app
goon-foss b3e1092175 feat(pornmike): browse scraper behind a cast gate + direct-mp4 extractor
pornmike.com puts everything in JSON-LD @graph -> ItemPage.mainEntity
(name, duration, uploadDate, actor[], genre[], keywords, description,
thumbnail), so the parser reads one JSON blob instead of scraping markup.
Cast is clean: the whole scene page carries exactly as many /pornstar/
links as there are actors, none of the sidebar pollution that got xxxfiles
rejected. 82% of sampled performers already carry a tpdb/stashdb ref and
19-20 of 20 channels are studios we already know.

Ingest is gated on a non-empty actor[]: 23% of the catalog has no cast and
those scenes could neither be attributed nor deduped. Tags and categories
come only from JSON-LD (keywords + genre), never from the HTML, which
carries 38 /category/ and 22 /tag/ nav and sidebar links per page.
Pagination is ?p=N only: the /N/ form 404s and ?page=N is silently ignored,
returning page 1.

Playback is the simplest in the portfolio: a plain <source> mp4 on twincdn
with no token, no query string and no expiry. Verified 206 on a Range
request from the VPS and from another machine in another country, both
without a Referer, so it is neither hotlink-guarded nor IP-bound and the
phone streams it directly.

Two honest caveats recorded in the module docstring: these are 5-12 minute
clips (median ~487s against 1800-2400s for the tubes we accepted), so for
the ~20% of the catalog from Tushy/Blacked Raw/Milfy/Anilos they will sit
as a short shadow next to full canonical scenes; and uploadDate is the
tube's import date, not the studio release, so pages past the second are
marked backfill.

Pilot ingest of 3 pages: 127 seen, 49 merged into existing scenes, 66 new,
0 errors; on page one the gate passed 42 of 62 links with zero scenes
missing cast, duration or studio.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-07-27 10:42:39 +02:00
..
api feat(pornbusy): browse scraper + loadvid POST-manifest producer 2026-07-27 08:45:00 +02:00
connectors feat(pornmike): browse scraper behind a cast gate + direct-mp4 extractor 2026-07-27 10:42:39 +02:00
enrich fix(movie-enrich): studio/performer disambiguation for generic TPDB titles 2026-07-05 11:31:38 +02:00
extractors feat(pornmike): browse scraper behind a cast gate + direct-mp4 extractor 2026-07-27 10:42:39 +02:00
models fix(watchporn): re-resolve stale CDN URL in-player, drop WebView page fallback 2026-07-05 01:18:08 +02:00
normalize feat(ingest): SQL phash match, tag inference + backfill, clip-store skip, browse tubes, watchdog 2026-06-01 15:07:35 +02:00
resolve fix(tags): collapse spelling-variant duplicate tags + prevent regrowth 2026-07-21 15:59:33 +02:00
scheduler feat(youperv): browse scraper + direct-mp4 extractor 2026-07-26 17:57:24 +02:00
templates feat(seo): public HTML SEO router + templates; add CLAUDE.md; ignore .nimbalyst 2026-05-31 16:29:59 +02:00
__init__.py Initial commit 2026-05-20 10:10:22 +02:00
auth.py Initial commit 2026-05-20 10:10:22 +02:00
config.py feat(movies): TPDB movie enrichment + dedup 2026-07-02 11:36:36 +02:00
db.py Initial commit 2026-05-20 10:10:22 +02:00
ingest.py fix(favorites): exclude bulk-backfill scenes from "+N new" (tube fake dates) 2026-07-01 16:04:01 +02:00
main.py feat(sources): 0-5★ ranking on Sites (freshness/metadata/plays) + playback telemetry 2026-06-22 10:00:59 +02:00