No description
Find a file
goon-foss b3e1092175 feat(pornmike): browse scraper behind a cast gate + direct-mp4 extractor
pornmike.com puts everything in JSON-LD @graph -> ItemPage.mainEntity
(name, duration, uploadDate, actor[], genre[], keywords, description,
thumbnail), so the parser reads one JSON blob instead of scraping markup.
Cast is clean: the whole scene page carries exactly as many /pornstar/
links as there are actors, none of the sidebar pollution that got xxxfiles
rejected. 82% of sampled performers already carry a tpdb/stashdb ref and
19-20 of 20 channels are studios we already know.

Ingest is gated on a non-empty actor[]: 23% of the catalog has no cast and
those scenes could neither be attributed nor deduped. Tags and categories
come only from JSON-LD (keywords + genre), never from the HTML, which
carries 38 /category/ and 22 /tag/ nav and sidebar links per page.
Pagination is ?p=N only: the /N/ form 404s and ?page=N is silently ignored,
returning page 1.

Playback is the simplest in the portfolio: a plain <source> mp4 on twincdn
with no token, no query string and no expiry. Verified 206 on a Range
request from the VPS and from another machine in another country, both
without a Referer, so it is neither hotlink-guarded nor IP-bound and the
phone streams it directly.

Two honest caveats recorded in the module docstring: these are 5-12 minute
clips (median ~487s against 1800-2400s for the tubes we accepted), so for
the ~20% of the catalog from Tushy/Blacked Raw/Milfy/Anilos they will sit
as a short shadow next to full canonical scenes; and uploadDate is the
tube's import date, not the studio release, so pages past the second are
marked backfill.

Pilot ingest of 3 pages: 127 seen, 49 merged into existing scenes, 66 new,
0 errors; on page one the gate passed 42 of 62 links with zero scenes
missing cast, duration or studio.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-07-27 10:42:39 +02:00
.github/workflows Initial commit 2026-05-20 10:10:22 +02:00
alembic fix(tags): collapse spelling-variant duplicate tags + prevent regrowth 2026-07-21 15:59:33 +02:00
app feat(pornmike): browse scraper behind a cast gate + direct-mp4 extractor 2026-07-27 10:42:39 +02:00
landing Mobile 0.1.9: OTA enable, WebView cookie-dismiss fix, porndoe connector 2026-05-22 11:20:57 +02:00
mobile fix: keep-screen-on podczas odtwarzania + on-demand miniaturki youporn 2026-07-26 17:00:31 +02:00
scripts fix(triage): theporndude candidate audit had two false-negative bugs 2026-07-27 09:37:51 +02:00
tests Mobile 0.1.9: OTA enable, WebView cookie-dismiss fix, porndoe connector 2026-05-22 11:20:57 +02:00
.env.example Initial commit 2026-05-20 10:10:22 +02:00
.gitignore chore: remove internal host/IP references from comments; untrack local dev notes 2026-07-11 18:59:38 +02:00
alembic.ini Initial commit 2026-05-20 10:10:22 +02:00
CONTRIBUTING.md Initial commit 2026-05-20 10:10:22 +02:00
DISCLAIMER.md Initial commit 2026-05-20 10:10:22 +02:00
docker-compose.yml fix(db): set shm_size 1g — parallel queries overflow default 64MB /dev/shm 2026-06-06 22:56:49 +02:00
Dockerfile Initial commit 2026-05-20 10:10:22 +02:00
LICENSE Initial commit 2026-05-20 10:10:22 +02:00
pyproject.toml Initial commit 2026-05-20 10:10:22 +02:00
README.md Initial commit 2026-05-20 10:10:22 +02:00
streamtape_sample.md Initial commit 2026-05-20 10:10:22 +02:00
theporndude_coverage.json theporndude audit: scorecards, coverage + raporty 2026-05-22 13:44:54 +02:00
theporndude_dump.json theporndude audit: scorecards, coverage + raporty 2026-05-22 13:44:54 +02:00
theporndude_final_report.md theporndude audit: scorecards, coverage + raporty 2026-05-22 13:44:54 +02:00
theporndude_free_tubes.json theporndude audit: scorecards, coverage + raporty 2026-05-22 13:44:54 +02:00
theporndude_full_audit_report.md theporndude audit: scorecards, coverage + raporty 2026-05-22 13:44:54 +02:00
theporndude_movies.json theporndude audit: scorecards, coverage + raporty 2026-05-22 13:44:54 +02:00
theporndude_movies_scorecard.json theporndude audit: scorecards, coverage + raporty 2026-05-22 13:44:54 +02:00
theporndude_resolved.json theporndude audit: scorecards, coverage + raporty 2026-05-22 13:44:54 +02:00
theporndude_scorecard.json theporndude audit: scorecards, coverage + raporty 2026-05-22 13:44:54 +02:00
theporndude_triage.json fix(triage): theporndude candidate audit had two false-negative bugs 2026-07-27 09:37:51 +02:00

Goon

Self-hosted aggregator for adult-content scene metadata. Indexes scenes from TheporndB, StashDB, and 30+ public adult tube sites; deduplicates across sources; serves an API + mobile (React Native) client for browsing and linking out to playback.

18+ ONLY · Self-hosted only · See DISCLAIMER.md before hosting an instance.


What it does

  • Multi-source ingest: pulls canonical scene/performer/studio metadata from TPDB and StashDB on a delta cron, merges duplicates by performer + title + date heuristics (perceptual hash + Levenshtein title distance).
  • Tube discovery: per-performer search across 30+ public adult tube sites (mainstream + aggregators). Each tube is scraped directly via HTTP — no proprietary API dependencies.
  • Stream resolution on demand: when a user clicks Watch, the API extracts a fresh m3u8/mp4 URL from the tube's page (or falls back to embed link for WebView playback). Mainstream tubes use yt-dlp; aggregator tubes use a generic P.A.C.K.E.R. unpacker for JWPlayer-based hosters (StreamWish/doodporn/mixdrop/...).
  • Mobile client (Expo / React Native): scene grid, performer pages, watch history, favorites, hold-to-preview animated thumbnails.
  • Performer-driven backfill: a continuous worker walks performers ordered by last_searched_at NULLS FIRST and back-fills tube scenes for the longest-stale performer first.

What it doesn't do

  • Host or store any media. Scene metadata + thumbnail URLs only.
  • Bypass paywalls, authentication, geo-blocks, or DRM.
  • Provide age verification, ToS gating, or moderation for public deployments. See DISCLAIMER.md.
  • Phone home. Sentry telemetry is opt-in (env var, empty by default).

Quick start

1. Run the backend (Docker)

git clone <repo-url> goon
cd goon
cp .env.example .env
# Edit .env:
#   - TPDB_API_TOKEN       (theporndb.net account → API tokens)
#   - STASHDB_API_KEY      (stashdb.org account → API keys)
#   - API_KEYS             (generate one: python -c "import secrets; print(secrets.token_urlsafe(32))")

docker compose up -d

Three services come up: db (Postgres 16), api (FastAPI on :8000, auto-applies migrations on startup), worker (APScheduler running TPDB/StashDB delta + performer-driven backfill).

Verify: curl localhost:8000/health{"status":"ok"}.

2. Install the mobile app (Android)

Download the latest debug APK from GitHub Releasesgoon-vX.Y.Z-debug.apk, install on your Android device (allow "Install from unknown sources" for the browser / file manager you used to download).

On first launch the app shows the age-gate disclaimer (must be accepted), then a login screen. Enter:

  • Backend URL: http://<your-backend-host>:8000 (e.g. your LAN IP, or http://localhost:8000 if running on the device — uncommon)
  • API key: one of the values you put in API_KEYS in .env

That's it.

Local Python (no Docker)

python -m venv .venv && . .venv/bin/activate    # or .\.venv\Scripts\activate on Windows
pip install -e .[dev]
cp .env.example .env                             # edit creds
alembic upgrade head
uvicorn app.main:app --port 8000

Worker (manual one-shot ingest)

# Foreground APScheduler with all jobs
python -m app.scheduler.worker

# One-shot:
python -m app.scheduler.worker --once --source=tpdb --limit=200
python -m app.scheduler.worker --once --strategy=performer-driven --top-n=20
python -m app.scheduler.worker --once --strategy=performer-driven \
    --performers="Lola Noir,Mia Malkova"

Building the APK locally

cd mobile
npm install
cd android
./gradlew assembleDebug
# output: mobile/android/app/build/outputs/apk/debug/app-debug.apk

Or just push a v* tag — GitHub Actions builds and attaches the APK to the Release (.github/workflows/build-apk.yml).

Sentry telemetry (optional)

Default behavior: no telemetry. Sentry only initializes when a DSN is present at runtime/build time.

To enable Sentry for your instance (errors only, no PII, no replay):

  • Backend: set SENTRY_DSN=https://... in .env (gitignored). Optionally SENTRY_ENVIRONMENT=production and SENTRY_TRACES_SAMPLE_RATE=0.1.
  • Mobile (local builds): create mobile/.env (gitignored) with EXPO_PUBLIC_SENTRY_DSN=https://.... Expo SDK 49+ auto-inlines EXPO_PUBLIC_* vars into the JS bundle at build time.
  • Mobile (CI builds): add a GitHub repository secret named SENTRY_DSN. The APK workflow exports it as EXPO_PUBLIC_SENTRY_DSN to gradle. Without the secret, the APK ships with telemetry disabled (forks of this repo don't inherit your DSN).

Sentry init is gated by if (SENTRY_DSN) { Sentry.init(...) } — empty DSN means the SDK is loaded as dead code but never sends a single request.


Configuration

All runtime config is environment variables (see .env.example for the full list). Highlights:

Var Default Required? Notes
DATABASE_URL postgresql+psycopg://goon:goon@localhost:5432/goon Yes Postgres 14+
TPDB_API_TOKEN empty For TPDB ingest Get from theporndb.net account
STASHDB_API_KEY empty For StashDB ingest Get from stashdb.org account
API_KEYS empty Recommended CSV of allowed API keys; empty = no auth (localhost-only)
SENTRY_DSN empty No Empty = no telemetry. Use your own DSN if you want crash reports.
LOG_LEVEL INFO No DEBUG for verbose tube scraping logs

Scheduler tuning (set to 0 to disable a job):

Var Default Description
GOON_SCHED_TPDB_HOURS 6 TPDB delta interval
GOON_SCHED_STASHDB_HOURS 6 StashDB delta interval
GOON_SCHED_PERFORMER_DRIVEN_HOURS 12 Top-N performer ingest
GOON_SCHED_PERFORMER_CONTINUOUS_SECONDS 15 Continuous backfill tick

Architecture (high level)

┌──────────┐   delta cron    ┌────────────┐
│  TPDB    │────────────────▶│            │
└──────────┘                 │            │
┌──────────┐                 │   ingest   │
│ StashDB  │────────────────▶│  pipeline  │──┐
└──────────┘                 │            │  │  cross-source
┌──────────┐  performer-     │            │  ▼   dedup +
│ ~25 tube │   driven   ┌───▶│            │ ┌─────────┐
│  sites   │  search    │    └────────────┘ │ Postgres│
└──────────┘────────────┘                   └─────────┘
                                                 │
   ┌─────────────────────────────────────────────┤
   ▼                                             ▼
┌──────────────┐                          ┌──────────────┐
│   FastAPI    │                          │   Worker     │
│  /scenes     │◀────── on Watch click ───│  scheduler   │
│  /performers │     resolve stream URL   │ (APScheduler)│
│  /playback   │     (yt-dlp / hoster     └──────────────┘
└──────────────┘      packer)
       │
       ▼
┌──────────────┐
│  Expo mobile │
│   (Android)  │
└──────────────┘

Key modules:

  • app/connectors/ — TPDB, StashDB, dooplay (movies), paradisehill (movies), direct_scrapers/ (25 tube discovery scrapers).
  • app/extractors/ — stream URL resolution per tube. yt-dlp wrapper + custom + generic embed-iframe + P.A.C.K.E.R. unpacker.
  • app/resolve/ — cross-source scene merging (phash, title similarity, performer overlap, release date window).
  • app/scheduler/ — APScheduler jobs + performer_driven.py (the core ingest strategy: completeness > recency).
  • mobile/ — Expo / React Native client.

Tube coverage

Discovery + stream resolution registered for ~33 sources:

Mainstream tubes: pornhub, redtube, xhamster, xvideos, xnxx, youporn, eporner, hqporner, sxyprn, porntrex, pornhat.

Aggregators / mirrors: xmoviesforyou, watchporn, siska, porn4days, porndish, xxxfreewatch, latestleaks, latestpornvideo, mypornerleak, porndittcom, hdporn92, sxyland, 0dayxx, perverzija, fpoxxx, porn00, pornxp, hdporngg, fullmovies, freshporno, shyfap.

Movie sites: paradisehill (primary) + dooplay mirrors (mangoporn, streamporn, pandamovies).

If you want to add another tube, see CONTRIBUTING.md.


Support the project

Goon is free, open-source, and ad-free. It stays that way because donations cover the VPS, the TPDB/StashDB tokens, and the time. Crypto only — mainstream processors refuse adult projects, even FOSS tooling.

In-app: Scenes → ♥ opens a screen with QR codes for Monero, Bitcoin, and USDT (TRC-20).

Addresses are hardcoded in mobile/src/lib/donate.ts so a compromised server cannot swap them mid-donation. Verify the value on-screen against the copy in this repo before sending.


Roadmap

Near-term:

  • Browse-by-performer + sort-by-studio
  • Multi-tag filter (AND / OR)
  • Continue-watching rail (position sync across devices)
  • Stash local-server bridge — sync favorites/watchlist with a self-hosted Stash
  • iOS sideload via TestFlight invite

Mid-term:

  • Web companion (read-only browser frontend over the same API)
  • BTCPay Server invoicing for one-time / recurring donations
  • Performer-alert notifications (server push when a favorited performer drops a new scene)

License

MIT — see LICENSE.