No description
Find a file
goon-foss bbdff6bc90 feat(reap): pomiar martwych zrodel per tub, z kontrola pozytywna i negatywna
Po sxyprn (34% martwych na 188 tys. zrodel) nasuwa sie przemielenie tak samo calej
reszty. Metoda sxyprn nie przenosi sie wprost:

* Twardy sygnal smierci (`HosterDead`) ma tylko 6 z 42 ekstraktorow. Dla reszty "nic nie
  zwrocil" znaczy naraz "skasowane", "zmienil sie markup" i "rate limit". Wygaszanie na
  tej podstawie schowaloby ZYWE sceny, jak przy porntrex (6479 scen po cichej zmianie
  markupu).
* Zywych zrodel jest ~3,9 mln, czyli ponad tydzien ciaglego pukania do wlasnej podazy.

Dlatego najpierw pomiar. Skrypt jest wylacznie do odczytu, niczego nie oznacza.

Martwe pada tylko na twardych sygnalach: HTTP 404/410 albo `HosterDead`. Pusty wynik
ekstraktora to "nieznane" i zrodla nikt nie rusza.

Kazdy tub dostaje DWIE kontrole, bo sam wynik proby niczego nie dowodzi: "0% martwych"
znaczy albo "tub zdrowy", albo "nie widzimy trupow".

* Pozytywna: zrodla scen, ktore realnie odegraly sie w ostatnich 14 dniach. Musza wrocic
  zywe. Lapie tuba, ktory zaczal oddawac 404 wszystkiemu bez sesji, czyli przypadek, w
  ktorym przemial skasowalby caly katalog. Prog to 5%, nie zero, bo czesc scen znika
  naprawde. Odsetek z kontroli odejmuje sie od wyniku proby.
* Negatywna: prawdziwy adres z numerem sceny podmienionym na absurdalny. Taka strona nie
  moze istniec, wiec musi wrocic martwa.

Negatywna od razu sie oplacila: pierwsza wersja pytala tylko o status HTTP i uznala
sxyprn za tuba, na ktorym sonda jest slepa (0/6), chociaz wiemy, ze ma tam 34% trupow.
Powod: sxyprn oddaje skasowany post jako HTTP 200, a `HosterDead` leci dopiero z
ekstraktora. Po przejsciu na `try_extract` negatywna daje 7/7.

Tuby resolvowane tylko w WebView na telefonie sa pomijane, bo z VPS sa nieweryfikowalne.
2026-08-27 14:36:48 +02:00
.github/workflows Initial commit 2026-05-20 10:10:22 +02:00
alembic fix(sites): sortuj po dacie premiery z fallbackiem na date dodania + napraw miniaturki 2026-08-20 16:25:53 +02:00
app fix(seekplayer): sprawdzaj czy adres CDN naprawde odpowiada, plus obsluga sciezki Tiktok 2026-08-27 08:59:28 +02:00
landing Mobile 0.1.9: OTA enable, WebView cookie-dismiss fix, porndoe connector 2026-05-22 11:20:57 +02:00
mobile fix(player): nie wstrzykuj naszego skryptu na stronach rodziny DoodStream 2026-08-27 09:10:16 +02:00
scripts feat(reap): pomiar martwych zrodel per tub, z kontrola pozytywna i negatywna 2026-08-27 14:36:48 +02:00
site docs(site): strona goon-foss.org do repo + aktualizacja o nowosci 2026-08-13 09:54:25 +02:00
tests Mobile 0.1.9: OTA enable, WebView cookie-dismiss fix, porndoe connector 2026-05-22 11:20:57 +02:00
.env.example Initial commit 2026-05-20 10:10:22 +02:00
.gitignore chore: remove internal host/IP references from comments; untrack local dev notes 2026-07-11 18:59:38 +02:00
alembic.ini Initial commit 2026-05-20 10:10:22 +02:00
CONTRIBUTING.md Initial commit 2026-05-20 10:10:22 +02:00
DISCLAIMER.md Initial commit 2026-05-20 10:10:22 +02:00
docker-compose.yml fix(db): set shm_size 1g — parallel queries overflow default 64MB /dev/shm 2026-06-06 22:56:49 +02:00
Dockerfile Initial commit 2026-05-20 10:10:22 +02:00
LICENSE Initial commit 2026-05-20 10:10:22 +02:00
pyproject.toml Initial commit 2026-05-20 10:10:22 +02:00
README.md Initial commit 2026-05-20 10:10:22 +02:00
streamtape_sample.md Initial commit 2026-05-20 10:10:22 +02:00
theporndude_coverage.json theporndude audit: scorecards, coverage + raporty 2026-05-22 13:44:54 +02:00
theporndude_dump.json theporndude audit: scorecards, coverage + raporty 2026-05-22 13:44:54 +02:00
theporndude_final_report.md theporndude audit: scorecards, coverage + raporty 2026-05-22 13:44:54 +02:00
theporndude_free_tubes.json theporndude audit: scorecards, coverage + raporty 2026-05-22 13:44:54 +02:00
theporndude_full_audit_report.md theporndude audit: scorecards, coverage + raporty 2026-05-22 13:44:54 +02:00
theporndude_movies.json theporndude audit: scorecards, coverage + raporty 2026-05-22 13:44:54 +02:00
theporndude_movies_scorecard.json theporndude audit: scorecards, coverage + raporty 2026-05-22 13:44:54 +02:00
theporndude_resolved.json theporndude audit: scorecards, coverage + raporty 2026-05-22 13:44:54 +02:00
theporndude_scorecard.json theporndude audit: scorecards, coverage + raporty 2026-05-22 13:44:54 +02:00
theporndude_triage.json fix(triage): theporndude candidate audit had two false-negative bugs 2026-07-27 09:37:51 +02:00

Goon

Self-hosted aggregator for adult-content scene metadata. Indexes scenes from TheporndB, StashDB, and 30+ public adult tube sites; deduplicates across sources; serves an API + mobile (React Native) client for browsing and linking out to playback.

18+ ONLY · Self-hosted only · See DISCLAIMER.md before hosting an instance.


What it does

  • Multi-source ingest: pulls canonical scene/performer/studio metadata from TPDB and StashDB on a delta cron, merges duplicates by performer + title + date heuristics (perceptual hash + Levenshtein title distance).
  • Tube discovery: per-performer search across 30+ public adult tube sites (mainstream + aggregators). Each tube is scraped directly via HTTP — no proprietary API dependencies.
  • Stream resolution on demand: when a user clicks Watch, the API extracts a fresh m3u8/mp4 URL from the tube's page (or falls back to embed link for WebView playback). Mainstream tubes use yt-dlp; aggregator tubes use a generic P.A.C.K.E.R. unpacker for JWPlayer-based hosters (StreamWish/doodporn/mixdrop/...).
  • Mobile client (Expo / React Native): scene grid, performer pages, watch history, favorites, hold-to-preview animated thumbnails.
  • Performer-driven backfill: a continuous worker walks performers ordered by last_searched_at NULLS FIRST and back-fills tube scenes for the longest-stale performer first.

What it doesn't do

  • Host or store any media. Scene metadata + thumbnail URLs only.
  • Bypass paywalls, authentication, geo-blocks, or DRM.
  • Provide age verification, ToS gating, or moderation for public deployments. See DISCLAIMER.md.
  • Phone home. Sentry telemetry is opt-in (env var, empty by default).

Quick start

1. Run the backend (Docker)

git clone <repo-url> goon
cd goon
cp .env.example .env
# Edit .env:
#   - TPDB_API_TOKEN       (theporndb.net account → API tokens)
#   - STASHDB_API_KEY      (stashdb.org account → API keys)
#   - API_KEYS             (generate one: python -c "import secrets; print(secrets.token_urlsafe(32))")

docker compose up -d

Three services come up: db (Postgres 16), api (FastAPI on :8000, auto-applies migrations on startup), worker (APScheduler running TPDB/StashDB delta + performer-driven backfill).

Verify: curl localhost:8000/health → {"status":"ok"}.

2. Install the mobile app (Android)

Download the latest debug APK from GitHub Releases → goon-vX.Y.Z-debug.apk, install on your Android device (allow "Install from unknown sources" for the browser / file manager you used to download).

On first launch the app shows the age-gate disclaimer (must be accepted), then a login screen. Enter:

  • Backend URL: http://<your-backend-host>:8000 (e.g. your LAN IP, or http://localhost:8000 if running on the device — uncommon)
  • API key: one of the values you put in API_KEYS in .env

That's it.

Local Python (no Docker)

python -m venv .venv && . .venv/bin/activate    # or .\.venv\Scripts\activate on Windows
pip install -e .[dev]
cp .env.example .env                             # edit creds
alembic upgrade head
uvicorn app.main:app --port 8000

Worker (manual one-shot ingest)

# Foreground APScheduler with all jobs
python -m app.scheduler.worker

# One-shot:
python -m app.scheduler.worker --once --source=tpdb --limit=200
python -m app.scheduler.worker --once --strategy=performer-driven --top-n=20
python -m app.scheduler.worker --once --strategy=performer-driven \
    --performers="Lola Noir,Mia Malkova"

Building the APK locally

cd mobile
npm install
cd android
./gradlew assembleDebug
# output: mobile/android/app/build/outputs/apk/debug/app-debug.apk

Or just push a v* tag — GitHub Actions builds and attaches the APK to the Release (.github/workflows/build-apk.yml).

Sentry telemetry (optional)

Default behavior: no telemetry. Sentry only initializes when a DSN is present at runtime/build time.

To enable Sentry for your instance (errors only, no PII, no replay):

  • Backend: set SENTRY_DSN=https://... in .env (gitignored). Optionally SENTRY_ENVIRONMENT=production and SENTRY_TRACES_SAMPLE_RATE=0.1.
  • Mobile (local builds): create mobile/.env (gitignored) with EXPO_PUBLIC_SENTRY_DSN=https://.... Expo SDK 49+ auto-inlines EXPO_PUBLIC_* vars into the JS bundle at build time.
  • Mobile (CI builds): add a GitHub repository secret named SENTRY_DSN. The APK workflow exports it as EXPO_PUBLIC_SENTRY_DSN to gradle. Without the secret, the APK ships with telemetry disabled (forks of this repo don't inherit your DSN).

Sentry init is gated by if (SENTRY_DSN) { Sentry.init(...) } — empty DSN means the SDK is loaded as dead code but never sends a single request.


Configuration

All runtime config is environment variables (see .env.example for the full list). Highlights:

Var Default Required? Notes
DATABASE_URL postgresql+psycopg://goon:goon@localhost:5432/goon Yes Postgres 14+
TPDB_API_TOKEN empty For TPDB ingest Get from theporndb.net account
STASHDB_API_KEY empty For StashDB ingest Get from stashdb.org account
API_KEYS empty Recommended CSV of allowed API keys; empty = no auth (localhost-only)
SENTRY_DSN empty No Empty = no telemetry. Use your own DSN if you want crash reports.
LOG_LEVEL INFO No DEBUG for verbose tube scraping logs

Scheduler tuning (set to 0 to disable a job):

Var Default Description
GOON_SCHED_TPDB_HOURS 6 TPDB delta interval
GOON_SCHED_STASHDB_HOURS 6 StashDB delta interval
GOON_SCHED_PERFORMER_DRIVEN_HOURS 12 Top-N performer ingest
GOON_SCHED_PERFORMER_CONTINUOUS_SECONDS 15 Continuous backfill tick

Architecture (high level)

┌──────────┐   delta cron    ┌────────────┐
│  TPDB    │────────────────▶│            │
└──────────┘                 │            │
┌──────────┐                 │   ingest   │
│ StashDB  │────────────────▶│  pipeline  │──┐
└──────────┘                 │            │  │  cross-source
┌──────────┐  performer-     │            │  ▼   dedup +
│ ~25 tube │   driven   ┌───▶│            │ ┌─────────┐
│  sites   │  search    │    └────────────┘ │ Postgres│
└──────────┘────────────┘                   └─────────┘
                                                 │
   ┌─────────────────────────────────────────────┤
   ▼                                             ▼
┌──────────────┐                          ┌──────────────┐
│   FastAPI    │                          │   Worker     │
│  /scenes     │◀────── on Watch click ───│  scheduler   │
│  /performers │     resolve stream URL   │ (APScheduler)│
│  /playback   │     (yt-dlp / hoster     └──────────────┘
└──────────────┘      packer)
       │
       ▼
┌──────────────┐
│  Expo mobile │
│   (Android)  │
└──────────────┘

Key modules:

  • app/connectors/ — TPDB, StashDB, dooplay (movies), paradisehill (movies), direct_scrapers/ (25 tube discovery scrapers).
  • app/extractors/ — stream URL resolution per tube. yt-dlp wrapper + custom + generic embed-iframe + P.A.C.K.E.R. unpacker.
  • app/resolve/ — cross-source scene merging (phash, title similarity, performer overlap, release date window).
  • app/scheduler/ — APScheduler jobs + performer_driven.py (the core ingest strategy: completeness > recency).
  • mobile/ — Expo / React Native client.

Tube coverage

Discovery + stream resolution registered for ~33 sources:

Mainstream tubes: pornhub, redtube, xhamster, xvideos, xnxx, youporn, eporner, hqporner, sxyprn, porntrex, pornhat.

Aggregators / mirrors: xmoviesforyou, watchporn, siska, porn4days, porndish, xxxfreewatch, latestleaks, latestpornvideo, mypornerleak, porndittcom, hdporn92, sxyland, 0dayxx, perverzija, fpoxxx, porn00, pornxp, hdporngg, fullmovies, freshporno, shyfap.

Movie sites: paradisehill (primary) + dooplay mirrors (mangoporn, streamporn, pandamovies).

If you want to add another tube, see CONTRIBUTING.md.


Support the project

Goon is free, open-source, and ad-free. It stays that way because donations cover the VPS, the TPDB/StashDB tokens, and the time. Crypto only — mainstream processors refuse adult projects, even FOSS tooling.

In-app: Scenes → ♥ opens a screen with QR codes for Monero, Bitcoin, and USDT (TRC-20).

Addresses are hardcoded in mobile/src/lib/donate.ts so a compromised server cannot swap them mid-donation. Verify the value on-screen against the copy in this repo before sending.


Roadmap

Near-term:

  • Browse-by-performer + sort-by-studio
  • Multi-tag filter (AND / OR)
  • Continue-watching rail (position sync across devices)
  • Stash local-server bridge — sync favorites/watchlist with a self-hosted Stash
  • iOS sideload via TestFlight invite

Mid-term:

  • Web companion (read-only browser frontend over the same API)
  • BTCPay Server invoicing for one-time / recurring donations
  • Performer-alert notifications (server push when a favorited performer drops a new scene)

License

MIT — see LICENSE.