goon/README.md

# Goon

Self-hosted aggregator for adult-content scene metadata. Indexes scenes from
TheporndB, StashDB, and 30+ public adult tube sites; deduplicates across
sources; serves an API + mobile (React Native) client for browsing and
linking out to playback.

> **18+ ONLY · Self-hosted only · See [DISCLAIMER.md](./DISCLAIMER.md) before
> hosting an instance.**

---

## What it does

- **Multi-source ingest**: pulls canonical scene/performer/studio metadata from
  TPDB and StashDB on a delta cron, merges duplicates by performer + title +
  date heuristics (perceptual hash + Levenshtein title distance).
- **Tube discovery**: per-performer search across 30+ public adult tube sites
  (mainstream + aggregators). Each tube is scraped directly via HTTP — no
  proprietary API dependencies.
- **Stream resolution on demand**: when a user clicks Watch, the API extracts
  a fresh m3u8/mp4 URL from the tube's page (or falls back to embed link for
  WebView playback). Mainstream tubes use yt-dlp; aggregator tubes use a
  generic P.A.C.K.E.R. unpacker for JWPlayer-based hosters
  (StreamWish/doodporn/mixdrop/...).
- **Mobile client** (Expo / React Native): scene grid, performer pages, watch
  history, favorites, hold-to-preview animated thumbnails.
- **Performer-driven backfill**: a continuous worker walks performers ordered
  by `last_searched_at NULLS FIRST` and back-fills tube scenes for the
  longest-stale performer first.

## What it doesn't do

- Host or store any media. Scene metadata + thumbnail URLs only.
- Bypass paywalls, authentication, geo-blocks, or DRM.
- Provide age verification, ToS gating, or moderation for public deployments.
  See [DISCLAIMER.md](./DISCLAIMER.md).
- Phone home. Sentry telemetry is opt-in (env var, empty by default).

---

## Quick start

### 1. Run the backend (Docker)

```bash
git clone <repo-url> goon
cd goon
cp .env.example .env
# Edit .env:
#   - TPDB_API_TOKEN       (theporndb.net account → API tokens)
#   - STASHDB_API_KEY      (stashdb.org account → API keys)
#   - API_KEYS             (generate one: python -c "import secrets; print(secrets.token_urlsafe(32))")

docker compose up -d
```

Three services come up: `db` (Postgres 16), `api` (FastAPI on `:8000`,
auto-applies migrations on startup), `worker` (APScheduler running TPDB/StashDB
delta + performer-driven backfill).

Verify: `curl localhost:8000/health` → `{"status":"ok"}`.

### 2. Install the mobile app (Android)

Download the latest debug APK from
[GitHub Releases](../../releases/latest) → `goon-vX.Y.Z-debug.apk`, install on
your Android device (allow "Install from unknown sources" for the browser /
file manager you used to download).

On first launch the app shows the age-gate disclaimer (must be accepted), then
a login screen. Enter:
- **Backend URL**: `http://<your-backend-host>:8000` (e.g. your LAN IP, or
  `http://localhost:8000` if running on the device — uncommon)
- **API key**: one of the values you put in `API_KEYS` in `.env`

That's it.

### Local Python (no Docker)

```bash
python -m venv .venv && . .venv/bin/activate    # or .\.venv\Scripts\activate on Windows
pip install -e .[dev]
cp .env.example .env                             # edit creds
alembic upgrade head
uvicorn app.main:app --port 8000
```

### Worker (manual one-shot ingest)

```bash
# Foreground APScheduler with all jobs
python -m app.scheduler.worker

# One-shot:
python -m app.scheduler.worker --once --source=tpdb --limit=200
python -m app.scheduler.worker --once --strategy=performer-driven --top-n=20
python -m app.scheduler.worker --once --strategy=performer-driven \
    --performers="Lola Noir,Mia Malkova"
```

### Building the APK locally

```bash
cd mobile
npm install
cd android
./gradlew assembleDebug
# output: mobile/android/app/build/outputs/apk/debug/app-debug.apk
```

Or just push a `v*` tag — GitHub Actions builds and attaches the APK to the
Release ([.github/workflows/build-apk.yml](./.github/workflows/build-apk.yml)).

### Sentry telemetry (optional)

Default behavior: **no telemetry**. Sentry only initializes when a DSN is
present at runtime/build time.

To enable Sentry for **your** instance (errors only, no PII, no replay):

- **Backend**: set `SENTRY_DSN=https://...` in `.env` (gitignored).
  Optionally `SENTRY_ENVIRONMENT=production` and `SENTRY_TRACES_SAMPLE_RATE=0.1`.
- **Mobile (local builds)**: create `mobile/.env` (gitignored) with
  `EXPO_PUBLIC_SENTRY_DSN=https://...`. Expo SDK 49+ auto-inlines `EXPO_PUBLIC_*`
  vars into the JS bundle at build time.
- **Mobile (CI builds)**: add a GitHub repository secret named `SENTRY_DSN`.
  The APK workflow exports it as `EXPO_PUBLIC_SENTRY_DSN` to gradle. Without
  the secret, the APK ships with telemetry disabled (forks of this repo don't
  inherit your DSN).

Sentry init is gated by `if (SENTRY_DSN) { Sentry.init(...) }` — empty DSN
means the SDK is loaded as dead code but never sends a single request.

---

## Configuration

All runtime config is environment variables (see [.env.example](./.env.example)
for the full list). Highlights:

| Var | Default | Required? | Notes |
|---|---|---|---|
| `DATABASE_URL` | `postgresql+psycopg://goon:goon@localhost:5432/goon` | Yes | Postgres 14+ |
| `TPDB_API_TOKEN` | _empty_ | For TPDB ingest | Get from theporndb.net account |
| `STASHDB_API_KEY` | _empty_ | For StashDB ingest | Get from stashdb.org account |
| `API_KEYS` | _empty_ | Recommended | CSV of allowed API keys; empty = no auth (localhost-only) |
| `SENTRY_DSN` | _empty_ | No | Empty = no telemetry. Use your own DSN if you want crash reports. |
| `LOG_LEVEL` | `INFO` | No | DEBUG for verbose tube scraping logs |

Scheduler tuning (set to `0` to disable a job):

| Var | Default | Description |
|---|---|---|
| `GOON_SCHED_TPDB_HOURS` | `6` | TPDB delta interval |
| `GOON_SCHED_STASHDB_HOURS` | `6` | StashDB delta interval |
| `GOON_SCHED_PERFORMER_DRIVEN_HOURS` | `12` | Top-N performer ingest |
| `GOON_SCHED_PERFORMER_CONTINUOUS_SECONDS` | `15` | Continuous backfill tick |

---

## Architecture (high level)

```
┌──────────┐   delta cron    ┌────────────┐
│  TPDB    │────────────────▶│            │
└──────────┘                 │            │
┌──────────┐                 │   ingest   │
│ StashDB  │────────────────▶│  pipeline  │──┐
└──────────┘                 │            │  │  cross-source
┌──────────┐  performer-     │            │  ▼   dedup +
│ ~25 tube │   driven   ┌───▶│            │ ┌─────────┐
│  sites   │  search    │    └────────────┘ │ Postgres│
└──────────┘────────────┘                   └─────────┘
                                                 │
   ┌─────────────────────────────────────────────┤
   ▼                                             ▼
┌──────────────┐                          ┌──────────────┐
│   FastAPI    │                          │   Worker     │
│  /scenes     │◀────── on Watch click ───│  scheduler   │
│  /performers │     resolve stream URL   │ (APScheduler)│
│  /playback   │     (yt-dlp / hoster     └──────────────┘
└──────────────┘      packer)
       │
       ▼
┌──────────────┐
│  Expo mobile │
│   (Android)  │
└──────────────┘
```

Key modules:

- [`app/connectors/`](./app/connectors/) — TPDB, StashDB, dooplay (movies),
  paradisehill (movies), [`direct_scrapers/`](./app/connectors/direct_scrapers/)
  (25 tube discovery scrapers).
- [`app/extractors/`](./app/extractors/) — stream URL resolution per tube.
  yt-dlp wrapper + custom + generic embed-iframe + P.A.C.K.E.R. unpacker.
- [`app/resolve/`](./app/resolve/) — cross-source scene merging (phash, title
  similarity, performer overlap, release date window).
- [`app/scheduler/`](./app/scheduler/) — APScheduler jobs +
  [`performer_driven.py`](./app/scheduler/performer_driven.py) (the core
  ingest strategy: completeness > recency).
- [`mobile/`](./mobile/) — Expo / React Native client.

## Tube coverage

Discovery + stream resolution registered for ~33 sources:

**Mainstream tubes:** pornhub, redtube, xhamster, xvideos, xnxx, youporn,
eporner, hqporner, sxyprn, porntrex, pornhat.

**Aggregators / mirrors:** xmoviesforyou, watchporn, siska, porn4days,
porndish, xxxfreewatch, latestleaks, latestpornvideo, mypornerleak,
porndittcom, hdporn92, sxyland, 0dayxx, perverzija, fpoxxx, porn00, pornxp,
hdporngg, fullmovies, freshporno, shyfap.

**Movie sites:** paradisehill (primary) + dooplay mirrors (mangoporn,
streamporn, pandamovies).

If you want to add another tube, see [CONTRIBUTING.md](./CONTRIBUTING.md).

---

## Support the project

Goon is free, open-source, and ad-free. It stays that way because donations
cover the VPS, the TPDB/StashDB tokens, and the time. **Crypto only** —
mainstream processors refuse adult projects, even FOSS tooling.

In-app: **Scenes → ♥** opens a screen with QR codes for Monero, Bitcoin, and
USDT (TRC-20).

Addresses are hardcoded in
[`mobile/src/lib/donate.ts`](./mobile/src/lib/donate.ts) so a compromised
server cannot swap them mid-donation. Verify the value on-screen against the
copy in this repo before sending.

---

## Roadmap

Near-term:

- Browse-by-performer + sort-by-studio
- Multi-tag filter (AND / OR)
- Continue-watching rail (position sync across devices)
- Stash local-server bridge — sync favorites/watchlist with a self-hosted Stash
- iOS sideload via TestFlight invite

Mid-term:

- Web companion (read-only browser frontend over the same API)
- BTCPay Server invoicing for one-time / recurring donations
- Performer-alert notifications (server push when a favorited performer drops a new scene)

---

## License

MIT — see [LICENSE](./LICENSE).