Goon — self-hosted aggregator for adult-content scene metadata. Indexes scenes from TPDB, StashDB, and 30+ public adult tube sites. Cross-source deduplication via perceptual hash + Levenshtein distance. FastAPI backend + APScheduler worker + React Native (Expo) mobile client. FOSS, ad-free, donation-funded. See README for details.
4 KiB
Contributing to Goon
Development setup
Goon backend is Python 3.12+, FastAPI + SQLAlchemy + APScheduler + Postgres. Mobile client is React Native + Expo.
Backend
# Create virtualenv
python -m venv .venv
. .venv/bin/activate # or .venv\Scripts\activate on Windows
# Install with dev extras
pip install -e .[dev]
# Bring up postgres (or use docker-compose; see README)
# Adjust DATABASE_URL in .env if needed
cp .env.example .env
# Run migrations
alembic upgrade head
# Run API
uvicorn app.main:app --reload --port 8000
# Run worker (separate terminal)
python -m app.scheduler.worker # full scheduler
python -m app.scheduler.worker --once --source=tpdb --limit=50 # one-shot ingest
Mobile
cd mobile
npm install
npm start # opens Expo dev server
Tests
pytest # full suite (~70 tests, <5s)
pytest tests/test_resolve_*.py -v
ruff check app/
mypy app/ # optional, CI-only
PRs must pass pytest + ruff check. Run them locally before pushing.
Code style
- Formatting: ruff (config in
pyproject.toml). Line length 100. - Type hints: required on public functions.
from __future__ import annotationsin every module. - Docstrings: write the why, not the what. Reference real bugs/incidents when explaining non-obvious code paths.
- Comments: only when the code can't speak for itself. Prefer renaming a variable over adding a comment that explains it.
- No dead code, no commented-out code, no TODO without an issue link.
- Polish or English in comments: existing code is mostly Polish in comments and English in code (function/class/var names). New code can be either, but be consistent within a file.
Adding a new tube extractor / scraper
If you want Goon to support an additional adult tube site:
-
Stream extractor (
app/extractors/tubes/): given a scene page URL, return a list ofStreamSource(m3u8/mp4 URLs with quality labels).- Mainstream tubes: try
_ytdlp.extract(yt-dlp covers ~30 tubes out of the box — just register the sitetag inapp/extractors/__init__.py). - WordPress-like tubes with embed iframe: register
_embed_iframe.extract. - Custom player / signed URLs / token rotation: write your own per-tube
module (see
hqporner.py,eporner.py,sxyprn.pyas references).
- Mainstream tubes: try
-
Discovery scraper (
app/connectors/direct_scrapers/): subclassBaseSearchScraper, setsitetag,_search_url_template,_scene_url_re. Most aggregator tubes can fit in 10-20 lines (seexmoviesforyou.py). -
Register the scraper class in
ALL_DIRECT_SCRAPERSinapp/connectors/direct_scrapers/__init__.py. -
Test with one performer name that you know has scenes on that tube:
python -m app.scheduler.worker --once --strategy=performer-driven \ --performers="Some Performer" --sitetags=<your-sitetag>
Database migrations
Use Alembic:
alembic revision -m "describe change" # new migration
alembic upgrade head # apply
alembic downgrade -1 # roll back one
Every migration must have a working downgrade(). We don't ship squashed
migrations — full history is the source of truth.
What we won't merge
- Adult-content moderation features (auto-tagging by detected acts, content filtering by performer attributes, etc.) — out of scope.
- Hardcoded credentials, API keys, or device IDs in source — must be env-driven.
- Bypassing tube paywalls / DRM / auth — Goon only scrapes publicly accessible search pages.
- Telemetry or analytics that report user activity to third parties.
Sentry is opt-in (
SENTRY_DSNempty by default). - Public deployment recipes (e.g. nginx config for an open instance). Goon is self-hosted only — see DISCLAIMER.md.
License
By contributing, you agree your contributions are licensed under the MIT License (see LICENSE).