Browse scrapers backfilling old catalogs stamp the tube's import/post date as
release_date, so old content (e.g. 83 MissaX classics via perverzija, ~3600/3 days
across eporner/youporn/etc.) fake-ranked as newest and flooded the favorites "+N".
NULLing the dates was a non-starter — the stub filter would hide 251k performer-less
scenes. Instead: a Scene.backfill flag marks bulk catalog imports; they stay visible but
never count as "new".
- scenes.backfill column (+ index, migration 0026); resolve_scene/_process_scene thread it.
- deep_crawl tags scenes from pages beyond the "latest" threshold (>2) as backfill;
latest pages + TPDB/StashDB delta stay genuine. Cursor reset re-sweeps page 1 so real
new content is always caught fresh.
- favorites +N (performers + studios) excludes backfill within the top-200 window.
- SceneOut exposes `backfill`; mobile NEW badge + NEW-first re-sort skip it (badge==count).
- Retroactive: tagged 439k existing scenes in bulk (>10 non-canonical / studio / day)
clusters. Device check: favorite-studios +N 11626 (naive) -> 236.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Filtered /scenes (tag/origin/q/studio/performer) ran exhaustive COUNT with
stub-filter EXISTS over 1.7M rows: TAG 5.1s, ORIGIN 4.9s, SEARCH 3.1s.
Mobile relied on `loaded < total` for infinite-scroll, making exact count
mandatory and ruling out approximate shortcuts.
Backend:
- SceneListOut gains has_more (bool) and total_capped (bool), both optional
for backward compat with old mobile
- Filtered count uses LIMIT _COUNT_CAP+1 (1000) subquery — cost is
O(min(matches, cap)) instead of O(all). Measured: TAG 5.1s→664ms,
SEARCH 3.1s→138ms, ORIGIN 4.9s→1.07s (also fixes SiteScenes showing
global count ~1M instead of per-site count)
- has_more from fetching per_page+1 rows (essentially free); extra row
stripped before serialisation
- Pure-default list (no filters at all) keeps TTL-cached full count
Mobile:
- getNextPageParam uses has_more ?? fallback to loaded<total
- Display shows "{total}+" when total_capped=true (5 screens)
Verified on emulator: tag "Big Tits" → "1000 scenes" loaded, no 500s,
backward compat confirmed (old APK works against new backend).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Goon — self-hosted aggregator for adult-content scene metadata.
Indexes scenes from TPDB, StashDB, and 30+ public adult tube sites.
Cross-source deduplication via perceptual hash + Levenshtein distance.
FastAPI backend + APScheduler worker + React Native (Expo) mobile client.
FOSS, ad-free, donation-funded. See README for details.
