Addresses the ultra-review findings on this branch:
Player (PlayerScreen.tsx): the new recoveryPending mirrored the fallback-chain guards
by hand and could deadlock into a permanent "Reconnecting" spinner with no way to Mark
broken — for gone (410) sources on IP-bound tubes (re-resolve bails before setting
reResolveDone) and for any post-load error on those tubes (re-resolve is initial-load
only). Derive one reResolveApplicable flag (IP-bound AND initial-load AND not-gone) and
use it for both the chain gate and the spinner, so gone/post-load errors fall through to
proxy/WebView or the terminal error card. Seek-recovery now falls through to the chain
when player.replace() throws instead of returning.
Quick-play (SceneDetail): the autoplay route param persisted and autoPlay={i===0} re-fired
when the source list reordered (e.g. after Mark broken drops the dead source), bouncing the
user into the player. Consume it once via onAutoPlayConsumed -> nav.setParams({autoplay:false}).
Backfill semantics: performer-driven direct-scraper "backward fill" now tags scenes
backfill=True (search-by-name pulls the whole old catalog); merge coalesces backfill
(keep AND drop) so a fresh scene merged into a dead dup keeps NEW; deep-crawl only tags
backfill on a tube's FIRST sweep (swept_once) so re-sweep catalog growth stays genuine;
pilot script tags backfill.
Perf/migration: migration 0026 is now idempotent (IF NOT EXISTS; prod got the column via
manual ALTER) and adds ix_scene_performers_performer_id (favorites count filtered
performer_id with no index); index also created on prod.
Cleanup: deleted dead FavoriteSceneRow (unused import in two screens, stale isNew without
the backfill guard); removed em-dashes from all lines this branch added (user CLAUDE.md
rule), including the user-facing changelog / Settings / player-overlay strings.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The umbrella Source.name for all direct tube scrapers (deep-crawl, browse-latest,
performer-driven) was "pornapp" — a misleading leftover from the removed external
porn-app API. It read like a dependency on a third-party "pornapp" service; it is
not — these are our own scrapers hitting 25+ tubes directly (kind=scraper,
origin tube:<sitetag>). Renamed to "tube-scraper" via a single SCRAPER_SOURCE_NAME
constant; DB row renamed in place (UPDATE name, same id) so all ingest_runs +
external_records history stays linked. No behavior change — external_id keying
(sitetag:url) and dedup are unaffected.
NOTE: playback_sources.origin "pornapp:<sitetag>" prefix is a separate legacy
format (resolve_playback parses it) and is intentionally left untouched.
Verified on prod: row renamed (0 stray "pornapp"), new runs land on "tube-scraper".
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
We ingested only ~3% of each browse tube's catalog (porndoe >62k scenes; we had 1959)
because tubes were hit only by performer-search + top-N browse. Pilot (porndoe pages
64-110): 1119 new scenes, 100% playable + 100% tagged, 0% canonical overlap (purely
additive — content not in TPDB/StashDB).
- app/scheduler/deep_crawl.py: round-robin over ALL_BROWSE_SCRAPERS, per-tube page cursor
in app/_state/deepcrawl_state.json (no DB migration), deep-paginate from the cursor,
idempotent (resolver skips known by raw_hash), mark 'exhausted' at catalog end then
reset cursors for an incremental re-sweep.
- _job_deep_crawl: hourly, 60 pages/run (~1860 scenes, ~22 min), wrapped in the 1h
hard-timeout; registered in build_scheduler (jobs=10).
- config: sched_deep_crawl_hours=1, deep_crawl_pages_per_run=60, deepcrawl_state_path.
- scripts/pilot_porndoe_deepcrawl.py: one-off pilot used to validate the approach.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
