fix(movie-enrich): studio/performer disambiguation for generic TPDB titles

Generic movie titles ("Monster Tits", "Pirates") map to many different TPDB
films with identical titles (different studios/casts, all title-score 1.0). The
old matcher searched per_page=10 and ranked by title only, so for a generic title
the correct film was often not even in the top 10, and among same-title
candidates it picked arbitrarily. Result: a ~95% no_match rate and silent
misattribution (e.g. "Monster Tits" by Venom Digital Media would get Galaxy
Productions' TPDB entry).

_best_match now:
  - searches per_page=40 (the right film for a generic title is often past top 10),
  - ranks title-gate survivors by a composite of title + studio similarity +
    performer overlap (our studio/cast from the primary source disambiguate which
    same-title film it is),
  - guards against misattribution: if we have a studio/cast signal and there is
    more than one near-identical-title candidate but the winner shares neither
    studio nor cast, return no_match instead of attaching a wrong same-title film.

Verified on prod data: a no_match-with-studio sample now matches 18/18 with the
correct studio (fixing Galaxy to Venom, Exquisite to Rodney Moore, and a no_match
to Cherry Boxxx), and an already-enriched sample keeps 16/18 identical picks with
the 2 differences being the same studio (benign TPDB duplicate). No wrong-studio
regressions observed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
goon-foss 2026-07-05 11:31:38 +02:00
parent e215b48255
commit 4baf6ea896

View file

@ -34,6 +34,8 @@ from sqlalchemy.orm import Session
from app.connectors.tpdb import TPDBConnector, _parse_movie
from app.models.movie import Movie, MovieExternalRef, MoviePerformer
from app.models.movie_playback_source import MoviePlaybackSource
from app.models.performer import Performer
from app.models.studio import Studio
from app.normalize.movies import normalize_movie
from app.normalize.text import normalize
from app.resolve.movie_merge import merge_movies
@ -56,33 +58,113 @@ def _cand_year(raw: dict) -> int | None:
return None
def _best_match(connector: TPDBConnector, movie: Movie, *, min_title: float) -> dict | None:
def _movie_studio_name(session: Session, movie: Movie) -> str | None:
if not movie.studio_id:
return None
st = session.get(Studio, movie.studio_id)
return st.name if st is not None else None
def _movie_performer_names(session: Session, movie: Movie) -> set[str]:
"""Znormalizowane nazwy performerów już przypiętych do filmu (do rozróżniania
kandydatów TPDB o tym samym tytule)."""
rows = session.execute(
select(Performer.name_normalized)
.join(MoviePerformer, MoviePerformer.performer_id == Performer.id)
.where(MoviePerformer.movie_id == movie.id)
).all()
return {r[0] for r in rows if r[0]}
def _cand_studio_name(raw: dict) -> str:
site = raw.get("site") or {}
name = site.get("name") if isinstance(site, dict) else site
return normalize(name or "")
def _cand_performer_names(raw: dict) -> set[str]:
return {
normalize(p.get("name"))
for p in (raw.get("performers") or [])
if isinstance(p, dict) and p.get("name")
}
def _best_match(
session: Session, connector: TPDBConnector, movie: Movie, *, min_title: float
) -> dict | None:
"""Najlepszy TPDB movie payload dla naszego filmu, albo None.
`token_sort_ratio` (a NIE token_set) na znormalizowanych tytułach: odporny na
inwersję ("Title, The" "The Title") i kolejność, ALE penalizuje różnicę długości,
więc krótki generyczny tytuł nie łapie dłuższego nadzbioru (bug: "Fantasies"
"Tara's Fetish Fantasies", bo token_set nagradza podzbiór). Precyzja > recall:
lepiej pominąć niż wzbogacić zły film. Guard roku ±2 (inny rok = inna edycja)."""
Tytuł to gate (`token_sort_ratio` >= min_title): odporny na inwersję i kolejność,
penalizuje różnicę długości (krótki generyczny tytuł nie łapie dłuższego nadzbioru).
ALE generyczne tytuły ("Monster Tits", "Pirates") mają w TPDB WIELE różnych filmów
o tym samym tytule (różne studia/obsady) i identycznym score 1.0. Sam tytuł ich NIE
rozróżni, a `per_page=10` często w ogóle nie zwracał właściwego (poprawny film bywa
poza top-10). Dlatego:
- `per_page=40` (właściwy film generycznego tytułu bywa dalej w wynikach),
- wśród kandydatów przechodzących title-gate wybieramy po KOMPOZYCIE:
tytuł + zbieżność studia + pokrycie obsady (nasze studio/obsada z primary
źródła jak paradisehill rozróżniają który to film),
- GUARD anty-misattribution: jeśli mamy sygnał (studio na naszym filmie) i jest
>1 kandydat o ~identycznym tytule, a zwycięzca NIE dzieli studia, to no_match
(lepiej nie wzbogacić niż przypiąć zły film o tej samej nazwie).
Precyzja > recall. Guard roku ±2 (inny rok = inna edycja)."""
query = (movie.title or "").translate(_DASH).strip()[:60]
if not query:
return None
my_norm = normalize(movie.title)
my_studio = _movie_studio_name(session, movie)
my_studio_norm = normalize(my_studio) if my_studio else ""
my_perfs = _movie_performer_names(session, movie)
best: dict | None = None
best_score = 0.0
for raw in connector.search_movies(query, per_page=10):
best_key: tuple[float, float, float, float] | None = None
best_studio_sim = 0.0
best_perf_overlap = 0.0
n_exact_title = 0
for raw in connector.search_movies(query, per_page=40):
cand_title = raw.get("title")
if not cand_title:
continue
score = fuzz.token_sort_ratio(my_norm, normalize(cand_title)) / 100.0
if score <= best_score:
tscore = fuzz.token_sort_ratio(my_norm, normalize(cand_title)) / 100.0
if tscore < min_title:
continue
cy = _cand_year(raw)
if movie.release_year and cy and abs(movie.release_year - cy) > 2:
continue # guard: inny rok → prawdopodobnie inny film (Taxi 2 ≠ Taxi Violeur 2)
best_score = score
best = raw
return best if best is not None and best_score >= min_title else None
if tscore >= 0.97:
n_exact_title += 1
cand_studio = _cand_studio_name(raw)
studio_sim = (
fuzz.token_set_ratio(my_studio_norm, cand_studio) / 100.0
if my_studio_norm and cand_studio
else 0.0
)
cand_perfs = _cand_performer_names(raw)
perf_overlap = (len(my_perfs & cand_perfs) / len(my_perfs)) if my_perfs else 0.0
# Kompozyt: tytuł jako baza, studio i obsada jako rozróżniacze (waga 0.6 każdy).
composite = tscore + 0.6 * studio_sim + 0.6 * perf_overlap
key = (composite, tscore, studio_sim, perf_overlap)
if best_key is None or key > best_key:
best_key = key
best = raw
best_studio_sim = studio_sim
best_perf_overlap = perf_overlap
if best is None:
return None
# Anty-misattribution: mamy czym rozróżnić (studio lub obsada na naszym filmie), jest
# >1 kandydat o ~identycznym tytule, a zwycięzca nie dzieli NIC (ani studia, ani
# obsady) → to prawie na pewno inny film o tej samej nazwie. Nie wzbogacaj.
my_has_signal = bool(my_studio_norm) or bool(my_perfs)
if (
my_has_signal
and n_exact_title >= 2
and best_studio_sim < 0.5
and best_perf_overlap < 0.5
):
return None
return best
def _fill_scalar_fields(movie: Movie, norm) -> None:
@ -123,7 +205,7 @@ def enrich_movie(
if already is not None:
return "skip"
raw = _best_match(connector, movie, min_title=min_title)
raw = _best_match(session, connector, movie, min_title=min_title)
if raw is None:
return "no_match"
ext_id = str(raw["id"])