MOTIF: Person-of-Interest Deepfake Detection Beyond 3DMM Coefficients
Video deepfakes targeting a specific individual, the Person-of-Interest (POI), are the most harmful ones, and, since a public figure is abundantly recorded, a detector can be built from genuine footage of that individual.
ProofPaper ↗
Key points
- Such detectors commonly describe a subject through a 3D Morphable Model (3DMM) and adopt its coefficients as a whole, so which part of that description carries the signal has never been measured.
- We dissect it, holding the encoder, the training corpus and the enrollment protocol fixed and varying only what the encoder observes.
- We further show that the dense surface the same fit returns, which these detectors discard, carries identity information that the coefficients do not, and that it helps precisely where they are weakest.
- We assemble the best configuration into MOTIF, a visual-only detector trained on real videos only, with no manipulated video and no POI-specific data.
Sources (1)
- [1]MOTIF: Person-of-Interest Deepfake Detection Beyond 3DMM CoefficientsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 10:51 AM
Video deepfakes targeting a specific individual, the Person-of-Interest (POI), are the most harmful ones, and, since a public figure is abundantly recorded, a detector can be built from genuine footage of that individual.
Such detectors commonly describe a subject through a 3D Morphable Model (3DMM) and adopt its coefficients as a whole, so which part of that description carries the signal has never been measured.
Extractive summary: sentences quoted from the sources.